Ding Haopeng, Yunfei Chen (2023). Facial acne recognition system based on machine learning. In: Intelligent Human Systems Integration (IHSI 2023): Integrating People and Intelligent Systems. AHFE Open Access, vol. 69, pp. 153–158. https://doi.org/10.54941/ahfe1002832
Facial acne plagues many people, causing appearance anxiety and even psychological problems. However, the skin detector or software using traditional image processing technology on the market cannot give consideration to both low cost and high precision. This research aims to develop a low-cost and efficient method to detect facial acne through machine learning. We use hundreds of facial acne patients' pictures collected on the network, use Photoshop to split into thousands of pictures of appropriate size and manually label them as data sets and verification sets, and train them in YOLOX model to finally identify and label skin problems such as facial pustules, acne marks, etc. through one person's facial photos. At present, we have run the system on the desktop (AMD R7 4800H+GTX1650) normally, using the latest YOLOX framework of the open-source YOLO series. In order to improve the learning quality under limited training data, image preprocessing including sharpening and flipping is introduced. The experimental results show that the recognition rate of this method for some skin problems can reach 80%. By further expanding the data set, it can achieve low-cost facial problem recognition. At the same time, this research is also a good case of applying deep learning technology to product design.
Acne vulgaris is a common chronic inflammatory skin disease of hair folli- cles and sebaceous glands in adolescence. Its symptoms mainly include acne, papules, pustules, nodules, cysts and scars, and it usually occurs in the face, back, chest and other parts rich in sebaceous glands of teenagers (Zhao, 2001). It is considered as a “puberty” disease because of its characteristics of “occurring in pre puberty and self-healing in late puberty”. However, with the acceleration of the pace of life, changes in diet habits and increased life pressure, the incidence rate of acne vulgaris among modern adolescents has gradually increased, and the condition is serious. There is a research report that acne will make patients lack self-confidence, leading to a decline in their quality of life, and in more serious cases can lead to depression, resulting in mental disorders (Lauren K D et al., 2011). About 8.8% of acne pati- ents have depression, and the depression rate of women is twice that of men (Uhlenhake E et al., 2010).
At present, most of the mainstream skin quality detection methods use large mechanical skin detection instruments, but this method has some limita- tions in real-time and portability. Therefore, how to objectively, conveniently and accurately detect and evaluate the quality of human face skin is an urgent problem to be solved in the medical beauty industry. In this paper, the human facial skin quality is taken as the research object, and the skin texture detection algorithm is studied. Based on the object detection algo- rithm YOLOX, a set of mobile face skin quality detection and evaluation system is developed, which realizes the goal of real-time and convenient skin quality detection. At present, we run this program on the desktop end (AMD R7 4800H+GTX1650) system. We use the aperture f/1.6, 26mm, 1.7m, and 12 megapixel cameras to capture images. We use the LED table lamp with a color temperature of 3900K, a color rendering index of 95, and a lighting power of 15w to illuminate. The data set of 2000 facial acne symptoms was obtained by using image processing software from the facial photos of 70 adolescents of different ages and genders collected through the network.
The rest of this paper is organized as follows. The next section will briefly introduce the background of this study. In section 3, I will elaborate on the proposed method, including the determination of target detection model, the design of system framework and the process of making data sets. Section 4 will introduce the experimental results and performance test results obtained so far. The fifth part summarizes the research work.
Background
For the research of irregular pigment deposits such as acne, spots and nevus, there are mainly traditional visual method, instrument detection relying on computer image processing and algorithm detection relying on computer vision. The traditional visual method is more accurate, but requires doctors to have rich experience. At present, the mainstream detection methods mainly rely on digital image processing strategies, using filtering, binariza- tion, color space conversion and other methods to detect facial skin. In 2020, Bose et al. (2020) used histogram gradient eigenvalue and Bootstrap aggre- gate decision tree classifier to detect face identity and facial spots. In the same year, Darmawan et al. (2020) realized the detection of facial acne type by extracting the values of R, G and B channels of acne in Indonesian skin images. In 2021, Ko et al. (2021) designed a skin analysis system based on image processing, which uses the noise based DBSCAN algorithm to achieve accurate classification of skin disease areas such as acne and spots.
With the emergence of deep learning, the development of algorithm dete- ction relying on computer vision has been greatly promoted (Li et al., 2016). Huang Shuting (2019) established a skin defect detection and evaluation system based on VGG16, which can evaluate skin problems such as wrinkles, spots, acne, and large pores on the skin surface. Tao Liuqing (2021), based on the lightweight convolutional neural network Mobile Net V3, studied the skin detection algorithm, developed a set of Android face skin quality dete- ction and evaluation system, and realized the goal of real-time and convenient skin quality detection.
Facial Acne Recognition System Based on Machine Learning 155
The Proposed Approach Target Detection Model Determination Through the above investigation, we found that the current machine learning ability has been enough to support the detection of facial acne, but what kind of target detection model to use for detection can ensure that the detection results can be accurate enough while having real-time and portability, which is a problem that needs to be solved.
In the use scenario, we hope to use an intelligent terminal with a camera, such as an iPad, a mobile phone or an intelligent beauty mirror, to take photos of the face, and then mark the acne on the face with a label box, and finally present it to the user. Therefore, in such a use scene, although the user is not deliberately fixed in the angle of taking pictures or using professional lights, it can still ensure that the pictures taken have similar specifications and clarity.
Through the research on current target detection algorithms, compared with two stage target detection algorithms such as R-CNN, YOLO series target detection algorithms using One stage are more suitable for mobile ter- minal deployment. Yolo algorithm uses a separate CNN model to achieve end to end target detection, directly divides the original image into small blo- cks that do not coincide with each other, and then obtains a feature map of the same size through convolution. Based on the above analysis, it can be considered that each element of the feature map is also a small block corre- sponding to the original image, and each element can be used to predict the targets whose center points are in the small block. This method can not only effectively save the time of recognition, but also recognize skin problems of different sizes on the face.
After comparing YOLOv5 and YOLOX target detection algorithms, we finally chose YOLOX target detection algorithm to recognize facial acne.
System Framework It is not enough to determine the target detection model used. YOLO series algorithms will adjust the input image to the size of 448 * 448 pixels. If the input whole face image is adjusted to such a definition, it will be difficult to identify subtle skin problems. If you use a macro camera to take pictures of local skin, although the system can clearly identify skin problems, it does not meet the other party’s pursuit of ease of use at the beginning of the system design.
Therefore, we must further adjust the image processing process: As shown in the flow chart below, the user will use the camera to take a picture of the left and right sides of the face, and then the two pictures will be sent to the system successively. The system will adjust the original photo to an image with a resolution of 1664 * 1248, then divide the photo into 12 pictures with a size of 416 * 416, and input each picture into YOLOX target detection algorithm for recognition processing. The final system will rearrange all the segmented images in the order before segmentation, and output the labeled images.
156 Ding and Chen
Figure 1: System framework flow chart.
Production of Data Sets The current facial data set resources are mainly concentrated in face reco- gnition, emotion recognition, etc., and there is no data set for facial skin problems. On the other hand, medical high-precision data set mainly collects skin mirror data such as melanoma, which is also unavailable. Therefore, we must create a data set for facial skin details.
Because of the COVID-19, it is more difficult to collect facial photos offline, so the pictures are basically collected through the network and the consent of the image owner is obtained. The specific data set production process is as follows: As shown in the flow chart: collect the facial photos of 70 teenagers with different degrees of facial acne through offline, online and other channels, use the image processing software (ps2018) to capture 500 screenshots (416 * 416) of facial skin acne symptoms from the photos, and use lambimg to mark the screenshots, which include: pustules, white heads, blackheads, moles, acne marks, old acne marks, papules, a total of 7 skin problems. Then use python script to flip and rotate the annotated screenshots to generate 2000 annotated pictures. Finally, put the annotations and pictures into the yolox model for training to get the training model.
Preliminary Results We will test the trained model to know the recognition accuracy. The test consists of five whole facial pictures. The artificial and training models are used to recognize the five facial pictures respectively, and the recognition rate and false recognition rate of pustules, white heads, blackheads, moles, pockmarks, old pockmarks and papules are calculated.
Facial Acne Recognition System Based on Machine Learning 157
Figure 2: Data set production flow chart.Table 1. Various skin problems in five manually marked pictures.
Table available in the original PDF paper.
1-a 2-a1 5-a1 6-b1 8-a1 pustule 7 9 1 5 3 white head 2 3 2 4 0 mole 1 3 4 4 6 acne mark 5 17 0 11 0 old acne mark 3 2 4 0 2 total 18 34 11 24 11
Table 2. Recognition results using YOLOX target detection algorithm.
Table available in the original PDF paper.
1-a 2-a1 5-a1 6-b1 8-a1 total pustule 100.00% 55.56% 100.00% 100.00% 100.00% 84.00% white head 100.00% 33.33% 0.00% 100.00% − 63.64% mole 0.00% 33.33% 75.00% 50.00% 16.67% 38.89% acne mark 100.00% 17.65% − 100.00% − 57.58% old acne mark 33.33% 50.00% 75.00% − 0.00% 45.45% Correct identification 83.33% - − 81.48% 36.36% 58.42% Misidentification 25.00% 26.67% − 4.35% 20.00%
The following table (Table 1) shows the skin problems in the five pictures manually marked.
The following table (Table 2) shows the recognition results using yolox target detection algorithm.
It can be found from the results that the recognition rate of pustule, white head and acne mark is more than 50%, while the recognition rate of other skin problems is low. This problem is mainly caused by the small data set. In this study, we encountered some difficulties in obtaining enough images of facial skin problems, because in order to avoid privacy problems, the photos
158 Ding and Chen
we collected must be approved by the photo owner. This inconvenience limits the acquisition of data sets to a certain extent.
CONCLUSION
This study provides a method to detect acne on human face through mach- ine learning. At the current stage, we have collected 70 facial acne pictures through the network, offline and other ways, and generated training sets from them for machine learning and accuracy verification. We use YOLOX target detection model to identify skin problems on the face of teenagers, including pustules, white heads, blackheads, moles, acne marks, old acne marks, and papules. In order to enable YOLOX model to more accurately identify skin problems in images, we have preprocessed the input and output photos to ensure the accuracy of recognition.
In the next research, we hope to cooperate with beauty hospitals or institu- tions to greatly expand the data set to increase the accuracy of identification. In addition, with the increase of the data set, we hope to be able to make more detailed identification of various skin problems, and combined with the research on the factors that cause facial acne, so that identification can estimate the recovery cycle of skin problems, and further improve the user experience.
This research has a certain reference value for the research of adolescent skin health. Our recognition system also provides a way of thinking for in- depth learning in daily life, and stimulates a practical application scenario of artificial intelligence.
REFERENCES
Bian Zhao. (2001). Clinical Dermatology, Nanjing: Jiangsu Science and Technology Press. Bose. P., (2020). A Method for Identification of Face of the Person and Presence of
Facial Spot in the Face. International Journal for Research in Applied Sciences and Biotechnology (IJRASB), Volume 7 No. 5. Darmawan. A., Rositasari. A, Muhimmah. I., (2020). The Identification System of Acne Type on Indonesian People’s Face Image, Conference Series: Materials Science and Engineering. IOP Publishing, 803(1). Huang Shuting, (2019). Human face skin defect detection and evaluation system.
Xi’an University of Electronic Science and Technology, MA thesis KO. J. and Cheoi. K. J., (2021). Image-processing based facial imperfection region detection and segmentation. Multimedia Tools and Applications, 1–14. Lauren, K D., O’Neill, J L., Feldman, S R. (2011). Acne in adolescents: quality of life,
self-esteem, mood, and psychological disorders. Dermatology Online Journal, 17 (1): 1. Li Yandong, Hao Zongbo, Lei Hang, (2016). Review of convolutional neural network research. Computer Application, 36 (09). Tao Liuqing. (2021). Facial skin quality detection and evaluation system. Xi’an
University of Electronic Science and Technology, MA thesis. Uhlenhake E, Yentzer B A, Feldman S R. (2010) Acne vulgaris and depression: a retrospective examination. Journal of Cosmetic Dermatology, Volume 9 No. 2, 59–63.
LISTED IN THE AHFE DIGITAL BOOKSTORE CATALOGUE
Explore the complete digital volume
Intelligent Human Systems Integration (IHSI 2023): Integrating People and Intelligent Systems
Read this individual research paper free through the original publisher, or explore the full digital proceedings book.
ISBN 978-1-958651-45-2
Explore this ISBN in the AHFE Bookstore ↗Listed in the July 2026 bookstore catalogue; search by ISBN to confirm current availability · Digital book purchase optional