Skinmesh: Machine learning for facial analysis

When we talk about machine learning (ML) in dermatology, the only thing that comes to mind is the melanoma detectors. But there are many other applications for machine learning in dermatology, even in the cosmetic industry. Cosmetic dermatology never gets much interest in the ML domain, despite being a highly lucrative industry.

skinmesh: machine learning for face analysis
AndyD at English Wikibooks, CC BY-SA 2.5, via Wikimedia Commons

Generative Adversarial Networks (GAN) is an interesting machine learning method with potential application in cosmetic dermatology. In simple terms, GAN’s are two machines, one trying to create something fake that the other is trained to identify. You can instantly see the application in cosmetic dermatology, where we try to make credible fakes. Google’s team does some excellent work on GANs in dermatology.

One downside of machine learning is the difficulty in making them available to users in a useful way. Some of the emerging techniques such as transfer learning are making the models small and robust enough to fit into ordinary devices such as mobile phones. TensorflowJS is one such javascript-based library of concise models, that can be instantly deployed on any device for some standard applications.

I recently experimented with TensorflowJS and found it easy, intuitive and useful. I made this simple react app to capture the coordinates of facial landmarks using the facemesh model. Skinmesh is a simple React component that uses the facemesh model from Tensorflowjs. The goal is to capture the facial landmarks for analyzing the facial symmetry for cosmetic treatments such as botulinum toxin injections and dermal fillers. This can be used in any front-end application to harvest facial coordinates for some useful predictions. Now I will work on the backend prediction algorithm. Does anybody want to jump in and collaborate?

On a different note, we have recently published a tool for capturing lesions and their distribution in a way conducive to machine learning. Do check it out!

And once again, here is the link to skinmesh!

Bell Eapen
Follow Me
About Bell Eapen 247 Articles
Techie Dermatologist, Information Systems PhD, Supporter of Open-Source Software, Machine Learning and AI geek, loves cricket, Canadian wine and beer. [Resume]

1 Trackback / Pingback

  1. Embeddings in healthcare: TypingDNA and Skinmesh - Bell Eapen

Leave a Reply

Your email address will not be published.


This site uses Akismet to reduce spam. Learn how your comment data is processed.