Problem: Many people are able to recognize the personality traits of the person they are talking to by their facial features. Experts in non-verbal communication can do this even with a photograph. But is it possible to teach artificial intelligence to do the same?
Solution: A cascade neural network was ‘fed’ a labelled dataset. It consisted of test results and 31,367 photos of 12,500 volunteers. After learning, an algorithm was able to predict five general personality traits. The neural network was better at recognizing conscientiousness as compared to the other four traits.
Psychology and artificial intelligence researchers from HSE University, Open University for the Humanities and Economics, and the Russian-British business start-up BestFitMe trained a cascade of artificial neural networks to recognize the ‘Big Five’ personality features based on photographs of human faces. The study has been published in Scientific Reports. The test dataset is available in the repository.
Physiognomy — the practice of assessing a person's character based on their appearance — has always been popular in European culture. Evolving from works by Ancient Greek philosophers Aristotle and Theophrastus, it reached its culmination during the 18-19th centuries in research by Johann Kaspar Lavater, Charles Darwin, and Cesare Lombroso. As these ideas were subjected to empirical trials in the 20th century, physiognomics was acknowledged as pseudo-science. Meanwhile, research done in recent years demonstrates that a correlation between facial traits and personality features does still exist.
First, it has been found that some psychological traits are determined genetically. The contribution of the genetic factor for the ‘Big Five’ features varies from 30 to 60%. Genes also determine the shape of skull bones, which determine certain facial features. It is believed that sexual selection has impacted the evolution and correlation between craniofacial and personality features. Females have looked for clear features of useful or safe personality traits, which has caused these traits to become sexually attractive (e.g., positive selection by forehead height), followed by inheritance.
In addition, the shape of the face, as well as the person’s behaviour is impacted by the prenatal and postnatal effect of hormones. The shape and size of cheekbones and the lower jaw, the face height-to-width ratio, as well as some other traits are visible indicators of the level of testosterone and oestrogen-two sex hormones. They are also responsible for risk appetite, aggressiveness, inclination to compete and dominate or, on the contrary, softness, compliancy, tenderness and considerateness.
Some studies have shown a correlation between facial traits and disposition to certain behavioural patterns, expressed as five basic personality features: extraversion, agreeableness, conscientiousness, neuroticism, and openness to experience.
People usually guess certain personality traits by facial features quite precisely. Professional psychologists and experts in non-verbal communication demonstrate an even higher correlation between their forecasts and the data of personality questionnaires. This means that artificial intelligence can also be taught this skill if a sufficient and well-labelled dataset is prepared for the training.
In their new study, the researchers from HSE University and the technology start-ups used social media to gather 77,346 photos of 25,202 volunteers. All photos were made using web cameras under controlled conditions (neutral facial expression, frontal view, eyes looking in the camera, good lighting, lack of makeup or jewellery). The respondents were also asked to complete an extended online questionnaire 5PFQ to outline their personality portrait and the degree of the ‘Big Five’ personality traits. After the incomplete questionnaires and invalid photos were removed, the final dataset was formed. It consisted of 12,447 completely valid descriptions of personality traits from the questionnaires and 31,367 photos. It included an average of 2.59 photos from females and 2.42 from males per volunteer questionnaire. The dataset was randomly split into two parts. The first part (90% of the data) was used as a training sample for the neural network. The second (10%) was a control sample used to evaluate the predictive capabilities of the algorithm.
Initially, the neural network was trained to differentiate the faces of different people and detect the face of the same person. Next, the algorithm was taught to decompose each image into 128 invariant features — regularly reoccurring individual traits. Inside the model, each invariant was presented as a vector in a 128-dimension space.
The obtained data was put in a multi-layer perceptron that used artificial neurons to compare the image invariants to personality traits. If they concurred, the data was ‘fixed’, but if there was a divergence, the error was re-entered in the neural network. Gradually, the artificial intelligence learned to match the facial traits to personality features with increasing precision.
The first published results are still far from ideal and rather look as a proof of concept. The coefficient of correlation between questionnaire data and the algorithm predictions varied from a low 0.14 to a reassuring 0.36. The network was best at assessing ‘conscientiousness’, with a correlation of 0.36 for male faces and 0.335 for females, while the worst indicators were in ‘openness to experience’. Interestingly enough, the algorithm was generally much better at predicting extraversion and neuroticism in women rather than in men.
The average effect size of 0.24 indicates that artificial intelligence can make a correct guess about the relative standing of two randomly chosen individuals on a personality dimension in 58% of cases as opposed to the 50% expected by chance. The 10% difference seems insignificant, but in fact, artificial intelligence outperforms humans in terms of prediction precision when judging by the facial traits of a stranger.
If the quality of the algorithm is further improved, it can potentially be used in recommendation systems of online shops and services. It might also provide good opportunities for HR departments by offering quick psychological diagnostics during Zoom interviews with job applicants. This method will be particularly effective for mass recruitment, such as taxi drivers, salespersons, cleaners etc. It can help exclude aggressive, mentally unstable, and unscrupulous people.
Another potential market is dating apps and websites, as well as services for psychological assessment of strangers on social media. This application could significantly improve women’s security when they date men they meet online.
Authors of the Study:
Evgeny Osin, Associate Professor, Deputy Head of the HSE International Laboratory of Positive Psychology of Personality and Motivation;
Denis Davydov, Open University for the Humanities and Economics;
Alexander Kachur, Artificial Intelligence LLC;
Konstantin Shutilov and Alexey Novokshonov, BestFitMe.