.jpg)
Artificial intelligence has no doubt made a massive impact in every industry making businesses more efficient and productive, and recruitment is no exception. Ingram Technologies has collaborated with Recrewty to explore new ways to refine the hiring process through the use of AI-driven chatbots.
This pioneering study is the result of a unique collaboration between two forward-thinking companies: Recrewty, a startup from Montenegro that develops tailored hiring automation solutions, and Ingram Technologies, an AI R&D company from Belgium. These two companies connected through the X2 deep tech project which aims to foster collaboration in cutting-edge technological advancements. Together, we’re exploring the emerging intersection of machine learning and AI to push the boundaries of recruitment technology.
In a recent study aimed at understanding AI’s ability to accurately infer personality traits and mitigate social desirability bias–a phenomenon that traditional psychometric tests are unable to detect, we produced promising results but there is still much to learn.
What made this pilot study particularly promising is that it is the first of its kind conducted in an underrepresented language, specifically Montenegrin, Serbian, Croatian, and Bosnian. AI-based tools are often developed and tested primarily in English, limiting their adaptability to other languages and cultural contexts.
Our study broke this barrier by deploying an AI chatbot tailored to these South Slavic languages. This expands the global applicability of AI-driven hiring technologies and opens new doors for their use in diverse linguistic settings.
Traditional psychometric tests have been a staple for many companies in their recruitment process. These tests are considered essential as they are able to measure critical personality traits such as conscientiousness, agreeableness, and openness. Testing these traits gives employers a data-driven way to evaluate candidates and aid in selecting the best candidate for the job.
Though the tests have proven to be very useful, they do not come without flaws. For example, these tests do not have strong defenses against a candidate who will “game” the system and answer questions in a manner they deem acceptable in the eyes of their potential employer.
In professional settings stakes are high so oftentimes candidates will consciously or unconsciously manipulate their responses to appear more conscientious or agreeable. This is a common phenomenon known as social desirability bias, this is crucial to detect as it can distort results and lead to a less-than-ideal hire.
Our study provides a fresh perspective on the utilization of AI chatbots to infer personality traits. To do this we came up with a simple yet powerful idea: why not use chatbots to analyse responses in a more dynamic, conversion-like environment ?
Doing this would reduce the risk of candidates faking their answers producing more accurate psychometric results. Due to a chatbot's ability to adapt in real-time it can analyse responses in real time unlike a static traditional questionnaire, mimicking a more natural interaction.
One key innovation in our research was the introduction of a “one-question-per-facet” approach. Instead of being bombarded with lengthy tests, candidates are asked one targeted question to measure each facet of a personality trait.
The results of this study showed that AI chatbot assessments were generally immune to social desirability bias. This breakthrough finding showed that AI tools could potentially offer a more accurate representation of a candidate's personality.
The chatbots’s psychometric properties were comparable to models used by the leading companies and studies, usually from English-speaking areas. Despite being in its pilot phase, our approach demonstrated that AI-driven assessments can hold up against traditional methods in terms of reliability and validity.
This study was conducted in a real-world hiring scenario across seven countries in the Balkan region,including industries such as IT, retail, and banking, providing valuable insights into the practical applications of AI in recruitment. This real world application provided a robust test of the chatbot’s capabilities in diverse industry contexts.
Candidates that participated in the study were compared to a control group of professionals outside of the hiring contexts. The chatbot was tasked with inferring personality traits while maintaining its integrity against the effects of social desirability bias.
To provide a more granular view of each personality trait our AI chatbot utilizes the Big Five model to assess traits like extraversion, agreeableness, conscientiousness, neuroticism, and openness. Traits such as extraversion and conscientiousness were easier to analyze by the chatbot while neuroticism and agreeableness served to be more complicated to measure.
One of the central hypotheses of the study was that AI chatbots would be less susceptible to social desirability bias compared to a traditional psychometric test. Our findings supported this hypothesis with results showing that AI-inferred scores were less influenced by social desirability. In traditional psychometric assessments, candidates in professional selection scenarios exhibited inflated scores on socially desirable traits, particularly conscientiousness.
AI chatbots maintained consistency across the control and experimental groups and showed no significant increase in socially desirable responses. These findings reinforce the potential of AI tools to provide a more accurate picture of a candidate's true personality in high-stakes environments.
While the chatbot performed well in assessing traits like extraversion and conscientiousness, there were challenges with other traits, such as neuroticism and agreeableness. These traits showed weaker psychometric properties in the chatbot assessments, indicating that more work is needed to fine-tune the AI’s capabilities.
We were able to gather psychometric properties that are on par with more established models through strategic interventions, such as the one-question-per-facet methodology. This innovation allowed us to pinpoint where the chatbot’s performance was strong and where it needed enhancement. We believe that this targeted approach will lead to continuous improvements in AI-driven assessments.
This pilot study marks an exciting step forward in the integration of AI and psychometrics in recruitment. By reducing social desirability bias, AI chatbots offer a promising alternative to traditional personality assessments, providing employers with more authentic insights into candidates. Moreover, the expansion of AI tools into underrepresented languages like Montenegrin, Serbian, Croatian, and Bosnian is a significant stride towards making AI technology accessible and effective in diverse global markets.
At Ingram Technologies and Recrewty, we’re committed to continuing our research and development in this area. Our goal is to continue creating AI tools that not only make hiring more efficient but also more equitable by ensuring that candidates are evaluated on their true potential, free from the distortions of bias.
As we move forward, we’ll be exploring larger-scale studies and refining our chatbot to improve its performance across all personality traits. The journey has just begun, and we’re excited to see where this technology will take us.
© Ingram Technologies
Ingram is an EU-based Artificial Intelligence R&D lab established in 2022.
We work on ethical, compliant and open AI-based software solutions for startups and SMEs.