New Tools Tackle AI Bias and Hallucinations: HP’s Integration with Galileo Luna
Gayathri Gopalan
HP has recently announced the integration of Galileo’s Luna with the Z by HP AI Studio. This integration empowers AI teams to build and deploy production-grade applications with greater confidence. This partnership combines HP’s robust AI platform and Galileo’s comprehensive evaluation and observability tools, including the Luna Evaluation Foundation Models.
At The Inclusive AI (TIA), we have long argued that bias and unreliability in AI systems are not just technical problems—they are ethical imperatives that demand urgent attention. The potential for AI to reinforce existing societal biases or generate misleading information poses serious risks to the fairness and trustworthiness of these increasingly ubiquitous systems.
In this article, we’ll justify why we at TIA believe this integration addresses the twin challenges of AI bias and hallucinations, and why we believe it could lead to more reliable, fair, and trustworthy AI applications across various industries.
Table of Contents
What Is AI bias?
In simple terms, AI bias occurs when machine learning algorithms produce unfair or prejudiced results. This happens when the algorithms make incorrect assumptions during the learning process. In today’s situation, where inclusivity is paramount, AI bias can be especially problematic because it can reinforce existing biases.
Two Examples of AI Bias
Facial Recognition Bias
Many people of color have been wrongly identified as suspects in criminal investigations that used facial recognition software. In this regard, Ifeoma Nwogu, a computer science professor at the University at Buffalo, says:
“Devices have been created in ignorance of the population they are going to interact with, intentionally or not. Even the cameras we use aren’t set up to measure and identify darker skin tones. I, personally, have to edit my photos and increase the exposure so that I’m not lost to the background.”
Facial recognition technology is trained to be accurate when the person in a photo has white skin. However, the darker the skin, the more error-prone the system becomes. For instance, a report published in the NY Times shows that up to 35% of dark-skinned women were wrongly identified by facial recognition software. These real-world biases invariably seep into artificial intelligence (AI), the fundamental framework that powers facial recognition technologies.
Gender Bias
A 2023 MIT study reveals that language models often exhibit gender bias when associating certain professions with specific genders. For example:
- Professions like “flight attendant,” “secretary,” and “physician assistant” are more strongly associated with feminine terms.
- Jobs such as “fisherman,” “lawyer,” and “judge” are more strongly associated with masculine terms.
This bias in language models can perpetuate harmful stereotypes about gender roles in various professions. These biases can have real-world consequences, potentially influencing hiring practices, career advice, or how people perceive certain professions.
What are AI Hallucinations?
AI hallucinations occur when an AI model, particularly a large language model (LLM), generates false or misleading information and presents it as fact. These hallucinations can range from minor inconsistencies to wholly fabricated information and are a common problem. Large Language Models (LLMs) and Large Multimodal Models (LMMs) are trained to predict the best string of text depending on your prompt. The more precise your prompt is, the more accurately ChatGPT or any other LLM will choose its text string.
LLMs and LMMs do not actually “know” anything. They are trained to respond to your queries as best as they can. If ChatGPT or any other LLM/LMM does not know the answer to your question, they will provide a nonsensical answer irrelevant to your query. An AI model does not possess factual accuracy and cannot identify falsified information.
For instance, ChatGPT knows that 2+2=4 because it has encountered this fact many times in its training data. If ChatGPT were trained to believe that 2+2=6, it would provide 6 as the answer instead of 4, since it lacks the ability to think independently.
AI hallucinations are an inevitable by-product of creating LLMs since these LLMs always try to respond to your queries by choosing an appropriate string of text.
Some Notable Examples of AI Hallucinations
Some famous examples of AI hallucinations include:
- Google’s AI chatbot Bard (now Gemini) falsely claimed that the James Webb Space Telescope took the first-ever pictures of an exoplanet outside our solar system. In reality, the first images of an exoplanet were taken in 2004, long before the James Webb telescope was launched in 2021.
- In a high-profile incident, lawyers used ChatGPT to prepare a legal brief and cited several cases the AI had completely fabricated. These non-existent cases appeared plausible and relevant to the argument but were entirely made up by the AI.
- When Meta demonstrated its Galactica AI model (designed for scientific research), it generated a paper about creating avatars and cited a fake paper on the topic. The citation appeared legitimate, attributing the non-existent paper to a real author in a relevant field.
- In some cases, AI models have generated entirely fictional historical events or figures. For example, an AI might describe a non-existent battle during World War II or invent a fictional monarch for a real country.
Why Does TIA Think This Integration Is Significant?
Now that we have learned about AI bias and AI hallucinations, it is time to justify the significance of HP AI Studio’s integration between Galileo’s Luna and Z.
AI Bias Detection
AI bias is a silent but powerful issue in developing and deploying AI systems. AI bias can lead to unfair and inaccurate outcomes, with significant ethical and practical implications. Integrating Galileo’s Luna with Z by HP AI Studio reduces bias in AI by providing advanced tools for detecting and correcting biases in your models. With these tools, you can thoroughly analyze your AI systems to identify any disparities in treatment or predictions across different groups. This process includes recognizing biases in your training data and model outputs, and adjusting to ensure more equitable results.
Developers can use Luna’s AI evaluation framework to detect and address bias in their AI models before deployment. Today, most of the AI frameworks are still at a nascent stage. In this vulnerable stage, AI could mirror our society, echoing societal imperfections (such as gender disparities) and reiterating misconstrued notions. By analyzing a model’s outputs and behaviors, Luna can identify potential biases related to protected characteristics, such as race, gender, and age. Once biases are identified, Luna offers actionable guidance for mitigation. Developers can refine their models using debiased datasets or adjust hyperparameters to reduce biased outputs. This trust framework implemented in Z by HP AI Studio aligns with TIA’s emphasis on transparency in AI systems.
This integration directly addresses the need for improved bias detection and mitigation through these measures. AI systems can inadvertently learn and amplify biases present in their training data. Luna’s continuous monitoring and evaluation capabilities allow for real-time detection of emerging biases, a feature TIA thinks is essential to creating trustworthy AI systems.
The customizable nature of Luna’s evaluation tools is particularly significant in light of TIA’s discussion on the complexity of defining and measuring fairness in AI. Different applications and industries may require different approaches to fairness, and Luna’s flexibility allows organizations to tailor their bias detection and correction strategies accordingly.
Furthermore, the integration’s focus on data security and control addresses a critical perspective on the importance of diverse, representative data sets in developing unbiased AI systems. By enabling organizations to optimize their proprietary data, this integration could lead to more nuanced, context-aware AI models that are less likely to perpetuate existing biases.
Lastly, we at TIA believe this integration could be crucial in reducing bias in AI systems. Providing tools that make bias detection more accessible and understandable could encourage collaboration between technical experts, ethicists, social scientists, and business leaders in addressing AI bias.
AI Hallucinations Detection
Luna is designed to identify when AI models produce outputs that deviate from expected or factual information. By comparing generated content against established data and patterns, Luna can flag instances where outputs exhibit signs of hallucination. This allows developers to address inaccuracies before deployment, ensuring that AI systems deliver reliable and accurate information.
The integration supports continuous monitoring of AI models in production, enabling real-time detection of hallucinations. Luna’s ongoing evaluation of model outputs facilitates immediate corrective actions, helping maintain the accuracy and reliability of AI systems over time. This proactive approach ensures that models remain effective and trustworthy throughout their operational lifecycle.
To mitigate hallucinations further, Luna emphasizes the importance of training AI models with high-quality and diverse datasets. By ensuring that models are developed using well-structured and representative data, the likelihood of generating nonsensical or inaccurate outputs is significantly reduced. This foundational step is crucial in preventing hallucinations from occurring in the first place.
Additionally, Luna also facilitates user interaction by offering guidance on improving model performance and reducing hallucinations. Its feedback mechanisms allow developers to refine their models based on real-world usage and identified errors, promoting a cycle of continuous improvement. This iterative process helps ensure that AI systems evolve and adapt to deliver more accurate and dependable results. According to Galileo, Luna can exceed industry benchmarks for detecting issues like hallucinations by up to 20%.
This level of accuracy is essential for building trust in AI systems and mitigating the risks associated with AI-generated misinformation. Furthermore, the ability to rapidly customize Luna for specialized enterprise use cases with over 95% accuracy is particularly promising. This flexibility allows organizations to fine-tune hallucination detection for their specific needs and contexts, further improving the reliability of their AI systems.
Our Final Thoughts
In conclusion, while the HP-Galileo Luna integration doesn’t solve all the challenges of AI bias and hallucinations, TIA sees it as a significant step in the right direction. It provides practical tools for bias detection and mitigation, promotes transparency, and could facilitate the kind of collaborative, multidisciplinary approach needed to create fairer, more ethical AI systems. As AI grows across industries, innovations like this will ensure these powerful technologies benefit society.
References
Afp. (2024, June 19). AI creating false stories about World War II, Holocaust: UNESCO. The Hindu. https://www.thehindu.com/news/international/ai-creating-false-stories-about-world-war-ii-holocaust-unesco/article68306857.ece
Ali, M. (2024, May 30). AI Identity crisis: Is your virtual assistant rocking a masculine beard or feminine elegance? The Inclusive AI. https://theinclusiveai.com/ai-identity-crisis-virtual-assistant/#ethical-issues-in-ai-development
Endicott, S. (2024, July 15). HP is here to combat hallucinations and bias when making AI models. Windows Central. https://www.windowscentral.com/hardware/computers-desktops/hp-is-here-to-combat-hallucinations-and-bias-when-making-ai-models
Gordon, R. (2023, March 3). Large language models are biased. Can logic help save them? MIT News | Massachusetts Institute of Technology. https://news.mit.edu/2023/large-language-models-are-biased-can-logic-help-save-them-0303
Kaiser, L. (2024, February 21). UB computer science professor weighs in on bias in facial recognition software. University at Buffalo. https://www.buffalo.edu/news/tipsheets/2024/ub-ai-expert-facial-recognition-expert-ifeoma-nwogu.html
Lohr, S. (2018, February 9). Facial recognition is accurate, if you’re a white guy. The New York Times. https://www.nytimes.com/2018/02/09/technology/facial-recognition-race-artificial-intelligence.html
Sharma, S. (2024, June 13). Towards Fairer Hiring Practices: Strategies for Mitigating Bias in AI-driven Recruitment — The. The Inclusive AI. https://theinclusiveai.com/mitigating-bias-in-ai-driven-recruitment/#building-a-fairer-future-strategies-to-mitigate-bias-in-aidriven-recruitment
Singh, M. (2024, June 7). Galileo releases ‘Luna’ to light up enterprise gen AI Evaluation. AIM Research | Artificial Intelligence Market Insights. https://aimresearch.co/generative-ai/galileo-releases-luna-to-light-up-genai-evaluation
Snoswell, A. J., & Burgess, J. (2022, November 29). The Galactica AI model was trained on scientific knowledge – but it spat out alarmingly plausible nonsense. The Conversation. https://theconversation.com/the-galactica-ai-model-was-trained-on-scientific-knowledge-but-it-spat-out-alarmingly-plausible-nonsense-195445
TheInclusiveAI. (2024, February 14). Home — the inclusive AI. The Inclusive AI. https://www.theinclusiveai.com/
Thorbecke, C. (2023, February 9). Google shares lose $100 billion after company’s AI chatbot makes an error during demo. https://edition.cnn.com/2023/02/08/tech/google-ai-bard-demo-error/index.html
Weiser, B. (2023, May 27). Here’s What Happens When Your Lawyer Uses ChatGPT. The New York Times. https://www.nytimes.com/2023/05/27/nyregion/avianca-airline-lawsuit-chatgpt.html
