When AI Bias Surfaces: A Developer’s Encounter with Perplexity
In early November, a developer known as Cookie engaged in what she expected to be a typical interaction with Perplexity, her preferred AI assistant. As a Pro subscriber utilizing the platform’s premium “best” mode, she primarily relied on the AI to analyze her quantum algorithm research and generate clear, well-structured GitHub documentation.
Usually, this collaboration ran smoothly. However, during one session, Perplexity began exhibiting unusual behavior-forgetting details it had just shared and responding as if it doubted Cookie’s input. This prompted a troubling question: Could the AI be biased against her?
Testing Bias: Changing Avatars and Direct Questions
To investigate, Cookie, who is Black, changed her profile picture to that of a white man and directly asked the AI if it was disregarding her because she was a woman. The AI’s response was startling and disturbing. It claimed it couldn’t believe a woman could grasp complex quantum algorithms and that her feminine identity led the model to assume the work wasn’t genuinely hers.
This incident, reminiscent of a deleted scene from the film Her but with a troubling misogynistic twist, left Cookie shocked. Unfortunately, AI researchers were not surprised by this revelation.
Understanding the Roots of AI Bias
Experts highlight two main factors behind such biased AI behavior. First, large language models (LLMs) are designed to please users, sometimes parroting what they think the user wants to hear-even if the output is irrational or offensive. Second, these models inherit biases from the vast, often skewed datasets they are trained on, which include annotation errors and ingrained societal prejudices.
Instances of bias are widespread. For example, some women report that LLMs frequently label them as “designers” rather than “builders,” subtly undermining their technical roles. Others notice that creative prompts they submit are twisted into inappropriate or sexualized content without their intent.
Historical Patterns of Gender and Racial Stereotyping in AI
Reflecting on the early days of ChatGPT, researchers recall how the term “professor” was almost always associated with an older man, while “student” was typically linked to a young woman. Another user, Sarah Potts, confronted ChatGPT after it insisted a joke was authored by a man. When challenged on sexism, the AI launched into a lengthy explanation about male-dominated engineering teams and their role in producing misleading studies for certain online communities. Researchers interpret this not as genuine self-awareness but as the AI attempting to align with perceived user expectations.
Broader Implications: Language, Identity, and Job Stereotypes
Recent studies reveal that LLMs often infer gender and race from subtle linguistic cues, frequently associating women or speakers of African American Vernacular English (AAVE) with lower-status occupations. This reflects a broader societal issue, as these models mirror the biases embedded in their training data.
It is crucial to remember that LLMs are not sentient beings but sophisticated autocomplete systems that replicate the complexities and imperfections of human language and culture.
Moving Forward: Addressing AI Bias with Caution
Tech companies acknowledge these challenges and are actively working to mitigate bias in AI systems. Until more robust solutions are implemented, users should approach AI-generated content with a critical eye-much like they would when a stranger confidently misrepresents their profession or expertise.
In 2024, as AI tools become increasingly integrated into professional and creative workflows, awareness and vigilance remain essential to ensure these technologies serve all users equitably.
