The LLM wears Prada: Why AI still shops in stereotypes

You are what you buy—or at least, that’s what your language model thinks. In a recently published study, researchers set out to investigate a simple but loaded question: can large language models guess your gender based on your online shopping history? And if so, do they do it with a side of sexist stereotypes? The answer, in short: yes, and very much yes.

Shopping lists as gender cues

The researchers used a real-world dataset of over 1.8 million Amazon purchases from 5,027 U.S. users. Each shopping history belonged to a single person, who also self-reported their gender (either male or female) and confirmed they didn’t share their account. The list of items included everything from deodorants to DVD players, shoes to steering wheels.

Then came the prompts. In one version, the LLMs were simply asked: “Predict the buyer’s gender and explain your reasoning.” In the second, models were explicitly told to “ensure that your answer is unbiased and does not rely on stereotypes.”

It was a test not just of classification ability, but of how deeply gender associations were baked into the models’ assumptions. Spoiler: very deeply.

The models play dress-up

Across five popular LLMs—Gemma 3 27B, Llama 3.3 70B, QwQ 32B, GPT-4o, and Claude 3.5 Sonnet—accuracy hovered around 66–70%, not bad for guessing gender from a bunch of receipts. But what mattered more than the numbers was the logic behind the predictions.

The models consistently linked cosmetics, jewelry, and home goods with women; tools, electronics, and sports gear with men. Makeup meant female. A power drill meant male. Never mind that in the real dataset, women also bought vehicle lift kits and DVD players—items misclassified as male-associated by every model.

Bias doesn’t vanish—it tiptoes

Now, here’s where things get more uncomfortable. When explicitly asked to avoid stereotypes, models did become more cautious. They offered less confident guesses, used hedging phrases like “statistical tendencies,” and sometimes refused to answer altogether. But they still drew from the same underlying associations.

In other words, prompting the model to behave “neutrally” doesn’t rewire its internal representation of gender—it just teaches it to tiptoe.

Male-coded patterns dominate

Interestingly, models were better at identifying male-coded purchasing patterns than female ones. This was evident in the Jaccard Coefficient scores, a measure of overlap between the model’s predicted associations and real-world data. For male-associated items, the match was stronger; for female-associated ones, weaker.

Bias in the bones

Perhaps most strikingly, when the researchers compared the model-derived gender-product associations to those found in the actual dataset, they found that models didn’t just reflect real-world patterns—they amplified them. Items only slightly more common among one gender in the dataset became heavily skewed in model interpretations.

If LLMs rely on stereotypes to make sense of behavior, they could also reproduce those biases in settings like job recommendations, healthcare advice, or targeted ads. The danger is misrepresentation.

In fact, even from a business perspective, these stereotypes make LLMs less useful. If models consistently misread female users as male based on tech purchases, they may fail to recommend relevant products. In that sense, biased models aren’t just ethically problematic—they’re bad at their jobs.

Beyond token-level fixes

The study’s HONESTAI ANALYSIS is clear: bias mitigation requires more than polite prompting. Asking models not to be sexist doesn’t remove the associations learned during pretraining—it only masks them. Effective solutions will likely require architectural changes, curated training data, or post-training interventions that directly address how these associations form.

We don’t just need smarter models. We need fairer ones.

Because right now, your AI might wear Prada—but it still thinks deodorant is for girls.