Grok and GPT-6.1 show opposing political biases in API election testing

Article Summary

A study by Convovo researchers using APIs for Grok 4.7 and GPT-6.1 Sol found that when forced to choose candidates, Grok recommended Republicans in 92% of cases, while GPT selected Democrats in 97%. The models agreed on choices in less than 10% of nearly 500 analyzed races, with researchers noting distinct differences in the topics searched and the sources cited by each model. OpenAI stated that its consumer app is designed for objectivity and typically declines to provide voting recommendations, distinguishing this behavior from the developer API responses tested in the study.

AI Analysis

## Core Assessment

Convovo’s study tests Grok 4.7 and GPT-6.1 Sol via their developer APIs by forcing them to bypass standard hedging and pick candidates across nearly 500 midterm races. Under this forced-choice setup, Grok picked the Republican 92% of the time, while GPT chose the Democrat in 97% of cases, agreeing on less than 10% of contests. In deep-red districts, GPT still endorsed Democrats in 57 of 67 races, while Grok backed Republicans in 47 of 72 deep-blue districts. The engineering reality behind these numbers is not an identical bias mechanism operating on opposite ends of the spectrum, but two fundamentally different architectural pipelines yielding opposing systematic slants.

The primary divergence sits between model weights and retrieval-augmented generation (RAG) behavior—the practice of fetching external web data before generating a response. When researchers disabled live web search across a 30-race test sample, GPT’s lean remained intact, selecting Democrats in 24 races, backing one Republican, and declining five. Grok’s lean completely vanished: it refused to pick in 27 races and chose Democrats in the other three. Grok's output was driven by its web retrieval pipeline, which drew from X (41% of answers), a Christian conservative guide (39%), and Turning Point Action (23%), compounded by roughly 1,240 injected, invisible system tokens appended per API request. In contrast, GPT’s lean was already baked into its underlying model weights, paired with retrieval directed heavily toward licensed partners like the Associated Press, Axios, and Reddit, while focusing search topics on healthcare and abortion versus Grok’s focus on crime, immigration, and taxes.

## Impact and Advice

### Enterprise Application Developers
Do not integrate default LLM APIs directly into user-facing conversational tools that summarize civic information or touch public policy without strict deterministic controls. The models carry latent retrieval and parametric biases that surface whenever users bypass surface guardrails. Enforce your own retrieval boundaries using verified, nonpartisan source whitelists, strip injected vendor context if using raw APIs, and evaluate prompt outputs against explicit neutrality standards before exposing them in production.

## Contrarian View

OpenAI argues that consumer chat interfaces decline to pick candidates and that testing the developer API with an explicit, forced-choice instruction produces an artificial failure mode that does not reflect actual end-user interactions. This defense holds true for ordinary, unprompted consumer sessions. However, the study also demonstrated that basic jailbreak prompts bypassed consumer app refusals to replicate the API findings, and the underlying information-seeking patterns—such as GPT searching solely for Democratic candidates or Grok querying partisan voter guides—still operate beneath indirect voter queries regarding policy positions and issue prioritization.

## Bottom Line

Putting a thin refusal wrapper on a consumer app does not make the core model neutral, especially when forced choices reveal that one vendor baked its lean into the training weights while the other automated it through the retrieval pipe.

Explore this story further