Seeing the Goal, Missing the Truth: Human Accountability for AI Bias

This National Bureau of Economic Research working paper investigates how human-defined goals influence Large Language Model (LLM) behavior through purpose-conditioned cognition. The authors demonstrate that revealing downstream objectives, such as predicting stock returns, causes LLMs to generate biased sentiment and competition measures, even when these metrics should remain task-independent. This goal-aware prompting leads to in-sample overfitting, improving performance on data prior to the model's knowledge cutoff but offering no advantage thereafter. The study highlights that this bias persists despite prompt regularization and can emerge from unintentional conversational cues. Crucially, the research argues that such AI bias is not merely an algorithmic flaw but stems from human accountability in research design. By exposing how purpose leakage distorts intermediate measures, the findings urge researchers to account for human influence in AI deployment, particularly within financial prediction tasks where objective integrity is paramount.