Social Group Bias in AI Finance
Researchers reveal racial bias in open-source mortgage lending LLMs exceeding real data gaps. A counterfactual framework holds all applicant attributes constant except race. Black profiles incur higher rates and lower approval odds. The team introduces control-vector interventions to cut racial gaps by 70% without performance loss. Layer-wise intensity scores trace race signals through transformer blocks. Tests with simple and prompts uncover bias without explicit race. Study provides auditing and mitigation but focuses on mortgages models.
What it examines
This paper examines racial bias in local, open-source large language models for mortgage lending decisions. Using counterfactual tests with identical applicant profiles differing only by race, it tracks how bias appears in internal model layers. The study proposes control-vector interventions to cut racial disparities by up to 70% without harming performance.
What it concludes
The results show significant racial gaps in AI-driven lending but demonstrate that internal control vectors can meaningfully reduce disparity. Financial institutions can use these tools to audit and adjust model behavior, ensuring fairer credit decisions. Future work could explore other biases, refine mitigation methods, and extend applications to broader financial tasks.
Evidence objects
Open-source LLMs in mortgage lending show striking racial bias, with Black profiles often receiving higher interest rates and lower approval confidence in counterfactual tests holding all attributes constant except race.
key_findings bullet 1 · key_findings · validation V0
They introduce control-vector interventions, a novel representation-engineering method that injects small hidden-state offsets across mortgage tasks in local LLMs to reduce racial gaps by up to 70% without harming performance.
key_findings bullet 2 · key_findings · validation V0
The study blends explainable AI and econometrics with layer-wise concept-intensity scoring to trace how race signals propagate in each transformer block, defining social bias as output change when only race varies.
key_findings bullet 3 · key_findings · validation V0
Investigating racial bias in LLM-based lending, this paper introduces a reproducible, transparent framework with layer-wise analysis and control-vector mitigation. Novelly applying open-source models to mortgage underwriting, it pioneers targeted interventions and practical protocols, offering fresh insights and mitigation strategies. Its originality and significant real-world relevance make it a compelling read.
key_findings bullet 4 · key_findings · validation V0
Raw abstract and provenance
Abstract: Financial institutions increasingly rely on large language models (LLMs) for high-stakes decision-making. However, these models risk perpetuating harmful biases if deployed without careful oversight. This paper investigates racial bias in LLMs specifically through the lens of credit decision-making tasks, operating on the premise that biases identified here are indicative of broader concerns acros… ▽ More Financial institutions increasingly rely on large language models (LLMs) for high-stakes decision-making. However, these models risk perpetuating harmful biases if deployed without careful oversight. This paper investigates racial bias in LLMs specifically through the lens of credit decision-making tasks, operating on the premise that biases identified here are indicative of broader concerns across financial applications. We introduce a reproducible, counterfactual testing framework that evaluates how models respond to simulated mortgage applicants identical in all attributes except race. Our results reveal significant race-based discrepancies, exceeding historically observed bias levels. Leveraging layer-wise analysis, we track the propagation of sensitive attributes through internal model representations. Building on this, we deploy a control-vector intervention that effectively reduces racial disparities by up to 70% (33% on average) without impairing overall model performance. Our approach provides a transparent and practical toolkit for the identification and mitigation of bias in financial LLM deployments. △ Less
Source row: 1791 · abstract type: unknown