This blog summarizes research trends based on published paper abstracts. Specific numbers or findings may contain inaccuracies. For scholarly rigor, always consult the original papers cited in each post.
Suppose a government agency builds a classifier to decide which tax returns to audit. The classifier is accurate: it identifies returns most likely to contain errors, maximizing revenue recovered per audit conducted. On standard machine learning metrics, the system performs well. Yet when researchers examine who actually gets audited, they find that the classifier disproportionately targets low-income taxpayers. The reason is not that the classifier is biased in any conventional sense. It is that auditing a low-income return is cheap and fast, while auditing a high-income return is expensive and slow. An accuracy-maximizing system, operating under a fixed budget, rationally concentrates its attention where returns on investment are highest per dollar spent on enforcement -- which means the poor.
This is not a hypothetical. It is the central finding of a study on IRS audit models by Black, Elzayn, and colleagues (2022), and it illustrates a broader pattern that connects three recent papers on AI and the digital economy. Each paper identifies a different mechanism by which well-designed, well-intentioned AI systems produce outcomes that widen inequality. Taken together, they suggest that the digital economy's fairness problem is not a bug to be patched but a structural feature of how intelligent systems interact with existing economic asymmetries.
The Research Landscape
The relationship between artificial intelligence and economic inequality has generated substantial research across economics, computer science, and public policy. Three streams are particularly relevant. First, macroeconomic modeling of AI's impact on labor markets, including the distribution of gains between capital and labor as automation advances. Second, the conceptual debate over whether AI should be designed to replace human capabilities (automation) or to complement them (augmentation), and what the choice between these orientations implies for wealth distribution. Third, the rapidly growing literature on algorithmic fairness, which has moved beyond narrow technical definitions of bias to examine how optimization objectives interact with structural inequalities.
These streams have largely developed in parallel. Macroeconomists model aggregate transitions without attending to the micro-level mechanisms by which algorithmic systems allocate burdens. Fairness researchers examine specific classifiers without connecting their findings to macroeconomic dynamics. The automation-versus-augmentation debate operates at a conceptual level that rarely engages with empirical evidence from deployed systems. Reading across these literatures reveals connections that no single stream captures on its own.
Paradox One: The Accuracy-Inequality Tradeoff
Black, Elzayn, and colleagues (2022), in a paper published at FAccT (DOI: 10.1145/3531146.3533138), examine the IRS's use of machine learning models to select tax returns for audit. The finding that anchors their analysis is counterintuitive: a more accurate classifier, by standard metrics, produces a less equitable distribution of audit burden across income groups. The reason lies in the differential cost of auditing different populations. Auditing a high-income return is approximately 41 times more expensive than auditing a low-income return, because high-income returns involve complex financial instruments, multiple income streams, and sophisticated legal structures that require specialized auditors and extended review periods. A model optimizing for total revenue recovered per audit dollar spent will therefore rationally concentrate audits on simpler, lower-income returns where the cost-per-dollar-recovered is lowest.
The paper evaluates standard algorithmic fairness interventions -- demographic parity, equal true positive rates, equalized odds -- and finds them inadequate for this problem. These metrics were designed for classification contexts where the cost of a positive prediction is roughly symmetric across groups. In tax auditing, the cost asymmetry is extreme: a false positive (auditing a compliant taxpayer) imposes real costs -- anxiety, time, professional fees -- that fall disproportionately on those least able to bear them. The authors find that the most effective intervention is not a fairness constraint at all, but a shift in the optimization objective itself: moving from classification (which returns to audit) to regression (estimating the dollar amount of underreported income). This reframing increases revenue recovery by a factor of 3.3 compared to the existing model while simultaneously reducing the concentration of audits on low-income filers, because it directs attention toward the absolute magnitude of tax noncompliance rather than the probability of any noncompliance.
<
| Claim | Source | Confidence | Note |
|---|
| More accurate classifiers concentrate audits on low-income taxpayers | Black et al. (2022), DOI 10.1145/3531146.3533138 | Stated | Central empirical finding |
| Auditing a high-income return costs approximately 41x more than a low-income return | Black et al. (2022), DOI 10.1145/3531146.3533138 | Stated | Key cost asymmetry reported |
| Standard fairness metrics (DP, TPR, EO) are inadequate for this context | Black et al. (2022), DOI 10.1145/3531146.3533138 | Stated | Evaluated and found insufficient |
| Classification-to-regression reframing increases revenue by 3.3x | Black et al. (2022), DOI 10.1145/3531146.3533138 | Stated | Compared to existing model |
| The regression approach matches oracle-level performance on equity | Black et al. (2022), DOI 10.1145/3531146.3533138 | Stated | Benchmarked against oracle |
This finding has implications that extend well beyond tax administration. Any domain where the cost of intervention varies systematically across populations -- healthcare, criminal justice, regulatory enforcement, social services -- is susceptible to the same dynamic. An optimization system that is blind to cost asymmetries will, in the pursuit of efficiency, reproduce and amplify whatever inequalities those cost structures encode.
Paradox Two: The Turing Trap
Brynjolfsson (2022), in a paper published in Daedalus (DOI: 10.1162/daed_a_01915), identifies a second structural paradox. The dominant paradigm in AI research -- what he calls Human-Level Artificial Intelligence (HLAI) -- orients the field toward building machines that replicate human capabilities. This orientation, Brynjolfsson argues, creates a systematic bias toward automation (replacing human labor) rather than augmentation (complementing human labor), because replicating a human capability is a more legible engineering target than inventing new capabilities that humans cannot perform alone. The result is what he calls the Turing Trap: the very success of AI research, measured by its ability to match human performance on increasingly complex tasks, accelerates the displacement of human workers and concentrates the resulting economic gains among the owners of capital.
The argument rests on a thought experiment Brynjolfsson attributes to the Daedalus tradition. If a technology can do exactly what a human can do, it competes directly with human labor and drives wages down. If a technology can do something no human can do -- or can dramatically enhance what humans do -- it creates new value that potentially benefits both capital and labor. The economic logic of augmentation is additive; the economic logic of automation is substitutive. Yet the incentive structures facing the three key actors -- technologists, firms, and policymakers -- all tilt toward automation.
Technologists pursue the Turing Test as a benchmark, which defines success as indistinguishability from human performance. Firms adopt AI to reduce labor costs, not to invent new capabilities. And policymakers, perhaps most consequentially, operate within a tax code that systematically favors capital over labor: in the United States, labor income is taxed at marginal rates up to 37 percent, while long-term capital gains are taxed at a maximum of 20 percent. A firm that replaces ten workers with an AI system reduces its tax burden. A firm that uses AI to make ten workers twice as productive does not. The tax code is, in effect, an automation subsidy.
Paradox Three: The Transition Trap
Korinek and Suh (2024), in an NBER working paper (DOI: 10.3386/w32255), model the macroeconomic dynamics of the transition to artificial general intelligence. Their framework treats the economy as a set of tasks distributed along a complexity spectrum, with AI systems progressively automating tasks from the simplest upward. The central variable is the rate at which AI capabilities advance relative to the economy's ability to create new tasks that require human judgment.
The paper presents four simulation scenarios. In the Business-as-Usual scenario, AI progress is gradual and the economy adapts through normal market mechanisms. In the Baseline AGI scenario, AI reaches general capability over approximately 20 years, producing an inverse-U trajectory for wages: initial increases as AI complements human labor on complex tasks, followed by a collapse as automation overtakes the creation of new human-complementary tasks. In the Aggressive scenario, the same transition compresses into roughly 5 years, leaving almost no time for institutional adaptation. In the Bout scenario, wages collapse sharply but recover over approximately 9 years as the economy restructures around new forms of human-AI complementarity.
The critical insight is that even in the most optimistic scenarios, the transition period produces severe distributional disruption. Workers whose tasks are automated early experience wage declines that may be permanent if they cannot retrain for tasks higher on the complexity spectrum. The compute-centric nature of the model -- where economic output becomes increasingly dependent on computational resources rather than human labor -- implies a structural shift in bargaining power from workers to the owners of compute infrastructure. The paper does not predict which scenario will obtain, but it demonstrates that the path between the current economy and a post-AGI economy passes through a valley of inequality whose depth and duration depend on variables that are, at present, deeply uncertain.
The Structural Pattern
Read together, these three papers reveal a common structural pattern. In each case, the source of inequality is not malice, bias, or technical failure. It is the interaction between an optimizing system and an asymmetric environment. The IRS classifier optimizes accuracy in an environment where audit costs are asymmetric across income groups. The AI research community optimizes for human-level performance in an environment where tax policy is asymmetric between labor and capital income. The macroeconomy adjusts to advancing AI capabilities in an environment where the ability to retrain is asymmetric across workers at different points on the skill distribution.
The common thread is that AI systems, by their nature, exploit whatever structure exists in their environment. When that structure encodes inequality -- as cost structures, tax codes, and skill distributions inevitably do -- optimization amplifies inequality. This is not a failure of the technology. It is what optimization means in an unequal world.
Open Questions
Several questions follow from this analysis. First, can the classification-to-regression reframing that Black et al. identify in tax auditing be generalized? Their finding suggests that the choice of optimization objective -- what the system is asked to maximize -- matters more than fairness constraints applied after the objective is set. Whether this principle extends to healthcare triage, criminal justice risk assessment, and welfare eligibility is an empirical question with significant policy implications.
Second, what would it take to shift the orientation of AI development from automation to augmentation? Brynjolfsson's analysis identifies the incentive structures that produce the Turing Trap, but the interventions he suggests -- changing tax policy, redirecting research funding, redefining benchmarks -- face well-known political economy obstacles. The firms best positioned to influence AI policy are precisely those whose business models depend on automation.
Third, how should societies prepare for the transition dynamics that Korinek and Suh model? The difference between the 20-year Baseline scenario and the 5-year Aggressive scenario is not merely a matter of speed but of institutional capacity. Educational systems, labor market institutions, and social safety nets that might adapt over two decades cannot adapt over five years. The policy question is whether preparation should target the most likely scenario or the most dangerous one.
Finally, there is a meta-question that cuts across all three papers: who decides what AI systems optimize for? The IRS classifier optimized for revenue per audit dollar. It could have optimized for equity-weighted revenue, or for deterrence, or for compliance rates across income groups. Each objective implies a different distribution of burdens and benefits. The choice of objective is not a technical decision. It is a political one -- and it is, at present, made largely by engineers and agency administrators rather than through democratic deliberation.
Closing
The three paradoxes examined here -- that accuracy produces inequity, that human-likeness produces displacement, and that transition produces disruption -- share a common lesson. The digital economy's fairness challenges cannot be resolved by making AI systems better at what they already do. They require changing what AI systems are asked to do. The IRS study demonstrates this concretely: the most effective intervention was not a fairness patch but a redefinition of the optimization target. Whether this insight can scale from tax auditing to the broader economy -- whether societies can collectively redefine what they ask intelligent systems to optimize -- may be the defining governance question of the next decade.
Suppose a government agency builds a classifier to decide which tax returns to audit. The classifier is accurate: it identifies returns most likely to contain errors, maximizing revenue recovered per audit conducted. On standard machine learning metrics, the system performs well. Yet when researchers examine who actually gets audited, they find that the classifier disproportionately targets low-income taxpayers. The reason is not that the classifier is biased in any conventional sense. It is that auditing a low-income return is cheap and fast, while auditing a high-income return is expensive and slow. An accuracy-maximizing system, operating under a fixed budget, rationally concentrates its attention where returns on investment are highest per dollar spent on enforcement -- which means the poor.
This is not a hypothetical. It is the central finding of a study on IRS audit models by Black, Elzayn, and colleagues (2022), and it illustrates a broader pattern that connects three recent papers on AI and the digital economy. Each paper identifies a different mechanism by which well-designed, well-intentioned AI systems produce outcomes that widen inequality. Taken together, they suggest that the digital economy's fairness problem is not a bug to be patched but a structural feature of how intelligent systems interact with existing economic asymmetries.
The Research Landscape
The relationship between artificial intelligence and economic inequality has generated substantial research across economics, computer science, and public policy. Three streams are particularly relevant. First, macroeconomic modeling of AI's impact on labor markets, including the distribution of gains between capital and labor as automation advances. Second, the conceptual debate over whether AI should be designed to replace human capabilities (automation) or to complement them (augmentation), and what the choice between these orientations implies for wealth distribution. Third, the rapidly growing literature on algorithmic fairness, which has moved beyond narrow technical definitions of bias to examine how optimization objectives interact with structural inequalities.
These streams have largely developed in parallel. Macroeconomists model aggregate transitions without attending to the micro-level mechanisms by which algorithmic systems allocate burdens. Fairness researchers examine specific classifiers without connecting their findings to macroeconomic dynamics. The automation-versus-augmentation debate operates at a conceptual level that rarely engages with empirical evidence from deployed systems. Reading across these literatures reveals connections that no single stream captures on its own.
Paradox One: The Accuracy-Inequality Tradeoff
Black, Elzayn, and colleagues (2022), in a paper published at FAccT (DOI: 10.1145/3531146.3533138), examine the IRS's use of machine learning models to select tax returns for audit. The finding that anchors their analysis is counterintuitive: a more accurate classifier, by standard metrics, produces a less equitable distribution of audit burden across income groups. The reason lies in the differential cost of auditing different populations. Auditing a high-income return is approximately 41 times more expensive than auditing a low-income return, because high-income returns involve complex financial instruments, multiple income streams, and sophisticated legal structures that require specialized auditors and extended review periods. A model optimizing for total revenue recovered per audit dollar spent will therefore rationally concentrate audits on simpler, lower-income returns where the cost-per-dollar-recovered is lowest.
The paper evaluates standard algorithmic fairness interventions -- demographic parity, equal true positive rates, equalized odds -- and finds them inadequate for this problem. These metrics were designed for classification contexts where the cost of a positive prediction is roughly symmetric across groups. In tax auditing, the cost asymmetry is extreme: a false positive (auditing a compliant taxpayer) imposes real costs -- anxiety, time, professional fees -- that fall disproportionately on those least able to bear them. The authors find that the most effective intervention is not a fairness constraint at all, but a shift in the optimization objective itself: moving from classification (which returns to audit) to regression (estimating the dollar amount of underreported income). This reframing increases revenue recovery by a factor of 3.3 compared to the existing model while simultaneously reducing the concentration of audits on low-income filers, because it directs attention toward the absolute magnitude of tax noncompliance rather than the probability of any noncompliance.
<
| Claim | Source | Confidence | Note |
|---|
| More accurate classifiers concentrate audits on low-income taxpayers | Black et al. (2022), DOI 10.1145/3531146.3533138 | Stated | Central empirical finding |
| Auditing a high-income return costs approximately 41x more than a low-income return | Black et al. (2022), DOI 10.1145/3531146.3533138 | Stated | Key cost asymmetry reported |
| Standard fairness metrics (DP, TPR, EO) are inadequate for this context | Black et al. (2022), DOI 10.1145/3531146.3533138 | Stated | Evaluated and found insufficient |
| Classification-to-regression reframing increases revenue by 3.3x | Black et al. (2022), DOI 10.1145/3531146.3533138 | Stated | Compared to existing model |
| The regression approach matches oracle-level performance on equity | Black et al. (2022), DOI 10.1145/3531146.3533138 | Stated | Benchmarked against oracle |
This finding has implications that extend well beyond tax administration. Any domain where the cost of intervention varies systematically across populations -- healthcare, criminal justice, regulatory enforcement, social services -- is susceptible to the same dynamic. An optimization system that is blind to cost asymmetries will, in the pursuit of efficiency, reproduce and amplify whatever inequalities those cost structures encode.
Paradox Two: The Turing Trap
Brynjolfsson (2022), in a paper published in Daedalus (DOI: 10.1162/daed_a_01915), identifies a second structural paradox. The dominant paradigm in AI research -- what he calls Human-Level Artificial Intelligence (HLAI) -- orients the field toward building machines that replicate human capabilities. This orientation, Brynjolfsson argues, creates a systematic bias toward automation (replacing human labor) rather than augmentation (complementing human labor), because replicating a human capability is a more legible engineering target than inventing new capabilities that humans cannot perform alone. The result is what he calls the Turing Trap: the very success of AI research, measured by its ability to match human performance on increasingly complex tasks, accelerates the displacement of human workers and concentrates the resulting economic gains among the owners of capital.
The argument rests on a thought experiment Brynjolfsson attributes to the Daedalus tradition. If a technology can do exactly what a human can do, it competes directly with human labor and drives wages down. If a technology can do something no human can do -- or can dramatically enhance what humans do -- it creates new value that potentially benefits both capital and labor. The economic logic of augmentation is additive; the economic logic of automation is substitutive. Yet the incentive structures facing the three key actors -- technologists, firms, and policymakers -- all tilt toward automation.
Technologists pursue the Turing Test as a benchmark, which defines success as indistinguishability from human performance. Firms adopt AI to reduce labor costs, not to invent new capabilities. And policymakers, perhaps most consequentially, operate within a tax code that systematically favors capital over labor: in the United States, labor income is taxed at marginal rates up to 37 percent, while long-term capital gains are taxed at a maximum of 20 percent. A firm that replaces ten workers with an AI system reduces its tax burden. A firm that uses AI to make ten workers twice as productive does not. The tax code is, in effect, an automation subsidy.
Paradox Three: The Transition Trap
Korinek and Suh (2024), in an NBER working paper (DOI: 10.3386/w32255), model the macroeconomic dynamics of the transition to artificial general intelligence. Their framework treats the economy as a set of tasks distributed along a complexity spectrum, with AI systems progressively automating tasks from the simplest upward. The central variable is the rate at which AI capabilities advance relative to the economy's ability to create new tasks that require human judgment.
The paper presents four simulation scenarios. In the Business-as-Usual scenario, AI progress is gradual and the economy adapts through normal market mechanisms. In the Baseline AGI scenario, AI reaches general capability over approximately 20 years, producing an inverse-U trajectory for wages: initial increases as AI complements human labor on complex tasks, followed by a collapse as automation overtakes the creation of new human-complementary tasks. In the Aggressive scenario, the same transition compresses into roughly 5 years, leaving almost no time for institutional adaptation. In the Bout scenario, wages collapse sharply but recover over approximately 9 years as the economy restructures around new forms of human-AI complementarity.
The critical insight is that even in the most optimistic scenarios, the transition period produces severe distributional disruption. Workers whose tasks are automated early experience wage declines that may be permanent if they cannot retrain for tasks higher on the complexity spectrum. The compute-centric nature of the model -- where economic output becomes increasingly dependent on computational resources rather than human labor -- implies a structural shift in bargaining power from workers to the owners of compute infrastructure. The paper does not predict which scenario will obtain, but it demonstrates that the path between the current economy and a post-AGI economy passes through a valley of inequality whose depth and duration depend on variables that are, at present, deeply uncertain.
The Structural Pattern
Read together, these three papers reveal a common structural pattern. In each case, the source of inequality is not malice, bias, or technical failure. It is the interaction between an optimizing system and an asymmetric environment. The IRS classifier optimizes accuracy in an environment where audit costs are asymmetric across income groups. The AI research community optimizes for human-level performance in an environment where tax policy is asymmetric between labor and capital income. The macroeconomy adjusts to advancing AI capabilities in an environment where the ability to retrain is asymmetric across workers at different points on the skill distribution.
The common thread is that AI systems, by their nature, exploit whatever structure exists in their environment. When that structure encodes inequality -- as cost structures, tax codes, and skill distributions inevitably do -- optimization amplifies inequality. This is not a failure of the technology. It is what optimization means in an unequal world.
Open Questions
Several questions follow from this analysis. First, can the classification-to-regression reframing that Black et al. identify in tax auditing be generalized? Their finding suggests that the choice of optimization objective -- what the system is asked to maximize -- matters more than fairness constraints applied after the objective is set. Whether this principle extends to healthcare triage, criminal justice risk assessment, and welfare eligibility is an empirical question with significant policy implications.
Second, what would it take to shift the orientation of AI development from automation to augmentation? Brynjolfsson's analysis identifies the incentive structures that produce the Turing Trap, but the interventions he suggests -- changing tax policy, redirecting research funding, redefining benchmarks -- face well-known political economy obstacles. The firms best positioned to influence AI policy are precisely those whose business models depend on automation.
Third, how should societies prepare for the transition dynamics that Korinek and Suh model? The difference between the 20-year Baseline scenario and the 5-year Aggressive scenario is not merely a matter of speed but of institutional capacity. Educational systems, labor market institutions, and social safety nets that might adapt over two decades cannot adapt over five years. The policy question is whether preparation should target the most likely scenario or the most dangerous one.
Finally, there is a meta-question that cuts across all three papers: who decides what AI systems optimize for? The IRS classifier optimized for revenue per audit dollar. It could have optimized for equity-weighted revenue, or for deterrence, or for compliance rates across income groups. Each objective implies a different distribution of burdens and benefits. The choice of objective is not a technical decision. It is a political one -- and it is, at present, made largely by engineers and agency administrators rather than through democratic deliberation.
Closing
The three paradoxes examined here -- that accuracy produces inequity, that human-likeness produces displacement, and that transition produces disruption -- share a common lesson. The digital economy's fairness challenges cannot be resolved by making AI systems better at what they already do. They require changing what AI systems are asked to do. The IRS study demonstrates this concretely: the most effective intervention was not a fairness patch but a redefinition of the optimization target. Whether this insight can scale from tax auditing to the broader economy -- whether societies can collectively redefine what they ask intelligent systems to optimize -- may be the defining governance question of the next decade.