Causal Claims in Economics
Welcome to our Project!
Economics is a dynamic field that has witnessed a profound transformation over the past four decades.
The discipline has shifted towards establishing causal relationships using advanced empirical methods—a movement known as the "credibility revolution."
At the heart of our project is the creation of the Causal Graph of Economics.
We have analyzed over 44,000 working papers from the National Bureau of Economic Research (NBER) and the Centre for Economic Policy Research (CEPR) using AI to map out the intricate network of causal claims that shape economic research.
Our Mission
Synthesize Causal Evidence from Economics (a global meta study) in Graphical format. See your causal graph here.
Make complex economic research accessible to scholars, students, and enthusiasts alike. Check out CClaRA- Causal Claims Research Assistant, fine-tuned on the causal knowledge graph from economics papers.
Key Findings of our paper
Full Working Paper Available here
Evolution of Empirical Methods
Significant Increase in Causal Claims: The average proportion of causal claims in papers rose significantly from approximately 5% in 1990 to around 28% in 2020, reflecting the impact of the credibility revolution in economics.
Growth in Causal Inference Methods and decline in Theoretical and Simulation Methods
Complex Narratives Linked to Top Publications:
Intricate Causal Narratives Enhance Publication and Citation Impact: Papers featuring intricate and interconnected causal narratives are more likely to be published in top-tier journals, particularly the top 5 journals, and receive more citations, especially within those journals.
Key Measures of Causal Narrative Complexity: Increases in the number of unique paths and the longest path length in causal knowledge graphs are positively associated with both publication in leading economics journals and higher citation counts. This highlights the value placed on depth and complexity in causal narratives.
Depth Over Quantity in Causal Claims: While the overall number of claims made is positively correlated with top journal publications, the number of causal edges alone does not show the same positive association with publication outcomes or citation counts. This suggests that depth over breadth in causal claims is valued.
Novelty Premium?
Novel Causal Relationships Enhance Publication but Not Citation Impact: Papers introducing novel causal relationships that have not been previously documented are more likely to be published in top 5 journals, indicating a premium on originality for publication success. However, this does not necessarily translate into higher citation counts once published.
Central vs. Peripheral Concepts
Specialized Topics Gain Recognition in Top Journals, but Central Topics Receive More Citations: The average eigenvector centrality of nodes is negatively associated with publication in top 5 journals, suggesting that papers engaging with less central, more specialized concepts are more likely to appear in the most prestigious journals. However, once published, papers focusing on more central concepts tend to receive more citations, including in top journals. This indicates a divergence between factors that enhance publication success and those that drive academic influence.
Balance Between Source and Sink Nodes
Top Journals Tend to Publish Papers with Multiple Causes Leading to Few Effects in Causal Claims: In the causal subgraph, top journals publish papers exploring multiple causal factors leading to fewer outcomes (many sources to few sinks), and such papers receive more citations.
Opposite Pattern for Non-Causal Relationships: For non-causal relationships in the full knowledge graph, papers focusing on few sources leading to multiple effects (few causes to many sinks) are more likely to be published in top journals.
Transparency and Replicability Concerns
Decline in Reporting Null Results: Reporting of null results declined from 15% in 1980 to around 8.6% in 2023, possibly reflecting increased pressure to produce significant findings and contributing to publication bias.
Increase in Use of Private Data: The use of private data doubled from about 4% in 1980 to above 8% in 2023, raising concerns about data accessibility, replicability, and transparency in economic research.
Our Approach
Leveraging a custom Artificial Intelligence (AI) pipeline, we process vast amounts of text to extract and structure causal relationships. Here's how we build the causal graph:
Data Collection: Gathering a comprehensive corpus of working papers from NBER and CEPR.
AI-Powered Extraction: Using our AI model to identify causal claims, empirical methods, and key economic concepts within each paper.
Standardization of Concepts: Mapping extracted variables to official Journal of Economic Literature (JEL) codes for consistency.
Construction of the Causal Graph: Connecting economic concepts through identified causal relationships to form a detailed causal graph.
Visualization: Creating graphical representations that illustrate how economic ideas are causally linked over time.
For the full research: Garg, P. & Fetzer, T. (2024), Causal Claims in Economics. Working Paper
Scroll down to explore some example graphs or find your own using our graph build tool!
Causal Graph of Landmark Economics Papers
Summary: Evaluates the impact of introducing microfinance in India, finding increased borrowing and investment but limited effects on consumption and development outcomes.
Key Relationships (JEL code in parenthesis; causal edges in orange):
Introduction of microfinance ➔ Households having a microcredit loan
Households having a microcredit loan ➔ New business creation
Households having a microcredit loan ➔ Investment in existing businesses
Investment in existing businesses ➔ Average monthly per capita expenditure
Microcredit ➔ Increased expenditure on durable goods
Microcredit ➔ Decreased expenditure on 'temptation goods'
Microcredit ➔ No significant change in development outcomes (health, education, women's empowerment)
Esther Duflo; Abhijit Banerjee; Rachel Glennerster; Cynthia G. Kinnan
Raj Chetty, Nathaniel Hendren, Patrick Kline, Emmanuel Saez
Summary: Analyzes U.S. intergenerational income mobility, identifying factors like less segregation and better schools that correlate with higher upward mobility.
Key Relationships (JEL code in parenthesis):
Parent income ➔ Child income rank
Lower residential segregation ➔ Higher upward mobility
Less income inequality ➔ Higher upward mobility
Better primary schools ➔ Higher upward mobility
Greater social capital ➔ Higher upward mobility
More stable family structures ➔ Higher upward mobility
Summary: Proposes that individual shocks to large firms significantly impact the entire economy's fluctuations, due to the disproportionate size of these firms.
Key Relationships (JEL code in parenthesis):
Idiosyncratic shocks to large firms ➔ Aggregate volatility
Idiosyncratic shocks to large firms ➔ GDP fluctuations
Fat-tailed distribution of firm sizes ➔ Idiosyncratic shocks do not average out
Economic activities of large firms ➔ Aggregate fluctuations
Granular residual ➔ Growth rate of GDP
Xavier Gabaix
Summary: Examines how access to imported inputs due to lower tariffs boosts Indian firms' product growth and performance by relaxing technological constraints.
Key Relationships (JEL code in parenthesis; causal edges in orange):
Declines in input tariffs ➔ Increased firm product scope
Lower input tariffs ➔ Improved firm performance
Increased availability of new imported inputs ➔ Relaxed technological constraints for domestic firms
Pinelopi K. Goldberg, Amit Khandelwal, Nina Pavcnik, Petia Topalova
These examples demonstrate how different research designs and topics result in varying causal graph structures. They showcase the utility of our approach in capturing and quantifying the complexity and novelty of economic research.
We build such causal graphs for all papers in economics in NBER and CEPR from 1980-2024.
The Causal Graph of Economics Literature
We construct a knowledge graph for each paper, where nodes represent economic concepts (JEL codes), and edges represent claims from a source node to a sink node.
Use of JEL codes is primarily for trackability, allowing us to group related concepts (e.g., cost of living, price level increases, inflation, deflation) into one (e.g. E31 - Inflation).
Claims are classified as causal if they are supported by causal inference methods such as Difference-in-Differences (DiD), Instrumental Variables (IV), Randomized Controlled Trials (RCTs), and others.
We use the proportion of causal edges in a paper to measure extent to which economists have increasingly adopted rigorous causal inference methods in their work, indicative of the credibility revolution.
Proportion of Claims in Paper that are Causal
By field, before and after 2000
Breakdown by Methods
Growth of "Mostly Harmless Econometrics"
Significant increase in the use of methods like Difference-in-Differences (from ~4% in 1980 to over 15% recently), Instrumental Variables, and Randomized Controlled Trials.
Decline in Theoretical Work The proportion of theoretical and non-empirical research has declined from approximately 20% in 1980 to under 10% in 2023.
Empirical Methods across Fields
Different fields have adopted methods at varying rates.
Health, Urban and Labour heavily utilize Diff-in-Diff.
Behavioural and Development Economics prominently feature RCTs
Theory is most prevalent in IO and Macro.
A Growing Project
This is just the beginning of our journey. Our project is continuously expanding, and we are committed to adding new features and insights to the website. Upcoming enhancements include:
Interactive Causal Graphs: Explore the web of economic concepts and their causal linkages interactively.
Expanded Data and Analysis: Incorporate more papers across disciplines and refine our AI methods for deeper insights.
Collaborative Opportunities: Engage with the research community to enrich our understanding of economic causality.
Questions?
Contact [team@causal.claims] to get more information on the project.
We would love to hear from you!