Reproduce an economics paper's findings from replication data with Claude
- Posted
- by Ethan Mollick
- Last checked
- Method:
- How we verify
Evidence: Article or interviewA newsletter, journalist or interview account of what happened.
Outcome
Given only the paper and its replication archive, Claude Sonnet 4.5 converted the STATA code to Python and reproduced the findings; the author spot-checked the results and GPT-5 Pro confirmed them.
Agent
Prompt Original prompt
Exact prompt as published in the source.
replicate the findings in this paper from the dataset they uploaded. you need to do this yourself. if you can't attempt a full replication, do what you can
Try this in:
Sends the Original prompt shown above. ChatGPT and Grok open with it filled in, and ChatGPT may send it straight away. Gemini opens empty, so we copy the prompt for you to paste. Every button also copies it.
The source used Claude (Sonnet 4.5). These are general chat tools, so results may differ.
Opens in a new tab. We don't see what you send.
These links are only a shortcut. We're not affiliated with OpenAI, Google or xAI, and your results may not match the source.
Optional stronger version
Our rewrite with extra checks. The poster didn't test it.
Goal: Replicate the main findings of this paper from its published replication data, doing as much as is feasible. Inputs (fill in or confirm): - The paper (PDF) and dataset files - Which tables or figures matter most - Software available (R, Python, Stata) Rules: - List the paper's key results first (table and coefficient) as your target. - Document every data cleaning step and assumption in a log. - Run the analysis yourself; compare each estimate with the paper and give the difference. - If full replication is impossible, explain exactly which steps blocked it. - Read the replication README before writing any code. - Keep the original variable names so the code can be compared line by line with the STATA files. - Show a table of paper value vs reproduced value for every coefficient you report. - Flag any result that differs beyond rounding and explain the likely cause. - Don't adjust specifications to get a closer match. If you get stuck: If variables are unclear, check the codebook and appendix before assuming. If a login, CAPTCHA or two-step code appears, stop and ask me to handle it; never guess or store passwords. Verify before you finish: Re-run the final script from scratch to confirm it reproduces your numbers. Output: A report: target vs replicated results table, code, log of choices and gaps. Done when: your table shows each key result with its replicated value and a clear match/mismatch.
Source
https://www.oneusefulthing.org/p/real-ai-agents-and-real-work
Related cases
- Fill missing industry details in a customer sheet with Zapier AgentsRecreated promptEvidence: Company case study
UK energy company egg set up a Zapier Agent that finds customers with missing industry details in a HubSpot-fed Google Sheet, searches for the information and updates the sheet; Zapier says it now takes about five minutes and egg no longer pays a contractor for it.
Poster says:About 5 minutes; contractor no longer needed
Zapier Agents · Data · Source date:
- Analyze every entry of a huge JSONL file with Claude Code subagentsRecreated promptEvidence: Described by poster
Claude Code previewed the data, trimmed fields, defined a schema and ran hundreds of Haiku subagents over ~45 minutes so every entry in a large JSONL file was analyzed.
Claude Code · Data · Source date:
The poster says he asked Grok to graph Iowa traffic deaths per month for the last five years, then California, to check whether a roadside death counter looked low; he shared the charts.
Grok · Data · Source date: