Real Agent Cases

Compare Claude Code, Codex and Gemini CLI on a 97k-message chat dataset

Posted
by Tyler Xie (@tylerxiety)
Last checked
Method:
How we verify

Evidence: Described by posterThe person describes what happened in words, without screenshots or video.

Outcome

The poster gave the same 97k-message WeChat dataset to Claude Code, Codex and Gemini CLI as analysts and published results with an evaluation table.

Agent

Claude Code, OpenAI Codex CLI, Gemini CLI

Prompt Recreated prompt

The source didn't share its exact prompt, so we wrote this from what it describes.

Analyze this ~97k-message chat dataset: clean it, compute statistics, visualize patterns and report findings.

Try this in:

ChatGPT Used in sourceGemini Used in sourceGrok

Sends the Recreated prompt shown above. ChatGPT and Grok open with it filled in, and ChatGPT may send it straight away. Gemini opens empty, so we copy the prompt for you to paste. Every button also copies it.

Opens in a new tab. We don't see what you send.

These links are only a shortcut. We're not affiliated with OpenAI, Google or xAI, and your results may not match the source.

Optional stronger version

Our rewrite with extra checks. The poster didn't test it.

Goal: Clean, analyze and visualize a ~97k-message chat dataset and report findings.

Inputs (fill in or confirm):
- Dataset path
- Questions of interest
- Privacy constraints
- Columns available (sender, timestamp, text) and time zone

Rules:
- Document cleaning steps.
- Compute key statistics.
- Make clear charts.
- Anonymize names in outputs.
- Remove system messages, duplicates and empty rows, and report how many were dropped.
- Cover activity over time, top participants, response times and common topics.
- Save every chart as PNG with a title, axis labels and data source.
- Keep raw data unchanged; write cleaned data to a new file.
- Remove or mask names before sharing outputs.
- Use the same questions for each tool.
- Time each tool and note errors.
- Compare results in one table.
- Run every tool on the same copy of the data in a separate folder so none can change the original.
- Check each tool's answers against a quick query you run yourself; flag any wrong numbers.
- Note cost or token use per tool if shown.

If you get stuck: If data is malformed, log and skip. If a login, CAPTCHA or two-step code appears, stop and ask me to handle it; never guess or store passwords.

Verify before you finish: Rerun from scratch to confirm results.

Output: Report with charts and code.

Done when: findings are reproducible.

Source

https://x.com/tylerxiety/status/1988989909429022850

Related cases