Lay out what was paid and owed in a vendor billing dispute with ChatGPT (it invented invoice numbers)
- Posted
- by Jamaal Glenn (LinkedIn)
- Last checked
- Method:
- How we verify
Evidence: Described by posterThe person describes what happened in words, without screenshots or video.
FailureThe source says the agent did not get the job done.
What went wrong: ChatGPT invented every invoice number; caught by a spot-check just before sending.
Outcome
A business owner fed ChatGPT all of the payment history, invoices and email threads from a vendor billing misunderstanding and asked it to set out exactly what had been paid and what was still owed. Five seconds before sending the email, he spot-checked a couple of invoice numbers and found they were all made up: they looked real and had the right format, but none existed. He didn't send it.
Agent
Prompt Recreated prompt
The source didn't share its exact prompt, so we wrote this from what it describes.
Run this task now, exactly as written below. Everything you need is here, so start straight away and only stop if something required is missing or a rule below tells you to. Goal: Reconcile what has been paid and what is still owed, using only the documents I provide. Fill in before running: - Invoices: [ATTACH] - Payment records: [ATTACH] - Email thread with the vendor: [ATTACH] Rules: - Use only invoice numbers, dates and amounts that appear word for word in my documents. - For every line, name the document and page it came from. - If something is missing or unclear, write 'not found' instead of filling it in. - Draft the email only after I approve the reconciliation table. - If a site asks for a login, a CAPTCHA or payment details, stop and hand control back to me. - Ask for my OK before you pay, book, send, submit or cancel anything. - Check facts that can change (prices, opening hours, dates, availability, rules) against the official source, and tell me anything you couldn't verify. Output: A reconciliation table with sources, then a draft email. You'll know it worked when: every number in the email matches a document I can point to.
Try this in:
This prompt has blanks to fill in, like [ATTACH]. We'll copy it instead of filling it in, so you can edit it before you send.
Opens in a new tab. We don't see what you send.
These links are only a shortcut. We're not affiliated with OpenAI, Google or xAI, and your results may not match the source.
Optional stronger version
Our rewrite with extra checks. The poster didn't test it.
Run this task now, exactly as written below. Everything you need is here, so start straight away and only stop if something required is missing or a rule below tells you to. Goal: Reconcile what has been paid and what is still owed, using only the documents I provide. Fill in before running: - Invoices: [ATTACH] - Payment records: [ATTACH] - Email thread with the vendor: [ATTACH] Rules: - Use only invoice numbers, dates and amounts that appear word for word in my documents. - For every line, name the document and page it came from. - If something is missing or unclear, write 'not found' instead of filling it in. - Draft the email only after I approve the reconciliation table. - If a site asks for a login, a CAPTCHA or payment details, stop and hand control back to me. - Ask for my OK before you pay, book, send, submit or cancel anything. - Check facts that can change (prices, opening hours, dates, availability, rules) against the official source, and tell me anything you couldn't verify. - Before sending, check every invoice number and amount against the original documents yourself. Output: A reconciliation table with sources, then a draft email. You'll know it worked when: every number in the email matches a document I can point to. Before you finish: re-read my goal and constraints, check your answer against each one, and list anything you assumed or couldn't confirm.
Source
Related cases
- Find a cheaper Medicare drug plan with ChatGPT after a premium rise, then confirm it in Medicare's Plan FinderRecreated promptEvidence: Described by posterBeginner-friendly
A Topretirements blogger's Medicare Part D drug plan wrote to say the 2027 premium would rise from $98.30 to $126 a month. They asked ChatGPT for cheaper options, giving their Florida ZIP code, their medications and preferred pharmacies. It suggested several, including a plan at $5.30 a month in their area, $1,448.40 a year less in premiums. They didn't switch on ChatGPT's word alone: as their insurance agent also advised, they checked it in Medicare's official Plan Finder, which confirmed the switch made sense, and enrolled. They get most prescriptions through the VA, so the cheapest premium suited them; they note others need to compare total yearly cost.
ChatGPT · Personal · Source date:
- Get a $199 membership refund for slow test results by citing the provider's own agreement with ChatGPTRecreated promptEvidence: Described by posterBeginner-friendly
Gokul N.'s blood test results from Superpower took 20 days. Gokul asked ChatGPT to check the refund policy, look at the dates and email the concierge. The email cited Section 14(a) of the membership agreement, set out the timeline with Superpower's own support replies, asked them to confirm when results had actually been available, and added a fallback asking for an exception if a different policy applied. Two days later Superpower confirmed a full $199 membership refund.
ChatGPT · Personal · Source date:
- Check a $3,802 surgery bill against insurance with Gemini and get a $269 consultation charge removedEvidence: Article or interview
Financial planner Mike Palmer's itemised statement for a surgery was $3,802 in confusing codes and offsetting entries; insurance covered $1,390, leaving $2,412 out of pocket. Palmer saved the bill details as a PDF without personal information and asked Gemini what the codes meant and whether the coverage was correct. Gemini explained each line, confirmed the write-offs showed in-network pricing, and flagged an office consultation fee as likely contested, since insurers usually bundle a same-day evaluation into the procedure. There had been no separate consultation, so Palmer called the provider to dispute it, and they removed the $269 fee.
Gemini · Personal · Source date: