Complete a Qualtrics survey with OpenAI Operator

Posted

Mar 13, 2025 by @Ingar30

·

Last checked

Oct 16, 2026

·

Method:

How we verify

Evidence: Screenshots or logsThe source shows screenshots, a transcript or the agent's output.

Outcome

The poster, a survey researcher, asked Operator to complete a Qualtrics survey they had created; they call the result very promising for Operator but very concerning for survey researchers.

Agent

OpenAI Operator

The source didn't share its exact prompt, so we wrote this from what it describes.

Run this task now, exactly as written below. Everything you need is here, so start straight away and only stop if something required is missing.

Goal: Test my survey end to end like a respondent would.

Fill in before running: Which survey link, and what kind of respondent should you act as?

What I'll give you:
- Survey link
- Respondent profile to follow
- Whether this is a test or live link

Rules:
- Answer every question consistently with the profile, using neutral test answers where the profile doesn't say.
- Note any question that is confusing, broken, forces an answer or has broken logic.
- Flag any question where a bot could answer easily, as a data-quality risk.
- Time how long the survey takes.
- Submit only if this is a test link, and never more than one response.

Output: Confirmation of completion and a list of problems spotted.

You'll know it worked when: the survey is completed and any issues are listed.

Run this task now, exactly as written below. Everything you need is here, so start straight away and only stop if something required is missing.

Goal: Test my survey end to end like a respondent would.

Inputs (fill in or confirm):
- Survey link
- Respondent profile
- Test or live link
- Number of test runs and branches to cover

Rules:
- Answer consistently with the profile.
- Note confusing or broken questions.
- Time the survey.
- Submit only on a test link.
- Use clearly fake but realistic answers (e.g. test@example.com) so test responses are easy to filter out.
- Record the question number and wording for every issue.
- Check skip logic: confirm each branch shows the questions it should and hides the rest.
- Note any question where an automated respondent could answer too easily, as a data-quality risk.
- Report mobile layout problems separately.

Checkpoint: Ask before submitting a live link.

If you get stuck: If a question blocks progress, report it. If a login, CAPTCHA or two-step code appears, stop and ask me to handle it; never guess or store passwords.

Verify before you finish: Check all branches you planned to test.

Output: Test report.

Done when: I know how the survey behaves for that profile.

Source

https://x.com/Ingar30/status/1900096859697262976