Real Agent Cases

Test five web tasks with Project Mariner

Posted
by @JulianGoldieSEO
Last checked
Method:
How we verify

Evidence: Shown on videoThe source includes a video or screen recording of the agent doing the task.

PartialThe source says it only partly worked, or had a significant problem a person had to deal with.

What went wrong: 1 of 5 tasks succeeded.

Outcome

In a test of 5 tasks, Project Mariner found YouTube video view counts, but failed to generate relevant video ideas, couldn't preview HTML properly, refused to log into email and couldn't send emails.

Agent

Project Mariner

Prompt Improved prompt

Strengthened by us to avoid the problem in the source. Not tested by the poster. It doesn't change what happened in the source.

Let's work this out together. I'll tell you what I'm after; help me sharpen the goal before you dive in, and check in with me before anything you can't undo.

Goal: Collect view counts for a set of YouTube videos and turn the top performers into new video ideas.

Questions to explore first: Which videos or channel, and what kind of ideas do I want (topics, titles, formats)?

What I'll give you:
- Video or channel links ([URLS])
- My niche and audience

Rules:
- Start by restating my goal and suggesting 2–3 ways to approach it; recommend one.
- Collect view counts first, with the date you checked and a link for each video.
- Base every idea on a specific top video and say which one inspired it.
- Keep ideas relevant to my niche; drop anything generic.
- Don't log into any account or send anything.
- If a step fails, say so and move on rather than stopping the whole task.

Output: A table of videos with view counts and dates, then 10 ideas each tied to a source video, plus 2–3 suggested next steps.

If anything here is unclear or missing, ask me instead of guessing.

You'll know it worked when: the counts match YouTube and every idea fits my channel.

Try this in:

This prompt has blanks to fill in, like [URLS]. We'll copy it instead of filling it in, so you can edit it before you send.

Opens in a new tab. We don't see what you send.

These links are only a shortcut. We're not affiliated with OpenAI, Google or xAI, and your results may not match the source.

The prompt they actually used (our rebuild)

The source didn't share its exact prompt, so we wrote this from what it describes.

Find the view counts for these YouTube videos and generate video ideas.
Optional stronger version

Our rewrite with extra checks. The poster didn't test it.

Let's work this out together. I'll tell you what I'm after; help me sharpen the goal before you dive in, and check in with me before anything you can't undo.

Goal: Collect view counts for a set of YouTube videos and turn top performers into new video ideas.

First: restate the goal in one sentence, suggest 2–3 ways to approach it, recommend one, and wait for my pick.

Inputs (fill in or confirm):
- Video or channel links
- Niche and audience
- The five tasks, written out

Rules:
- Record view counts and dates from each video page.
- Rank by views per day since publish.
- Find patterns in top performers.
- Suggest ideas based on patterns.
- Run each task separately.
- Log what worked and failed.
- Never log in to email without me.
- Note where it stalled.
- Give links for every video.
- Report each task as done, partly done or failed, with one line on why.
- Generate video ideas only from the data you collected, not general ideas.
- Check that any HTML you create opens and renders correctly.
- Never send email without my explicit OK.

What went wrong before, and how to avoid it:
- In the source test only 1 of 5 tasks succeeded: keep each step small, report failures, and continue.

If you get stuck: If a task step fails, report it and continue with the rest. If a login, CAPTCHA or two-step code appears, stop and ask me to handle it; never guess or store passwords.

Verify before you finish: Check view counts for a sample.

Output: Table and ideas, plus next steps.

Done when: I get grounded ideas from real numbers.

Source

https://x.com/JulianGoldieSEO/status/1927077282041999503

  • Summarize English and Tamil YouTube videos with CometImproved promptEvidence: Described by posterPartial

    Comet Assistant summarized English YouTube conversations quickly and well, but on a Tamil discussion it had to play the whole video and made a few mistakes, missing some crucial points.

    Perplexity Comet Assistant · Research · Source date:

  • Find links to content with Project Mariner (no links returned)Improved promptEvidence: Described by posterFailure

    Asked to find links to content, Project Mariner found the content but wouldn't return the links, which the poster said made the result not very helpful.

    Project Mariner · Research · Source date:

  • Report global NAND market share with Gemini Deep ResearchImproved promptEvidence: Screenshots or logsFailure

    Asked for a simple report on global NAND market share, Gemini Deep Research returned figures the poster called an incredible failure when compared with Sandisk's investor-day slide and Claude's answer to the same query.

    Gemini Deep Research · Research · Source date: