The second opinion I could not get

Two black roof rats looking out of a hammock together, a second pair of eyes -- part 9 of Notes from building this site

By · 8 min read · Last saved

This article comes with a free Claude skill

The rule this story ends with is packaged as agent-clean-room. It is one small file you drop into Claude Code so your own assistant works this way too. It is public domain: copy it, change it, no attribution and no permission needed. How to install it is at the end of this article.

I wanted a second model to look at something.

The reasoning behind that is ordinary. When one assistant does most of the work and then checks its own work, the check is worth very little, because it is performed by the thing being checked, using the same assumptions that produced the thing. So the plan was to hand a design to a different model and ask it to disagree.

It refused. Not rudely, and not with an argument: it simply declined to engage with the request at all, and produced nothing.

I assumed I had asked badly and rewrote the question. It refused again. I made the question shorter, then more formal, then split it into two. Refused, refused, refused.

Then I pasted the same question into an ordinary chat window with that same model, and got four paragraphs of exactly the thoughtful disagreement I had been asking for.

What Claude Code sends with every subagent prompt

What I had not understood is that a coding assistant does not send your question. It sends your question wrapped in a description of where you are standing.

Before my words, the request carried the name of the branch I was on, the list of files I had changed, the subjects of my last several commits, and the contents of the instruction files that sit in the top of the project. All of that is genuinely useful when the assistant is helping me, and none of it is something I typed.

Now consider what those things say on this project. The branch names, the file names and every recent commit message are about breeding rats: pedigrees, litters, which animals are related to which and by how much.

A safety check does not read the question and the surroundings separately. It scores the whole thing that arrives. So a mundane request about page layout was being judged in the company of a wall of text about animal breeding, and it kept coming out on the wrong side of the line.

The model was fine. The envelope was the problem, and the envelope was one I had never seen, because I did not write it.

The part worth generalising is that the conclusion I had reached, that model is unusable for me, is the conclusion almost everybody reaches. It is wrong, and it is expensive, because it ends the investigation at the exact point where it was about to become cheap.

The fix is to ask from somewhere that has nothing to say

Run the second model as a plain background process, started from a scratch folder outside the project: no version control, no instruction files, no configuration. Then hand it a brief that contains everything it needs.

Three things are doing the work there, and only the first is obvious.

Outside version control. No repository means no branch, no diff, no commit log. This is most of the effect on its own.

No instruction files. Project instructions are picked up automatically from wherever you started. A scratch folder has none to pick up.

A brief that stands alone. The reviewer cannot see your code, so the brief has to carry the whole problem. That sounds like a cost and it is actually the largest single benefit: writing a brief that makes sense to someone with no context forces you to state the problem properly, and a fair amount of the value arrives before you send anything.

The line this must not cross

Removing your branch name from a question about page layout is legitimate. Your branch name was never part of the question.

Rewording the substance of a question until a check stops objecting is a different act, and I want to give the practical reason rather than the moral one, because the practical reason is the one that actually stops you.

Whatever comes back is a response to what you sent. If you soften the question, you get a confident answer to the softened version. You will then act on it as though it addressed the real one. You have not obtained a second opinion at all. You have obtained a second opinion about something else, and the fact that it reads as reassuring is precisely what makes it dangerous.

When a subject really is declined, there are two honest moves: find a reviewer that will take it, or do the work yourself and write down that no second opinion was obtained.

Then find out where the boundary actually is

A clean room stops irrelevant context causing refusals. It does not tell you which of your real subjects a given reviewer will engage with, and that is the thing you need in order to plan.

So we mapped it: small, honest questions across the areas we actually work in, each one framed in its own real terms, with every result written down including the refusals.

That is also where we made the mistake worth passing on. Four probes went well, and we wrote the conclusion down as though it were settled. It was not. The fifth case contradicted it and cost a day.

The fix is a single habit: keep the count attached to the claim. “Engaged four times out of five, all on one afternoon” stays useful for months, because it tells the next reader how much weight to put on it. “It handles this fine” is the same information with the load-bearing part removed, and it will be quoted back at you long after it stopped being true.

And check what actually answered you

Running a model as a background process puts several layers between you and it, and the text that comes back cannot tell you what produced it. A model asked to be something else will say it is that thing, in the output, convincingly. Read the identity from the run’s own metadata instead, and check every turn rather than the first one.

Two failures are specific to launching a process rather than typing into a window, and both look like success:

Nothing at all. The process finishes cleanly and writes an empty file, or writes an error into a file nobody opens. If your wrapper checks only that the process exited without complaint, it will happily record a review that never took place. Empty output is a failure. Always.

Stopping halfway. A run can produce genuinely good work and then stop mid-thought. Keep the real part, say plainly where it ended, and finish the rest yourself under your own name. What you must not do is quietly run it again until something completes, and then describe the whole thing as that model’s work.

The rule

  • Ask from a scratch directory, outside version control, with a brief that stands alone.
  • Strip the surroundings, never the substance. An answer to a disguised question is an answer to the disguise.
  • Map the boundary deliberately, and keep the sample size attached to what you concluded.
  • Verify which model answered, from metadata, on every turn.
  • Treat empty output as failure, because it is the one kind of failure that reports success.

Get the skill

Everything above is generic. None of it is about rats, and none of it is specific to this site. So the rule is also published on its own as a Claude skill: a single Markdown file that an AI coding assistant reads and applies when the situation comes up.

The file: agent-clean-room/SKILL.md

Or take the whole set:

git clone https://github.com/blonderoofrat/agent-skills

Installing it in Claude Code. Copy the skill’s folder into one of these, so the file ends up at .../skills/agent-clean-room/SKILL.md:

  • ~/.claude/skills/ (available in every project on your machine)
  • your-project/.claude/skills/ (that one project only)

Claude reads it at the start of the next session and applies it when what you are doing matches the description at the top of the file. You can also ask for it by name.

Using a different assistant? The file is plain Markdown with a two-line header. Paste the body into whatever system prompt, rules file or instructions file your tool uses. Nothing in the rule itself is Claude-specific.

Licence: CC0, public domain. Copy it, adapt it, ship it in commercial work, no attribution required. These are deliberately frozen snapshots rather than a maintained project, so if one is wrong for your situation, change it. That is easier than asking us to.


Part of Notes from building this site: articles about working practices that exist because something here went wrong first. The rule above is also published on its own, as a free public-domain instruction file for AI coding assistants, the agent-clean-room skill, in blonderoofrat/agent-skills on GitHub.

Filed under News about this site