Breathing Is Not Your Strongest Habit

On 7 August, I opened the first reading from Sense, the tool I am building to measure how I work with AI coding agents. Right at the top sat its headline finding:

“Do more: you report the concrete bugs you observed. Helped 40 of 40 times.”

I stared at the line for a few seconds and typed my reply into my notes: “how is this a behaviour or habit worth talking about? it is like saying breathing is your strongest habit helping you live.”

When you open a report about your own work, you start with a natural assumption: a summary of what you do most often will tell you what you are good at. Software trained you to expect this. Your phone’s screen-time report tells you which app stayed open for twelve hours this week. Your music app’s year-in-review congratulates you for playing the same track eighty times. Most reports I have used rank what I did most. You nod because the numbers are accurate, close the tab, and change nothing about how you work tomorrow.

I had hoped for a sharp mirror of my judgment: where my decisions moved a session forward, and where my blind spots caused rework. “40 of 40” taught me nothing. Telling an AI agent about a bug you just saw on your screen is basic participation. Everyone who builds with AI does it every day.

What happened

A day earlier, on 6 August, I had asked for one simple number that shows whether I am getting better or worse at working with AI, the way WHOOP gives a daily recovery score.

The first version of Sense read 146 of my sessions: 101 in Claude Code and 45 in Codex. It found 649 moments, and 646 of the quotes behind them matched the original sessions word for word.

The reading ran from start to finish and still said nothing. I wrote down the question I could not answer: “what is this number, why should i care? what are you measuring, helping me understand and hence improve”.

The first version had no standard separating routine activity from judgment. It ranked my habits by how often they appeared, and in anyone’s record the most frequent action is the one everybody performs. Rank people by frequency and everyone looks excellent at breathing. It also rated 93% of all moments as “helped”, so there was no room left at the top for a hard call made well.

That is also where my WHOOP comparison broke down. WHOOP reads states: recovery, strain and sleep. Our first version only counted activity, and counting how many times I typed a bug report could never produce a meaningful score.

What we checked

Instead of inventing another rating scale, we spent 7 to 9 August reading how professions where mistakes cost lives have rated human judgment at work for decades.

We went to three published handbooks: the ANTS System Handbook for anaesthetists (University of Aberdeen, 2012), the NOTSS Handbook for surgeons (Royal College of Surgeons of Edinburgh, 2019), and the pilot competencies in the IATA/ICAO/IFALPA Evidence-Based Training Implementation Guide (Edition 2, 2024).

What surprised me first was how little they care about volume. They score how well something was done, not whether it happened. In all three, competent work sits in the middle of the scale. ANTS and NOTSS score from 1 to 4, and airlines set their own range, often five points. Doing your regular job properly puts you in the middle. The top mark is kept for getting ahead of a problem before it grows.

In the ANTS handbook, 3 is “Acceptable”, and the top rating reads:

“4 – Good: Performance was of a consistently high standard, enhancing patient safety; it could be used as a positive example for others.”

Its markers for “Good” include that an anaesthetist “keeps ahead of the situation” and “takes action to avoid or mitigate potential problems”.

The pilot framework has a whole competency for managing automation, written for autopilots long before coding agents existed. Two of its observable behaviours read like a checklist for working with Claude Code or Codex today: “Selects appropriate level and mode of automation in a timely manner” and “Effectively monitors automation, including engagement and automatic mode transitions”.

Research from the operating theatre shows why counting hides the moments that matter. In one study of 10 recorded robot-assisted operations:

“Of 4,583 examined utterances, 329 (7%) were risk-related.”

That is from Kunkes and colleagues in Surgical Endoscopy, 2022. The other 93% was not about risk. The judgment shows in the 7%.

Reading the handbooks also exposed a mistake in my own research process. On 7 August, I had commissioned an AI research report to summarise these scales. It said competent work sits at 2 of 4. In the handbooks themselves, 2 means “Marginal”, a cause for concern, and acceptable work is 3 of 4.

Had we trusted the summary, our whole scale would have treated a cause for concern as competence. Since then, I treat an AI research report as a list of places to look, and the original document gets read before a number goes into the product.

One more test: could simple “happened or did not happen” marks work? We counted four of them across 287 of my recorded sessions in Claude Code and Codex, twice, with identical results.

Looking at it, I saw the flaw straight away. Three of the four marks measured what the AI did. Only the fourth, pressing Escape in 21% of sessions, pointed to a decision of mine.

What it says about your judgment

Your judgment with AI does not show in the things you repeat every session. Handing an agent a task happens every time. Watching it edit a file happens almost every time.

In my own work, judgment carries weight in the few moments when a session starts to drift: when I notice the agent going round in circles on the same fix, when I press Escape to stop it, or when I reject its plan and ask for a simpler one.

A report that only counts your activity tends to flatter you. In 1996, Avraham Kluger and Angelo DeNisi published a meta-analysis in Psychological Bulletin covering 607 effect sizes on feedback. Feedback made performance worse in over a third of cases, and it did the most harm when it pointed attention at the person rather than the task.

A report that ranks your habits is about you. A useful reading of your work stays on the task at the moment something could go wrong, and shows whether you spotted it early or let the automation run. As I told myself that week, while we threw out the first scoring model: “Nothing is built with 100% perfection the first time.”

Try it on your last ten sessions

You do not need Sense to test this on your own work tonight.

1. Open the transcripts from your last ten sessions with an AI agent.

2. Skip the routine prompts and find the three moments that changed the outcome: a hard stop, a direct “no”, or a change of plan.

3. For each one, write down what you noticed and did in that exchange that you do not do the rest of the time.

How often you do something says little about your judgment. How well you handle the few moments that matter says a lot. When you look back at your last ten sessions, which single stop saved you from the biggest mess?

What went into this post

• 146 sessions read, 649 moments found, and 646 quotes checked word for word against the original sessions
• 287 sessions counted twice, with identical results
• Three published rating handbooks read: ANTS, NOTSS and the pilot competency framework
• One claim in an AI research report corrected against the original handbooks
• Ideas dropped: ranking habits by how often they happen, and a single score built from activity counts

Sources

• Kunkes T. et al. “Influence of hierarchy on risk communication during robot-assisted surgery: a preliminary study.” Surgical Endoscopy 36(5):3087–3093, 2022.
• ANTS System Handbook v1.0, University of Aberdeen, 2012.
• NOTSS Handbook v2.0, Royal College of Surgeons of Edinburgh, 2019.
• IATA/ICAO/IFALPA Evidence-Based Training Implementation Guide, Edition 2, 2024 (pilot competencies OB 3.5 and 3.6).
• Kluger A.N. and DeNisi A. “The Effects of Feedback Interventions on Performance.” Psychological Bulletin 119(2):254–284, 1996.