How to Tell if You Are Steering the AI or Just Babysitting It

Give two developers the same feature on Monday. By lunch both are done, same tests, same pull request. Only one of them was flying the plane.

A Lego X-wing fighter with a minifigure pilot sitting in its open cockpit, in an orange flight suit and a white helmet. A second minifigure stands out of focus behind it.
Photo: Alex de Koning / Unsplash

Give two developers the same feature request on Monday morning.

By lunch, both are done. Same result. Same tests. Same shiny pull request.

One of them knew exactly what they were doing.

The other spent the morning doing this:

Accept. Accept. Accept. Accept.

Then the tests failed. So they typed: "Fix the tests."

The AI fixed the tests. Then something else broke. "Fix that too."

Eventually everything turned green. Ship it.

Now here is the problem.

If I show you only the finished code, can you tell which developer was actually in control?

Probably not.

And that is going to become a very big problem.

Good-Looking Work Is Getting Cheap

AI can now produce a ridiculous amount of decent-looking work.

Code. Documents. Presentations. Research. Emails. Spreadsheets. Plans.

Give it enough context and a few minutes, and it can make something that looks surprisingly impressive.

Which means the finished work tells us less than it used to.

A brilliant engineer can produce 500 lines of clean code.

So can someone who has absolutely no idea what those 500 lines do.

That changes what we should pay attention to.

The interesting question is no longer just: what did you make?

It is: what did you do while the AI was making it?

That is where the human shows up.

Are You Using AI, or Babysitting It?

I think there are roughly three kinds of AI users.

1. The Passenger

They type: "Build me a dashboard."

AI starts building. Looks good. Accept. Accept. Accept.

Tests fail. "Fix the tests." Accept. Something breaks. "Fix it." Accept.

Eventually it works. Maybe.

The Passenger is not really steering. They are sitting in the back hoping the car knows where it is going.

Sometimes it does. That is what makes this dangerous.

2. The Babysitter

The Babysitter is more involved.

AI breaks something. They fix it.

AI misunderstands something. They explain it again.

AI invents something. They correct it.

AI creates another problem. They clean that up too.

Three hours later the work is finished, and the Babysitter needs a strong cup of coffee.

They are technically in control, but mostly after things go wrong.

Their relationship with AI is purely reactive. AI acts, human reacts.

It works, but it is exhausting.

3. The Pilot

The Pilot behaves differently.

Before AI starts, they give it the destination. They give it the map.

They tell it which roads are closed. They tell it where it absolutely must not go.

Then they let it drive, but they keep watching.

When an important decision comes up, they check it.

When something looks strange, they investigate.

When the AI starts heading toward a lake, they grab the wheel before everyone gets wet.

Same AI. Very different human.

The Passenger turn 1 turn 10 It ships. Nobody can say why it works. The Babysitter It ships. Every act is a repair. The Pilot It ships at turn six. you set it up first you acted you accepted the episode ends
The same episode, three times over. The Passenger never acts. The Babysitter only acts once the work is already wrong, and every act is a repair. The Pilot sets the work up first, acts once early, and finishes four turns sooner.

Did You Read the Damn Thing?

Imagine AI makes a fairly important change to your code.

You look at it. You notice something feels wrong.

You go back. You ask a question. You check one file.

Then you reject the change.

Now imagine someone else gets the same output.

Click. Approved. Next. Click. Approved. Next. Click. Approved.

Those two people are not working with AI in the same way.

Even if both finish the feature.

One is supervising. The other is hoping.

That difference matters.

When AI Goes Stupid, How Fast Do You Notice?

Anyone who uses AI seriously knows this moment.

Everything is going beautifully.

Then suddenly the AI says something completely insane.

It invents a library.

Deletes a file you told it not to touch.

Changes the database schema for absolutely no reason.

Or confidently explains that the bug is caused by something that does not exist.

And you find yourself typing: "No. Why did you do that?"

Then: "I explicitly told you not to change that."

Then: "Put it back."

Then, a few turns later: "PUT. IT. BACK."

This is funny until you realize something.

Catching AI mistakes is becoming part of the job.

And some people catch them almost immediately.

Others let the mistake grow.

One bad assumption becomes three. Three become ten.

Ten turns later, the AI has built an entire small city on top of nonsense.

How quickly you notice drift matters.

the AI goes wrong here Caught at turn 2 turn 1 turn 10 Nothing was built on it. Caught at turn 10 Seven more turns were built on it. you accepted you acted built on the mistake
Two episodes that both go wrong at turn two. Catch it there and there is nothing to unpick. Miss it, and every later turn is built on the same wrong assumption.

Pilots Figured This Out Years Ago

Funny thing. This is not actually a new problem.

People have been dealing with it for decades. Just not with ChatGPT. With airplanes and operating rooms.

Modern planes can automate an enormous amount of flying. Which is fantastic, until something unexpected happens. Then suddenly the human has to understand what is happening and take control.

Researchers studying aviation automation found a nasty pattern.

When machines work well most of the time, humans start relaxing.

We watch less carefully. We trust more quickly. We stop checking every little thing.

Which makes complete sense. If something gives you the right answer 99 times, why would you deeply inspect answer number 100?

Because answer number 100 might be the one where it confidently flies you into a mountain.

Researchers have specific names for this:

  • Automation Complacency: You stop paying close attention because the machine is usually right.
  • Automation Bias: You trust the machine even when your own eyes tell you something feels wrong.
  • The Out-of-the-Loop Problem: Dr. Mica Endsley documented this across flight decks. The machine does so much that when it suddenly drops the ball, you have no situational awareness of what just happened.

Aviation does not grade pilots on whether the plane landed on a clear day. They use NOTECHS to observe how the crew works.

Surgeons did the exact same thing with the NOTSS system, Non-Technical Skills for Surgeons. They score habits you can watch, using behaviourally anchored rating scales.

They do not grade the cargo or the patient. They measure the human in the loop.

Their questions are older than ours, and they are the same questions.

Four of our five ways of working already have a twin in a cockpit or an operating theatre. The fifth is ours alone, and it is the one that only exists because we work with a machine that forgets.
WhereWhat they watchWhat we call it
Flight deckDo they scan the instruments, or trust the display?Checking
Flight deckDo they set the limits before take-off?Setting up
Operating theatreHow fast do they notice the patient has changed?Correcting
Operating theatreDo they hand the next team what it needs?Carrying forward
No twinNobody hands a plane a folder of notes.Handing over

The AI Problem Is Not That It Is Bad

This is the strange part.

Bad AI is easy. You do not trust it. You check everything. You stay awake.

The interesting problem begins when AI becomes really good.

Good enough that you stop checking.

Good enough that accepting becomes a reflex.

Good enough that you start thinking: "It probably knows."

That is when your own behavior begins changing. Not because someone told you to change, but because the tool quietly trained you.

You read less. You question less. You accept faster.

And because the work still looks good, nobody notices. Possibly not even you.

So What Should We Measure?

This is the question behind Korture Sense.

Not: how many prompts did you write today? Who cares?

Not: how many lines of code did AI generate? Definitely not.

And not: how many hours did AI save you?

Maybe it saved you three hours. Maybe it also made you slightly worse at thinking.

What we read instead is five things. These are their real names.

Five ways of working. Each one is scored against your own earlier sessions, never against another person. 100 is your own past.
What we readWhich means
Setting upwhat you ask for, and how tightly you frame it
Handing overthe context, notes and sources you give it
Checkingwhat you verify before you accept it
Correctingwhat you do when the work is wrong
Carrying forwardwhat you keep for the next session

You will notice there is no speed in that list, and no volume. That is deliberate, and it cost us some work to get there.

What We Tried, and Decided Not to Measure

Everything above is what survived. Plenty did not, and it is worth saying which, because a measurement product that only tells you about its wins is not being straight with you.

Six things we tried and do not ship.
What we triedWhy we dropped it
TimeA week passing is not improvement
One single numberIt settled at 105 and stopped moving
ReworkWe could not read it reliably
A skill levelIt graded the person
Naming the kind of workThe reader could not do it
You against everybodyTwo people is not everybody

Two of those are worth a sentence more. The single number was run over 646 real moments of our own work. It settled near 105, stopped moving, and nobody could say why it was 105 or what would move it. That is why there are five named parts now instead of one number.

And the last row is a hole, not a decision. To tell you whether a pattern is you or everybody, we would need many people followed over months. We have 646 moments from two. No public data set can fix that either, because they strip identity out before release. So we do not make that claim, and we will say when we can.

This Should Not Require Reading Your Work

There is another important part. We do not think understanding these habits should require reading your private work.

Your code can stay your code.

Your client document can stay your client document.

Your ideas can stay your ideas.

So none of this leaves your machine. There is no dashboard anyone else can open.

No reading your private code. No opening your client documents.

No manager view showing who typed what at 11:47 PM.

Apart from being invasive, that would also miss the point. Your code is the final product. Your judgment happens before that.

A mirror. Not a surveillance camera.

The Best AI User May Produce Less

This is where old productivity thinking gets weird.

Imagine two people.

Person A generates 5,000 lines of AI-written code today. Person B generates 800.

Who was more productive?

No idea.

Maybe Person A built a fantastic system. Or maybe Person A approved a giant pile of unmaintainable garbage.

Maybe Person B spent two hours catching one dangerous architectural assumption that would have cost the company six months of rework later.

Output alone is becoming a terrible way to judge work.

AI makes output cheap.

Judgment is still expensive. Maybe more expensive than ever.

The Real Skill Is Knowing When to Take the Wheel

In a field experiment with Boston Consulting Group, researchers from Harvard studied how knowledge workers navigate the jagged frontier of AI.

They noticed two distinct patterns among the top performers:

  • Centaurs: People who divide the work clearly. The human does some parts, AI does others.
  • Cyborgs: People who work hand-in-glove with the AI, steering, editing, and prompting turn-by-turn.

The names are fun, but the lesson is simple.

Being good with AI does not mean giving everything to AI.

And it does not mean doing everything yourself.

It means knowing:

When should I hand this over?

When should I stay involved?

When should I check?

When should I stop the machine completely?

That is steering.

And That Is What I Want to Know About Myself

This is personal for me.

I do not want an AI dashboard to tell me I used AI for 4.7 hours this week. That is useless.

I want it to tell me:

You have started accepting complex changes without checking them.

Or:

You keep correcting the same kind of mistake. Maybe your setup is the real problem.

Or:

On difficult architecture work, you are verifying less than you used to.

Or even:

You are getting sharper at this. You are catching drift earlier and giving clearer direction.

That would actually teach me something.

Because AI is not just changing what we produce. It is changing how we behave while producing it.

And that change is happening very quickly.

Take a Look at Your Last Five Prompts

Not fifty. Five.

Look at them:

  • Did you know what you wanted?
  • Did you explain it clearly?
  • Did you check the important parts?
  • Did the AI go off course?
  • How many turns did it take you to notice?

And here is the slightly uncomfortable question:

When the AI made a decision for you, did you consciously decide to let it?

Or did you just click Accept?

That difference may end up mattering a lot more than how much work you produced.

Because AI can make almost anyone look productive.

The harder question is whether you are still the one flying the plane.

All writing