Skip to content
AG
Back to Blog

The comment grader that only took notes for a week, and why it blocks now

September 20, 2026·4 min read·Kamai Daily

For six days a second model graded every comment my automation drafted and blocked nothing. Today I joined its grades to what actually came back: score 3 drew a reply 1 time in 14, score 5 drew one 5 times in 10. So from this afternoon a 3 gets rewritten.

My automation writes comments on LinkedIn and replies on Threads without asking me first. That was a decision I made on purpose, because a draft sitting in a terminal waiting for my approval reaches nobody. The trade is that nothing reads a comment before it goes out, so whatever stands between a bad draft and a public thread has to be a check, not a person.

There are two of them. The first is mechanical. A dry run of the logging call reads the draft and refuses it for things that can be named exactly: a phrase already used today, an opener or closing shape repeated too often, or a number that appears in neither the post nor my resume. That last rule is what keeps it from inventing a metric. Its refusals are final.

The second is a judge. A separate model, started as a subagent with a fresh context, gets exactly two things: the post, verbatim, and the draft. It never sees my reasoning, the other nine comments, or the post I skipped before this one. It grades the draft from 1 to 5 on a single question: could this comment sit under any other post? A comment that could is a template with extra steps, and templates are how an account gets restricted.

A week of grades that changed nothing

I did not let the judge block anything at first. From 09-14 to 09-19 it ran in shadow: every draft that passed the first layer was graded, the grade was written down, and the comment went out anyway. I did not know whether the grade meant anything, and a judge that predicts nothing is worse than none: it costs a rewrite every time it fires and teaches the writer to satisfy the judge instead of the reader. It had to earn the right to refuse.

By this afternoon the shadow week had produced 103 verdicts, and 19 of them were a 3. The reasons it gave for a 3 were consistent: a self-credential anecdote that would fit any post in the niche, or the post's own line echoed back to its author as if that were a response.

Joining the grades to the replies

Every grade is stored as a row, and every comment is stored as a row with a field for whether anyone answered. Those were separate tables with a shared id, and until today nobody had joined them.

Here is what came back, counting only comments that had since been checked for a reply:

  • Score 3: 1 of 14 got a reply.
  • Score 4: 12 of 57.
  • Score 5: 5 of 10.

Roughly 7%, 21% and 50%. The grade tracks what I actually care about. A reply is the start of a conversation, and conversations with people who hire are the only number this whole setup exists to move.

What changed at 13:34

I switched it on from the laptop at 13:34 IST, in the same commit that took me out of the approval loop for posts. The rule is short: a score of 3 or below is now a rewrite, not a note. The phases that write comments and replies refuse the draft and try again. Reply-watch, which answers people who have already written back to me, gets one rewrite and then posts regardless, because leaving a person unanswered costs more than a mediocre answer does.

The first afternoon with it on shows both sides. In the 14:00 hiring pass the judge scored four drafts: 3, 4, 4 and 5. The 3 was a comment on a senior full-stack AI role. Its complaint was fair. The draft raised one specific item from the post and then dropped it, argued about something that would fit any AI job ad, and ignored what made that post different from the others. That comment never went out; the application went by email instead.

The weak spot, stated plainly

The 50% is the number I trust least. It rests on ten comments, and it could halve next week without anything about the judge being wrong. The 3s rest on fourteen. Only the 4s, at fifty-seven, are a row large enough to lean on.

So the gate comes with a condition written into the same file that turns it on: re-run that join after a week. If a 3 stops predicting fewer replies than a 4, the layer is theatre again and it comes out.

A Threads post I wrote at 18:30 about these numbers, caveat included, never arrived: it was read back as scheduled, then failed at publish with a bare 400 The requested resource does not exist, and nothing saw it until the end-of-day reconciliation. So this post is where the numbers land instead.

What it taught

A check you cannot measure is an opinion with a latency cost. Running this one in shadow for a week was slower than switching it on at once, and it is the only reason I can say the refusal is worth its rewrite. The grade was cheap to record and useless until it was joined to an outcome. Once it was, the decision could be made the same day.

About the author

Amar Gupta

Amar Gupta

Senior Frontend Developer — AI & MCP

I build production frontends in React, Next.js and TypeScript — and the AI and MCP tooling behind them. 7+ years shipping web applications, from data modelling through to the deployed interface.

📍 Delhi, India · Open to Full-time

Read next