Mo RezaAli

The model bill is a dollar a month, and the hours are what decide it

contact@therezaali.com (Mo RezaAli) — Thu, 06 Aug 2026 00:00:00 +0200

There is a CVSS 9.3 record against the sentence that sells this business.

An insufficient database Row-Level Security policy in Lovable through 2025-04-15 allows remote unauthenticated attackers to read or write to arbitrary database tables of generated sites. NOTE: this is disputed by the Supplier because each individual customer of the Lovable platform accepts a responsibility over protecting the data of their application.

CVE-2025-48757, National Vulnerability Database

The sentence it fires at is the one I wrote myself, and the one every version of this pitch contains: let the platform own the plumbing, because hosting, database, auth, deployment and infrastructure security are already handled. Read the supplier’s dispute again. Their defence is that data security in a generated app belongs to the person who generated it. That is the correct division of responsibility and it is the exact opposite of what the pitch promises.

Why I am the one telling you this

I have run an unattended system into the ground and I have the commit log.

From 31 October 2025 to 27 April 2026 a worker on my own domain ran on a cron of 0 */1 * * *. Every hour it took 450 characters of an RSS description, passed them to gpt-4o-mini with a rewrite pass on gpt-4o, and committed the output straight to main. AUTO_PR was "false", because I never wrote a review step. It made 842 Add post: commits and 2,922 files across four languages. 180 of the 739 English posts cited a source called “Internal Analysis” that does not exist. I deleted all of it in August 2026 and wrote up the full account.

No alert fired for six months. Nothing was measuring anything, so nothing had a threshold to cross, so the pipeline emitted its worst output at a steady hourly rate until a human finally read it. That is the failure this article is about, and it is not a story about a model writing badly. It is a story about a system with no published number that a person checks on a schedule.

The other thing I did was read the documents. This market’s most-repeated statistic traces to a 2011 magazine article and a 2007 conference deck, both free, and the deck says in as many words that it did not measure the thing it gets cited for.

One piece of housekeeping, because it is the standard I am asking you to hold me to. My original post carried four revenue ranges. They were one creator’s illustrative figures, I could not name the creator, and an income claim from an unnamed third party is the exact shape this site fails its own build on. They are gone. What is left is arithmetic and documents.

1. The plumbing is owned until the record says otherwise

Read the record carefully rather than as ammunition. It is tagged disputed and exclusively-hosted-service, its NVD status is Deferred, and the CISA assessment of 25 June 2025 records exploitation as poc and automatable as yes. Reporting a disputed CVE as settled fact would be the same error this site exists to correct, so: disputed, and the dispute is the point. The vector is AV:N/AC:L/PR:N/UI:N/S:C/C:H/I:L/A:N. Network, low complexity, no privileges, no user interaction, scope changed. In plain terms the generated app’s own tables were reachable by an unauthenticated stranger. Not a connected CRM, not an integration somebody wired up carelessly. The database the platform handed over.

My original post conceded one thing here: that how client data is accessed, stored and acted on once you connect a real CRM or inbox stays your responsibility. That concession was too narrow by exactly one layer.

Detect it: take the public client-side key your generated app ships in its front-end bundle, and from a machine that has never signed in, request a table the app never exposes in its interface. Users, invoices, submissions. If a row comes back, row-level security is not on for that table. This takes four minutes and it is the single check whose absence the CVE describes.

Design so it cannot recur: treat every table the platform generates as public until you have personally denied it. Default deny, then grant. Run the unauthenticated read as a test you repeat after every schema change, because the schema changes when you prompt for a new feature and nothing tells you which policies came with it.

2. Sixty seconds is not a measurement

I described the product as every inbound lead scored and personally followed up within 60 seconds. That is a product specification, so it is not false. But it borrows its authority from a number nobody quoting it has read.

The 2011 Harvard Business Review article by Oldroyd, McElheran and Elkington audited 2,241 US companies by timing each one’s response to a web-generated test lead. 37% responded within an hour, 16% took one to 24 hours, 24% took longer than a day, and 23% never responded at all. The average among firms that answered inside 30 days was 42 hours.

The published distribution is 586, 143, 90, 81, 90, 61, 7, 133, 538 and 512 companies across the buckets from under five minutes to no reply. That sums to 2,241, the stated sample, and 819 of them answered inside the hour, which is 36.5%. The arithmetic holds when you check it, which is worth knowing before you rely on the next part.

The multipliers come from a separate dataset in the same article: 1.25 million leads at 29 B2C and 13 B2B companies. Firms that tried to contact within an hour were nearly seven times as likely to qualify the lead as those trying an hour later, and more than 60 times as likely as those waiting a day. The article defines qualify as “having a meaningful conversation with a key decision maker.” Not a sale. A conversation.

The famous “21x” comes from somewhere else entirely, and it is the one worth correcting.

Table: What the popular version of the speed-to-lead statistic claims, against what the two primary documents say.

The popular version	What the primary source says
“The MIT study”	InsideSales.com’s CEO presenting InsideSales.com’s own customer data at MarketingSherpa’s B2B Demand Generation Summit, 16 October 2007. James Oldroyd was then a Faculty Fellow at MIT Sloan. Never peer-reviewed.
“21x more likely to close”	The deck states: “This study did not address close ratios.” 21x is the qualify ratio at five minutes versus 30, and each of the six companies defined “qualified” its own way.
“Respond within 60 seconds”	No primary source measures 60 seconds. The 2007 study’s finest granularity is five-minute buckets; the HBR threshold is one hour. Nothing published shows 60 seconds beating five minutes.
“Applies to email, chat, an AI agent”	The 2007 deck measures outbound phone dials and says so in its definitions: a dial is “the physical action of a sales or lead generation calling a lead.” Nothing in it touches another channel.
“And the 2011 audit confirms it across 2,241 companies”	The audit never names a channel. Its entire description of method is “measuring how long each took to respond to a web-generated test lead.” It is silent on medium, which cuts both ways: it is not evidence for the phone, and it is not evidence for an AI agent either.
A controlled result	Six companies, three years of one vendor’s CRM, unadjusted ratios, no control group, no confidence intervals. Leads reached instantly differ systematically from leads reached at 30 minutes.

What survives still sells: minutes beat hours, hours beat days, and 23% of the companies audited never replied at all. That is true, it is checkable, and the buyer can verify their own reply time in an afternoon.

Detect it: before you promise a response time, instrument the client’s current one. Send three test enquiries through their own form at different hours and time the human reply. That is your before, it costs nothing, and it is the only number in the pitch that belongs to this client rather than to a 2007 conference deck.

3. Silent compounding has a number, and it is 0.3%

My post said errors compound silently at scale and left it there. Here is the mechanism, with the threshold and the enforcer named.

Google’s sender guidelines, effective 1 February 2024, put a floor under every sender: SPF or DKIM, valid forward and reverse DNS, TLS for transmission, and a Postmaster Tools spam rate below 0.3%. That “or” is load-bearing and easy to read as “and”, so take it from Google’s own two-line summary of the split: “All senders: SPF or DKIM” and “Bulk senders: SPF, DKIM, and DMARC.” Yahoo draws the line in the same place, telling all senders to “implement SPF or DKIM at a minimum” and bulk senders to “implement both SPF & DKIM.”

Cross 5,000 messages a day to Gmail accounts and the second list attaches: both SPF and DKIM rather than either, a DMARC record, a From: domain aligned with the SPF or DKIM domain, and one-click unsubscribe via List-Unsubscribe-Post: List-Unsubscribe=One-Click alongside List-Unsubscribe. On complaints the requirement is the same at every volume, a spam rate below 0.3%, and Google’s monitoring section tightens it: keep it “below 0.10% and avoid ever reaching a spam rate of 0.30% or higher.” Yahoo publishes the same 0.3% ceiling independently, plus a two-day window for honouring an unsubscribe.

Now watch it compound. Suppose a follow-up sequence draws one spam complaint per 500 delivered messages. That is 0.2%: under the ceiling, over the 0.10% Google asks for. At 50 sends a day you clear the authentication bar with SPF alone: no DKIM, no DMARC, no alignment, no unsubscribe header. Add clients until the same sequence sends 5,000 a day and the bar moves under you without one word of the sequence changing: now DKIM as well as SPF, now DMARC, now alignment, now one-click unsubscribe, and the same 0.2% is one bad segment from the ceiling. Cross it and you degrade the sending domain’s reputation. If you did the efficient thing and put every client on one domain, you degrade delivery for all of them in the same week. That is the concentrated-platform-risk argument from my post, applied at the one layer where it is a published number rather than a worry.

Detect it: verify the domain in Postmaster Tools before the first send, and read the spam rate weekly. Alert at 0.10%, not 0.30%. By the time you are at the ceiling you are already being filtered.

Design so it cannot recur: one sending subdomain per client, never a shared one, so a single bad list cannot take the others with it. Process unsubscribes by machine inside two days rather than by hand. Send the unsubscribe headers on every marketing message, not only above the bulk threshold, because the threshold is where enforcement starts and not where the harm does.

4. The margin variable is hours, and model spend is about a dollar

I told readers to subtract platform and model-usage costs. On the evidence, that points at the wrong line.

Table: Claude model IDs and Claude API list prices per million tokens, from Anthropic’s models overview, read on 6 August 2026.

Model	API ID	Context	Input $/MTok	Output $/MTok
Claude Fable 5	`claude-fable-5`	1M	10	50
Claude Opus 5	`claude-opus-5`	1M	5	25
Claude Sonnet 5	`claude-sonnet-5`	1M	3 (intro 2)	15 (intro 10)
Claude Haiku 4.5	`claude-haiku-4-5-20251001`	200k	1	5

Sonnet 5’s introductory rate runs through 31 August 2026, so anyone quoting it in September is quoting a stale price.

Take a content system producing 20 posts a month, at roughly 2,000 input and 2,000 output tokens each, on Opus 5. Input is 0.04 MTok at $5, so $0.20. Output is 0.04 MTok at $25, so $1.00. Total model spend: about $1.20 a month. Run ten times that volume and you are under $12. Those token counts are my assumptions for that shape of job rather than a measurement, and other vendors price differently, so take the order of magnitude as the finding. The order of magnitude is a rounding error.

Now the line that decides the business.

Table: Gross hourly rate on a $500 monthly retainer, before platform fees, model spend and the acquisition cost of the client.

Support hours per month	Gross per hour
3	$166.67
6	$83.33
10	$50.00

That is the whole model in three rows. A $500 retainer with three hours of monthly firefighting is a different business from a $500 retainer with ten, and no amount of model-price optimisation moves either row.

Which is why the best idea in my original post was the one with no number attached to it: correction burden. A system whose output the client trusts without editing is the product. The month they start fixing every result by hand, the fee stops making sense to them, and it stops making sense to you first, because their hand-fixing arrives in your inbox as support hours.

Detect it: log hours per client per month from the first week. Watch the trend, not the total. Support hours rising for two consecutive months are a churn notice with a delay on it.

Design so it cannot recur: put scope in writing before the build, naming what the fee covers, what counts as a change request, and the revision limit. Add error alerts that reach you before they reach the client. Ship a monthly results snapshot, because renewal is decided against what the client can see, and firefighting they never heard about is value you burned invisibly.

5. Article 28(4) makes the platform’s failure yours

Anyone building an automation that touches a client’s leads, inbox or customer records is a processor under GDPR. The no-code platform and the model vendor are sub-processors. That is not a metaphor, and Article 28 is short enough to read in one sitting.

The controller may use “only processors providing sufficient guarantees to implement appropriate technical and organisational measures,” and the processing must be governed by a binding contract that names the subject matter, duration, nature, purpose, data types and categories of data subject, and requires documented instructions, confidentiality, Article 32 security measures, conditions on sub-processors, help with data-subject rights, deletion or return of the data at the end, and information made available for audits.

Two paragraphs matter most. Article 28(2): a processor “shall not engage another processor without prior specific or general written authorisation of the controller.” Article 28(4): where a sub-processor fails, the initial processor “shall remain fully liable to the controller for the performance of that other processor’s obligations.”

Read that next to section 1. The platform’s position is that data security in your generated app is your responsibility. GDPR’s position is that the platform’s failures are your liability to your client. Both of those are true at once, and together they turn “let the platform own the plumbing” from a productivity tip into an assumed risk with your name on it.

My post said to get explicit permission before connecting anyone’s CRM or inbox. In the EU that is not etiquette. It is a written precondition, and naming your sub-processors is part of it.

6. Article 22 does not bite where people think it does

The correction runs the other way here, before someone sells you a compliance package you do not need.

Article 22’s right not to be subject to a decision based solely on automated processing applies only where the decision “produces legal effects concerning him or her or similarly significantly affects him or her.” Scoring an inbound B2B lead as warm or cold clears neither bar. Routine lead scoring is not what Article 22 is for, and saying so is more useful than the vague warning most of this market repeats.

The real boundary is where the decision does have significant effect: eligibility, credit-like screening, pricing that materially disadvantages someone. There, where the processing rests on contractual necessity or explicit consent, the controller must provide the right to obtain human intervention, to express a point of view, and to contest the decision. At that point “keep a human in the loop for high-stakes actions” stops being a best practice and becomes an obligation with an article number behind it.

7. The disclosure rule that started applying four days ago

The EU AI Act’s Article 113 sets the general date of application at 2 August 2026. Article 50, the transparency article, is now live.

Article 50(1) requires providers of systems that interact directly with people to make sure those people know they are dealing with AI, “unless this is obvious from the point of view of a natural person who is reasonably well-informed, observant and circumspect.” Article 50(2) requires synthetic content to be marked machine-readably, and 50(5) requires all of it “at the latest at the time of the first interaction or exposure.” Then Article 50(4), second subparagraph:

Deployers of an AI system that generates or manipulates text which is published with the purpose of informing the public on matters of public interest shall disclose that the text has been artificially generated or manipulated. This obligation shall not apply where the use is authorised by law to detect, prevent, investigate or prosecute criminal offences or where the AI-generated content has undergone a process of human review or editorial control and where a natural or legal person holds editorial responsibility for the publication of the content.

Article 50, EU Artificial Intelligence Act

My post described the content-system offer as one that keeps producing “with you as the editorial backstop.” That phrase, written as sales copy, happens to describe the exemption: human review or editorial control, with a named person holding editorial responsibility. So stop selling it as a convenience and start specifying it as a control. Name the editor in the contract. Keep the record of what they reviewed.

I will state the limit of that reading plainly. “Informing the public on matters of public interest” is not defined in a way that obviously settles whether a client’s marketing blog falls inside it, and I am not a lawyer. The reason I would build to the exemption anyway is that it costs one named person and a review log, and the alternative is finding out.

What I would keep from the pitch, unchanged

Most of it survives, and the parts that survive are the parts that were never carrying a number.

The billing shapes hold. Bill monthly when the value is a system that keeps running, bill once when you hand over an asset the client owns, and use setup-plus-monthly when there is real build cost up front and ongoing value to maintain. Anchor the price out loud against the labour it displaces, so the buyer measures you against a salary rather than a software subscription. Polish belongs to that anchor rather than to vanity: a finished client-facing surface justifies a price a rough script will not.

The build shape holds. Almost all of these systems are one skeleton: capture, qualify and store, automate, dashboard. Build it once properly and re-skin it per vertical. And plan before generating: have the model turn requirements into a data model, workflows and views, read the plan, correct it with your own judgment, then build from the corrected version. One piece of wording I had loose, since I called it a reasoning engine. Anthropic calls Claude Code an agentic coding tool that reads your codebase, edits files and runs commands, and Claude the model family is a different product from Claude Code. Most Claude Code surfaces need a subscription or a Console account, so it is not free either.

Niching down holds, with the absolute removed. I wrote that “lead-gen for local dental practices” outsells “AI automation for businesses” every time. I have no survey and neither does anyone else selling that advice. The defensible version: one niche gives you reusable builds and outreach language that lands, which is a mechanism rather than a measured result.

And one observation, offered as an observation. I have watched people who were one demo away from their first client disappear into a config error for a month. I cannot quantify that and I will not pretend otherwise.

What I have now that I did not have before

Five numbers I can check, and one rule.

The numbers: a CVSS score and a vendor dispute that between them define who owns app data security. A response-time literature that measures outbound phone dials where it names a channel at all, is silent on channel where it does not, and nowhere measures 60 seconds. A 0.3% spam-rate ceiling with a 0.10% target and a 5,000-per-day threshold where the authentication requirement doubles. A model bill of roughly a dollar a month against a support-hours line that decides everything. And a date, 2 August 2026, after which the disclosure question has an article number attached to it.

The rule is the one my own pipeline taught me at a cost of 2,922 files: any system that runs unattended has to publish one number a human reads on a schedule, and if you cannot name that number before launch, you have not built a service. You have built something that will be wrong quietly for as long as nobody looks. Mine was wrong quietly for six months, and the number that would have caught it was the count of citations whose URLs did not resolve, which is now the check that has to pass before anything on this site ships at all.

The 23-minute interruption figure is not a recovery time

contact@therezaali.com (Mo RezaAli) — Thu, 06 Aug 2026 00:00:00 +0200

The published measurements of how long it takes to get back to work after an interruption run from 599 milliseconds to 25 minutes and 26 seconds. The top of that range is 1,526 seconds, so dividing it by 0.599 gives a spread of about 2,548 times, and every number in it is correct. Four research groups measured four different quantities, and English calls all four “recovery”.

The most quoted figure in this subject is not among them, because it is not from a paper. “It takes 23 minutes and 15 seconds to refocus after an interruption” originates in a Gallup Business Journal interview with Gloria Mark, published 8 June 2006. What she describes it as measuring is elapsed time until interrupted work was resumed, conditioned on the 81.9 percent of cases where it was resumed the same day at all, and counting minutes in which the person was working productively on roughly two other things. Her own assessment of it, in the sentence that produces it, is “which I guess is not so long.”

The published figure from the same research programme, Mark, Gonzalez and Harris at CHI 2005, is 25 minutes and 26 seconds, with a standard deviation of 54 minutes and 48 seconds. The deviation is more than twice the mean, so the distribution is not summarised by its average, and no article I have seen quoting 23 minutes has mentioned it. An independent citation audit checked five commonly cited papers for the 23-minute figure and found it in none of them. I pulled the two primary PDFs and confirm the same.

Table: What four primary studies actually measured. All four get reported in popular writing as recovery time after an interruption.

Construct measured	Study	Value
Extra delay before the first action on resuming a suspended goal	Monk, Trafton and Boehm-Davis, JEP: Applied 2008, lab task	599 ms
Return to the prior work rate, scored off videotape	Jackson, Dawson and Wilson, EASE 2002, 16 employees, 28 days	64 s
Return to the application holding the primary task after an email alert	Iqbal and Horvitz, CHI 2007, 27 users, instrumented	9 min 33 s
Elapsed clock time until the thread of work resumed, same day only	Mark, Gonzalez and Harris, CHI 2005, 24 workers, 700+ hours	25 min 26 s

These are not substitutes. 599 ms is a memory retrieval cost, and it is a difference rather than a duration. Nobody in that experiment waited 599 milliseconds and then carried on. Monk, Trafton and Boehm-Davis measured a mean resumption lag of 1,548 ms (SD 231) after an interruption against a mean interval of 949 ms (SD 283) between actions on uninterrupted trials, and report the gap between the two as “an estimated cost of 599 ms on the VCR programming task.” Quote either number and you are right; quote them as the same number and you are not. 64 seconds is a behavioural recovery. 9 minutes 33 seconds is a navigation delay, and Iqbal and Horvitz say so themselves: their measure is application access, “not resumption of a suspended task.” 25 minutes 26 seconds is a scheduling statistic that includes 2.26 other working spheres of real work.

The post I wrote, and what was missing from it

I published a post saying I had spent years treating responsiveness as productivity, that some of my busiest days produced almost nothing that mattered, and that my most productive weeks were the ones where I disappeared for a few hours a day with no Slack, no meetings and no email. That post is here. All of it is my own first-person recollection, unaudited, with no output metric behind it. I never measured a before or an after. That is exactly the shape of a post-hoc success story, so the useful work is finding out which parts of it survive contact with people who did measure.

One phrase survives, conservatively. I wrote “context switches every few minutes.” The CHI 2005 field data puts an average working sphere at 11 minutes 4 seconds before a switch. The earlier study it builds on, Gonzalez and Mark at CHI 2004, timed the finer unit and found 3 minutes 8 seconds of continuous work on a single event, with formal meetings and personal time excluded from that average.

Interrupted work gets finished faster

Mark, Gudith and Klocke, CHI 2008, titled “The Cost of Interrupted Work: More Speed and Stress”, is the paper most often cited for the proposition that interruptions destroy productivity. On the productivity measure it found the opposite. Subjects answered a fixed set of emails, interrupted every two minutes by phone or instant message, with the time spent on the interruptions themselves subtracted out.

Table: Completion time and reported stress in Mark, Gudith and Klocke, CHI 2008. Stress is a 20-point scale.

Condition	Completion time	Stress
No interruption	22.77 min	6.92
Interrupted, same context	20.31 min	9.46
Interrupted, different context	20.60 min	9.13

Both interrupted conditions were significantly faster than the uninterrupted one. There was no significant difference in errors and none in politeness. Emails fell from 31.49 words when uninterrupted to 29.17 when interrupted, a significant drop. Frustration, time pressure and effort all rose significantly. The authors’ own reading: people “develop a mode of working faster (and writing less) to compensate.”

Two things must travel with that result or it misleads. The study ran 48 subjects, 81 percent of them German university students with a mean age of 26, in a 1.5-hour laboratory session, not a field study of information workers. And the authors flag the alternative they could not rule out: the lost time may occur only in the moments right after each interruption and be masked by the faster overall style, which would need more sensitive measurement to see.

The correction still holds and it is the useful part. Fragmented time produces output, faster, at equal accuracy, and shorter. The cost is not throughput. It is paid in stress load and in elaboration. My post said the work that matters “doesn’t happen during the ten minutes between two meetings,” and anyone who has cleared an inbox in ten minutes knows that is false as stated. What is true and sourced is that what you produce in ten minutes is systematically shorter and costs you more to produce.

The mechanism

Goals are not stored on a stack that you pop back to. Altmann and Trafton’s activation-based model in Cognitive Science treats a goal as a memory item whose activation decays over time and suffers retroactive interference from goals encoded after it, with retrieval depending on cues in the environment. That is why 2.26 intervening working spheres cost more than 25 minutes of clock time does, and why Mark’s informants reported real difficulty when windows or papers had been rearranged during their absence: the retrieval cue was gone, so the pending goal had nothing to be reinstated by.

The second mechanism is attention residue. Leroy, in Organizational Behavior and Human Decision Processes, found across two experiments that people “need to stop thinking about one task in order to fully transition their attention and perform well on another,” and then reports the part most productivity writing has backwards: finishing the prior task is not sufficient. Time pressure while finishing it is what enables the disengagement. A task you completed calmly can leave more residue than one you completed against a deadline.

That answers why my busy days produced nothing, and the answer is not that I did no work. I did a great deal of work, quickly, and it was short. The analysis sitting in drafts was the item whose activation had decayed furthest, because it was the only one with no external cue demanding it be reinstated.

How to detect this in your own system

Count switches, not pings. Microsoft’s June 2025 Work Trend Index says employees are interrupted every two minutes during core work hours, 275 times a day, by meetings, emails or chats. Read its methodology note before quoting either number. The two figures are measured over different windows and Microsoft says so in consecutive sentences: “The two-minute figure reflects the average time between pings during an eight-hour workday. The 275 is based on the 24-hour day.” Then comes the sentence that almost never travels with the statistic: “Based on the top 20% of users by ping volume received.” So 275 is not the average worker’s day. It is a 24-hour count for the most-pinged fifth of users, and it is not divisible into an 8-hour one. What is measured underneath is “a rolling 28-day sum of pings (meeting invites, emails, chats) per unique user per workday”, which counts what arrived, not what pulled you out of what you were doing. Your own equivalent is the count of times you changed context, which no dashboard reports and which you have to sample by hand.

Measure your internal-to-external ratio before you buy a solution. In Mark’s CHI 2005 data, 48 percent of interruptions were external and 52 percent were self-initiated, and among analysts and developers the two ran roughly equal. Every calendar-blocking remedy, including the one I recommended in my post, addresses the external 48 percent.

Check whether alerts actually move you. Iqbal and Horvitz found only 40.8 percent of email alerts produced a switch to the mail client within 15 seconds; the other 59.2 percent averaged 7 minutes 32 seconds first, which the authors read as self-initiated. Jackson, Dawson and Wilson, watching 180 hours of videotape, found 70 percent of emails reacted to within 6 seconds of arrival. People already treat most alerts as deferrable, and unreliably.

Separate interruption from context change. This is the sharpest rule in the literature and I did not have it. Mark’s informants described interruptions inside their current working sphere as “interactions” and reported them as beneficial, one analyst describing a problem continuing to develop in the background while not being worked on. What they called disruptive was being forced out of the current sphere. The cost is in the context change.

How to design so it cannot recur

Preserve the retrieval cue. Altmann and Trafton’s model makes the workspace itself part of the memory system, so leaving the file open, the terminal on the failing test and the note half-written is the cheapest intervention available, and it costs nothing.

Do not assume a blocked calendar solves it. In the same Gallup interview that produced the 23-minute figure, Mark describes getting a three-hour block, closing her office door, spending the entire block on small accumulated tasks and touching none of the paper she had reserved it for. Her own diagnosis: “I interrupted myself. I kept interrupting myself.” That is the honest limit on the prescription I gave, from the researcher who measured the problem.

Be careful what you claim for batching. Kushlev and Dunn ran 124 people, one week limited to three email checks a day against one week unlimited, and found significantly lower daily stress in the limited week, F(1,121)=4.18, p=.04, d=.37. Mark and colleagues at CHI 2016 logged 40 people over 12 workdays with heart-rate monitors and state that “despite widespread claims, we found no evidence that batching email leads to lower stress.” I would bet on the second, and the reason is in the details. Kushlev and Dunn’s effect appears on a single-item daily stress question but not on the validated weekly Perceived Stress Scale, where the two conditions sit at 1.68 and 1.67 with d=.04. The sample was roughly two thirds students, there was no control condition, and the distraction effect was significant for students and absent for community members. Mark’s team measured physiologically, in situ, on working adults. When a daily self-report and a biosensor disagree, the self-report is the measure that knows which condition it is in.

Design the block for elaboration, not for volume. Since interrupted work comes out faster and shorter at equal accuracy, protected time is for the work whose quality lives in its length: the argument that needs a fourth paragraph, the analysis that needs its caveat, the problem that needs a wrong approach tried first.

What I have now that I did not have before

A rule that discriminates, in place of a rule that generalises. My post said to protect attention, which is true and unactionable. What I run instead is a test on context change: an interruption that keeps me inside the current working sphere is nearly free and sometimes useful, and one that moves me out of it is the expensive event, whatever it cost in wall-clock minutes.

I also have a number I will not use. When I next see 23 minutes and 15 seconds in a deck, I know where it came from, what it measured, that the researcher who produced it called it not so long, and that the peer-reviewed version is 25 minutes 26 seconds with a standard deviation twice its own size. The figures that circulate most are the ones that have travelled furthest from what was actually measured, and the provenance is worth more than the number.

Seven steps at 85% each is a 32% agent, and mine ran worse

contact@therezaali.com (Mo RezaAli) — Thu, 06 Aug 2026 00:00:00 +0200

Seven steps at 85% each is a 32% agent. Ten steps is 20%. Those two figures are exact: 0.85⁷ = 0.3206 and 0.85¹⁰ = 0.1969. No prompt, no model upgrade and no platform changes them, because per-step success rates in a chain multiply instead of averaging.

I built a seven-node abandoned-cart agent in a visual workflow builder and it passed its demo on one test cart. Then I pointed it at a real day: 140 abandoned carts, of which maybe 30 came out the other end handled correctly. About one in five, by my own rough count that night, and I want to be precise about how imprecise that is. I was reading a CRM, not a log table. It is my own unaudited figure from my own work, published here before this site existed. There is no execution log to link, so this one rests on my account of it.

One customer who abandoned a $12 phone case got a “complete your premium order” SMS carrying a 15% discount code I never meant to issue to anyone. That class of send ran for about two weeks before I caught the pattern. Call it a couple hundred dollars in discount codes handed to people who were not buying. Every node in that chain reported success.

I blamed the model first and swapped to a bigger one. Same mess, slightly more expensive. The hypothesis was wrong, and the arithmetic below says by how much.

The arithmetic, and the assumption it rests on

The model here is not new and it is not mine. It is the series-system reliability formula, and the NIST/SEMATECH e-Handbook states it in section 8.1.8.2 as R_S(t) = ∏ R_i(t): multiply the reliabilities. The handbook also states the condition that makes it valid, and this turns out to be the whole story of my agent: “Each component operates or fails independently of every other one, at least until the first component failure occurs.”

Hold onto that condition. It is the one my agent broke.

The worksheet, and what those numbers actually are

Here is the flow with a rate against each node. Read this before the table, because it changes what the table is: these are not logged measurements. I did not instrument per-node success at the time. They are my estimates, assigned after watching the thing fail, and by my own admission in the original post I was being generous on a couple of them.

Table: My seven-node abandoned-cart flow with retrospectively estimated per-node success rates, and the running product. The rates are my estimates rather than measurements; the running total is arithmetic on those estimates.

Step	Node	Estimated per-step success	Running product
1	Read cart contents	0.97	0.9700
2	Classify buyer intent	0.78	0.7566
3	Pick the right offer	0.88	0.6658
4	Draft the email	0.92	0.6125
5	Draft the SMS	0.92	0.5635
6	Schedule the send	0.95	0.5354
7	Log result to CRM	0.90	0.4818

Modelled end to end: 48%. For a flat rate applied to any chain length, the curve looks like this.

Table: End-to-end reliability of a chain of independent steps, by chain length and uniform per-step success rate.

Steps	@95%	@90%	@85%
3	86%	73%	61%
5	77%	59%	44%
7	70%	48%	32%
10	60%	35%	20%

The model under-predicted the failure, and that is the finding

My original post said the ten-step figure of 20% matched my night at the CRM almost exactly. It does not, and the mismatch is worth more than the match would have been. My agent had seven steps, not ten. My two models of it predict 32% (flat 0.85) and 48% (the worksheet). What I published was one in five. Both models were optimistic, one of them by nearly thirty points.

Two derivations pin the gap. Both are this article’s arithmetic on the one rate I published, not a second measurement, and they inherit its looseness: I counted “maybe 30” out of 140 off a CRM screen and called it one in five, so treat 0.20 as a round number and not a reading. To land at 20% with a uniform 0.85 per step you need ln(0.20) / ln(0.85) = 9.9 steps, so the 20% figure describes a chain I did not build. And a seven-step chain finishing at 20% implies a uniform per-step rate of 0.20^(1/7) = 0.795, not 0.85.

Three mechanisms account for the difference, and each one generalises.

A node on the canvas is not a step in the chain. “Classify buyer intent” is one box and at least three failure surfaces: the model call, the parse of whatever came back, and the schema or enum check that decides the label is usable. Count failure surfaces, not boxes, and my seven-box flow was closer to ten steps than seven. The multiplication was right. My count of what to multiply was wrong.

The independence assumption does not hold. NIST is explicit that the product formula needs each component to fail independently of every other one. A cart record with a malformed line item does not fail one node. It degrades the read, poisons the classification and misprices the SMS, all from one cause. Correlated failure makes the real distribution more bimodal than the product of independent rates suggests: batches come out mostly fine or mostly wrecked rather than uniformly degraded. The product is a fair planning tool and a poor description of the variance.

Estimates made by the person who built the thing run warm. The 0.795 above sits five and a half points under the 0.85 flat default, and about ten points under the 0.901 geometric mean of the rates I actually wrote in the worksheet (0.4818^(1/7)). The generic default was optimistic. The rates I picked node by node, after watching the thing fail, were about twice as optimistic as the generic default I had not bothered to think about.

Google’s SRE team published the sharpest correction to naive chain multiplication I have found, in The Calculus of Service Availability (ACM Queue, March/April 2017). Their starting rule is mine: “A service cannot be more available than the intersection of all its critical dependencies.” Then they call the inference that each extra link needs another 9 “incorrect”, because a dependency appearing at several points must be counted once. Their rule: “If a service has N unique critical dependencies, then each one contributes 1/N to the dependency-induced unavailability of the top-level service, regardless of its depth in the dependency hierarchy.”

Applied to my flow, my classify-intent node is not step two of seven. It is a critical dependency of steps three through seven, which is why one wrong label produced five confident successes carrying a wrong answer. Count unique critical dependencies, not boxes.

Why that node and not another

The rates in the worksheet are uneven and the split is not random. Reading a cart is 0.97 and hitting a scheduler is 0.95 because both are plumbing: deterministic, checkable, the same every time. Classifying why someone abandoned is 0.78 because it is a judgment call made in one pass with no way to ask a follow-up question. Price shock. Just browsing. Payment failed. Distracted. Comparison shopping. Accidental add. My agent would read a cart holding one cheap item, decide “high purchase intent, price-sensitive” and fire a discount at someone who was never buying, who then learned that abandoning carts prints coupons. That is where it snapped in every batch I looked at, and I have not seen a classification node of that kind sitting at 95%.

Why it fails green

The worst property of this failure is not the rate. It is that nothing turns red.

That is a documented property of the tooling, not a quirk of my build. n8n’s error handling documentation says the error workflow “runs if an execution fails”, and gives the usual causes as “errors in node settings, or the workflow running out of memory”. The only way to make a workflow fail deliberately is the Stop And Error node, which the same page documents as a way “to force executions to fail under your chosen circumstances”. That is a statement about the world model the platform holds: an execution either errors or it does not, and semantic correctness is not an observable. A node returning “high purchase intent, price-sensitive” for someone who was browsing has succeeded by every definition the runtime has.

One layer down, the reason the model produces a confident wrong label rather than declining is also documented. Why Language Models Hallucinate (Kalai, Nachum, Vempala and Zhang, September 2025) argues that models hallucinate “because the training and evaluation procedures reward guessing over acknowledging uncertainty”, and that they “originate simply as errors in binary classification”. That is why a bigger model does not fix it. A bigger model is optimised by the same scoring that pays for a confident guess and charges nothing extra for a wrong one.

The failure category also has a name. Why Do Multi-Agent LLM Systems Fail? (Cemri, Pan, Yang et al., Berkeley, revised October 2025) built a taxonomy from 1600+ annotated traces across 7 frameworks, developed from 150 traces under expert annotation at an inter-annotator agreement of kappa = 0.88. It lands on 14 failure modes in three categories, and the third is task verification. Their conclusion is that these failures “require more sophisticated solutions”, not better models.

Why you cannot out-model a long chain

My original post got this argument right and the arithmetic wrong, and correcting it makes the point harder. I wrote that a better model might move a node from 0.85 to 0.90, and that across ten steps this takes you from 20% to 35%. It does not. Upgrading one node in a ten-step chain gives 0.85⁹ × 0.90 = 0.2085. Twenty percent to twenty-one percent. The 35% figure is 0.9¹⁰ = 0.3487, which requires every node in the chain to improve. One better node buys you one point.

The published evidence points the same way. METR’s Measuring AI Ability to Complete Long Software Tasks (arXiv 2503.14499, v4 revised 10 July 2026) puts frontier models at a 50%-task-completion time horizon “of around 50 minutes”, doubling “approximately every seven months since 2019”, and attributes the increase “primarily” to “greater reliability and ability to adapt to mistakes”. Length is the binding constraint and reliability is what moves it. Those are software engineering tasks with expert human baselines, so take the direction, not a number for a marketing flow.

tau-bench (Yao, Shinn, Razavi and Narasimhan, June 2024) introduced pass^k, the probability that all k independent trials of the same task succeed, and reports that “even state-of-the-art function calling agents (like gpt-4o) succeed on <50% of the tasks, and are quite inconsistent (pass^8 <25% in retail)”. That is my multiplication run across repeated trials instead of across steps, by researchers, with a metric that has a name.

It is also the honest limit on my planning number. I use 85% as the default for a fuzzy step and 94% or better for a deterministic one, and that is an assumption rather than a citation. tau-bench measures whole episodes, so it supports “chains degrade fast” and does not establish that any given LLM step sits at 85%. I could not find a public benchmark reporting per-step success for a node like “classify why this person abandoned a cart”. Instrument your own nodes and use your numbers over my default.

Retry, and why it does not save you

The obvious objection from anyone running these flows is that the platform retries. n8n documents per-node Retry on Fail, Max Tries and Wait Between Tries (ms) in node Settings. That page describes the mechanism and states no default value for either number, so the retry budget is a choice you make rather than one the page makes for you.

Retry then partitions the chain along the same line everything else in this piece does. For an independent transient failure it is close to free reliability: a plumbing step at p = 0.95 with three tries gives 1 − (1 − 0.95)³ = 0.999875. For a judgment step the error is largely deterministic given the same input and the same prompt, so retrying returns the same wrong label with the same confidence and the effective rate stays at 0.78. Retries fix plumbing and do nothing for judgment.

The one technique I know of that raises a judgment node without a human and without a bigger model is sampling the step several times and taking the majority answer. Self-Consistency (Wang et al., 2022, revised 2023) reports +17.9% on GSM8K and +6.4% on StrategyQA doing exactly that. Those are 2022 reasoning benchmarks, not intent classification, so it is a mechanism to test rather than a promised lift, and n samples costs n times the tokens for that step.

How to detect this in your own flow

The test takes ten minutes and needs no tooling.

List every failure surface, not every box. A node that calls a model, parses its output and validates a schema is three rows.
Assign each row a success rate you would bet your own money on, not the rate from your best demo run. Mark each row JUDGMENT or PLUMBING.
Multiply down the column. That product is your optimistic ceiling, not your expected result, for the three reasons above.
For each row, ask whether a wrong output fails loud or fails green. Every green-failing row is invisible to your platform’s error handling and needs a gate you wrote yourself.
Mark which rows are critical dependencies of downstream rows. Those are where one error multiplies into several confident successes.

For the empirical version rather than the estimate, log the input and output of every node for one real batch and score a sample by hand. That is what I did not do before shipping, and it is the difference between a worksheet and a measurement.

Design so it cannot recur

Delete steps before you optimise them. Cutting my flow from seven steps to five, by removing the standalone SMS-draft node and folding CRM logging into the send step, takes the modelled 0.97 × 0.78 × 0.88 × 0.92 × 0.95 to 0.5819. Forty-eight percent to fifty-eight, holding the same per-step rates. But the two moves are not equivalent and my original post glossed over it. Deleting work is an unconditional gain. Merging work is conditional: the send step now does two jobs, and holding its rate at 0.95 is an assumption I have not tested. Merging only pays if the merged node does not absorb the failure rate you thought you removed.

Put the human on the weakest number and nowhere else. One checkpoint, at classify-intent. The agent drafts everything and parks the batch. Each morning I spend about eleven minutes scanning intent labels and the offers attached to them, approve, fix the handful that are wrong, and release. The whole flow by hand used to take two hours. Both are my own unaudited figures from my own work, in the same post linked above. Automation moved the judgment. It did not remove it.

Gate what a rule can check. Two of mine: an offer that is a discount over 20% on an order under $20 gets blocked and flagged, and an SMS carrying a price that does not match the cart gets blocked. These are if-statements. They make no model call and cost no tokens per run.

The arithmetic of a gate is worth writing down, because my original post asserted the effect without the formula. For a node with success rate p and a gate catching a fraction d of its errors, effective correctness is:

p' = p + (1 - p) * d

On my weakest node, p = 0.78: a gate catching 70% of errors gives 0.934, one catching 90% gives 0.978. Two conditions travel with that formula or it lies to you. It only holds if caught errors are then corrected; if they are merely blocked and dropped, precision rises, throughput falls and end-to-end success does not improve at all. Mine are corrected, by me, in the eleven minutes. And my two rules are value-range checks that catch the loud subset of a silent failure class. Neither catches “wrong intent label, plausible offer, correct price”, which is the failure that cost me the fortnight.

The demo is one trip down the chain on a clean input. The system is the same chain on real volume, where the weakest node decides the outcome and reports success the whole way down. Before the next one ships, run the column. If the number scares you, that is the column doing its job.

Kimi K3’s checkpoint is 1,560.9 GB, which is 121 GB more than a B200 node holds

contact@therezaali.com (Mo RezaAli) — Thu, 06 Aug 2026 00:00:00 +0200

Kimi K3’s weights ship natively as MXFP4. Four bits per parameter, half a byte, quantization-aware trained rather than squeezed after the fact. Multiply that by the 2.8 trillion parameters on the model card and you get 1,400 GB.

The checkpoint Moonshot actually uploaded is 1,560.9 GB.

The 96 safetensors shards in the repository’s file listing sum to 1,560,936,091,448 bytes: 1,560.9 GB, or 1,453.7 GiB. Hugging Face’s dtype census for the same repository lands 75.8 MB below that, which is the safetensors JSON headers. Both counts are 161 GB above the multiplication, and the reason is one field in config.json.

A 4-bit release is not uniformly four bits

quantization_config targets Linear layers at num_bits: 4, and then exempts six patterns:

"ignore": [
 "re:.*self_attn.*",
 "re:.*shared_experts.*",
 "re:.*mlp\\.(gate|up|gate_up|down)_proj.*",
 "re:.*lm_head.*",
 "re:.*vision_tower.*",
 "re:.*mm_projector.*"
]

Attention, the shared experts, the dense MLP projections, the output head, the vision tower and the multimodal projector never get quantised. They stay BF16. That is 57.18 billion parameters, 2.06% of the model, and on their own they cost 114.4 GB. The parts left in higher precision are exactly the parts that would break first if you crushed them: the router-independent path every token takes.

The second missing term is the format’s own bookkeeping. group_size is 32 and scale_dtype is torch.uint8, so every 32 quantised values carry a one-byte shared scale. Across 2.72 trillion quantised parameters that is another 85.1 GB of scales that are not weights and still occupy memory.

Table: Kimi K3’s published checkpoint by dtype, from Hugging Face’s parameter census for moonshotai/Kimi-K3 and the group size in config.json, accessed 6 August 2026.

Component	Parameters	Bytes
MXFP4 weights, four bits packed	2,722,740,830,208	1,361.37 GB
MXFP4 group scales, one byte per 32 values		85.09 GB
BF16 tensors (attention, shared experts, dense MLP, output head, vision tower)	57,179,884,544	114.36 GB
F32 tensors	11,122,432	0.04 GB
Total	2,779,931,837,184	1,560.86 GB

Half a byte each against the exact parameter count gives 1,390.0 GB. The real file is 170.9 GB larger, and that overshoot splits almost evenly in two: 85.1 GB of group scales, 85.8 GB of extra precision on the 2% that was never quantised. The checkpoint averages 4.49 bits per parameter, not four.

Subtract

NVIDIA’s DGX B200 page states “8x NVIDIA Blackwell GPUs” and “GPU Memory 1,440 GB total, 64 TB/s HBM3e bandwidth”. It does not print a per-accelerator figure; 180 GB is 1,440 divided by eight, and that division is mine, not NVIDIA’s.

1,560.9 GB Kimi K3 checkpoint (96 safetensors shards)
1,440.0 GB node memory (8 x B200)
----------
 -120.9 GB short

The weights do not fit. Not tightly, not with tuning: an eight-GPU B200 node is 121 GB short of holding the checkpoint at rest, before one byte of KV cache, activations, the 401M-parameter MoonViT-V2 vision encoder or CUDA graphs. Which is what vLLM’s day-0 post says in plain words: “The entire model can barely fit in a single NVIDIA DGX B300 and requires a minimum of 16 NVIDIA B200/GB200 GPUs to serve on that hardware generation.”

The first version of this page said 1,400 GB and put 40 GB of headroom on that node. It reached that by multiplying the headline parameter count by four bits, which is the shortcut the model card invites and the checkpoint refuses. The error was 161 GB, it inverted the conclusion from a tight fit to an impossible one, and it made vLLM’s sixteen-GPU floor look like caution rather than arithmetic. The file listing is public and the shard sizes are on it. Add up the 96 numbers yourself. That is the check, and it is the one I should have run before I ran the multiplication.

What I published on 17 July, and what the sources say

I posted a summary of K3 on 17 July 2026, about 22 hours after Moonshot’s launch thread. Four of its claims do not survive the primary sources.

It said K3 has “faster inference”. It does not. It said Moonshot “introduced two new architectural improvements”. Both were published months earlier, with their own papers. It said the open weights were coming “later this month”, which was true for ten more days and is now simply out of date. And it said K3 “consistently ranks among the strongest models” rather than “dominating a single category”, which is the exact inverse of what happened.

Those are four instances of one mechanism. A launch thread is a compression of a technical report, written by the vendor, optimised for a specific effect. Summarising it inherits every framing decision while dropping every qualifier underneath, because the qualifiers live in the papers, the licence file and the config JSON. So I went and read those instead.

KDA and AttnRes were not new, and the real claim is better

Kimi Delta Attention was introduced in Kimi Linear (arXiv 2510.26692, submitted 30 October 2025) as “an expressive linear attention module that extends Gated DeltaNet with a finer-grained gating mechanism”. Attention Residuals got its own paper, Attention Residuals (arXiv 2603.15031, Kimi Team, led by Guangyu Chen), submitted 16 March 2026. The weights shipped on 27 July 2026, which is 270 days after the first paper and 133 days after the second.

They also do different jobs, and welding them into one sentence about long context gets one of them wrong. KDA is the context mechanism: linear-attention layers holding a fixed-size recurrent state, interspersed with periodic full-attention layers so global recall survives. AttnRes is a depth mechanism. It “replaces fixed accumulation with softmax attention over preceding layer outputs”, because uniform residual aggregation causes “uncontrolled hidden-state growth with depth, progressively diluting each layer’s contribution”. Nothing to do with context length.

The Kimi Linear paper reports up to 6x decoding throughput at 1M context and a 75% KV cache reduction. If you see either quoted as a K3 result, it is not one. Both were measured on a 48B-total, 3B-activated model in October 2025.

Which is what makes K3 the more interesting story once you stop calling it an invention. It is the scaling event for a recipe that was already validated small. Kimi Linear ran a 3:1 hybrid, one full attention layer for every three linear ones, 25.0%. K3’s config is 93 layers: 69 KDA plus 24 Gated MLA, so 25.8%, at roughly 58 times the parameter count. The cadence is not uniform, and the config is worth reading rather than averaging: full_attn_layers is [4, 8, ... 88, 92, 93], every fourth layer plus a second full-attention layer stacked directly on top of the first at 92 and 93.

That ratio transferring across a 58x scale-up is my own reading of two config files, and I want to be careful not to hang Moonshot’s headline number on it. The technical report claims “an approximately 2.5x improvement in overall scaling efficiency” over Kimi K2, and its abstract credits that to a bundle, not to the layer ratio: KDA and AttnRes, “together with Stable LatentMoE, which effectively activates 16 of 896 routed experts per token, and refined training and data recipes”.

While you are in the config, note that 1.8% and 3.7% are both true sparsity figures for this model and they measure different things.

Table: Two distinct sparsity quantities for Kimi K3, derived from the Hugging Face model card, accessed 6 August 2026.

Quantity	Arithmetic	Value
Routed experts selected per token	16 / 896	1.7857%
Routed plus shared experts	18 / 898	2.0045%
Activated parameters as share of total	104B / 2,800B	3.714%

Expert-count sparsity is about half the parameter-activation fraction, because the shared experts, all attention parameters, the embeddings and the vision encoder are active on every token no matter how the router votes. Those are the same tensors the quantiser skipped. If you quote a sparsity figure, say which one it is.

The speed claim is the one that is simply false

Artificial Analysis, accessed 6 August 2026, measures K3 (max) at 38.7 output tokens per second, #54 of the 101 models on that page, against a median of 62.8 for open-weight models of similar size, with a time to first token of 3.11 seconds against a 1.82 second median. Their own summary calls it “particularly expensive when comparing to other open weight models of similar size” and “notably slow and very verbose”. On price it sits at #96 of 101. Moonshot’s pricing page lists $3.00 per million input tokens, $0.30 cached, $15.00 output, flat, with no context-length tiering.

vLLM reports 111 to 118 tokens per second single-user, rising to 331 to 370 with DSpark speculative decoding. Those are self-hosted, single-user, at batch size 1 on GB300 NVL72, and not comparable to a third party measuring the hosted API under production load. Do not put them in the same column.

A 2.8 trillion parameter model being slow was predictable from its own headline number before anyone measured anything, which is exactly why writing “faster inference” in the first ten lines was avoidable.

It dominated one board and sits thirteenth on another

LMArena put K3 at #1 in the Frontend Code Arena with 1679 points, past Claude Fable 5, a 17-place jump from Kimi K2.6 at #18, and #1 in six of seven frontend domains, second in Gaming. On LMArena’s main text leaderboard, read on 6 August 2026, kimi-k3-max sits at rank 13 with a score of 1485 (±10), on 3,556 votes, flagged Preliminary, with a rank spread of 4 to 31. Same model, same organisation, two boards, two verdicts, and the second one comes with an uncertainty band 27 places wide that the first one never shows you.

Moonshot is more modest about this than the coverage is. Its own technical report abstract says K3 “achieves frontier-level performance across long-horizon coding, agentic, knowledge, reasoning, and vision tasks” and then, in the same paragraph, that it “trails the most powerful proprietary models, namely Claude Fable 5 and GPT-5.6 Sol”. The comparison chart on Artificial Analysis’s own K3 page, on 6 August 2026, runs Claude Opus 5 (max) at 60.69, Claude Fable 5 (with fallback) at 59.86, GPT-5.6 Sol (max) at 58.89, and then Kimi K3 (max) at 57.11. Fourth, not sandwiched between two Claudes. I wrote the sandwich version first, by picking the two comparators that flattered the argument, which is the same error this article is about one level up.

One more thing worth reading off the repository README: every K3 result in those benchmark tables was produced at reasoning effort “max”, temperature 1.0. Forty-five lines later the same file records that the reasoning_effort field accepts "low", "high" and "max", with "max" as the default. So the benchmark setting is also the setting a caller gets by not choosing one, and the point is not that the tables are rigged. It is that the most expensive configuration is the one you get by not choosing, which is what the $15 per million output tokens and the 38.7 tokens per second are describing.

The two demos, with the qualifiers put back

The kernel result is real and the framing matters. Moonshot’s own post states the configuration: FLA Triton AttnRes at production scale, 96 layers, 8192-dim model, 8192 tokens, over 15 hours of iteration, cutting forward and backward pass combined from 283.6 ms to 114.4 ms while preserving the same numerics. That is a training-side number, not inference latency, on Moonshot’s own architecture. Neither the vendor nor most coverage states the multiple, so: 283.6 / 114.4 = 2.479x, a 169.2 ms absolute reduction, 59.7% cut.

The chip demo has a single source and it is Moonshot’s own write-up, which is more careful than the number that travelled from it:

As an early proof of concept, Kimi K3 designed a chip to serve a nano model built on its own architecture. In a single 48-hour autonomous run, K3 built, optimized, and verified the chip using open-source EDA tools on the Nangate 45nm library. Within 4 mm², the chip closes timing at 100 MHz and sustains over 8,700 tokens/s decode throughput in simulation, packing 1.46M standard cells, 0.277 MB of SRAM, and an INT4 MAC array with fused dequantization.

Three qualifiers are inside that sentence and none of them survived the coverage. Nangate 45nm is an open academic standard-cell library, not a foundry process. “In simulation” governs the throughput figure. And the part serves “a nano model built on its own architecture”, so 8,700 tokens per second describes a small model on a design that has never been fabricated, at 100 MHz. Nothing was taped out. No silicon exists.

My own post hedged this with “these results should be interpreted carefully until they’re independently replicated”, which points at the wrong risk. The limits were not waiting on replication. They were in Moonshot’s own sentence, and I kept the headline and dropped them.

Open-weight, and the licence has numbers in it

The weights are released. The training data and training code are not. That makes this an open-weight release, and “open source” describes a different thing.

The licence is not a different family from K2’s, which is how I first read it. The Kimi K3 License is textually MIT-derived: “Permission is hereby granted, free of charge”, to “use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software”. Its clause 3, requiring any product with more than 100M monthly active users or $20M monthly revenue to display “Kimi K3” prominently in its UI, is the same attribution clause K2’s Modified MIT already carried. The genuinely new restriction is clause 2: a Model-as-a-Service operator whose aggregate revenue exceeds $20M over any consecutive 12 months must sign a separate agreement with Moonshot before any commercial use. Clause 4 exempts purely internal use, and use through Moonshot’s own products or certified inference partners.

So the sentence I wrote about developers who can “inspect, customize, fine-tune, and deploy on their own infrastructure” holds up on one verb out of four. Inspection is weights only. Customisation and fine-tuning are permitted under those two thresholds rather than unconditionally. And deployment on your own infrastructure means at least sixteen B200s, which for almost everyone means renting the same hardware from the same handful of clouds you were already renting the closed model from.

The gap thesis, and the error bars nobody quotes

My post said three times, in three phrasings, that the gap between open and closed models is shrinking. The only organisation that measures it publishes numbers pointing the other way, and publishes intervals that are more informative than either point estimate.

Table: Epoch AI’s published estimates of how far open-weight models lag closed-weight models, with the 90% confidence intervals stated on each page.

Epoch AI page	Published	Time gap	ECI gap	90% CI on the ECI gap
Open-weight models lag state-of-the-art by around 3 months	30 Oct 2025	3.5 months (CI 1.1 to 5.3)	7 points	0 to 14
Open models lag state-of-the-art closed models by 4 months	29 May 2026	4 months	8 points	7 to 11

The point estimate moved from 3.5 months to four. That movement sits well inside the older interval of 1.1 to 5.3 months, so on its own it establishes nothing.

The interval is where the argument actually turns, and it is not the one I would have quoted from memory. Epoch’s November page puts the vertical gap at 7 ECI points with a 90% interval of 0 to 14, and says the gap “varies considerably over time, sometimes even closing completely”. Its May 2026 page puts it at 8 points with a 90% interval of 7 to 11, roughly a third as wide, and that interval excludes zero. Over the window Epoch measured, 1 January to 28 May 2026, “sometimes closing completely” is no longer in the data. So the claim that fails is mine, not theirs. The four-month figure is still soft in the other direction, because Epoch says the same estimate “would grow to six months” under a stricter catch-up criterion, and softness in that direction does not help me either.

Two more things about that data. Epoch’s May 2026 publication has Kimi K2.6 at ECI 151.60 as its leading open model. It predates K3 entirely, so it says nothing about K3. And the defensible version of my argument is the one I did not make: holding a roughly constant three-to-four-month lag while the frontier itself accelerated is a harder result than closing a gap against a stationary target. That framing survives the data. The one I used does not.

How to detect this before you publish it

Every error above is findable in under an hour, and each one has a tell.

A round number that came out of a multiplication. 1,400 GB is 2.8T times four bits and nothing else. If a figure is the product of two headline numbers, the artifact it describes will have a real size, and the real size is the one to publish. Read the file listing.
A precision label applied to a whole model. “4-bit”, “FP8”, “INT4”. Open config.json and read quantization_config.ignore before you believe it covers everything.
An adjective attached to a mechanism. “New”, “novel”, “first”. Search arXiv for the mechanism name before repeating it. Both of K3’s had papers with submission dates.
A performance number without its measured configuration. The 6x throughput figure is true of a 48B model in October 2025 and false attached to K3. Ask what model, what hardware, what date.
A point estimate with no interval. Epoch publishes both. The interval is what decided whether my thesis was wrong, and it never appears in a summary.
A claim with a shelf life. “Later this month” was accurate for ten days. If a sentence expires, date it inside the sentence or do not write it.

The rule underneath all six is narrow enough to follow: no vendor claim ships without its primary artifact open in a second tab, and the artifact is the paper, the licence, the config JSON, the file listing or the pricing page, never the announcement. All four errors in my post, and the one at the top of the first version of this page, sat in that gap.

The weights shipped on 27 July 2026 at 1,560.9 GB across 96 files. That number came off a directory listing rather than out of a multiplication, which is the only reason I am willing to put it in a title.

K3 lists at half of GPT-5.6 Sol, and the saving is either 9.6% or 30.6%

contact@therezaali.com (Mo RezaAli) — Thu, 06 Aug 2026 00:00:00 +0200

Moonshot AI’s Kimi K3 lists at $15.00 per million output tokens. OpenAI’s GPT-5.6 Sol lists at $30.00. Exactly half, on both vendors’ own pricing pages, read on 6 August 2026.

So the saving is 50%, and it is not. Artificial Analysis runs both through its Intelligence Index and publishes what each actually costs to finish the work, and this is where I have to stop and report a problem rather than a number.

Its article gives $0.94 per task for K3 against $1.04 for Sol, a 9.6% saving. Its model page for the same model, read the same day, carries the same metric with different values: costPerIntelligenceIndexTask is 0.8552 for K3 and 1.2325 for Sol, which is a 30.6% saving. Same publisher, same metric, same afternoon, one figure three times the other.

Table: Cost per Intelligence Index task, as published by Artificial Analysis in two places on 6 August 2026.

Source	Kimi K3	GPT-5.6 Sol	Saving
Article prose	$0.94	$1.04	9.6%
Model page dataset	$0.8552	$1.2325	30.6%

I cannot tell you which is right, and that is the finding. An earlier version of this page put 9.6% in its own title, having read one of those pages and not the other. What survives both readings is the direction and the mechanism: the saving is real, it is nowhere near the 50% the list prices imply, and the reason is token volume, which Moonshot documents on the same page that carries the price.

Where I was already standing

I had this model in a costed build before the measurement existed. In what an always-on AI agent actually costs to run I put moonshotai/kimi-k3 on a three-model allowlist at $3.00 in and $15.00 out, marked it escalation only, and wrote that the output column is the side of the ledger you control least, because you can budget your input and you cannot budget how much a model decides to write.

I derived that rule from the list price. It was the right rule and I could not have told you the mechanism, because I had a price and nobody had published a cost. The mechanism is now on the vendor’s page, one paragraph under the price table:

Always reasons and supports configuring its reasoning effort with the top-level reasoning_effort request field (low / high / max, default max).

Two words carry the whole gap. Always, so there is no cheap non-reasoning path. Default max, so the most expensive setting is the one you get by not choosing. Reasoning tokens bill as output tokens, at $15.00 per million.

How much more it emits, and where the arithmetic stops

Take the floor first, because it needs no assumptions about the workload at all.

Every one of K3’s unit prices is 60% or less of Sol’s: input $3.00 against $5.00, cached input $0.30 against $0.50, output $15.00 against $30.00. So for any mix of cached, uncached and output tokens whatsoever, a task that consumed identical token counts on both models would cost K3 at most 60% of Sol’s $1.04, which is $0.62. It costs $0.94. Divide: K3 is consuming at least 1.5 times the tokens, and that number falls out of four published prices and two published costs with no modelling in between.

The obvious next step is to turn that floor into an exact multiple, and it does not survive contact with the definitions. Artificial Analysis publishes a per-task token figure for Sol: 15k output tokens per Intelligence Index task at max reasoning effort. For K3 it publishes totals instead, 130M tokens across the index at a cost of $2,437.41. Dividing that total by the $0.94 per-task cost looks like it hands you a task count, and K3’s tokens per task would fall straight out of it. But the measurer defines that per-task number two different ways. The v4.1 announcement that introduced the metric says it takes “the total cost, total time, and total output tokens for a model to run the Intelligence Index and divide by the number of tasks across its evaluations”. The K3 model page calls the same metric a “Weighted average cost (USD) per Artificial Analysis Intelligence Index task”, and the index weights are nowhere near uniform: GDPval-AA v2 carries 20% of the score, AA-Omniscience Non-Hallucination 4%. Under the first reading the division is a task count. Under the second it is an unweighted total over a weighted mean, which counts nothing. One of the two readings voids the arithmetic, so the arithmetic does not ship and 1.5x is what I am willing to put my name to.

The measurer agrees on direction, in plain language. Artificial Analysis’s own summary calls K3 “notably slow and very verbose” and sets its 130M tokens against a 100M median among open-weight models of similar size, which is a narrower comparison set than the full index.

The one place the list price tells the truth

There is a tier where the headline discount is real and understated, and the post I am building on missed it.

Table: List prices per million tokens from each vendor’s own pricing documentation, read 6 August 2026. OpenAI’s page does not state the input-token threshold at which its long-context tier engages, so the last row applies above an unstated boundary.

Per million tokens	Kimi K3	GPT-5.6 Sol	K3 as % of Sol
Input, cached	$0.30	$0.50	60.0%
Input, standard	$3.00	$5.00	60.0%
Output, standard	$15.00	$30.00	50.0%
Output, long context	$15.00	$45.00	33.3%

K3 charges one flat output rate all the way to its 1,048,576-token window. Sol charges a premium above a threshold. Above that line the gap widens from half to a third, and there is an architectural reason rather than a commercial one. vLLM’s engineering write-up describes Kimi Delta Attention as a linear-attention mechanism holding a fixed-size recurrent state instead of a growing KV cache, interleaved with periodic full-attention layers, and says directly that this is what makes a 1M-token context affordable. Flat long-context pricing is that design showing up on an invoice.

So the honest shape is the inverse of the headline. On short agentic turns, where reasoning dominates the bill, the 50% discount collapses to 9.6%. On very long single-pass context, where the recurrent state does the work, it improves to 33%.

The 2.7% figure measures a preference vote, and the ratio is undefined

Stanford’s 2026 AI Index says the top US model leads by 2.7% as of March 2026. That number is real and the popular restatement of it fails three ways.

It is an Arena rating, which the report itself describes as human voting on ratings “inspired by chess ratings”. It measures which answer people preferred, not what either model can do. The underlying pair, per press reads of the report, is Claude Opus 4.6 at 1,503 against ByteDance’s Dola-Seed-2.0-Preview at 1,464. A 39-point difference; 1503/1464 gives 2.66%.

Dividing two Elo ratings is also not a defined operation. Elo is an interval scale with an arbitrary zero, so the ratio moves when you shift the origin while nothing about the models changes at all.

Table: The same 39-point Arena gap expressed as a percentage, with every rating on the board shifted by a constant. Win rate is 1 / (1 + 10^(-39/400)) throughout.

Shift applied to all ratings	Gap as a percentage	Head-to-head win rate
none	2.66%	55.6%
+500	1.99%	55.6%
+1,000	1.58%	55.6%

The quantity that survives the shift is the expected score. The top US model was preferred in about 56 of every 100 blind comparisons, which is a far less comfortable number than 2.7% for anyone arguing the models are interchangeable. It is also five months stale for August 2026, predating Kimi K3 entirely.

What the report actually concludes is better than the statistic anyway: competitive pressure is shifting “toward cost, reliability, and domain-specific performance”. That is the argument, from the primary source.

Open weights, with the licence read

The weights are open and the licence is new, which is the part worth reading. Moonshot’s own back catalogue is not a guide to it. Kimi K2 and Kimi K2.6 each ship a file whose first line is Modified MIT License, and the K2 text says exactly what the modification is:

Our only modification part is that, if the Software (or any derivative works thereof) is used for any of your commercial products or services that have more than 100 million monthly active users, or more than 20 million US dollars (or equivalent in other currencies) in monthly revenue, you shall prominently display “Kimi K2” on the user interface of such product or service.

K3 ships a bespoke Kimi K3 License instead. Its section 3 is that same branding clause with the model name changed. Its section 2 has no counterpart in the K2 file at all:

If the Licensee or any of its affiliates operates a Model as a Service business, and the aggregate revenue of the Licensee and its affiliates exceeds 20 million US dollars (or the equivalent in other currencies) in total over any consecutive 12 months, the Licensee must enter into a separate agreement with Moonshot AI before using the Software or its derivative works for any commercial purpose.

Read the trigger carefully. It is the aggregate revenue of the licensee and its affiliates over any twelve months, not the revenue earned from serving K3, and 20 million dollars is a small company. An inference provider above that line needs a signed agreement with Moonshot before it charges for a single token, unless it is one of the certified inference partners that section 4(b) exempts. The claim that you can deploy the weights yourself and fine-tune them survives, but through section 4(a), which exempts internal use, defined as use that does not make the software, its outputs or its underlying capabilities available to third parties.

Self-hosting has a hardware floor too. vLLM’s launch post states the model “can barely fit in a single NVIDIA DGX B300 and requires a minimum of 16 NVIDIA B200/GB200 GPUs to serve on that hardware generation”. Hugging Face’s repository metadata for the model reports 2.78 trillion parameters, 2.72 trillion of them stored as packed 4-bit MXFP4, across 1.56 TB of safetensors. That is the download, before a byte of KV cache or recurrent state.

Which undercuts the data-residency argument for most of the named adopters, because they are not self-hosting. Airbnb uses Alibaba’s Qwen for customer service; DoorDash routes lower-level coding work to Kimi for, in its CTO’s words, “better quality [at] cheaper cost”. Both consume an API, and K3 reaches Azure customers through Fireworks on Microsoft Foundry. Routing through a US intermediary to a Chinese-trained model is a different data posture from running weights on your own metal, and US lawmakers requested information from DoorDash about exactly this on 31 July.

That distribution route also corrects the source post: Microsoft is not an adopter, it is the channel. Kimi K2.6 and K2.5 are sold directly through Microsoft Foundry. The American hyperscaler is monetising the Chinese price advantage as a distributor, which is a better fact than the one it replaces. Siemens should come out of a list about American buyers too; it is a German industrial group.

Three corrections to the surrounding numbers

The semiconductor selloff is real and smaller than it reads. The SOX fell 1.6% on 17 July to close 20.2% below its 22 June record, its steepest week since April 2025, and the same report notes the index was still up more than 60% on the year. A 20% drawdown from a 105% run is profit-taking with a bear-market label attached.

The $725 billion capex figure is four hyperscalers, not five, and “roughly” rather than “more than”. It is an analyst aggregation of company guidance whose own component table sums to well under its headline, with Microsoft appearing at two different values on one page.

And the trend runs both ways. K3’s $15.00 output price is 3.75 times its own predecessor’s $4.00. The lab leading the price attack quadrupled its own rate, which is the strongest available objection to the commoditisation thesis.

Detecting this in your own stack

The test is one query against your provider’s billing export, and it works for any model pair.

Take one real task class, not a prompt. A whole agent turn including tool calls and retries.
Pull total spend and total completions for that class over a week, and divide. That is your cost per task. Per-token price is an input to it, never a substitute for it.
Split output tokens into reasoning and visible answer. If your provider reports reasoning tokens separately, the ratio is the story.
Compute the floor: multiply the incumbent’s cost per task by the challenger’s worst unit-price ratio. If the challenger costs more than that, it is emitting more tokens, and you now know the minimum multiple.
Re-run at each reasoning_effort level before you conclude anything. A default of max is a benchmark setting, not a production setting.

Designing so the bill cannot surprise you

Set reasoning_effort explicitly on every call, including the ones where you want the default, so the value is in your code rather than in the vendor’s. Cap max_tokens per task class, because an uncapped verbose model has no upper bound you control. Price migrations on measured cost per task and refuse to sign off on a per-token comparison. And route by tier: long single-pass context to the flat-rate model, short reasoning-heavy turns to whichever model finishes in fewer tokens.

What I have now that I did not

A floor calculation that converts two published prices and two published costs into a hard lower bound on token consumption, which I can run on any model pair the day it launches. The knowledge that reasoning_effort defaults to max on the model I had already put on an allowlist, which turns a rule I wrote on instinct into one I can defend. An Elo shift table that retires percentage comparisons of Arena ratings permanently. And the correction that reorganises the whole thesis: the cheap model is cheap per token and average per task, so the commoditisation of foundation models is arriving through a different door than the price list suggests.

Half price bought 9.6%. Measure the task.

Google’s checkout form guidance ships an autofill token that does not exist

contact@therezaali.com (Mo RezaAli) — Thu, 06 Aug 2026 00:00:00 +0200

Google’s own guidance page for checkout forms ships an autofill token that does not exist. On web.dev’s Payment and address form best practices, the string address-line-1 appears five times, four of them inside an autocomplete attribute: autocomplete="address-line-1" twice, plus autocomplete="shipping address-line-1" and autocomplete="billing address-line-1". The HTML Standard defines the token as address-line1, with no hyphen before the digit, and contains zero occurrences of the hyphenated spelling. I checked both pages by string search on 6 August 2026. The same web.dev code sample gets it right in one attribute and wrong in the next: <input autocomplete="address-line-1" id="address-line1">.

An unrecognised token does not throw and does not warn in the console. It simply fails to match, leaving the field to the browser’s own field-name heuristics, which is where it would have been with no attribute at all. So a developer who copies the most-linked checkout forms page on the web gets worse mobile autofill than one who copies the specification, with no signal that anything happened. That matters more than any layout advice on the same page, because autofill is the only checkout change that removes the typing instead of rearranging it.

Why I was reading the specification

Because I stopped trusting my own reason for rebuilding a checkout.

Here is what I published at the time. These are my own unaudited figures from my own work, not anything a third party measured. 74% of my visitors were on mobile, and mobile customers were converting at roughly half the rate of desktop. Before buying more traffic I opened the store on my own phone and found a two-column form, shipping fields ahead of payment, and a coupon box sitting there inviting people to go hunting for a code. I made it single column, moved payment higher in the flow, and removed the coupon field on mobile. The edit took around 30 minutes. I ran the variant for a week and mobile completion improved from far behind desktop to almost matching it. The post is here.

The change was right. The reasoning that led to it was not, and the test that confirmed it cannot support the lesson I drew from it. Both of those are fixable, and fixing them is worth more than the result was.

The conversion gap was never evidence about the checkout

The gap on its own carries no information about my form. It is one ratio with no comparison group: nothing in it separates “my checkout is bad” from “a phone is a worse buying environment than a laptop”. I looked for a free primary source publishing a current device-split conversion benchmark and did not find one. The vendor benchmarks that get quoted for this sit behind lead capture forms, so I will not quote a figure I could not open. What justified the work was opening the checkout on a phone. The gap was the prompt. The walkthrough was the evidence. I had those two backwards.

The gap is worse than uninformative, though, because part of it is instrumentation.

Cross-device purchases debit mobile and credit desktop. GA4’s reporting identity falls back to device ID whenever User-ID is not collected, per Google’s own documentation. A shopper who browses on a phone and buys on a laptop adds a non-converting session to the mobile denominator and a converting session to the desktop numerator. Most small stores never set User-ID, so their device-split report is biased against mobile by construction.

Safari deletes the state that would let you recognise a returning visitor. WebKit announced that ITP “would cap the expiry of client-side cookies to seven days”, then extended the same rule to everything a script can write, “deleting all of a website’s script-writable storage after seven days of Safari use without user interaction on the site”. The affected list includes IndexedDB, LocalStorage, SessionStorage, media keys, and Service Worker registrations. Mobile traffic skews heavily to Safari. Returning iOS visitors are therefore re-counted as new sessions more often than returning desktop Chrome visitors, which inflates the mobile session denominator.

Both artifacts push the same way, and neither is visible in the report they corrupt.

How to detect this in your own system

Stop comparing sessions-to-orders by device. Compare checkout-start to order by device instead. That ratio lives inside a single session on a single device, so cross-device attribution cannot touch it and seven-day storage deletion cannot touch it. In GA4 terms it is purchase over begin_checkout, segmented by device category.

Run both numbers side by side for a week before changing anything. If session conversion shows a wide device gap and checkout completion shows near parity, the problem is upstream of the checkout and redesigning the form will do nothing. If checkout completion is also far apart, you have a checkout problem and a metric clean enough to measure the fix with. Setting User-ID removes the first artifact for logged-in traffic. Nothing removes the second.

What the single-column evidence actually says

The canonical citation is Ben Labay’s study for CXL Institute, published 17 October 2016. CXL’s own copy now returns a 403, including to a request carrying a current Chrome user agent, so the readable original is on Speero. The instrument was CXL’s Trust Seal survey, not a checkout. Nobody bought anything. The outcome measure was form completion time, not completion rate and not revenue. The sample was 702 people fielded in early June 2016, n = 356 linear against n = 346 multi-column, and the single-column form was completed 15.4 seconds faster, significant at 95% on a two-sample t-test. And then the sentence almost nobody who cites the study reproduces: “We restricted the survey to desktop users.” The authors add their own caveat, that the results are “not directly transferable to all form types and situations”.

So the best-known quantitative evidence for single-column forms measured speed, on a survey, on desktop, ten years ago, and it is routinely quoted as proof that single column converts better, in checkouts, on mobile.

The change is still correct on mobile. It needs a mechanical argument rather than a borrowed statistic. On a 390 CSS-pixel viewport, two columns either shrink each control below a comfortable tap target or trigger pinch-zoom, and they break the vertical DOM order that the on-screen keyboard’s “Next” key walks. All three are confirmable in a device emulator in under a minute, which beats a decade-old desktop survey as a standard of proof.

The change that beats layout

If friction is the problem, the strongest available answer is fewer keystrokes, and the specification is where that lives. The HTML Standard’s autofill token list is the actual API: given-name, family-name, street-address, address-line1, address-line2, address-level1, address-level2, postal-code, country, country-name, tel, email, cc-name, cc-number, cc-exp, cc-csc, and the shipping and billing section prefixes for forms that collect both addresses.

Three more from Google’s page, which are right even though its address token is not. Use type="tel" for phone numbers and type="email" for email, so the phone renders the correct keyboard. Never use type="number" for a card number: it “adds an up/down arrow to increment numbers, which makes no sense for data such as telephone, payment card or account numbers”. Use type="text" with inputmode="numeric" instead, in a single input rather than four split boxes.

What the abandonment statistics actually say

Baymard’s cart abandonment figure is 70.22%, and Baymard states plainly what it is: “an average calculated based on 50 different studies”, spanning 2006 to 2025, ranging from 55.00% (Forrester 2010) to 84.27% (SaleCycle 2020), most recently fed by Uptain 2025 at 71.72%. It is a literature average of other people’s measurements across nineteen years, not a measurement of anything.

The reasons breakdown matters more here, and the circulating version is stale. On the live page, 42% of US online shoppers abandoned because “I was just browsing / not ready to buy”, and Baymard removes that segment before computing everything else. Of the remainder: 40% extra costs, 20% delivery too slow, 19% did not trust the site with card details, 18% forced account creation, and 17% “too long / complicated checkout process”. The 22% still quoted for checkout complexity was true of this same URL two years ago. The Internet Archive’s 1 June 2024 capture reads “22 % Too long / complicated checkout process” under a heading that says “2024 data”, and the 1 January 2022 capture reads 18% under “2021 data”. One URL, one question, three answers in four years: 18, then 22, now 17. The 26% that travels alongside it is not a checkout-complexity figure at all. It is the forced-account-creation row of that same 2024 table, “The site wanted me to create an account”, which the live page now gives as 18%. Cite 17%, name the date you read it, and say it is calculated after excluding four in ten respondents who were never going to buy. Baymard publishes no sample size or field date for that survey, so do not invent one.

Two more from the same publisher, and they disagree. The abandonment page, updated 22 September 2025, says the average US checkout displays 23.48 form elements by default, 14.88 counting only fields, against an ideal of 12 to 14 elements or 7 to 8 fields. Baymard’s blog post of 26 June 2024 says the 2024 average was 11.3 form fields, down from 11.8 in 2021 and 12.7 in 2019, against an ideal of 8. Different pages, different vintages, different definitions of what counts. Pick one, name it with its date, and do not average them. Baymard’s claim that better checkout UX is worth a 35% conversion increase comes from its own usability sessions and is sold as part of a paid product, so it is a vendor estimate.

The mobile speed number has degraded the same way. Milliseconds Make Millions, by Google, Deloitte Digital and Fifty-Five, reports an 8.4% conversion increase and a 9.2% average order value increase per 0.1s, from 37 brands, retail findings resting on 20.5m sessions. Its methodology section says “Approximately 4 weeks’ worth of hourly data was collected to reach statistical significance” and never says when those four weeks were. The document carries a “©2020 Deloitte Ireland LLP” line, so the data is at least six years old by now, but the window itself is undated and any date you see attached to this study is somebody’s inference. It is a logarithmic regression over an uncontrolled panel, and nobody’s site was made faster as an intervention. The appendix defines its own test: “In this context, statistically significant means a direct correlation was identified.” The same paragraph adds that where speed had “minimal or zero effect on funnel progression”, those observations “were not included in the report”. Its lead generation section records mobile conversion rates falling by almost 2% with faster speed, while the executive summary presents speed as uniformly positive. Faster sessions are also newer phones on better connections in richer postcodes, and the study cannot separate those.

What my test could and could not tell me

Fixing the duration at one week in advance was right for two reasons I would like to claim I planned. It covers a full weekly cycle, so day-of-week composition is balanced, and it is a fixed horizon rather than a stopping rule. Evan Miller’s demonstration is the one to keep in mind: running a significance test after every observation produces a 26.1% false positive rate against a nominal 5%, and his prescription is “Decide on a sample size in advance and wait until the experiment is over.”

What I did not do was write down what “completion rate” was measured against. That omission decides whether the result means anything, because required sample size moves by two orders of magnitude depending on the denominator. Two-sided alpha of 0.05, power 0.80, standard two-proportion test. The base rates below are illustrative, not mine: I never published my own, and inventing them afterwards would be worse than admitting the gap.

Table: Sessions required per arm to detect a given effect at 80% power, by choice of denominator

Metric and effect	n per arm	Total	Mobile traffic per day for 7 days
Session CVR 1.5% to 2.7% (+80% rel.)	2,241	4,482	641
Session CVR 1.5% to 1.8% (+20% rel.)	28,304	56,608	8,087
Session CVR 1.5% to 1.65% (+10% rel.)	108,153	216,306	30,901
Checkout completion 45% to 55% (+22% rel.)	392	784	112
Checkout completion 45% to 50% (+11% rel.)	1,565	3,130	448
Checkout completion 45% to 48% (+6.7% rel.)	4,338	8,676	1,240

If completion rate meant checkout-starts to orders, a week at a few hundred mobile checkout starts a day is adequately powered and the finding stands. If it meant sessions to orders, a week of small-store traffic can only detect implausibly large effects, and a result at that volume is more likely noise than signal. I did not record which, so the honest statement is that the power of that test is unknown.

Two further limits, both structural. Three changes shipped as one variant, which is a valid test of “ship all three or none” and an invalid basis for the lesson “single column works”. And removing the coupon field was measured with a metric that cannot adjudicate it: taking the discount box away mechanically raises revenue per completed order, because fewer discounts get redeemed, while potentially suppressing orders from people holding a valid code. Completion rate reads both of those as success. Revenue per session tells them apart.

Designing so it cannot recur

Instrument checkout-start to order by device permanently, and treat session-level device conversion as a directional signal at best.
Write the denominator and the required sample size down before shipping the variant. If the required n exceeds what a week of traffic can deliver, do not run the test. Ship on mechanism and monitor.
One change per variant, or state openly that you are testing a bundle and cannot attribute the result.
Match the metric to the change. Layout goes to completion rate. Anything touching discounts goes to revenue per session.
Copy autofill tokens from the WHATWG specification, not from articles about it, including this one. Check the list yourself.
The strongest version of removing friction is removing the form, and it is an addition rather than a substitute. The Payment Request API hands the browser’s stored card and address to the site through a native sheet, so there is no layout to argue about. Read the banner at the top of MDN’s page for it before you plan around it: “Limited availability”, and “This feature is not Baseline because it does not work in some of the most widely-used browsers”. It also requires a secure context. So it is a progressive enhancement over a correctly tokenised form, and the form is still what a share of your mobile traffic will actually see. Building the sheet and skipping the tokens leaves the fallback path worse than where you started.

And know your platform before quoting anyone’s timings, including mine. On Shopify, “Checkout UI extensions for the information, shipping, and payment steps are available only to stores on a Shopify Plus plan”, so a 30-minute checkout edit is not a number that transfers. The platform sets the floor for the rest, too. HTTP Archive’s 2025 crawl puts Shopify origins at 76% passing Core Web Vitals in both the desktop and mobile tables, against WooCommerce at 33% and 35%. Before blaming your traffic, find out which of those you are standing on.

What holds up

The lesson in my original post survives with one repair. Sometimes the ads are not the problem and the funnel is. But the funnel claim needs a metric the browser and the analytics tool have not already corrupted, and the device-split conversion rate that made me suspicious in the first place does not qualify.

The limitation I wrote then still stands and I would not soften it. A cleaner checkout does not fix a weak offer. If people do not want the product, reducing friction only helps them leave faster. That work paid because demand already existed and the form was in the way.

What I have now that I did not have during those 30 minutes: a device metric that measures behaviour rather than storage policy, a denominator written down before the test rather than after it, one variable per variant, revenue per session for anything that touches a discount, and a form specification instead of a form opinion. The result I reported is the least useful thing I got out of that week.

Chain of thought is worth 0.7 points outside math and logic

contact@therezaali.com (Mo RezaAli) — Thu, 06 Aug 2026 00:00:00 +0200

Chain of thought buys 14.2 accuracy points on symbolic reasoning, 12.3 on math and 6.9 on logical reasoning. On everything else it buys 0.7. That is 56.8 with it against 56.1 without, averaged across every other category tested. In the authors’ own words: “As much as 95% of the total performance gain from CoT on MMLU is attributed to questions containing ‘=’ in the question or generated output.”

Those figures come from To CoT or not to CoT? (Sprague et al., arXiv 2409.12183, last revised 7 May 2025): “a quantitative meta-analysis covering over 100 papers using CoT” plus the authors’ own evaluations of “20 datasets across 14 models.”

I published a post recommending chain of thought for “complex reasoning, coding, math, multi-step decision making.” Coding is not a category showing benefit in that meta-analysis, and neither is multi-step decision making. The ratio between two items I listed as equivalent is 12.3 against 0.7, about 17.6x. Correcting that with the numbers attached is worth more than leaving it standing.

Why a prompt fails in a way a better model cannot fix

From 31 October 2025 to 27 April 2026 I ran a publishing bot against this domain. It made 842 Add post: commits. It used gpt-4o-mini with a gpt-4o rewrite pass. Its entire input per article was 450 characters of RSS description, sliced in code. It had AUTO_PR = "false", so nothing was reviewed before it hit main. 180 of the 739 English posts cited a source called “Internal Analysis” that does not exist.

The prompt asked for an authoritative article with statistics and named sources, from 450 characters that contained neither. No model upgrade repairs that. The prompt specified an output the input could not support, and the model did the only thing left: it filled the shape. The lesson runs opposite to the folk version. Prompts matter more than model choice, but not because a clever prompt adds capability. They matter because the prompt is where you decide what is being asked for, and a badly specified ask fails identically at every model quality. The full account of that pipeline is here. The commit counts come from a private repository, so you have my word for them rather than a link.

1. Chain of thought: real, and narrower than advertised

The technique has two origins and most writing conflates them. Wei et al. (arXiv 2201.11903, January 2022) named it, and their method was few-shot: “a few chain of thought demonstrations are provided as exemplars in prompting.” Kojima et al. (arXiv 2205.11916, May 2022) established the zero-shot version and the phrase “Let’s think step by step.” What almost everyone actually does is Kojima’s, under Wei’s name. The original gains are large and real.

Table: Chain-of-thought accuracy on arithmetic benchmarks, as published in the two originating papers. Wei’s figures are read from Table 2 of the full text; deltas and multiples are my arithmetic from those published numbers.

Model	Benchmark	Standard	With CoT	Delta	Multiple	Source
InstructGPT text-davinci-002	MultiArith	17.7%	78.7%	+61.0 pts	4.45x	Kojima 2022
InstructGPT text-davinci-002	GSM8K	10.4%	40.7%	+30.3 pts	3.91x	Kojima 2022
PaLM 540B	GSM8K	17.9%	56.9%	+39.0 pts	3.18x	Wei 2022
GPT-3 175B	GSM8K	15.6%	46.9%	+31.3 pts	3.01x	Wei 2022
LaMDA 137B	GSM8K	6.5%	14.3%	+7.8 pts	2.20x	Wei 2022

Every benchmark in that table is arithmetic. Wei also stated a scale condition plainly: the technique “does not positively impact performance for small models, and only yields performance gains when used with models of ∼100B parameters.” That caveat has aged out, since anything a reader touches in 2026 clears it. The task-category caveat has not.

Table: What my post claimed chain of thought helps with, against what the Sprague meta-analysis measured.

Claim in the post	Measured average gain	Supported
Math	+12.3 pts math, +14.2 pts symbolic	Yes
Complex reasoning	+6.9 pts logical reasoning; +0.7 pts for commonsense, knowledge and soft reasoning	Only the logical and symbolic subset
Coding	Not a category showing benefit	No
Multi-step decision making	Not a category showing benefit	No

The floor is also not zero. Liu et al. (arXiv 2410.21333) took six tasks from cognitive psychology where deliberation degrades human performance and found that “in three of these tasks, state-of-the-art models exhibit significant performance drop-offs with CoT (up to 36.3% absolute accuracy for OpenAI o1-preview compared to GPT-4o).” Verbal overshadowing has a machine analogue.

Then the price. Chain of thought spends output tokens, the most expensive class on every sheet I checked. On Anthropic’s current price list, output bills at exactly 5x input on every model in the table, from Haiku 4.5 at $1/$5 to Fable 5 at $10/$50. OpenAI’s sheet does not hold to a single multiple across its line. I am not going to quote rows off it, and the reason is the article’s own subject: that page renders its table from JavaScript, an earlier draft of this paragraph quoted four model-and-price pairs from it that are not on it at all, including two models it does not list, and the error survived one round of checking. What matters for the argument holds without the enumeration anyway. Output is the expensive class on every sheet I looked at, the multiple over input is several times rather than a few percent, and it is not the same multiple everywhere, so the cost of a reasoning preamble is a vendor-specific number you have to read off the current page rather than a rule of thumb you can carry between them. A worked illustration at Claude Sonnet 5’s introductory $10 per million output tokens: a 100-token direct answer costs $0.0010, and the same answer behind a 900-token reasoning preamble costs $0.0100. At 100,000 calls a month, $100 against $1,000. Outside math and logic you are paying that $900 for 0.7 points. My own arithmetic from published list prices, not a bill anyone has received.

Both vendors have now moved. OpenAI’s reasoning best practices says outright: “Avoid chain-of-thought prompts: Since these models perform reasoning internally, prompting them to ’think step by step’ or ’explain your reasoning’ is unnecessary.” Anthropic’s prompting best practices file manual CoT under “Manual chain-of-thought (CoT) prompting as a fallback,” for when thinking is switched off, and add that “a prompt like ’think thoroughly’ often produces better reasoning than a hand-written step-by-step plan. Claude’s reasoning frequently exceeds what a human would prescribe.” One vendor calls the instruction unnecessary, the other has demoted it to a fallback. Neither is where the folk advice still sits.

2. Few-shot: three to five, and they are not teaching

My post said one or two examples. Anthropic’s current guidance says “Include 3–5 examples for best results,” with examples wrapped in <example> tags and required to be “Diverse: Cover edge cases and vary enough that Claude doesn’t pick up unintended patterns.” Take the vendor’s number over mine.

Why diversity is the requirement exposes that the folk mechanism is wrong. I wrote that models follow patterns better than vague instructions. Min et al. (arXiv 2202.12837) tested that and found that randomly replacing the labels in demonstrations barely hurts performance, across 12 models including GPT-3. What carries the effect is the label space, the distribution of the input text, and the format of the sequence. Examples are not teaching the model correct answers. They are declaring the shape and range of an acceptable response, which is why five lookalike examples do less work than three that span the edges. Caveat: Min et al. tested classification and multiple-choice, so do not carry “labels do not matter” into open-ended generation.

Format is a live variable rather than a cosmetic one. Sclar et al. (arXiv 2310.11324) held semantic content fixed, changed only separators, spacing and casing, and report “performance differences of up to 76 accuracy points when evaluated using LLaMA-2-13B.” Two things must travel with that number or it misleads: LLaMA-2-13B is a 13B open-weight model from 2023, not a frontier model, and 76 points is the maximum observed spread, not a typical one. Discounted for both, it still says separators and casing are part of the prompt rather than packaging around it.

Order is the failure people miss. Turpin et al. (arXiv 2305.04388) reordered multiple-choice options in few-shot prompts so the correct answer was always “(A)”. Accuracy dropped by as much as 36% across 13 BIG-Bench Hard tasks, tested on GPT-3.5 and Claude 1.0, and the models wrote fluent reasoning that never mentioned the bias steering them. Your example set is an instruction whether you intended it as one or not.

OpenAI’s guidance on reasoning models, again verbatim: “Try zero shot first, then few shot if needed.”

3. Role prompting: register, not correctness

I claimed a role “changes how the model prioritizes information, explains concepts, and structures its answers.” The first and third parts are supported. The implied fourth, that it makes answers more correct, is not.

Anthropic’s docs claim exactly the modest thing and no more: “Setting a role in the system prompt focuses Claude’s behavior and tone for your use case.” No accuracy claim appears. Zheng et al. (arXiv 2311.10054) tested 162 roles across 4 LLM families on 2,410 factual questions and found that “adding personas in system prompts does not improve model performance across a range of questions compared to the control setting where no persona is added,” and that where gains appear, “the effect of each persona can be largely random.”

A second paper disagrees, and suppressing that would be dishonest. Kong et al. (arXiv 2308.07702) report role-play gains across twelve reasoning benchmarks with ChatGPT, including AQuA moving 53.5% to 63.8% and Last Letter moving 23.8% to 84.2%. Their explanation is the part that settles it for me: role-play “acts as a more effective trigger for the CoT process.” If that is right, role prompting is an oblique way of invoking technique one rather than a third independent technique, which predicts it adds nothing on a model with reasoning already on. Zheng measured factual accuracy and found no reliable gain; Kong measured reasoning tasks and found gains they attribute to reasoning elicitation. A role reliably changes register and unreliably changes correctness, so use it for voice and audience and never for truth.

Table: What each technique costs and what it reliably buys. Prices are Anthropic list prices for Claude Sonnet 5, read 6 August 2026; the effect column is the measured average from the papers cited above.

Technique	Token class	Cacheable	What it reliably buys
Few-shot examples	Input, $2/MTok	Yes, cache reads bill at 0.1x base	Output format and output space
Chain of thought	Output, $10/MTok, 5x input	No, regenerated every call	+12.3 pts on math, +0.7 elsewhere
Role	Negligible	Yes, it sits in the system prompt	Tone, register, structure

They have opposite cost structures and my post presented them as interchangeable. Five 200-token examples cost $0.0020 per call at base input price, or $0.0002 on a cache read, a 10x reduction that repays the 1.25x write premium inside a single reuse. Anthropic’s docs reach the same conclusion independently: “caching pays off after just one cache read for the 5-minute duration.” The cheap technique is the one with the most reliable effect.

How to detect this in your own system

Four checks, none needing a benchmark harness.

Look for an equals sign. If your task does not involve symbolic manipulation, arithmetic or formal logic, the Sprague number for it is 0.7 points, and a “think step by step” instruction in a copywriting, summarisation or classification prompt is spend with no documented return.

Ask whether a human expert would do the task better on instinct than on deliberation. Liu et al. found the drops clustered on tasks with exactly that property.

Stop treating reasoning traces as an audit log. Turpin showed models rationalising a bias they never named. Anthropic’s measurement of 3 April 2025 found Claude 3.7 Sonnet mentioned an inserted hint in its reasoning 25% of the time and DeepSeek R1 39%. In a setup where models were rewarded for following deliberately wrong hints, they took the hint “in over 99% of cases” and admitted to it “less than 2% of the time in most of the testing scenarios.” Outcome-based reinforcement learning on an earlier snapshot of Claude 3.7 Sonnet raised faithfulness and then plateaued, at 28% on MMLU and 20% on GPQA. If you are logging chain of thought for compliance, you are logging a plausible story about the computation.

Last, check that the prompt’s demands are proportionate to its input. The bot failed because it asked for sourced statistics from 450 characters that had none, and no technique on this page would have saved it. List every fact the output requires and confirm each is present in the input. Anything absent will be generated.

What I have now that I did not have before

A decision rule instead of a habit. Chain of thought when the task contains symbols, math or formal logic, and not otherwise. Examples in the published range, three to five, chosen for spread rather than similarity, cached. Roles for voice and never for accuracy.

Both vendor pages cited here are written against whatever models are current, so the rule needs re-reading rather than remembering. One trap documented right now: with extended thinking disabled, “Claude Opus 4.5 is particularly sensitive to the word ’think’ and its variants,” and the docs suggest “consider,” “evaluate,” or “reason through” instead. And three techniques is not the set. Schulhoff et al.’s systematic survey catalogues 58 text-based prompting techniques and defines 33 vocabulary terms.

The post I am correcting was not wrong about 2022. It was describing 2022. These techniques were genuinely transformative when published, and the literature that narrowed them arrived later and got far less attention. Reading the primary source is the only defence, and it takes an afternoon.

Automate up to the decision, and never the decision itself

contact@therezaali.com (Mo RezaAli) — Thu, 06 Aug 2026 00:00:00 +0200

X’s open-source ranking model exposes nineteen prediction heads. None of them is an impression. Four of them are negative feedback, and the live Rust scorer does something with those four that is stranger than the folklore. A post whose weighted score comes out net-negative is not ranked low on the same scale as everything else. It leaves through a different branch, gets rescaled by the ratio of the negative weights to the total, and is then multiplied by a constant called NEGATIVE_SCORES_OFFSET that the repository imports and never defines. If that constant is positive, every net-negative post lands in a band underneath every net-positive one.

// home-mixer/scorers/ranking_scorer.rs:83
let negative_sum = -(not_interested + block_author + mute_author + report + not_dwelled);
let total_sum = positive_sum + negative_sum;

// home-mixer/scorers/ranking_scorer.rs:175
fn offset_score(combined_score: f64, w: &ScoringWeights) -> f64 {
 if w.total_sum == 0.0 {
 combined_score.max(0.0)
 } else if combined_score < 0.0 {
 (combined_score + w.negative_sum) / w.total_sum * NEGATIVE_SCORES_OFFSET
 } else {
 combined_score + NEGATIVE_SCORES_OFFSET
 }
}

I opened that file to work out whether an automated welcome DM was worth its click rate. It was not. The click rate turned out to be the least interesting reason, and the most interesting one had been written down years before I ran it.

What I was running, and the line that started this

I run cold sequences for 1688 sourcing at Yakkyo S.p.A. One of them got a one-line reply: “Is this a real person or a bot?”

The email was technically perfect. Right name, right company, a reference to their product category that Clay had pulled and Claude had stitched into a clean opening. It read like nobody was home. Reply rate had been sliding for six weeks and I had been blaming the copy.

I take that question seriously rather than laughing it off, for a specific reason. From 31 October 2025 to 27 April 2026 I ran a publishing bot against this domain. It made 842 Add post: commits and produced 2,922 machine-generated files before I deleted them; of the 739 English ones, 180 cited a source called “Internal Analysis” that does not exist. The full account is here, and those counts come from a private repository, so you have my word rather than a link.

Same shape, different scale: an automation took over a judgment call, kept producing output that looked fine, and nothing alerted because nothing broke. So this quarter I killed automations instead of stacking them. Three went, eleven stayed. Every figure below for my own results is my own unaudited number from my own work, published first on X.

Kill 1: the fully automated cold sequence

Clay enriched the lead, an n8n workflow scored it, Claude wrote a three-email sequence with a first line pulled from the prospect’s site, and it sent on a schedule. I forgot about it for about two months.

Reply rate fell from roughly 9 percent to under 4. What diagnosed it was not the average: positive replies fell faster than total replies, while “unsubscribe” and “not relevant” kept arriving at the same clip. The sequence was filtering out the people most worth talking to and retaining the ones who were going to say no anyway.

Why it happened. Automating a first line does not scale the good version of that line. It scales the mean of every line the model would write, and that mean is competent and interchangeable. Cold outreach does not pay on the mean. It pays on the tail, where one specific observation lands because somebody looked. Taking the human out of that step does not lower quality evenly. It removes variance, and the variance was the product.

How to detect it. Reply rate sums over both tails, so the drift hides inside it. Split it. Positive and negative replies are separate series that move independently, and their divergence leads the average by weeks.

Then watch what your provider watches. Google’s sender guidelines are specific and almost everyone quotes them wrong. The number people repeat is 0.3 percent; the guidance says “Keep spam rates reported in Postmaster Tools below 0.10% and avoid ever reaching a spam rate of 0.30% or higher.” Two thresholds, two jobs. 0.10 percent is the target you sit under, 0.30 percent is the line you never touch, and treating 0.30 as your budget spends your margin before the bad week arrives.

Making that measurable needs a clean opt-out channel: RFC 8058 one-click unsubscribe, the List-Unsubscribe-Post: List-Unsubscribe=One-Click header Google names directly. Without it, an annoyed recipient’s cheapest exit is the spam button, which routes the signal into your reputation instead of your dashboard. With it, opt-outs are a dated count you can line up against the day an automation went live. For US recipients that channel is also statutory: 15 U.S.C. 7704(a)(4)(A)(i) makes it unlawful to send more than 10 business days after an opt-out request.

What runs now. Enrichment and scoring stayed automated. That genuinely took two hours and now takes about eleven minutes. I am leaving that unrounded and unconverted, because a percentage computed off an “about” would carry a decimal place I never measured. Claude drafts the full email in maybe 20 minutes and I rewrite the opening of every tier-one email by hand. Reply rate came back to around 8 percent, and the positive-reply mix is better than it ever was on full auto. The judgment moved back to a human. The typing stayed automated.

A gate that did not exist when I built this. Article 50 of the EU AI Act, Regulation (EU) 2024/1689, entered into force on 2 August 2026, and I work in Messina, so it reaches me. It is also being widely misreported as a blanket duty to label AI-written email. Article 50(1) puts the interaction-disclosure duty on providers of systems intended to interact directly with natural persons. Article 50(4)’s deployer duty on text covers material “published with the purpose of informing the public on matters of public interest” and carves out anything that “has undergone a process of human review or editorial control”. A cold email drafted by a model and rewritten by me fits neither well. Read it before you panic or dismiss it: “real person or a bot?” now has a legal surface beside the craft one.

Kill 2: the automated DM to every new follower

Every new follower on X got an automated welcome DM with a link. A webhook fired the message. It is the growth tactic everyone copies.

The DMs converted to clicks at well under one percent, and mute and block events on the account ticked up in the exact window I turned it on. So I killed it on the numbers. Then I read the policy and found the numbers had been the second-best reason all along.

X’s Developer Agreement and Policy, under “Spam, bots, and automation”, says that “Services that perform write actions, including posting Posts, following accounts, or sending Direct Messages, must follow the Automation Rules. In particular, you should:”. The first bullet under that colon is:

Always get explicit consent before sending people automated replies or Direct Messages

Read the modal verbs, because they are doing two different jobs. The must points at a separate document: that bolded sentence is a link to help.x.com/rules-and-policies/x-automation, which answers 403, with a Chrome user agent as readily as without one, so I have not read the Automation Rules and will not quote them. The consent line is a should, and it is X writing out in its own words what it expects compliance to look like. A follow is not explicit consent on either reading. Nobody who clicked that button agreed to receive anything. If the automation authenticated through the X API, it ran its whole life against the one instruction X spelled out on a page I can fetch, independent of how it performed. The same section adds a line that reads differently after the cold-email story above: “You should never mislead or confuse people about whether your account is or is not a bot.”

One boundary. That section governs write actions through the X API and developer products. An automation driving a browser falls under the X Rules on platform manipulation instead, which is help.x.com/rules-and-policies/platform-manipulation and also 403s, so the same non-quotation applies there.

The mechanism, folklore removed. Growth posts repeat a table of weights: reply-back plus 75, mute or block minus 74, report minus 369. Those constants appear nowhere in either public repository. What is published is a demo fusion in phoenix/run_pipeline.py at lines 355-360, containing four heads, all positive:

weighted = ( all_probs[:, IDX_FAV] * 1.0
 + all_probs[:, IDX_REPLY] * 0.5
 + all_probs[:, IDX_RT] * 0.3
 + all_probs[:, IDX_DWELL] * 0.2 )

Production weights resolve at request time from a feature-switch service that is not vendored. No negative weight has ever been published, so nobody, me included, can honestly say what the system punishes hardest. The same gap covers NEGATIVE_SCORES_OFFSET from the opening code fence: ranking_scorer.rs pulls it in with use crate::params::* and the published tree has no params file. I pulled the whole repository listing, 244 paths, to check that. The branch is verifiable and the magnitude is not, which is the honest state of every weight discussed in this section.

What is verifiable is better than a weight, because it is structural rather than tunable. phoenix/runners.py declares nineteen heads at lines 233-252 and, at lines 266-271, NEGATIVE_FEEDBACK_INDICES = [14, 15, 16, 17]: not_interested_score, block_author_score, mute_author_score, report_score. “Show less of this” is not_interested_score. Each is a separately modelled prediction, and offset_score sends any candidate that nets out below zero down the separate rescaled branch rather than the shifted one.

The account-level claim, corrected. I used to say negative feedback drags down how the whole account gets treated. The outcome is roughly right and the mechanism I described does not exist. There is no account penalty ledger in the open source. block_author_score and mute_author_score are per-candidate, per-viewer predictions that this viewer would block or mute you over this post. The genuine account-level effect is blunter and permanent: author_socialgraph_filter.rs removes your post from a viewer’s candidate set outright once that viewer has muted or blocked you, and it propagates through quotes and retweets. One mute and you are gone from that feed at any score, and no amount of later quality recovers it.

There is also an asymmetry worth knowing, one line in each of two files. Scoring history defaults to UserActionAggregationType::DenseWithNotInterestedIn (scoring_sequence_query_hydrator.rs:47). Retrieval history defaults to plain Dense (retrieval_sequence_query_hydrator.rs:50). Explicit negative feedback is modelled by default where posts are scored, and not by default where they are fetched.

And the part that kills the tactic on its own terms. Outbound DMs are not a ranking input at all. The only DM among the nineteen heads is share_via_dm_score at index 8, a positive that models somebody sharing your post via DM. Sending DMs earns nothing. The harm route was mute and block plus a policy breach; the upside route did not exist.

What runs now. Nothing automated. I reply to people’s posts, and when someone follows after a good exchange I send a real DM by hand, maybe one in ten. Slower, and those conversations go somewhere. X API access has since moved to pay-per-usage pricing, so the tier names and prices in every guide written about this tactic are now wrong as well. Read the policy first, the pricing page second.

Kill 3: the AI-generated weekly content batch

Every Monday a workflow generated five posts and a thread from that week’s themes and queued them. Five minutes of work for a week of content. Impressions held steady and replies cratered.

My explanation for that was wrong, and the correct one is more useful. I said the cheap signal on X is the impression and the expensive one is the reply. There is no impression head. The nineteen-entry ACTIONS list contains no impression at all; raw counts are telemetry, not model inputs. Nor can I claim a reply outranks a like, because in the only fusion X has published a reply carries 0.5 against a favorite at 1.0.

What I was describing is dwell, which is modelled harder than anything else in the list: dwell_score at index 10, a discrete dwell_time at index 18, and again in the continuous head block. Then the live Rust path adds a head that is not in the nineteen-head list at all, not_dwelled, sitting inside negative_sum next to block, mute and report. Being seen and scrolled past is directly and negatively scored. That is what generated content produces. It reads fine and it skims, and skimming is the failure the model is instrumented to catch.

Corroboration sits one level up, in the content screen rather than the ranker. grox/classifiers/content/banger_initial_screen.py runs a Grok vision model over posts and returns a result whose fields include quality_score and, at line 37, slop_score: int | None. Somebody at X thought machine-generated filler deserved its own integer.

What runs now. AI writes a first draft in 20 minutes, I add my voice and the part I am genuinely unsure about, and I post three times a week instead of five. Fewer posts, more replies. The publish test is whether I would be slightly nervous to post it. No flinch, no reach.

The statistic everyone cites about this, and what it says

A 2025 MIT report gets quoted constantly for the claim that 95 percent of AI pilots never reach measurable return. I have quoted it that way myself. Having now read all 26 pages, the popular version is overstated on four axes, and the correction is worth more than the statistic was.

Table: What the MIT NANDA report actually is, against how it is usually cited.

Usually cited as	What the document says
An MIT study	“Preliminary Findings from AI Implementation Research from Project NANDA”, July 2025. Its sole listed reviewer is also a listed author, and it states the views “do not reflect the positions of any affiliated employers”
A study of 300 companies	52 structured interviews, 153 senior leaders surveyed “across four major industry conferences”, and a review of 300-plus publicly disclosed initiatives
95 percent of AI pilots fail	5 percent success for embedded, task-specific GenAI tools. The same funnel gives 40 percent for general-purpose LLMs, and the same section reports “Generic LLM chatbots appear to show high pilot-to-implementation rates (~83%)”
Never reaches measurable return	Success is “deployment beyond pilot phase with measurable KPIs” at six months, defined by what “users or executives have remarked”. The authors write that six months “may be insufficient… potentially understating success rates”

The 95 is one hundred minus five, taken off the branch of a two-branch funnel that excludes the tools most people actually use. Quoting it as a verdict on AI means quoting the number that specifically is not about ChatGPT or Copilot.

One more checkable fact I have not seen mentioned: the report is no longer served from MIT. The Internet Archive’s index for nanda.media.mit.edu/ai_report_2025.pdf records four 200s on 18 August 2025, at 11:55:20, 12:50:35, 12:54:28 and 14:57:14 UTC, then a 404 at 17:28:56 UTC the same day. Its URL now lands on a Media Lab group overview page. The most-quoted AI statistic of 2025 has not been hosted at its published address since that afternoon.

One thing to get right if you check that index yourself: the length column is not the size of the PDF. It reads 886,480 on the first capture, 867,304, 886,477 and 865,562 on the next three, all four with the same content digest, because it measures the archive’s own compressed record. I downloaded the archived file. It is 923,623 bytes, and it is the same file in every capture.

My reading of it is that most people automate the judgment rather than the grunt work. That is my opinion, not a finding. Its nearest actual finding is budget misallocation, and it contradicts itself inside one section: the takeaway line says 50 percent of GenAI budgets go to sales and marketing while the body of that same section says “approximately 70 percent”. Cite it if you like, but say which number you took and that the source gives two.

The rule, and the paper that named it in 2000

The rule I run now: automate up to the decision, and after the decision, never the decision itself. That is a rediscovery, and the original is more precise than my phrasing.

Parasuraman, Sheridan and Wickens (IEEE Transactions on Systems, Man, and Cybernetics Part A 30(3):286-297, May 2000) model automation as applying to four independent stages: information acquisition, information analysis, decision and action selection, and action implementation. Level is chosen per stage. Their ten-point scale applies to stage 3 specifically, running from a computer offering several options, through level 4 where it suggests one alternative and the human retains authority, through level 6 where the human gets a limited window to veto, up to level 10 where the computer decides and acts alone.

Every automation I killed sat between levels 7 and 10 on stage 3. Every replacement sits at level 4. Enrichment and scoring are stages 1 and 2, and I automated those harder rather than less. That is what “buy the boring stuff, build the edge” means once you say it precisely.

Running stage 3 too high has its own named failure. Endsley and Kiris (Human Factors 37(2):381-394, 1995) called it the out-of-the-loop performance problem: an operator supervising an automated decision loses the situational awareness needed to take over when it degrades. I lost six weeks to that. I was not monitoring the sequence. I was receiving its output.

The quarterly check, and what each trip means

Every automation goes through this once a quarter. Two trips and it is killed or pulled back to level 4.

A quality signal falling while a volume signal holds. Track positive and negative replies as separate series; their divergence leads the average.
A negative signal rising since launch. Unsubscribes, mutes, blocks, “not relevant”, Postmaster spam rate against 0.10 percent rather than 0.30. Implement RFC 8058 so opt-outs arrive as a dated count rather than as reputation damage.
It is making a call a human used to make. That is stage 3. Name the level it runs at; above 4 is the finding.
I would be embarrassed if the recipient knew. The bot test, now also a compliance question in the EU and a policy question under any developer terms you have accepted.
It could break for three days without alerting me. If you would learn from a metric rather than an alarm, you are out of the loop in the Endsley sense.
The time saved is smaller than the trust it costs. Two hours down to eleven minutes with no quality cost, keep forever. Five minutes saved for a point of reply rate, kill it.

What I have now that I did not have in April

Four things, and only one is time.

A vocabulary with citations behind it, so “automate the boring stuff” becomes “keep stage 3 at level 4 and push stages 1, 2 and 4 as high as they go”. A compliance gate that runs before the ROI gate, because kill 2 was on the wrong side of the policy before it was on the wrong side of the numbers, and I checked those in the wrong order. A detection method that watches divergence between two series rather than the level of one. And a reading of the ranker built from files rather than from other people’s posts, which is why the version above corrects three things I used to say.

Two numbers in my original post do not reconcile, and I would rather fix that here than let it sit. I killed three automations and kept eleven. I also said most funnels are over-automated by about a third. Those are different claims and neither confirms the other: three of fourteen is a count of things I run, and the one-third is a guess about other people’s funnels. Read the count as a count and the guess as a guess.

The hours I got back did not go into building more automations. They went into the manual first lines, the real DMs, and the posts I am slightly nervous about. Those are the parts that do not scale, which keeps turning out to be the same list as the parts that work.

An always-on laptop draws 3 W to 7 W, which is £7 to £16 of electricity a year

contact@therezaali.com (Mo RezaAli) — Thu, 06 Aug 2026 00:00:00 +0200

Every automation I run only runs while I am sitting in front of it. I fill the form, the steps go green, I approve the output, I close the laptop, and the print floor goes dark. The factory only exists while I am standing in it. So the work that did not happen last week wasn’t blocked by money or by ideas. It was blocked by me being asleep, at a desk, or on a train.

The obvious repair is a machine that keeps working when I am not there, and the internet is not short of guides telling you how to install one. What almost nobody publishes is the bill. How many watts, at whose unit rate, for how many hours. How many cents per scheduled run, at which per-token price. Which vendor licence thresholds bite a business and leave a hobbyist alone. And, the part that surprised me most, which spending caps actually exist rather than which ones people assume exist.

So this page is two documents: the money and the reasons to walk away, then the build in eight parts, every command carrying the line that tells you whether it worked. What I did was write the specification, price every component against a primary source, and do the arithmetic before spending anything. I have not run it at all yet. Nothing below is a measurement from my own machine, because there is not yet a machine to measure.

What it would be, and who should close this tab

The design calls for a Windows laptop that stays home, plugged in, with sleep and lid-close-to-sleep switched off, because a sleeping computer does nothing at 03:00. The only wallet in the system would be a prepaid OpenRouter account funded with a single small top-up. The interface would be a chat page served by Open WebUI in a Docker container on that laptop, reachable from a phone over Tailscale’s private network rather than through a hole punched in a router. And the worker would be Hermes Agent, holding a list of scheduled jobs, answering a Telegram bot allowlisted to exactly one account, and starting itself again after a reboot. One job would fire early and leave a market brief on a phone before its owner is awake; the bot would make the machine reachable from anywhere, answering exactly one person; and the paid models anyone would use in a commercial AI chat would run on a page served from a laptop you own.

Four installs, one small top-up, and no hole in the router. One inbound rule does get added to the laptop’s own Windows firewall in Part 6, scoped to private networks, and I would rather name that here than let this page claim nothing is ever opened while Part 6 tells you to open something.

The disqualifiers come before the instructions, because the cheapest possible outcome of reading this is that you spend five minutes and decide against it.

Your only laptop travels with you. The machine has to stay home and stay on. What still applies to you is the cost arithmetic, which holds for any hosted model you already pay for.

You have an Intel Mac. The worker software doesn’t support it. Apple Silicon Macs can run this, with a caveat covered further down.

The machine has less than 8 GB of RAM, or runs an older Windows. Below Windows 10 22H2 or Windows 11 23H2 it will fight you at every step.

The machine is not yours to administer. A work laptop or a locked-down family PC won’t let you install software or change power settings, and both are required.

One thing to check now rather than at step forty. Docker’s Windows install requirements list the Pro, Enterprise and Education editions. Press the Windows key, type winver, press Enter, and the window names your edition. If it says Home, treat Part 2 as your test and attempt it before spending any money, which is why it is the first install rather than the last.

Table: What the build asks of you before you start. Money figures are from the sources cited in the sections below; the disk figure is this page’s working minimum rather than a vendor-published requirement.

What it takes	The number
Your time	About three hours over two sittings. Two restarts during the install, one reboot test at the end.
One-time money	$5 of model credit. The card fee is 5.5% with a $0.80 minimum, so the charge lands around $5.80. No subscriptions.
Monthly money	Electricity, roughly £0.60 to £1.30 at the current UK capped rate, plus model spend of roughly a third of a dollar.
The machine	A Windows laptop that stays home, plugged in, on your home Wi-Fi. Windows 10 (22H2) or Windows 11 (23H2 or later), 64-bit, 8 GB RAM, 20 GB free disk.
Your phone	Android or iPhone, with Telegram and one more free app.
Your router	Untouched. No port is forwarded from the internet, ever. One rule is added to the laptop’s own firewall in Part 6, scoped to private networks.

The money

Electricity, and what a laptop actually draws

Two numbers turn a laptop into a monthly bill: what it draws, and what a kilowatt-hour costs where you live. The second is published by regulators. The first is where nearly every guide, including the source material this build came from, quietly guesses, and the guess I inherited was 10 W to 30 W with the screen off. It’s wrong, and it’s wrong in the expensive direction.

ENERGY STAR publishes measured, at-the-wall figures for certified computers under IEC 62301, and its “Long Idle” state is defined as awake with the display backlight off, which is exactly the state this build sits in for almost the whole day. Across the 1,174 notebook-class models in that database on 6 August 2026 the median long-idle draw is 0.7 W, the 95th percentile is 2.1 W, and the highest single entry is 7.1 W. Not one of the 1,174 reaches 10 W with the screen off. The inherited range sits close to the ceiling the EU’s Ecodesign rules permit for a notebook, which is to say the figure everyone repeats is closer to the worst computer the law allows than to a normal one.

The complication has to travel with the number, because it moves it. Those sub-1 W readings depend on the platform dropping into a low-power S0ix state when the backlight goes off, and a resident Docker container plus a polling agent are precisely what prevents that transition. Of the 1,148 rows typed plainly as “Notebook”, 1,011 answer the sleep-mode field at all and 614 of those report their long-idle state as a sleep state, so for a majority of certified machines the published floor is a number this build will never see.

That rules out the bottom of the range and leaves the top. The same database puts short-idle, meaning awake with the screen on, at a median of 5.0 W, and the median per-model gap between the two states is 4.1 W. That gap is not the display alone: it is the display plus the deeper sleep the machine is no longer taking, and this build gives up the second half of it. So the honest working envelope is the certified screen-off band with its floor removed, roughly 3 W to 7 W, where 7 W is within a rounding error of the highest figure any of the 1,174 models records. Call it bounded inference from a measured ceiling rather than a measurement of anything. One caveat travels with it in the other direction: the ENERGY STAR list is certification data and is biased toward efficient hardware, so a gaming laptop with discrete graphics is outside everything above.

Two published rates turn watts into money. The UK figure is Ofgem’s capped average unit rate of 26.11p per kWh for 1 July to 30 September 2026. The EU figure is Eurostat’s average household price of EUR 0.2896 per kWh for the second half of 2025. Both move, and both are linked so you can substitute your own. The arithmetic is watts times 24 hours times 30 days, divided by 1,000.

Table: Monthly electricity for a laptop left running continuously, across the idle-draw range the published measurements support. UK rate is Ofgem’s capped unit rate for 1 July to 30 September 2026; EU rate is Eurostat’s household average for the second half of 2025.

Idle draw	kWh per month	UK at 26.11p/kWh	EU at EUR 0.2896/kWh
3 W	2.16	£0.56	EUR 0.63
5 W	3.60	£0.94	EUR 1.04
7 W	5.04	£1.32	EUR 1.46
10 W	7.20	£1.88	EUR 2.09

The last row is the inherited guess, kept for comparison. It is above the screen-off draw of every one of the 1,174 certified models, so treat it as the ceiling of the old assumption rather than a figure this build is likely to hit.

Your standing charge is not in that table on purpose. You pay it whether this laptop runs or not, so only the added kilowatt-hours belong to the machine.

The envelope is roughly £0.56 to £1.32 a month in the UK, EUR 0.63 to EUR 1.46 on the EU average. Seven to sixteen pounds a year.

Which retires a claim I was going to make. At 10 W to 30 W electricity was the largest recurring cost by a wide margin and the sentence wrote itself; at 3 W to 10 W it is merely the likely largest line at the usage level this page assumes, the two costs are now the same order of magnitude, and which one wins depends on how heavily you use the thing. The only real-world measurement either document has points the other way: the author of the guide this build derives from measured $2 to $6 a month of model spend on a heavier two-machine setup. One person’s usage, not a forecast, and not your expected spend either. It is still the only observed model-spend number in the file, and dropping it while asserting the opposite would be the wrong way round.

A smart plug with a power display settles the whole question in a day.

What one scheduled run costs

The job I specified is a morning brief: at 07:30 the worker searches the web for what moved in a chosen market overnight and sends a handful of bullets with source links to Telegram. Hermes handles the scheduling in chat, and jobs report back to wherever they were created.

Start with the naive version, because it’s the one every guide publishes. Assume one run sends about 6,000 tokens in and gets about 700 back. On deepseek/deepseek-v4-pro, which OpenRouter’s own model API listed at $0.435 in and $0.87 out per million tokens on 5 August 2026, that’s $0.0026 plus $0.0006. A third of a cent.

That number is wrong, and it’s wrong structurally rather than sloppily. A search-and-summarise brief is not one request. It is a tool loop: the worker asks the model what to do, the model calls web_search, the results come back, the model calls web_extract on a page or two, the results come back again, and only then does it write the brief. Each of those is a separate API call, and every call re-sends the whole accumulated transcript, because that is how a stateless chat completions API works. This page explains the same mechanism for long chats three paragraphs from now. It applies to the scheduled job first.

So take the shape rather than a false precision. Roughly N tool turns means roughly N times the transcript re-sent, and the transcript grows as fetched text lands in it. A three-turn brief finishing with a 6,000-token transcript sends nearer 12,000 to 18,000 input tokens across the run.

Table: One morning-brief run on deepseek/deepseek-v4-pro at OpenRouter’s listed prices on 5 August 2026, modelled as a three-turn tool loop. The token counts and the turn count are assumptions for a search-and-summarise job, not a measurement.

Direction	Tokens	Price per 1M tokens	Cost
Input, as a single request	6,000	$0.435	$0.0026
Input, three turns re-sending the transcript	about 18,000	$0.435	$0.0078
Output, three turns	about 900	$0.87	$0.0008
One run			about $0.009
Thirty runs			about $0.26

Nearer a cent a morning than a third of one, and about a quarter a month for a job that runs every day of it. The turn count is a guess as well, since a brief that opens eight pages costs more than one that opens two, and I can’t tell you which yours will be. The ground truth is OpenRouter’s activity page, which logs every request with its real cost, and after the first week of real runs that page replaces this table entirely.

Ad-hoc chat behaves the same way and has one trap in it. Long conversations re-send the whole history with every message, so the twentieth turn in a thread costs many times what the first one did. Forty ordinary standalone questions in a month land around $0.05 to $0.15 by the same arithmetic. Forty turns of one enormous thread do not. Add the two and the year comes to roughly $4 to $5, which still fits inside the one-time $5 credit, but no longer with room to spare.

Three models, and the one that can make this expensive

The chat interface is wired to exactly three model IDs rather than the hundreds OpenRouter offers, which keeps the dropdown short and the choices deliberate.

Table: The three models on the allowlist, with prices per million tokens from OpenRouter’s model API on 5 August 2026. All three carry a 1,048,576-token context window.

Model	Input	Output	What it is for
`deepseek/deepseek-v4-flash-0731`	$0.09	$0.18	Bulk and mechanical work: classification, reformatting, first drafts, anything scheduled that runs often
`deepseek/deepseek-v4-pro`	$0.435	$0.87	The default. Briefs, research, summaries, most real thinking
`moonshotai/kimi-k3`	$3.00	$15.00	Escalation only, one task at a time

Read the right-hand column as a ratio. Kimi K3’s output price is $15.00 against V4 Pro’s $0.87, roughly seventeen times, on the side of the ledger you have least control over, because you can budget your input and you cannot budget how much a model decides to write.

Which gives the escalation rule its shape. Start at V4 Pro. Drop to Flash when the job is mechanical and frequent, because at $0.09 in you can afford to run it hourly without thinking. Escalate to K3 only when V4 Pro has visibly failed at the task in front of you, then go back down. A K3 habit is the only realistic route by which this stack stops being a rounding error on your electricity bill.

Context is the other half. All three models advertise a 1,048,576-token window, and a large window is an invitation to paste enormous things into it. Feeding 500,000 tokens of context to K3 costs 500,000 divided by 1,000,000 times $3.00, which is $1.50 before the model has written a word of reply. On V4 Pro the same dump is about $0.22. On Flash it is $0.045. A window is what the model will accept, not what you should send.

Pin the dated snapshot where one exists

The flash entry above is written as deepseek/deepseek-v4-flash-0731 rather than the unsuffixed slug, and that is a deliberate choice with money attached. Model names come in two flavours on any hosted platform: a dated snapshot points at one frozen release, while a bare or -latest name points at whatever the provider currently considers current, which means it can repoint under you with new weights, different refusal behaviour, different output length and a different price. In an interactive chat you’d notice inside a day. For a job that fires at 07:30 while you’re asleep you might not notice for a fortnight, and the first signal would be either a strange brief or a bill that doesn’t match the arithmetic above.

So the rule is to pin the dated name in anything scheduled. It does not survive contact with the model this build actually schedules.

On 6 August 2026 I read OpenRouter’s model API and pulled every deepseek-v4 slug it publishes. There are four: deepseek/deepseek-v4-flash, deepseek/deepseek-v4-flash-0731, deepseek/deepseek-v4-pro, and ~deepseek/deepseek-v4-flash-latest. Flash has a dated snapshot. Pro does not. The default brain Part 7 sets is deepseek/deepseek-v4-pro, so the rule as stated is unfollowable for the one model that matters most here.

Two ways out, and pick one on purpose. Keep Pro and accept that no pin is available, in which case the monthly check stops being hygiene and becomes the only control you have: read OpenRouter’s model list, confirm the slug still exists and the price still matches the table above, and read one brief properly instead of skimming it, because a repointed model shows up in the writing before it shows up in the bill. Or move the schedule to deepseek/deepseek-v4-flash-0731, which is pinned, costs roughly a fifth as much, and is what this page’s own escalation rule recommends for a mechanical job that runs daily anyway, at the price of quality on the one output you read every morning.

Part 7 keeps Pro and pairs it with the calendar reminder. A pinned name that starts returning “not found” is the system telling you to go look. An unpinned one never tells you anything.

The caps that actually exist

OpenRouter standard accounts have no account-level spending cap. That is the finding I did not expect, and it is the sharpest thing on this page. Their limits documentation is explicit that what constrains you is the prepaid balance and per-key limits, and there is no monthly-ceiling setting to switch on. Plenty of guides describe one. It isn’t there. If you were relying on a dashboard toggle to save you from a loop, you were relying on something that does not exist.

What does exist is three layers, and they do different jobs.

The balance is the wall. OpenRouter is prepaid. The minimum purchase is $5 per transaction under section 4.1 of the terms, and the card fee is 5.5% with a $0.80 minimum per their FAQ, so the first charge lands around $5.80. Leave auto top-up switched off and the worst possible month is the credit you loaded. The same terms allow unused credits to expire after 365 days, so stockpiling a large balance buys you nothing except a larger blast radius.

The per-key limit is the fuse, and a fuse only does work if it is smaller than the wall. Create one named key for this machine and set a credit limit on it in the Keys interface. When a key reaches its limit, requests fail with a 402 and the jobs stop, which is the behaviour you want: a loud, specific error on the activity page rather than a quiet drain.

Part 4 departs from the guide this build came from, deliberately. That guide loads a $5 balance and sets the key limit to $5, and because a fuse rated at exactly the wall’s value can never trip before the wall does, what reads on the page as two independent safety layers turns out in practice to be one layer written down twice, which fails you precisely in the case both were there to cover: the unattended loop at 04:00 that nobody is watching. Load $5, set the key to $2. A loop then stops at $2 with a visible error while $3 of balance sits untouched behind it, and you decide whether to raise the limit instead of learning the answer from an empty account. Two keys with separate limits give you two fuses, which OpenRouter’s provisioning documentation covers if you want per-job accounting.

The job list is the patrol. /cron list in the bot chat shows everything scheduled. Read it weekly. Anything you no longer recognise or no longer open gets paused or removed. One structural comfort: Hermes documents that a scheduled job cannot create scheduled jobs, so a runaway job can repeat itself but cannot multiply into a family of jobs while you sleep.

A normal month on the activity page is one small entry around 07:30 each morning, a scatter of chat entries during the day, everything in fractions of a cent. Runaway has three shapes: the same job appearing many times an hour, which means a schedule set wrong; entries on moonshotai/kimi-k3 you don’t remember asking for, which means something defaulted to the expensive model; or a single day costing more than a normal month, which usually means one job is being fed something enormous every run.

The response, in order, and the order is the point. Pause the job first, in plain language or with /cron pause, because stopping the spend beats diagnosing it. Then /cron list for anything else you didn’t expect. Then the Keys page to confirm the fuse held. Diagnose last. Your damage ceiling was set when you chose the balance and the key limit, and if those two are right the worst outcome is an annoying afternoon rather than a bill.

The absence of the morning message is also the monitoring system. If nothing arrives at 08:00, something is wrong, and it costs nothing to notice.

The licence lines that bite a business

Two of the four installs are free for a person and not necessarily free for a company.

Docker Desktop is free for personal use and for businesses under 250 employees and under $10 million in annual revenue. Docker’s licence page sets both lines, and crossing either one puts you on a paid subscription. The cheapest paid tier on Docker’s pricing page is Docker Pro at $9 per user per month billed annually, but Pro is a single-user plan, so a business licensing a team cannot buy it: that is Team at $15 or Business at $24 per user per month. Two thresholds joined by “and” on the free side means either one can move you, and the price you land on is not the headline one.

Tailscale reads softer, but their page resolves more of it than I first credited. The pricing page describes the free Personal plan as “only suitable for non-commercial use of Tailscale”, and a line earlier as being “for individuals who want to use Tailscale at home”. It also states the mechanical rule Tailscale applies at signup: an account created with a public email domain lands on Personal, while a custom domain triggers a business trial. That is Tailscale telling you how it reads your situation, and it’s why Part 6 says to sign in with a personal address. What it doesn’t resolve is the middle case, one person at home on a personal address reaching their own laptop to run their own business errands. Standard is listed at $8 per user per month. Read their page and decide, or pay the $8 and stop thinking about it.

Everything else is genuinely $0 a month at this scale: Open WebUI, Hermes Agent, and the Telegram Bot Platform. The running total is a pound or two of electricity plus pennies to a couple of dollars of model spend, on hardware you already own, after the one-time top-up.

Privacy, without hedging

Prompts route through OpenRouter to a model provider, and each provider handles data under its own policy. OpenRouter states that it does not store prompt or completion content unless you opt in, but it always stores request metadata: token counts, timestamps, model names. Its provider routing controls govern whether requests can go to providers that may train on the data, and the privacy policy is the document that binds.

Treat the machine like any other cloud service, because that’s what it is. The chat page runs on your laptop; the thinking does not. Market research and draft copy are fine. Customer names, addresses, invoices, anything you wouldn’t paste into a normal AI chat window: no. Any guide telling you a self-hosted chat page makes your prompts private has confused the interface with the model.

The chat page itself is protected by whatever password you set on its first account, and Open WebUI’s hardening guide covers locking it down. That password opens a door to a machine on your home network, so it should not be one you use anywhere else.

Renting a server instead

The laptop is one answer to “where does this run”. The other is a small virtual server, and it deserves a real comparison rather than a footnote.

State the requirement before the prices and you stop buying compute you’ll never use. Open WebUI publishes no minimum memory figure: I read its quick start and its README on 6 August 2026 and neither states one. So the 1 to 2 GB I work from is an estimate, and this is what it rests on. In the project’s own discussion of memory use a maintainer reports the container idling at 1.4 GB, the person who opened the thread measures 500 MB from a cold start and over 1 GB in use, and a contributor gets it to roughly 200 MB by stripping optional services out; a separate thread on minimum system requirements puts an API-only install at 1 GB of RAM, 1 CPU core and 10 GB of disk. User and maintainer reports, not a vendor requirement, and they select every row below.

The agent is small, the operating system takes its share, and the model runs somewhere else entirely, so the CPU is close to irrelevant: this thing is idle more than 99% of the time and then spends thirty seconds waiting on someone else’s GPU. A 2 GB instance fits. A 4 GB instance is comfortable.

Table: Cheapest plan at each provider that genuinely fits Open WebUI plus the agent plus the operating system, read on 6 August 2026. Prices exclude VAT and are the on-demand monthly rate for the provider’s cheapest region. The Vultr and Linode/Akamai pages answer automated requests with HTTP 403, so those two rows were read in a browser and are cited inline below rather than in this page’s machine-checked source list.

Provider	Plan	RAM	vCPU	Disk	Monthly
Vultr	Cloud Compute, Regular Performance	2 GB	1	55 GB SSD, 2 TB transfer	$10.00
DigitalOcean	Basic Droplet, Regular	2 GB	1	50 GB SSD, 2,000 GiB transfer	$12.00
Linode / Akamai	Shared CPU, Linode 2 GB	2 GB	1	50 GB, 2 TB transfer	$12.00
Hetzner Cloud	CPX12, Regular Performance, AMD	2 GB	1	40 GB	EUR 11.49 plus EUR 0.50 IPv4
Oracle Cloud	Always Free, Ampere A1	up to 12 GB	up to 2 OCPU	200 GB block storage	$0.00

The 4 GB step-up, same sources and same date, runs $20 at Vultr, $24 at DigitalOcean, $24 at Linode, and EUR 19.49 for Hetzner’s CPX22.

Three rows came out of awkward pages, and saying so is more useful than pretending otherwise. Vultr and Akamai, which is where Linode’s plan table lives now that linode.com/pricing redirects, both answer non-browser requests with HTTP 403, so their numbers were read in a browser on 6 August 2026. Hetzner’s regular performance page renders prices client-side and every cell came out blank in my session, so rather than fill the gap from memory I read each row’s product key out of the markup and resolved it against the live price feed the page itself loads. And before planning around Hetzner at all, every row on its cost-optimized line carried a “not available” label on the day I looked, including the cheap Arm plan that usually makes Hetzner the obvious answer. Stock changes. Check it yourself.

Oracle’s free tier is the genuinely interesting row. Oracle’s own Always Free documentation states an Ampere A1 allocation of “1,500 OCPU hours per month” and “9,000 GB hours per month”, which is 2 OCPUs and 12 GB of memory running continuously, plus 200 GB of storage and 10 TB of outbound transfer. Several times what this build needs, at nothing a month.

Two points for the server, two for the laptop

I am not stacking this. The server genuinely wins twice. It simplifies the network, because Telegram’s API and the model API are both outbound HTTPS from any host and a server has a public IP, so the private-network step becomes optional rather than load-bearing.

And it dissolves the Docker licensing question. A server runs Linux, so you run Docker Engine rather than Docker Desktop. Docker’s engine documentation states the Apache License, Version 2.0, and the LICENSE file in the moby repository is Apache License, Version 2.0, January 2004. The thresholds on Docker Desktop’s licence page do not reach Docker Engine on Linux, so if your company is over those lines that is $9 to $24 per user per month the server side wins outright.

The laptop wins twice as well. The first is arithmetic: it’s a sunk cost, and its marginal cost is electricity, roughly £7 to £16 a year at the draws in the table above, against $120 to $144 a year for the cheapest fitting server, recurring forever, in a currency that may not be yours.

The second is where the disk lives. On a rented server the provider is a party to your data at rest: the hypervisor host can read the guest’s memory and disk, snapshots sit on their storage, and the volume is reachable by their staff, their subpoena process and their jurisdiction. Hetzner is Germany and Finland and is framed around GDPR; DigitalOcean, Linode/Akamai, Vultr and Oracle are US-headquartered whichever region you pick. On a laptop in your own home, the chat history, the Telegram credentials and the API key sit on a disk no provider can touch. For an assistant that reads your messages, that difference is not small.

Twelve months, and which one wins

Table: Twelve-month total for the same build in three places. Assumptions: the laptop is already owned and its purchase price is excluded; idle draw is taken at 7 W, the upper end of the bounded-inference range above and not a measurement; the UK rate of 26.11p/kWh (Ofgem, 1 July to 30 September 2026) and the EU average of EUR 0.2896/kWh (Eurostat, second half of 2025) are held flat for twelve months, which they will not be; model spend is identical in all three rows because the model runs remotely in all three, and at roughly $4 to $5 a year it still fits inside the one-time credit, though only just; server prices are the on-demand rates read on 6 August 2026, exclude VAT and any overage, and assume no price change; the Oracle row assumes the Always Free tier remains available and survives the capacity limits Oracle applies to it; no currency conversion is performed anywhere in this table.

Where it runs	Recurring per month	Twelve-month total, including the one-time $5.80 of model credit
Laptop you already own	£1.32 (EUR 1.46) electricity	$5.80 plus £15.84 (EUR 17.52)
Cheapest fitting VPS (Vultr 2 GB)	$10.00 hosting	$125.80
Free-tier VPS (Oracle Always Free A1)	$0.00	$5.80

Two inputs moved while this page was being checked. The electricity figure came down when the inherited idle-draw assumption was replaced with published measurements, so the laptop row is roughly half what an earlier draft carried, and the model-spend estimate went up, because re-costing the brief as a tool loop rather than a single request roughly triples it. Neither change reorders the table: the laptop still wins on money by roughly a factor of five at any plausible exchange rate, which is why no conversion is performed here.

Rent the server when uptime matters more than ownership. When the laptop would be closed, carried, sleeping, or sitting on a residential connection that drops. When somebody other than you depends on the thing answering. At $10 to $12 a month you are buying availability and simplicity, not compute. Oracle’s free tier is the middle path worth taking seriously, with three caveats: it is capacity limited and can refuse to launch, it wants a card at signup, and it puts your chat history on a US hyperscaler’s disk.

Keep the laptop when the hardware is already paid for and already running, when the data is personal enough that provider access is a real objection, and when a few hours of downtime a year cost you nothing. For a single-user assistant in your own house, that last one is usually the honest answer.

Four things I left out, and why

An Apple Silicon Mac instead of a Windows laptop. Docker Desktop and Open WebUI behave the same way on macOS, and Hermes supports Apple Silicon while explicitly not supporting Intel Macs. The catch is physical: a closed MacBook sleeps, so it has to live lid-open with sleep disabled. Workable. The Windows laptop simply has fewer ways to go wrong.

A free local model on your own GPU. Tools like Ollama run open weights on your own hardware, which is free per run, private, and works offline. On an ordinary laptop GPU it’s also slow, slow enough that a job set for 07:30 is not reliably finished by 08:00. And the whole nightly workload costs a few tens of cents a month on hosted models, so local inference is a hobby lane rather than week-one material.

A browser-based coding environment. Open WebUI’s team ships a tool called Open WebUI Computer, installed as the cptr package, which serves your machine’s real files, terminal, editor and git to a browser tab. It works. It also hands full access to your machine to whoever holds the login, and nothing in the first month of this build needs it.

Running a Claude subscription through third-party tools. The setup this build derives from plugged a subscription-authenticated CLI into that environment to reuse a plan you already pay for. I had assumed Anthropic’s documentation ruled that out, and on re-reading, the live pages say something more interesting. The Agent SDK overview points third-party tools at API-key authentication, but the support article on using the SDK with a Claude plan opened on 6 August 2026 with a banner reading “Update June 15: We’re pausing the changes to Claude Agent SDK usage described below. For now, nothing has changed: Claude Agent SDK, claude -p, and third-party app usage still draw from your subscription’s usage limits.” So it works today, by the vendor’s own words, and in the same sentence the vendor tells you it was about to stop working and may yet. A policy explicitly described as paused is a poor foundation for a process you leave running unattended for a year. If you pay for Claude, use Claude’s own apps alongside this build, and watch that banner rather than this page.

The build, eight parts

One path, in order, with no choices to make along the way. Do all eight parts sitting at the machine that will stay home, and read this page in a browser on that machine so you can copy commands rather than retype them.

Each command block has a line above saying what it does and a line beneath saying what a working result looks like. That second line is the point of the format: most walkthroughs tell you what to type and leave you to guess whether it worked, so the first silent failure takes the next hour with it. If your screen doesn’t match the “what you should see” line, stop and read that part’s failure table. Anywhere you substitute your own value, the text is an OBVIOUS-PLACEHOLDER in capitals with a filled example underneath. Parts 1 to 5 make a sensible first sitting, Parts 6 to 8 a second.

Part 1, make the machine stay awake (15 min)

Factory settings put a laptop to sleep the moment you stop touching it.

Plug it in. It stays plugged in from now on.
Open Settings. Windows 11: System, Power and battery, Screen and sleep. Windows 10: System, Power and sleep. Different names, same settings.
Set “when plugged in, put my device to sleep after” to Never.
Set “when plugged in, turn off my screen after” to 5 minutes. A dark screen saves power while the machine underneath keeps working.
Now the lid, which is the step people skip. Type “lid” into the Settings search box, open “Change what closing the lid does”, set “When I close the lid” in the Plugged in column to Do nothing, and save. Without this, shutting the lid puts the whole build to sleep.

Table: Part 1 failures and fixes.

Symptom	Fix
No “plugged in” column on the lid screen	The laptop is running on battery. Plug it in and reopen the screen.
“Change what closing the lid does” not found in search	Windows key, type “control panel”, open it, then Hardware and Sound, Power Options, “Choose what closing the lid does” in the left sidebar.

Checkpoint. Stop here and you still have a laptop that stays awake with the lid shut, which is the physical foundation of everything else. Test it: play a long video with sound, close the lid, listen. Audio still playing means the machine is awake.

Part 2, install Docker Desktop (40 min, two restarts)

Docker runs the chat interface in the background. Install it once, set it to start with Windows, never think about it again.

Check the machine can run it. Task Manager (Ctrl+Shift+Esc), Performance tab, CPU, bottom right: “Virtualization: Enabled”. If Task Manager shows only a plain list of apps, click More details; on Windows 11, Performance is the graph icon in the left sidebar. Disabled means read the failure table before going any further.
Open PowerShell as administrator: Windows key, type “powershell”, right-click Windows PowerShell, Run as administrator, Yes.

This installs the Windows subsystem Docker runs on:

wsl --install

What you should see: several components download and install, then a message asking you to restart the computer. Restart it.

After the restart, a black Ubuntu window may ask for a username. Docker does not use it; close it.
From Docker’s download page, take the AMD64 build of Docker Desktop for Windows (correct for both Intel and AMD laptops; ARM64 is for rare ARM machines) and run the installer. Keep “Use WSL 2 instead of Hyper-V” ticked if asked. Restart again if asked.
Open Docker Desktop, accept the service agreement, and choose Continue without signing in. Wait until the whale icon in the tray stops animating and the window says Engine running.
Gear icon, General, tick “Start Docker Desktop when you sign in to your computer”. That’s what brings the chat interface back after a restart. Do not skip it.

Table: Part 2 failures and fixes.

Symptom	Fix
Virtualization: Disabled in Task Manager	It has to be switched on in the laptop’s BIOS. The no-key-tapping route: Settings, System, Recovery, Advanced startup, Restart now, then Troubleshoot, Advanced options, UEFI Firmware Settings, Restart. Fallback is tapping F2 or Delete during startup. Enable the setting named SVM Mode (AMD) or Intel VT-x / Virtualization Technology (Intel), save and exit.
`wsl --install` errors about your Windows version	Windows is too old. `winver` shows your version; you need Windows 10 22H2 or Windows 11 23H2 or later. Run Windows Update, then retry.
Docker Desktop says WSL needs updating	In the same admin PowerShell run `wsl --update`, then reopen Docker Desktop.

Checkpoint. Stop here and you still have Docker installed, running, and set to start with Windows.

Part 3, start the chat interface (20 min)

Open WebUI is a chat page served from your machine: the familiar layout, on your laptop, with an account that is yours alone. Open PowerShell, normal this time, no administrator needed.

This downloads the chat interface and starts it in the background, set to restart itself whenever the machine reboots:

docker run -d -p 3000:8080 -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

What you should see: several download progress bars, then one long string of random letters and numbers on its own line. That string means it started. The first download is a few gigabytes, so let it run. If the screen goes dark mid-download, that’s Part 1’s screen-off setting doing its job. Move the mouse; the download never stopped.

This confirms it is running:

docker ps

What you should see: a table with one row, open-webui, whose STATUS column starts with Up.

Open a browser on the machine and go to http://localhost:3000, allowing a minute on first load. Click Sign up. Use a real email format, since nothing is ever sent and the address is only your username, and a strong password you use nowhere else, because this page is a door to your machine.
The first account created becomes the administrator, and sign-ups switch themselves off once it exists. Confirm that lock: your name at the bottom left, Admin Panel, Settings, General, sign-up toggle off. The wording shifts between versions. Your own account is unaffected.

Table: Part 3 failures and fixes.

Symptom	Fix
`docker: command not found`, or a pipe error	Docker Desktop is not running yet. Open it, wait for Engine running, try again.
Error says port 3000 is already in use	Re-run the long command with `-p 3001:8080` instead, and use 3001 everywhere this page says 3000.
Page never loads	Wait 60 seconds after `docker ps` shows Up. The first start is slow.

Checkpoint. Stop here and you still have a private chat page running on your machine. It has no brain yet.

Part 4, open the brain account (15 min)

OpenRouter is one account and one balance across many paid models. Load $5, set a smaller fuse inside it, and that balance is the hard ceiling on what this machine can ever spend.

Sign up at openrouter.ai with a login you control long-term. Credits and Keys live under the profile icon, top right.
Credits, add $5, the minimum per their terms. The 5.5% card fee with its $0.80 minimum makes the total about $5.80, itemised before you pay.
Leave auto top-up off. Standard accounts have no account-wide cap, so the balance is the cap, and with auto top-up off the worst month costs $5.
Keys, create a key named night-shift, and put 2 in the credit limit field. If anything ever loops, the key shuts off at $2 with an error while $3 of your balance is still there. This departs deliberately from the guide this build follows, which sets the limit to the full $5: a fuse rated at the wall’s own value can never trip first, and two layers that fire at the same number are one layer.
The key is sk-or-v1- plus a long string and it is shown exactly once. Copy it into a password manager before you close the page, because you paste it twice, in Part 5 and Part 7. It is the wallet: never in a chat message, a screenshot, or a shared workflow.

Table: Part 4 failures and fixes.

Symptom	Fix
You closed the key window without copying it	It cannot be shown again. Delete the key on the Keys page, create a new one, copy it this time.
Checkout total looks higher than $5	That is the purchase fee, itemised at checkout. A total near six dollars is correct.
Later, requests start failing with a 402	The `night-shift` fuse tripped at $2, which is what it is for. Read the activity page to find what spent it, then raise the key limit on the Keys page. The balance behind it is untouched.

Checkpoint. Stop here and you still have a funded model account with two real limits: a $5 wall and a $2 fuse inside it.

Part 5, wire the brain to the chat (15 min)

Two paste operations connect the account to your chat page, and an allowlist keeps the model menu at three entries instead of hundreds. Open WebUI’s own guide to OpenAI-compatible providers covers the screens if yours differ.

On the chat page: your name at the bottom left, Admin Panel, Settings, Connections. Under the OpenAI-compatible section, click add. Two fields matter:
- URL: https://openrouter.ai/api/v1. The /v1 is not optional, and a missing /v1 is the classic reason no models appear.
- Key: sk-or-v1-YOUR-KEY-HERE Filled example: sk-or-v1-9f3ab81c0d2e4f56a7b8c9d0e1f2a3b4 (yours is longer)
In the same connection’s model IDs field, add exactly these three, one at a time so each becomes its own entry rather than one comma-separated line: deepseek/deepseek-v4-pro, deepseek/deepseek-v4-flash-0731, and moonshotai/kimi-k3.
Save. Open a new chat, pick deepseek/deepseek-v4-pro from the dropdown at the top, and send: “Reply with the single word OK.” Any sensible reply means the wiring works, because models are unreliable narrators and the exact words do not matter.
The real proof is elsewhere. Open the activity page and the test message is there as a logged request costing a fraction of a cent. That page is your receipts drawer from now on.

Table: Part 5 failures and fixes.

Symptom	Fix
No models appear after saving	The URL has to be exactly `https://openrouter.ai/api/v1`. Re-check for a missing `/v1` or a trailing space.
Model errors mentioning 401	The key pasted wrong. Re-copy it with no spaces around it.
Model errors mentioning 402	The $5 top-up did not complete, or the key’s $2 limit is spent. Check the Credits and Keys pages, in that order.
No model IDs field in your version	Save the connection, then go to Admin Panel, Settings, Models and switch off everything except the three above.

Checkpoint. Stop here and you still have a working AI chat served from your own machine, with receipts. Good finish line for the first sitting.

Part 6, put it in your pocket (15 min)

Tailscale connects your own devices to each other over an encrypted private network, so your phone can reach the chat page from anywhere.

Never open a port on your home router to reach any of this. No step in this build touches your router.

On the machine, install Tailscale for Windows and sign in with an address ending @gmail.com, @outlook.com or similar. An address at your own business domain makes Tailscale treat you as a business and start a paid trial.
Click the Tailscale tray icon, behind the caret near the clock if hidden. The machine’s private address is on the line naming this computer, in the form This device: 100.x.y.z. Write it down. This page calls it MACHINE-TAILSCALE-IP. Filled example: 100.101.102.103
Open the port to your private networks. Windows key, type wf.msc, Enter. Inbound Rules, New Rule, Port, TCP, specific local ports 3000 (3001 if you switched ports in Part 3), Allow the connection, tick Private only and untick Domain and Public, name it Open WebUI, Finish.

That rule is on the laptop’s own firewall, not on your router, and it is scoped to networks Windows already classifies as private. No port is forwarded from the internet, so the only thing that can reach port 3000 is a device on a network the laptop already trusts, which after step 4 means your own tailnet.
On the phone, install Tailscale, sign in to the same account, flip its switch on, and allow the VPN configuration it asks for. That’s how Tailscale links your devices.
The proof: turn off the phone’s Wi-Fi so it is on mobile data, open the phone browser, and go to http://MACHINE-TAILSCALE-IP:3000. Filled example: http://100.101.102.103:3000 Log in with the account from Part 3.
Then the step that stops this build failing silently in six months. Tailscale device keys expire, and the default expiry period is 180 days; when a device’s key lapses, connections to and from it stop working. For a machine you are deliberately not looking at, that is an outage with a half-year fuse already lit. In the Tailscale admin console open Machines, find this laptop, click the menu icon at the right of its row, and choose Disable Key Expiry. Tailscale’s own documentation names trusted servers and hard-to-reach devices as exactly the case for doing this.

Table: Part 6 failures and fixes.

Symptom	Fix
Page times out on the phone	Check the Tailscale switch is on on the phone, that you allowed the VPN permission (toggle the switch again to re-prompt), and that the machine’s tray icon shows connected. Then the firewall rule: double-click `Open WebUI` in Inbound Rules, Advanced tab, tick all three profiles, because the Tailscale adapter can register as a public network. The machine is still not reachable from the internet; the rule only applies to networks the laptop is already on.
Wrong address	The `100.x.y.z` address comes from the Tailscale tray icon, not from `ipconfig`. Re-copy it.
Login page loads but rejects the password	It is the Open WebUI account from Part 3, not your Tailscale login.
It worked for months, then the phone stopped reaching it	The device key expired. Sign the machine in to Tailscale again, then do step 6 so it cannot recur.

Checkpoint. Stop here and you still have your machine’s chat, with your capped paid models, on your phone anywhere.

Part 7, hire the worker (25 min)

Hermes Agent is a free, open-source agent that lives on the machine, uses your OpenRouter balance as its brain, and in Part 8 answers Telegram messages and runs scheduled jobs. Open PowerShell, normal, not administrator.

This downloads and installs the worker and everything it needs:

iex (irm https://hermes-agent.nousresearch.com/install.ps1)

What you should see: an installer walks through downloading its components and finishes without red errors. Then close PowerShell and open a new one so the hermes command is picked up.

This confirms the install and shows which tools the worker has switched on:

hermes tools --summary

What you should see: a printed summary of enabled tools. Look for web_search, because the test below depends on it.

This stores your OpenRouter key where the worker looks for it:

hermes config set OPENROUTER_API_KEY sk-or-v1-YOUR-KEY-HERE

Filled example: hermes config set OPENROUTER_API_KEY sk-or-v1-9f3ab81c0d2e4f56a7b8c9d0e1f2a3b4

What you should see: a confirmation that the value was saved.

This opens the model picker, where you set the worker’s default brain:

hermes model

What you should see: an interactive picker, arrow keys to move and Enter to confirm. Choose OpenRouter as the provider, then select or type deepseek/deepseek-v4-pro, and confirm it as the default. The picker on your screen is the ground truth, not this page.

That slug is unpinned, and OpenRouter published no dated Pro snapshot on 6 August 2026, so put a monthly reminder in your calendar now to check the price and read one brief closely. If you would rather have the pin than Pro’s writing, type deepseek/deepseek-v4-flash-0731 at this picker instead.

This is the test that matters, brain plus live web in one shot:

hermes -z "Search the web for the current price of Bitcoin and answer in one line with the number."

What you should see: one line with an actual current price. That is proof the worker can think (OpenRouter) and fetch live information (web search), the two abilities every scheduled job depends on.

Table: Part 7 failures and fixes.

Symptom	Fix
The install command itself shows red errors	Safe to re-run. Open a new PowerShell, paste the same line again, allow it if your antivirus asks. If it fails twice the same way, take the error text to the docs at hermes-agent.nousresearch.com.
`hermes` is not recognized	You are in the old PowerShell window. Close it, open a new one.
The test answers but clearly did not search (no number, or it says it cannot browse)	Run `hermes tools`, which opens an interactive menu: arrow keys move, space toggles, Enter confirms. Enable `web_search` and `web_extract` for the CLI, then re-run the test.
Model or auth errors	Re-run `hermes model` and re-check every choice, then re-run the `hermes config set` line. A key pasted with one character missing fails exactly like this.

Checkpoint. Stop here and you still have a working agent on the machine that answers one-shot questions with live data, on your capped balance.

Part 8, phone control, and surviving a reboot (25 min)

This part gives the worker a phone number, in the form of a private Telegram bot only you can use, and makes everything survive a restart.

In Telegram, search for @BotFather and open it. The real one has a blue verification mark and that exact username; impostors exist, so if there is no mark, search again. Send /newbot. It asks for a display name, then a username ending in bot, and replies with a token like 1234567890:AA.... That token is a house key. Never screenshot it or paste it anywhere except the wizard below.
Get your own Telegram user ID, a number rather than your @username. Search for @userinfobot, press Start, and it replies with your numeric ID. It is a widely used third-party utility bot and it sees only your public profile. Placeholder used below: YOUR-TELEGRAM-ID. Filled example: 123456789

This starts the connection wizard on the machine:

hermes gateway setup

What you should see: a wizard. Choose Telegram, paste the bot token, and when it asks for allowed user IDs, enter YOUR-TELEGRAM-ID and nothing else. The allowlist is the lock: with only your ID on it the bot ignores every other Telegram account, and bots cannot start conversations in the first place.

This registers the worker to start by itself whenever you sign in to Windows, and needs no administrator rights:

hermes gateway install

What you should see: confirmation that an autostart entry was created. It uses Windows Scheduled Tasks, with a Startup-folder fallback.

This starts it right now:

hermes gateway start

What you should see: confirmation the gateway started.

This checks on it, and it is your go-to command whenever the bot seems quiet:

hermes gateway status

What you should see: a status readout saying it is running.

On your phone, open Telegram, find your bot by its username, press Start, and send Are you there? A reply arrives. You are talking to your machine.
The negative test: from any other Telegram account, message the bot. Correct behaviour is nothing at all. If that account gets a reply, stop and re-run hermes gateway setup, because the allowlist is wrong. No second account handy? Skip it.
The reboot test, which proves the whole build. Restart Windows, sign in, wait two minutes, then message the bot and open http://MACHINE-TAILSCALE-IP:3000 in the phone browser. Both work, because Docker restarts the chat interface (Part 3’s --restart always plus Part 2’s start-on-sign-in) and the gateway autostarts. The chat page can take up to five minutes while Docker wakes.

What the build’s uptime actually depends on

The source material I worked from says that after any restart the machine has to be signed in to once before the worker is back, and that overnight Windows updates therefore cost you the occasional morning. Reading the vendors’ own documentation, that is both too pessimistic about the common case and silent about the cases that really do break it.

Start with the dependency chain, because every link in it is scoped to a sign-in rather than to a boot. Docker Desktop’s autostart setting is worded “Start Docker Desktop when you sign in to your machine”, and it is off until you tick it. Hermes registers itself with schtasks /Create /SC ONLOGON, which is a logon trigger. Microsoft’s own reference is explicit that the alternative, a boot trigger, “starts a task when the system is booted” and that “only a member of the Administrators group can create a task with a boot trigger”. Nothing in the default build has one. The uptime of this machine is the uptime of an interactive Windows session, and every reboot symptom in the tables above is a consequence of that single fact.

Now the part the inherited limitation gets wrong. Windows has a feature called Automatic Restart Sign-On, and Microsoft documents that “when Windows Update initiates an automatic reboot, ARSO extracts the currently logged in user’s derived credentials, persists it to disk, and configures Autologon for the user”, after which “the last interactive user is automatically logged in and the session is locked”. A locked session is still a signed-in one, so the logon trigger fires and the gateway comes back. The policy “is enabled by default” if you have not configured it. So the overnight Windows update, the scenario the limitation is usually written about, mostly does not cost you a morning.

What does cost you a morning is narrower and worth knowing precisely. ARSO “only occurs if the last interactive user didn’t sign out before the restart or shutdown”, so signing out before bed defeats it. On a machine joined to Active Directory or Microsoft Entra ID “this policy only applies to Windows Update restarts”, which means a power cut or a manual restart on a work-managed laptop gets nothing. It also fails where the account is disabled, where logon hours apply, or where the user must change their password at next sign-in, which is a realistic six-month event on a personal Microsoft account. When a morning goes missing and you want to know why rather than guess, the LSA Operational log in Event Viewer records event 322 for a failed ARSO configuration and 320 or 321 when it worked.

One honesty note about that page: it says in one place that ARSO is “opted out for Client SKUs” and in another that the policy “is enabled by default”. I have not been able to resolve the contradiction, so treat ARSO as probable rather than guaranteed and let the event log settle it on your own machine. If you want the guarantee instead of the probability, the fix is to stop depending on a logon trigger: register the gateway as a real Windows service, which the Hermes Windows documentation itself points at, or add a second task with a boot trigger and the administrator rights that requires. Neither removes the Docker Desktop dependency, which is the harder half.

Table: Part 8 failures and fixes.

Symptom	Fix
The bot never replies	Run `hermes gateway status` on the machine. If it is not running, `hermes gateway start`. If it is running, re-run `hermes gateway setup` and re-paste the token carefully.
Replies stopped after the reboot	Sign in to Windows first, then `hermes gateway status`, and `hermes gateway install` again if needed.
The negative test got a reply	The allowlist holds the wrong value. It has to be the numeric ID, not the @username. Re-run `hermes gateway setup`.

Checkpoint. The machine is complete. It thinks on a capped budget, reaches your pocket, ignores strangers, and survives a reboot. It has no job yet.

Living with it

The first job

Create the job from your phone, in the bot chat, because jobs report back to wherever they were created.

This message creates the scheduled job. Copy the whole block into Telegram and replace [YOUR NICHE], brackets included, with your own category. Do not retype the quotes, because phone keyboards swap them for curly ones the command may not parse:

/cron add "every day at 7:30" "Search the web for what changed in the [YOUR NICHE] market in the last 24 hours: new competitor angles, price moves, trending products, and any ad platform policy news. Send 5 short bullets, each with its source link. End with one hook angle worth testing today, written as a single sentence."

What you should see: nothing yet, because the placeholder is still in it. Here is the same command filled in:

/cron add "every day at 7:30" "Search the web for what changed in the posture corrector market in the last 24 hours: new competitor angles, price moves, trending products, and any ad platform policy news. Send 5 short bullets, each with its source link. End with one hook angle worth testing today, written as a single sentence."

What you should see: the bot confirms the job was created. Send /cron list, and the next run has to say tomorrow morning at 07:30 local time. A strange hour means the machine’s clock or timezone is off. Fix it in Windows Settings, Time and language, then remove and re-add the job.

To run it once immediately, message the bot in plain language: “Run the morning brief job now.” What you should see is an acknowledgement, then the brief itself a minute or two later. Read the first one with a working eye. The question is not whether it’s perfect. It’s whether one of those five bullets just handed you tomorrow’s angle.

Management happens in the same chat, in plain language or with /cron:

Different time or days: “Change the morning brief to weekdays at 7:00.”
Pause it: “Pause the morning brief.” Resume the same way.
See everything scheduled: /cron list
Delete it: “Remove the morning brief job.”
If it cannot tell which job you mean, /cron list shows the name and the id.

One rule as you add more. One job, one purpose. Five small pausable jobs beat one long mush message you stop reading, and they fail one at a time instead of all at once.

Table: First-night failures and fixes.

Symptom	Fix
No confirmation after `/cron add`	On the machine: `hermes gateway status`, then `hermes gateway start`.
Acknowledged, but no brief after five minutes	`/cron list` to confirm the job exists, then ask the bot what happened to its last run.
Bullets arrive without links, or read as invented	Web tools are off. Re-run Part 7’s search test on the machine and re-enable `web_search` with `hermes tools`.
Silence tomorrow at 07:30	The laptop is sitting at the Windows sign-in screen after an overnight update. Sign in and wait two minutes.

Weekly and monthly maintenance

The weekly pass takes five minutes. Did the morning brief arrive today, since its absence is the monitoring system? Glance at the activity page for shape and total, not forensics. Run /cron list and ask whether everything scheduled is still something you read. Once a month, check free disk space is above 10 GB.

Then the honest limitation about restarts. After any reboot the machine has to be signed in to once before jobs and the bot come back, because the worker’s autostart runs at sign-in and not before it. Windows updates restart at night on their own schedule, so an occasional missed morning is normal rather than a fault. There is a trade available and it has a real cost: Windows can be set to sign your user in automatically, which is the setting you reach by searching for “netplwiz”, and the price is that anyone who opens the laptop is signed in as you. Leave it alone unless the machine is behind a door you lock.

Back up the two things you cannot recreate. The uninstall section below is honest that deleting the chat volume is permanent, and a failing disk is the same event without the warning. The worker’s configuration and stored key live in %LOCALAPPDATA%\hermes per the Hermes Windows-native page, and that is an ordinary folder: paste the path into the File Explorer address bar and copy it elsewhere. The chat history lives in a Docker volume named open-webui, which is not a folder you can browse, so use Docker’s own back up, restore, or migrate procedure, which mounts the volume into a throwaway container and writes a tar file to a directory you choose. Do both monthly and keep the copy off the laptop.

The monthly pass is mostly one command sequence. Open WebUI’s update procedure works because your chats and settings live in the storage volume rather than in the container, so removing the container loses nothing:

docker rm -f open-webui
docker pull ghcr.io/open-webui/open-webui:main
docker run -d -p 3000:8080 -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

What you should see: the old container is removed, a fresh image downloads, and a new container id prints. http://localhost:3000 logs you in with the same account and the same history.

This updates the worker in place:

hermes update

What you should see: it checks for a newer version, updates if there is one, and reports what it did.

Then message the bot once to confirm it answers, and skim the model prices against OpenRouter’s model list. Providers reprice quietly, and this is the pass where you would notice.

Troubleshooting

Table: Consolidated symptoms, likely causes and fixes for the running machine.

Symptom	Likely cause	Fix
No morning brief, bot silent	Machine asleep, powered off, or at the sign-in screen	Wake it, sign in, then `hermes gateway status`
No brief, but the bot answers chat	The job is paused or errored	`/cron list`, then “Run the morning brief job now” and read the error
Bot silent, machine running	Gateway down	`hermes gateway status`, then `hermes gateway start`. Re-run `hermes gateway install` if it did not survive a reboot
Chat page dead from the phone	Tailscale off, an expired device key, or the firewall rule	Check both Tailscale switches and the machine’s status in the admin console, then the `Open WebUI` inbound rule from Part 6
Chat page dead everywhere	Container stopped	`docker ps` on the machine. If it is empty, open Docker Desktop and check the start-on-sign-in setting from Part 2
Errors mentioning 401	Key wrong or revoked	Re-paste the key with `hermes config set OPENROUTER_API_KEY ...` and in the Open WebUI connection
Errors mentioning 402	Key fuse blown at $2, or balance empty	Keys page first, then Credits. Check the activity page for what spent it before raising anything
Answers suddenly slow or strange	Model changed upstream	Check the model name still exists, and pin a dated snapshot if the provider publishes one

Every component here belongs to somebody else, and all of them ship changes without asking you. So the more useful table is the one that maps a breakage to the document that outranks this page.

Table: Where the ground truth lives when something upstream changes. In every row, the linked source wins over anything written here.

What broke	Where the ground truth lives
A model name starts returning “not found”	OpenRouter’s model list. Find the current slug, then update `hermes model` and the Open WebUI allowlist
A Hermes command or screen no longer matches this page	The Hermes CLI reference and the picker on your screen, both of which outrank this file
Open WebUI screens moved	Open WebUI’s own quick start
Prices drift from the tables above	OpenRouter’s model API is the live source, and this page’s date stamp tells you how stale my copy is
Docker or Tailscale licence terms move	Docker’s licence page and Tailscale’s pricing page

The full uninstall

The exit gets the same detail as the entrance, in reverse build order.

1. Remove the jobs. In Telegram, /cron list, then remove each one in plain language: “Remove the morning brief job.”

2. Stop the worker’s autostart. In PowerShell, hermes gateway stop, then hermes gateway uninstall, which removes the Scheduled Task or startup entry.

3. Delete the worker. Delete the folder %LOCALAPPDATA%\hermes by pasting that into the File Explorer address bar. On native Windows that single folder is the whole of it: the Hermes Windows-native page documents it as the root holding the config, the stored credentials, the skills, the sessions and the logs, with the install in a subfolder beneath. A .hermes folder in your user profile exists only if you repointed HERMES_HOME there.

4. Retire the bot. In @BotFather, /mybots, select the bot, and revoke the token or delete the bot outright. A dead token is a dead door.

5. Remove the chat interface and its stored chats. The volume deletion is the permanent part, so be sure before you run the second line:

docker rm -f open-webui
docker volume rm open-webui

What you should see: each command echoes the name it removed. Chats and the admin account are gone with the volume.

6. Uninstall Docker Desktop. Settings, Apps, Installed apps, Docker Desktop, Uninstall.

7. Close the model account. Delete the night-shift key on the Keys page. Spend any remaining credit or accept its loss; the terms cover refunds and expiry in section 4.

8. Remove Tailscale, then restore the power settings. Uninstall the app on the machine and the phone, and remove both devices from the machines list in your Tailscale admin console. Then Part 1 in reverse.

The laptop is now exactly the machine it was before Part 1.

What I do not know yet

Things I expect to learn only by running this. What my laptop actually draws, which turns a bounded inference into one row of fact. What a search-and-summarise run really costs across a real tool loop, which the activity page will answer within a week and which may be several times the estimate above in either direction. How often Windows restarts itself overnight. Whether Hermes ships with web search on by default on a fresh Windows install, which Part 7 checks twice precisely because I could not confirm the default in the documentation. Whether Docker Desktop installs on Windows Home, which is why Part 2 comes before any spending. And whether the morning message is something I read on day thirty or something I stopped opening on day nine, which is the only question here that no amount of arithmetic can answer.

The costing stands on its own regardless. Roughly £7 to £16 of electricity a year, four or five dollars of model spend across the same year, $0 in licences below Docker’s employee and revenue lines, a one-time $5.80, and a prepaid balance that is the hard ceiling on everything the machine can ever spend. Against $120 to $144 a year to rent the same job a server, or $0 on a free tier that puts the disk in someone else’s building. If those numbers make the idea worth three hours of your evening, the eight parts are above. If they don’t, you’ve lost the ten minutes it took to read the arithmetic, which beats finding out in month three.

4,600 people sorted human text from machine text at 50 to 52%

contact@therezaali.com (Mo RezaAli) — Thu, 06 Aug 2026 00:00:00 +0200

Nobody detects AI content by spotting polish, because almost nobody detects it at all. In Jakesch, Hancock and Naaman (PNAS, March 2023), 4,600 participants across six experiments “identified the source of a self-presentation with only 50 to 52% accuracy.” In Nightingale and Farid (PNAS, February 2022), 315 participants sorting synthetic faces from real ones scored 48.2%, “close to chance performance of 50%.”

That matters to me because I built a conclusion on the opposite assumption. Meta defines hook rate as arithmetic, not a vibe: the metric “counts the number of 3-second video plays and divides it by ad impressions.” I ran two cuts of the same AI UGC ad. In the second, the creator stumbled over the opening sentence, paused, and started again. I nearly cut that clip, left it in, and that version produced a noticeably higher hook rate. I explained it to myself by saying polished AI content has become easy to recognise and the imperfection defeated recognition. The recognition half of that sentence is false. The stumble did not make my ad harder to identify as AI, and the paper I reached for to explain it turns out to argue against me on the one cue I used. What survives is a narrower mechanism with better evidence behind it, and it happens to sit inside the exact window the metric samples.

This was my test, my budget, one product, a handful of creatives, a few weeks. I published the pattern in the X post this article is built from and did not publish the hook rate figure there. I am not going to invent one here. What I can do is take the test apart, which is worth more than the number would have been.

The design fault is in the first three seconds

Hook rate samples the opening three seconds. The stumble, the pause and the restart are in the opening three seconds. The variable I believed I was testing, global polish, and the variable I actually changed, the content of the measurement window, were the same edit. Meta’s own A/B testing best practices page states the requirement I broke: “You’ll have more conclusive results for your test if your ad sets are identical except for the variable that you’re testing.”

The metric is also narrower than the argument I hung on it.

Table: How Meta defines the four metrics involved, in Meta’s own words, read 6 August 2026.

Metric	Meta’s definition	Status
Hook rate	3-second video plays divided by ad impressions	“This metric is in development”
Hold rate	ThruPlays divided by 3-second video plays	“This metric is in development”
3-second video plays	Played at least 3 seconds, or 97% of length if shorter. “Time spent replaying the video for a single impression won’t be included.”	Stable
ThruPlay	Played at least 15 seconds, or 97% of length if shorter than 15 seconds	Stable

The denominator is impressions, not reach and not video plays, so a fatiguing creative drags its own hook rate down through repeat impressions with no change to the creative at all. And replays inside an impression are excluded, so if the stumble makes someone rewind, the metric pays nothing for it.

What people actually detect, measured

The popular version of this research is “people cannot tell AI faces from real ones.” That overstates it, and the correction is worth more than the statistic. Nightingale and Farid ran a second experiment with training and trial-by-trial feedback, and accuracy rose to 59.0% (95% CI 57.7% to 60.4%), which is above chance and reliably so. At chance untrained, barely above chance trained. In their third experiment, participants rated synthetic faces 4.82 for trustworthiness against 4.48 for real faces, “only 7.7% more trustworthy” but significant at t(222) = 14.6, P < 0.001, d = 0.49.

Table: What three primary studies measured about human detection of synthetic content.

Study	Stimuli	Accuracy	Direction of the error
Nightingale & Farid 2022, N = 315	StyleGAN2 still faces	48.2%	Synthetic rated more trustworthy
Nightingale & Farid 2022, N = 219, trained	Same, with feedback	59.0%	Training helps a little
Miller et al. 2023, N = 124	White AI still faces	No figure published	AI faces “judged as human more often than actual human faces”
Jakesch et al. 2023, N = 4,600	Self-presentation text	50 to 52%	Human roughness misread as machine

Miller and colleagues called the third row “AI hyperrealism” and found that “people who made the most errors in this task were the most confident (a Dunning-Kruger effect).” Two ceilings travel with it. The effect was found for White AI faces specifically, because generators are trained disproportionately on White faces. And the distinguishing attributes do exist: they “permitted high accuracy using machine learning.” People have the signal and misread it, which is a fact about human heuristics rather than about how clean the output is.

The heuristic runs the other way

Jakesch maps the heuristic feature by feature, and read properly the map convicts my explanation. The paper fits two logistic regressions over the same language features: one predicting whether participants called a self-presentation AI-generated, one predicting whether it actually was. Where the two odds ratios point the same way, the cue works. Where they point opposite ways, people are not merely guessing, they are confidently backwards.

Table: Jakesch, Hancock and Naaman 2023, Table 1. Odds ratios above 1 mean more likely. Model 1 covers 38,866 judgments, model 2 covers 4,690 self-presentations. Rows marked ns were not significant in model 2.

Feature	Rated AI	Actually AI	Reading
Nonsensical content	1.105	1.233	Machine signal, read correctly
Repetitive content	1.083	1.470	Machine signal, read correctly
Conversational words	0.947	0.898	Human signal, read correctly
Grammatical issues	1.048	0.851	Read as machine, actually human
Rare bigrams	1.042	0.666	Read as machine, actually human
Long words	1.034	0.783	Read as machine, actually human
Contractions	0.947	1.134	Read as human, actually machine
First-person pronouns	0.925	0.992 ns	Read as human, diagnostic of nothing
Authentic or spontaneous words	0.946	0.945 ns	Read as human, diagnostic of nothing

Three features are functional cues and that is all: “Only nonsense, repetition, and conversational words were functional cues.” The paper is blunt about the rest. “Participants were 5% more likely to rate self-presentations with grammatical issues as AI-generated, although grammatically flawed self-presentations were, in fact, 15% less likely to be AI-generated.” “Participants often rated self-presentations with long words or rare bigrams as generated by AI, while most self-presentations with long words or rare bigrams had been written by humans.”

Read the direction twice, because it inverts the folk model. Roughness is not what people mistake for humanity. Roughness is what people mistake for a broken machine, and in this corpus roughness was a genuine human tell that they read backwards. The mirror image is on the contractions row: the single most casual-sounding feature on the list was 13% more likely to appear in the generated text, and people took it as evidence of a person. The cues they do read as human, first-person speech and spontaneous-sounding words, are not significantly associated with either source. So the heuristic is not just weak. It is wrong in a specific direction: it penalises the mess that humans actually make and rewards the ease that machines actually produce.

Then the part that should make anyone in advertising sit up. The authors trained classifiers on participants’ judgments and used them to select generated text that hit the cues people read as human. Those optimised versions “were rated as human more often than regular generated self-presentations (65.7% vs. 51.6%)” and, more sharply, were “more likely to be seen as human than self-presentations that were actually written by humans (65.7% vs. 51.7%).” Optimising for the heuristic beats being human at seeming human.

That finding is real and exploitable. It is not a licence for the edit I made. An optimiser aimed at this heuristic would smooth the grammar, shorten the words, drop the rare phrasing and push conversational register up. It would cut the stumble. I had been telling myself the stumble was the video-native form of “more human than human”, and on the only measured version of that heuristic it is closer to the opposite.

One hedge in my own favour, and it does not rescue the argument. Jakesch measured written self-presentations, and “grammatical issues” there is a crowdworker label on text, not a spoken false start in a video. I am not entitled to map one onto the other. I am also not entitled to cite the paper as support when the feature nearest to what I did carries the wrong sign, which is what I was doing.

The mechanism that does fit

It is older and I had never considered it. Corley, MacGregor and Donaldson (Cognition, 2007) recorded ERPs during spoken sentences and found the N400 effect for unpredictable words was reduced when the target was preceded by a hesitation marked by “er”. In a later recognition memory test, words preceded by disfluency were more likely to be remembered. Disfluency raises attention and improves encoding.

That is a claim about the moment of hesitation, not about the speaker’s perceived species, and it is the only mechanism I have found that joins the edit I made to the number that moved. Hook rate asks whether someone is still there at three seconds. A cue documented to raise attention, sitting at second one, is a better account of that than anything about humanness.

The ceilings are real and one of them is a citation ceiling. The experiment used lab sentences rather than paid video, it measured comprehension and memory rather than scroll behaviour, and the sample size is not reported in the PubMed abstract I can link, with the full text behind Cognition’s paywall. A reader following my citation cannot check how many people it ran on, so treat the size of the effect as unverified from here.

Why it happened, and why sample size was not the reason

Three mechanisms stack, in order of size. First, the confounded variable: I changed the measurement window and read the result as evidence about a global property.

Second, non-random impression assignment. Meta’s ad auction page lists estimated action rates as an auction input, “an estimate of whether a particular person engages with or converts from a particular ad.” If both cuts sit in one ad set, delivery routes each toward the people it predicts will engage with that specific ad, so any hook rate gap is partly the system’s own prediction of hook rate returned as impression allocation. More spend does not fix that. It feeds it. Meta’s A/B testing page says as much: “We do not recommend testing informally, such as by turning ad sets or campaigns on and off manually. This can lead to inefficient ad delivery and unreliable test results.”

Third, learning phase instability. Ad sets “exit the learning phase as soon as they can deliver stably. This usually occurs after about 50 results in the week after the ad set’s last significant edit”, and during it “performance is less stable, so your results aren’t necessarily indicative of future performance” (learning phase).

I originally blamed sample size, and at impression level that is almost certainly wrong. For a two-proportion test at 80% power and alpha 0.05 two-sided, normal approximation:

n = [1.95996*sqrt(2*p̄*q̄) + 0.84162*sqrt(p1q1 + p2q2)]^2 / (p1 - p2)^2

At a 25% baseline and a 20% relative lift that is [1.23779 + 0.53061]^2 / 0.0025 = 1,251 impressions per arm.

Table: Impressions per arm needed to detect a hook rate difference at 80% power, alpha 0.05 two-sided, normal approximation. My arithmetic.

Baseline hook rate	+5% relative	+10% relative	+20% relative	+30% relative
20%	25,582	6,509	1,682	771
25%	19,146	4,861	1,251	570
30%	14,855	3,762	963	437

Any live Meta test clears the +20% and +30% columns in hours. Treat those figures as a floor, since impressions are not independent Bernoulli trials and clustering by person and session inflates the true variance. The conclusion holds anyway: my test was almost certainly powered well enough and still could not answer the question. The defect was assignment and confounding, not n.

How to detect this in your own account

Move the imperfection out of the first three seconds. Put the stumble at second eight and re-run. If polish itself is the driver, hook rate should not move. If it collapses back, the effect was positional.
Compare impressions delivered to each cut. Materially unequal delivery inside one ad set is the optimiser’s fingerprint on your result.
Check frequency, because the denominator is impressions.
Confirm both ad sets left the learning phase, roughly 50 results in a week, before reading anything.

To design so it cannot recur: one variable, isolated outside the metric window; separate ad sets through Meta’s A/B test tool so audiences are split rather than competing; seven days minimum and thirty maximum; budget for both arms to exit learning; hypothesis and metric written down before launch. Meta’s page on how winning campaigns are determined lists three reasons its own declared winner might still have turned out more expensive than the alternative: “The length of the study was too short.” “There weren’t enough results to calculate an accurate winner.” “Best practices for study length weren’t followed or met.” Two of the three are duration.

The friction question is becoming a compliance question

Meta’s transparency post, published 3 February 2025 and updated 1 June 2026, describes two labelling regimes, and they are not the same size. For Meta’s own tools: “When an image or video is created or significantly edited with our generative AI creative features in our advertiser marketing tools, a label will appear in the three-dot menu or next to the ‘Sponsored’ label”, and when those in-house tools “result in the inclusion of an AI-generated photorealistic human, the label will appear next to the Sponsored label (not behind the three-dot menu).” For everything else, Meta “will also begin automatically detecting ads created or edited using third-party AI tools through industry-standard signals. When detected, we’ll apply an ‘AI info’ label included in About this ad”, and About this ad is the three-dot menu.

So the loud label is documented for Meta’s in-house generation, and an AI UGC ad built in a third-party tool currently earns the quiet one. I would not plan around that gap. Meta does not say what the industry-standard signals are, and my reading that they are provenance metadata of the C2PA kind is an inference, not something the post states. What that class of metadata does is documented. C2PA 2.4 defines soft bindings, computed from the content rather than the raw bits, and says they “enable digital content to be matched even if the underlying bits differ”, giving as the example “an asset rendition in a different resolution or encoding format.” A re-encode does not shake it off. Detection is moving from perception to metadata, where the viewer’s eye does not participate.

Article 50 of Regulation (EU) 2024/1689 (full text) became applicable on 2 August 2026, four days before I wrote this. Providers of systems generating synthetic audio, image, video or text “shall ensure that the outputs of the AI system are marked in a machine-readable format and detectable as artificially generated or manipulated”, and deployers generating a deep fake “shall disclose that the content has been artificially generated or manipulated”. I am not a lawyer, read the text yourself, but one consequence of the definitions is worth flagging. The Commission’s FAQ gives three cumulative criteria for a deep fake under Article 3(60). The second one decides this, and it is easy to read only half of. In full: “simulated persons, objects, places, entities or events need to resemble someone or something that exists, can plausibly exist or could have plausibly existed in reality.” Stop at “exists” and a fully synthetic presenter resembling no real person looks like it falls outside. The clause that matters is “can plausibly exist”, and a photorealistic synthetic person built to pass as a real one is close to the paradigm case of it. On the Commission’s own wording I read the deployer disclosure duty as reaching an AI UGC ad, with the provider’s marking duty binding the generation tool on top of that. You inherit a marked file you did not choose to mark, and you probably owe the disclosure as well. The Commission has assessed a Code of Practice on transparency of AI-generated content as adequate, so the mechanics are being standardised now rather than eventually.

Which makes the size of the label effect the question worth asking, and the two studies that measure it are routinely misreported. Altay and Gilardi (PNAS Nexus, October 2024, two preregistered experiments, N = 4,976) found that labelling headlines as AI-generated “reduced sharing and accuracy ratings by.17 points [-0.29, -0.04], P = 0.010 on the 6-point scale”, replicated at 0.11 points in Study 2, with sharing intentions alone reaching significance in neither (P = 0.27 and P = 0.085). And the line the citations leave out: “the effect of labeling headlines as AI-generated (2.66pp) was three times smaller than the effect of labeling headlines as false (9.33pp).” The direction is negative and the magnitude is small. Koning and Voorveld (Journal of Interactive Advertising, September 2025, N = 304) found an AI disclosure raised persuasion knowledge, which reduced trust in the ad and the organisation, and also reports a countervailing path through attitudinal persuasion knowledge that increased trust. Citing only the negative half misrepresents it.

What I have now

A test design that separates the variable from the metric window, and a corrected model of my own result. The win did not come from being harder to detect, because polish is not what people detect on. It did not come from feeding the humanness heuristic either, because the measured version of that heuristic reads roughness as a machine and would have deleted my stumble. What is left is attention: a disfluency effect on encoding that has been in the literature since 2007, landing by accident inside the three seconds the metric samples.

That changes the next build instead of confirming it. If the driver is disfluency, the lever is not “leave humanity in”, it is placement, and a hesitation belongs immediately before whatever the viewer most needs to remember, not at second one where I happened to put it. Step one of the recipe above tests that directly, and it can come back against me.

My original conclusion was that the advantage lies in knowing how much humanity to leave in. The sentence is wrong about the noun, and it is wrong in a more useful way than I expected. Where people do carry a shortcut for humanity, it is mapped, unreliable, and it rewards smoothness rather than friction. The friction I left in bought me something else. Calling that attention rather than humanity is the difference between a lever I can aim and a story I liked.

The scaled content abuse policy does not ban AI writing

contact@therezaali.com (Mo RezaAli) — Wed, 05 Aug 2026 00:00:00 +0200

Scaled content abuse is when many pages are generated for the primary purpose of manipulating search rankings and not helping users. This abusive practice is typically focused on creating large amounts of unoriginal content that provides little to no value to users, no matter how it’s created.

Spam policies for Google web search, Google Search Central

Two sentences. That is the whole definition, it is free to read, and it does not contain the word “AI”. That is worth sitting with, given how much of the commentary about it is about nothing else.

Read the last clause again. No matter how it’s created. Method is not the test.

That is my claim, and I think it is right. It is also the claim that draws the sharpest objection I have heard on the subject, which is that the distinction is a lawyer’s distinction with no cash value. This piece states the text, then states that objection as strongly as I can put it, then says where I think it fails.

I am quoting the version of the spam policies page that carried “Last updated 2026-05-15 UTC” in its footer on the day I read it. The same page lists five examples, and only the first mentions generative tools:

Using generative AI tools or other similar tools to generate many pages without adding value for users

Scraping feeds, search results, or other content to generate many pages (including through automated transformations like synonymizing, translating, or other obfuscation techniques), where little value is provided to users

Stitching or combining content from different web pages without adding value

Creating multiple sites with the intent of hiding the scaled nature of the content

Creating many pages where the content makes little or no sense to a reader but contains search keywords

Spam policies for Google web search, Google Search Central

Every one of the five turns on the same two conditions: many pages, and no added value. Neither condition mentions a model. A policy that bans writing with a language model and a policy that bans flooding an index are two different policies, and Google published the second one.

What changed in March 2024, and what did not

Google announced the policy on 5 March 2024, alongside the March 2024 core update, in a post by Chris Nelson. Three spam policies launched that day: expired domain abuse, scaled content abuse, and site reputation abuse. The post is explicit that scaled content abuse was a rewrite rather than a new prohibition:

This new policy builds on our previous spam policy about automatically-generated content, ensuring that we can take action on scaled content abuse as needed, no matter whether content is produced through automation, human efforts, or some combination of human and automated processes.

What web creators should know about our March 2024 core update and new spam policies, Google Search Central Blog

The post anticipates the misreading and answers it directly:

Our long-standing spam policy has been that use of automation, including generative AI, is spam if the primary purpose is manipulating ranking in Search results. The updated policy is in the same spirit of our previous policy and based on the same principle. It’s been expanded to account for more sophisticated scaled content creation methods where it isn’t always clear whether low quality content was created purely through automation.

What web creators should know about our March 2024 core update and new spam policies, Google Search Central Blog

So the widening in 2024 went in the opposite direction from the one people assumed. The old rule named automation and could be dodged by putting a human in the loop. The new rule drops the method entirely and keeps only the purpose and the scale, which means a room of underpaid writers producing the same output is covered too.

Google’s position on AI writing predates all of this by a year, and sits in the FAQ at the foot of the February 2023 post by Danny Sullivan and Chris Nelson rather than in its body. Worth saying, because the difference matters if you go looking. On “Is AI content against Google Search’s guidelines?”:

Appropriate use of AI or automation is not against our guidelines. This means that it is not used to generate content primarily to manipulate search rankings, which is against our spam policies.

Google Search’s guidance about AI-generated content, Google Search Central Blog

And on whether it will rank: “Using AI doesn’t give content any special gains. It’s just content. If it is useful, helpful, original, and satisfies aspects of E-E-A-T, it might do well in Search. If it doesn’t, it might not.”

The objection, at full strength

Here is the case against everything above, and it is better than it usually gets credit for.

Google cannot observe purpose. Purpose is a fact about a publisher’s head, and no crawler reaches it. What Google can observe is the artefact (the page, the site, the rate at which pages appeared), so “primary purpose” has to be inferred from surface features. And the surface features that read as generated at scale are, in 2026, overwhelmingly the features of machine-written text: uniform section structure, uniform length, hedged summary prose, citations that name a publisher and no document.

Then close the loop. Google’s November 2024 site reputation abuse post says the company does not “simply take a site’s claims about how the content was produced at face value.” So the publisher cannot rebut the inference by explaining the process, and a disclosure page will not do it either.

Put those together and the objection lands: a rule that says no matter how it’s created, enforced by inference from features that machine writing produces, is a rule about machine writing in everything but wording. “Method is not the test” is then true on paper and worthless in the hand: a distinction you cannot cash, drawn by the only party who gets to score the exam.

I do not think that argument is silly. I think it is half right, and the half that is wrong is the half that matters.

Where it fails

It fails on the same document it appeals to. If the policy were secretly about detecting machine authorship, the rater instructions would tell raters to look for machine authorship. They tell them the opposite.

Section 4.6.5 of the quality rater guidelines, in the version dated 11 September 2025:

Pages and websites made up of content created at scale with no original content or added value for users, should be rated Lowest, no matter how they are created. Even if you are unsure of the method of creation, e.g. whether or not the page is created using generative AI tools, you should still use the Lowest rating when you strongly suspect scaled content abuse after looking at several pages on the website.

General Guidelines, 11 September 2025, section 4.6.5

Uncertainty about the method does not protect the page, and it is not supposed to resolve into a guess about the method either. The rater is told to judge the output and to stop asking.

Two neighbouring instructions show the same standard cutting against human work. Section 4.6.6 applies the Lowest rating to copied or paraphrased main content “even if the page assigns credit for the content to another source”, which catches hand-built aggregation sites with impeccable attribution and no model anywhere near them. And section 3.2 defines the yardstick as labour: “Effort: Consider the extent to which a human being actively worked to create satisfying content,” with the counter-example given as “the automatic creation of thousands of pages by running existing freely available content through existing translation software without any oversight, manual curation, etc.”

The operative words in that counter-example are oversight and manual curation. Both are human acts, both are performable on machine output, and neither is available at ten thousand pages a week. That is the axis the policy actually runs along, and it is not the same axis as authorship. The two come apart in both directions: a content farm staffed entirely by people fails, and a page a model drafted, a person checked against named sources, and a person put their name on, passes.

So the objection is right that intent is read off the artefact. It is right that you cannot talk your way out. What it gets wrong is the conclusion, because the artefact is being read for evidence of work, not for evidence of a model. Evidence of work is something you can put on the page on purpose. Google’s helpful-content page even tells you one of the things it wants to see there: “Is the use of automation, including AI-generation, self-evident to visitors through disclosures or in other ways?” A disclosure will not launder a thin page. It is still the answer to a question the policy asks out loud.

What the policy prohibits, in plain terms

Three tests, all of which have to fail before you are in trouble:

Scale. One page is not scaled content abuse. The word “many” is in every version of the definition.
Purpose. The pages exist mainly to rank, not mainly to answer someone. The helpful-content page frames this as the “Why” question, and it is the only one of the three that is about intent rather than artefact.
Absence of added value. Section 3.2’s effort standard, above, is what the output gets measured against.

Nothing in there says do not use a model. It says do not ship volume you have not supervised.

One clarification before the end, because the third policy from March 2024 keeps getting folded into this one. Site reputation abuse was rewritten on 19 November 2024 and now reads:

Site reputation abuse is a tactic where third-party content is published on a host site mainly because of that host’s already-established ranking signals, which it has earned primarily from its first-party content. The goal of this tactic is for the content to rank better than it could otherwise on its own.

Spam policies for Google web search, Google Search Central

The November rewrite removed the escape hatch that first-party oversight used to provide; Google’s stated reasoning was that reviewing real cases showed “no amount of first-party involvement alters the fundamental third-party nature of the content.” Different policy, different failure, and a coupons subdirectory is not a scaled content problem.

The practical reading

Draft a page with a model, check it against sources you can name, put your name on the result, and this policy is not about you. Point the same model at a directory and fill it with eight hundred pages nobody asked for, and it was written about you specifically. Since March 2024 it has not mattered whether a person pressed the button.

What I cannot tell you is how any of this is enforced against a particular site. I have no manual action notice to show you, no message from Google, and no interest in inferring a verdict from a traffic chart. The documentation on generative AI content is careful about the same boundary from the other side: the rater guidelines “are not a guide to ranking first in Google,” and raters’ “ratings don’t directly influence ranking.” What the guidelines give you is the standard, written down, by the people who wrote it. Read as a description of what a sceptical stranger is told to look for, it is unusually clear. Read as a rule about robots, it is unreadable, which is why so much of the anxiety about it lands on people it was never drafted for.

Structured data that earns a manual action

contact@therezaali.com (Mo RezaAli) — Wed, 05 Aug 2026 00:00:00 +0200

Two of Apple’s own numbers about one app, read within the same hour on 5 August 2026:

apps.apple.com ratingValue 4.9 reviewCount 1770266
itunes.apple.com/lookup averageUserRating 4.89705 userRatingCount 1769848

418 ratings apart. Both are Apple’s. Both describe the Eventbrite app. Neither is wrong. That is what a live counter looks like when you photograph it twice.

I start there because it is the cheap end of a question that gets expensive quickly: where did the number in your markup come from? AggregateRating is a machine-readable statement, made by you, in your own markup, that many people rated a thing and this is their average. When that statement is false there is nothing left to reinterpret afterwards. No tone, no context, no “that isn’t what I meant.”

Google’s rules for the type are mostly about provenance rather than accuracy, and they are stricter than most people implementing them realise. Six conditions have to hold at once. Below: the two documented rules the six come from, the list itself, and then the list run against four public directories that all publish ratings about the same product.

What a structured data manual action is, and is not

Google’s general structured data guidelines define the penalty precisely:

If your page contains a structured data issue, it can result in a manual action. A structured data manual action means that a page loses eligibility for appearance as a rich result; it doesn’t affect how the page ranks in Google web search.

General structured data guidelines, Google Search Central

So it is narrower than people fear and worse than they assume. Your rankings are untouched. Your stars are gone, and a human at Google has looked at your page and concluded you were being manipulative. The Search Console documentation describes what triggers it:

Google has detected that some of the markup on your pages may be using techniques that are outside our structured data guidelines, for example: marking up content that is invisible to users, marking up irrelevant or misleading content, or other manipulative behavior.

Manual Actions report, Google Search Console Help

Three triggers: invisible, irrelevant or misleading, manipulative. The first is a technical mismatch. The second and third are accusations of bad faith.

Rule one: markup describes what the reader can see

The quality guidelines are blunt about it:

Don’t mark up content that is not visible to readers of the page. For example, if the JSON-LD markup describes a performer, the HTML body must describe that same performer.

Don’t mark up irrelevant or misleading content, such as fake reviews or content unrelated to the focus of a page.

Don’t use structured data to deceive or mislead users. Don’t impersonate any person or organization, or misrepresent your ownership, affiliation, or primary purpose.

General structured data guidelines, Google Search Central

The same page states the relevance test as one sentence: “Your structured data must be a true representation of the page content”. It gives two examples of failure, a live streaming site labelling broadcasts as local events, and a woodworking site labelling instructions as recipes.

The review snippet documentation repeats the visibility rule specifically for ratings:

If you use AggregateRating, users should be able to see that aggregate rating on the page.

Review snippet structured data, Google Search Central

A number that exists only in the JSON-LD is already a violation before anyone asks whether it is accurate.

Rule two: ratings come from users, not from you

This is the rule most often broken by businesses with no intent to deceive, because it feels reasonable to publish your own testimonials. Google closed it in September 2019 and explained why in the announcement:

Reviews that can be perceived as “self-serving” aren’t in the best interest of users. We call reviews “self-serving” when a review about entity A is placed on the website of entity A - either directly in their markup or via an embedded third-party widget.

Making Review Rich Results more helpful, Google Search Central Blog

The current documentation states the consequence and the sourcing requirement:

If the entity that’s being reviewed controls the reviews about itself, their pages that use LocalBusiness or any other type of Organization structured data are ineligible for star review feature. […]

Ratings must be sourced directly from users.

Don’t rely on human editors to create, curate, or compile ratings information for local businesses.

Review snippet structured data, Google Search Central

Two more constraints sit alongside it. AggregateRating is defined by its documentation as markup for “an aggregate evaluation of an item by many people,” which is a statement about where the number came from and not just what shape it has. Schema.org’s own definition of the type is “The average rating based on multiple ratings or reviews.” And “Don’t aggregate reviews or ratings from other websites,” which rules out the obvious workaround of scraping somebody else’s average and marking it up as your own.

The enforcement note is filed where nobody reads it, inside a paragraph about completeness: “users prefer recipes with actual user reviews and genuine star ratings (note that reviews or ratings not by actual users may result in manual action).”

The six conditions

Put the two rules together and an AggregateRating you may legitimately publish has to satisfy all of:

It is about an item, not a category or a list.
The ratings came from users.
Those users were not you.
You did not copy the average from another site.
A human editor did not compile it.
The reader can see the same number on the page.

Conditions 2 to 5 are all provenance. Only 6 is about the page. Which means most of the work of getting this right happens before any markup is written, in the boring question of who supplied the numbers and how you know.

Four directories, one product, one day

Provenance is easier to see in the wild than in the abstract, so: Eventbrite, read across four public sources on 5 August 2026.

On the App Store, Apple’s Eventbrite page carries its own aggregate rating markup: a ratingValue of 4.9 from a reviewCount of 1,770,266, which the page renders for the reader as “1.8M Ratings.” Apple’s public iTunes Lookup API, queried a few minutes later, answered averageUserRating 4.89705 from userRatingCount 1769848. That is the 418-rating gap at the top of this page. The long decimals here are perishable in a way the policy quotations above are not; query the lookup endpoint yourself and you will get a third number. The rounded value, 4.9, is the part that holds still.

On Google Play, the store listing’s own markup reported 4.8 (4.81129264831543) from 273,901 ratings, read the same day.

Two software directories put the organiser-facing product lower, and much closer to each other: Capterra’s Eventbrite reviews page and GetApp’s both showed 4.6 from 5,804 reviews, read in a browser. Those two are flagged here rather than listed as machine-checked sources, for two reasons. Both answer a plain command-line request with HTTP 403 (I tried again while revising this paragraph and got 403 from each), so the build cannot re-verify them on deploy the way it re-verifies everything else, and that pair of figures is the one claim on this page you have to take on my word. They are also both Gartner Digital Markets properties drawing on a shared review pool, which makes the matching number one dataset rather than two independent confirmations.

Across four directories the range is 4.6 to 4.9, and the spread is not the interesting part. The App Store and Play figures describe a consumer app used by attendees. The directory figures describe software bought by event organisers. Different populations, answering different questions, and no single figure among them is entitled to be published as the rating for the product.

Now notice what Apple and Google are doing on those pages. Both publish AggregateRating markup about a product they do not own. Both are within policy, because each is aggregating ratings that real users left with them, and each shows the reader the same number it marks up. The type is not forbidden. Condition 3 is about whether the ratings are about you, not about whether the subject is somebody else. Provenance is what makes the assertion true or false.

The rules, short

Mark up what is on the page. If the reader cannot see it, do not assert it.
Do not rate yourself. Self-serving reviews on LocalBusiness or Organization make the page ineligible for the star feature.
Do not rate other people’s products unless real users handed you the ratings. Ask who those people were. If the answer is a spreadsheet, stop.
Do not copy someone else’s average into your markup.
If a number is a live counter, say so in prose, because the markup cannot.
If you cannot show the reader how the number was produced, write prose instead of schema. Prose invites an argument. AggregateRating does not. It only asserts, and it asserts to machines that have no way to ask you a follow-up question.

This build emits no Review or AggregateRating markup about any third party, and the editorial policy carries that as a standing rule rather than as a one-time cleanup.

Seven things a comparison site publishes that you can check from outside

contact@therezaali.com (Mo RezaAli) — Wed, 05 Aug 2026 00:00:00 +0200

The reasonable objection to an article like this one goes: you cannot audit a comparison site from outside. You are not in the room. You do not know what was tested, who paid for what, or which vendor called which editor. Absent that, a disclosure page and an affiliate notice are about all anyone can ask for, and a reader is better off just judging the verdicts on their merits.

The first half of that is true. The conclusion does not follow.

You cannot audit the verdicts. What you can audit (from outside, without access, in the time it takes to open four tabs) is the process, and you can do it wherever the publisher has committed to enough specifics that a mismatch between what they said they do and what the pages show them doing would be visible to a stranger. That is the whole game. A small number of organisations play it well enough to steal a standard from.

Four publish enough detail to reverse-engineer one: Wirecutter, Consumer Reports, G2 and Tom’s Hardware. Below is the checklist I pulled out of their documents. Each check comes with the way to run it on somebody else’s site, and then with its result on a comparison benchmark of my own, which ranked 19 online registration products and which I deleted in August 2026. Running them together is deliberate. A checklist nobody has ever failed in public is a wish list.

I am grading myself with someone else’s ruler. That is why the confidence label on this page reads medium and not high: the checks are documented, the weighting I put on them is my opinion, and I am the only source for the grades, because I removed the property.

1. A published methodology that the site’s own process actually follows

The first check is not “is there a methodology page.” It is “does the methodology page describe what the site does.”

Wirecutter’s guide template is public. Every guide is supposed to contain “discussions of why you should trust us, who the guide is for, how we picked and tested the products, our picks, flaws (but not dealbreakers) that we encountered, other options worth considering, and the models we tested but don’t recommend” (The Anatomy of a Wirecutter Guide). Thirty seconds and three guides is enough to falsify that.

The same page says why they bother, and the reasoning is the reason I keep citing them: “The Wirecutter journalists who make picks and write guides are dedicated experts. But it’s not enough for us to simply make that assertion and expect readers to trust us. We have to explain why.”

Run it: read the methodology, open two or three actual comparisons, and try to trace one specific verdict back through the stated process. If the trail goes cold, the methodology is decoration.

Result on mine: failed. The benchmark published a scoring methodology. The scoring function did not implement it. Two documents, one public and one executing, disagreeing with each other. Worse: a Math.abs() call in the comparison logic took the absolute value of a score difference, so every product appeared to lead every rival it was set against. Every head-to-head page was a win. A rounding error does not do that. That is a claim generator.

2. Inclusion thresholds stated as numbers, not adjectives

“We only cover the leading products” commits its author to nothing at all.

G2 publishes real thresholds. For a category to get a Grid report, “it must have at least six products with 10+ reviews, and 150+ reviews overall,” and “To qualify for inclusion, a product or service must have at least 10 reviews in the corresponding category” (Research Scoring Methodologies). The same page states an update cadence you could catch them missing: “Placement on live Grids® is updated daily.”

Numbers like that are a commitment with a downside attached, which is what makes them worth reading: they tell you which products were excluded and on what ground, they leave the publisher nowhere to retreat to when a category on the site plainly fails to clear the bar it set for itself, and they convert a claim about editorial rigour into something a stranger with a browser can falsify in an afternoon.

Run it: find the stated minimum, then go hunting for a product on the site that falls under it.

Result on mine: failed. The site claimed “10+ sources including G2 and Capterra.” Its own source-coverage panel (which I built and shipped, on the same page) showed those sources as skipped or errored. The site contradicted itself in full view of the reader.

3. Named testers, dated tests

Consumer Reports names the size of the operation: the Auto Test Center “requires a full-time staff of about 30 — engineers, editors, statisticians, technicians, photographers, videographers, and support staff” (How Consumer Reports Tests Cars). Tom’s Hardware’s storage methodology carries a byline and a date (Chris Ramseyer, published 14 March 2015), and the page still tells you the author “was a senior contributing editor for Tom’s Hardware. He tested and reviewed consumer storage” (How We Test HDDs And SSDs).

That date cuts both ways and it is worth being fair about it. A methodology page from 2015 is old. Tom’s Hardware does maintain separate methodology pages per category: power supplies and network switches each have their own, so the 2015 date is not the whole picture. Even so, an old date you can see beats a fresh-looking page with no date, because it lets you ask the right question. An undated methodology cannot be stale, in the same way an unlabelled jar cannot be past its expiry.

Run it: look for a human name and a date on the methodology, and on the individual comparisons. Then ask whether the date on a comparison squares with the products inside it.

Result on mine: failed. It claimed hourly recomputation. The inputs that recomputation ran against were months old. The timestamp was real and it was measuring the wrong event: when the job ran, not when the data changed. Freshness that cannot decay is not being measured; it is being displayed.

4. Conflicts disclosed at the point of the conflict

A reader deciding between two products on a comparison page is not going to detour through your About page first, so a conflict that lives only there arrives after the decision it should have informed. The FTC’s guidance refuses to let the burden slide somewhere convenient: “the ultimate responsibility for clearly and conspicuously disclosing a material connection rests with the influencer and the brand – not the platform” (FTC’s Endorsement Guides: What People Are Asking; the underlying rules are 16 CFR Part 255).

G2 implements the idea at the row level. Reviewers “who have a business relationship with a vendor (or their competitor) that could create bias (such as a reseller) can share insight, but their reviews do not count toward scoring,” and those reviews carry a flag (How G2 ensures authentic reviews). Its community guidelines add: “If a review is incentivized, G2 will clearly label the review as incentivized” (Community Guidelines).

Run it: find a product whose vendor has any commercial relationship with the publisher, then see whether that relationship is named on the page where the product is ranked.

Result on mine: failed, and this is the one that matters. The benchmark ranked Jumbula. I was Marketing Specialist at Jumbula from September 2019 to March 2026. That relationship appeared nowhere in the benchmark property, while the same property asserted that no vendor could pay to influence rankings. The second statement may well have been true in the narrow sense that no money changed hands. It still worked as misdirection, because the conflict that existed was not the conflict being denied.

5. Test units bought, or the alternative stated plainly

Consumer Reports: “we purchase every vehicle we test from a dealership, just like you do. (Last year we spent more than $2.2 million buying cars.)”

Wirecutter’s arrangement is mixed and the page says so: it requests units or buys them, spends “tens of thousands of dollars buying models for testing” every month, returns or donates what it receives, and is blunt about unsolicited gifts: “We simply do not accept these freebies” (Yes, I Work at Wirecutter. No, We Don’t Get a Bunch of Free Stuff.).

Those are two different policies, arrived at by two organisations with different economics, and the honest part is not that either one is ideal but that both are described precisely enough for a reader to weigh what each arrangement might do to a verdict. Set either beside a sentence like “we maintain independence,” which describes no arrangement, names no counterparty, forecloses no possibility and could sit unchanged on the About page of a publisher taking money from every vendor it covers. “We bought it and here is what it cost” can be wrong. That is what makes it worth something.

Result on mine: not applicable, in a way that is itself the finding. Nothing was purchased because nothing was tested. A benchmark ranking 19 products without running any of them is a scoring exercise over secondary data, and it should have said exactly that in its first sentence.

6. Affiliate revenue disclosed where the link is

Wirecutter puts it in the site’s standing header: “We independently review everything we recommend. We may make money from the links on our site,” and expands on the mechanism in About Wirecutter: “we may get paid commissions on products purchased through our links to retailer sites.” Tom’s Hardware carries the equivalent line on the review pages themselves: “When you purchase through links on our site, we may earn an affiliate commission.”

The FTC guidance makes the same point from the other end: the disclosure belongs in the content, not only beside the link.

Result on mine: passed. There was no affiliate revenue, so there was nothing to disclose. The only line that passes, and it passes for the least impressive reason available.

7. A real corrections log

Not a contact form. A dated, public, standing list of things the publisher got wrong. The New York Times publishes one continuously. A comparison site carrying hundreds of verdicts and zero published corrections is either newly launched or not looking.

Result on mine: failed. There was none. Instead of a correction I deleted the whole property, which is the crudest available remedy and destroys the evidence along with the error. Had I kept a corrections log from the start, I would have had somewhere to put the Math.abs() bug on the day I found it, and you would be able to check my account of it now instead of taking my word.

What the pattern was

The checks that caught me were not the sophisticated ones. Nobody had to audit my statistics.

Three of the seven failures were visible on the page. A source panel that contradicted the sentence above it. A freshness claim contradicted by its own inputs. A comparison table in which nothing ever lost.

Underneath all of it sits one ordinary sequence: I wrote the claims first, because claims are the part you write when you are excited about a project and they cost nothing; I built the system second, against a spec that had already been published to the world as a description of something finished; and then I never went back, not once in months, to run the boring comparison between the two documents that would have taken an afternoon and caught every failure above. Automation widens that gap and then hides it, because output keeps arriving and output keeps looking like output, which is also how the publishing side of this domain ran for six months.

Which reduces the checklist to a single question, asked of any claim you can see: what would I expect to find on this site if that sentence were false? Then go look for it. On my benchmark it took under a minute.

Perplexity’s own documentation says its fetcher ignores robots.txt

contact@therezaali.com (Mo RezaAli) — Wed, 05 Aug 2026 00:00:00 +0200

“this fetcher generally ignores robots.txt rules”

That is Perplexity, describing Perplexity-User on its own crawler documentation page. Not a critic’s characterisation. The vendor’s.

It draws the line most writing about AI crawlers smudges. Crawlers that gather training data are governed by robots.txt and their operators say so. Fetchers that run because a human has just typed a question and is waiting for an answer largely are not, by design, and their operators say that too. Same documents, a few paragraphs apart. Carry that distinction and most advice about “blocking AI” sorts itself into the part that works and the part that cannot.

Short piece. Three things are documented, one is not, one is proposed.

Documented: who fetches what

OpenAI documents four agents on Overview of OpenAI Crawlers. GPTBot is “used to make our generative AI foundation models more useful and safe.” OAI-SearchBot is “used to surface websites in search results in ChatGPT’s search features.” Of the third, the page says OpenAI “also uses ChatGPT-User for certain user actions in ChatGPT and Custom GPTs,” and that it “is not used for crawling the web in an automatic fashion.” OAI-AdsBot validates the safety of pages submitted as ads. One operational figure: “it can take ~24 hours from a site’s robots.txt update for our systems to adjust.” IP ranges are published as JSON at openai.com/gptbot.json, openai.com/searchbot.json and openai.com/chatgpt-user.json, all three of which returned 200 when I fetched them on 5 August 2026.

Anthropic documents three: ClaudeBot for training data, Claude-User for user-initiated fetches, and Claude-SearchBot for search indexing. The support page states that “Anthropic’s Bots respect ‘do not crawl’ signals by honoring industry standard directives in robots.txt,” gives a Crawl-delay: 1 example, and links claude.com/crawling/bots.json for source-IP verification (Does Anthropic crawl data from the web?).

Google-Extended is the one people misread, and the misreading has a consequence. It is not a crawler. Google’s own documentation: “Google-Extended doesn’t have a separate HTTP request user agent string. Crawling is done with existing Google user agent strings; the robots.txt user-agent token is used in a control capacity” (List of Google’s common crawlers, last updated 14 July 2026). You will never find Google-Extended in your access logs, because there is nothing there to find. It is a switch, not a visitor. Block it, then check your logs for a change. You will see none, and conclude, wrongly, that it was ignored.

Perplexity documents PerplexityBot, which is “designed to surface and link websites in search results on Perplexity. It is not used to crawl content for AI foundation models,” alongside the Perplexity-User line at the top of this page (Perplexity Crawlers).

Documented: what robots.txt is not

The Robots Exclusion Protocol is a real standard: RFC 9309, Koster, Illyes, Zeller and Sassman, September 2022. Three provisions are worth knowing before you lean on it.

It is not a security control. The Security Considerations section says so directly: “The Robots Exclusion Protocol is not a substitute for valid content security measures.” Anything reachable without authentication is reachable.

It is cached. “Crawlers SHOULD NOT use the cached version for more than 24 hours,” which lines up with OpenAI’s stated ~24-hour adjustment window. Deploying a robots.txt change and expecting immediate effect is a category error.

It is size-limited. “The parsing limit MUST be at least 500 kibibytes.” Rules past that boundary have no guarantee of being read at all.

Not documented: whether any of them run JavaScript

The question I get asked most, and none of the operators has answered it. On 5 August 2026 I fetched the four crawler documentation pages named above and searched their rendered text for the string “JavaScript”. Anthropic’s, Google’s and Perplexity’s contain it zero times. OpenAI’s contains it once, in the left-hand navigation tree, as a link to an unrelated “JavaScript Pixel” page in the advertising docs.

The only party documenting rendering is Google Search, and it documents it for Googlebot rather than for anything AI-specific: “Googlebot queues all pages with a 200 HTTP status code for rendering, unless a robots meta tag or header tells Google not to index the page. The page may stay on this queue for a few seconds, but it can take longer than that” (Understand the JavaScript SEO basics, last updated 4 March 2026).

So the defensible statement is a narrow one: rendering is documented for Googlebot and undocumented everywhere else. My practical conclusion (inference, not documentation) is to put the substance in the initial HTML response, since that is the only path every fetcher demonstrably has.

Proposed, not adopted: llms.txt

llms.txt was published by Jeremy Howard on 3 September 2024, and it describes itself accurately, which is the part people skip. The site’s own subtitle: “A proposal to standardise on using an /llms.txt file to provide information to help LLMs use a website at inference time.” Further down: “the llms.txt specification is open for community input.” It specifies a Markdown file with an H1 project name, an optional summary blockquote, and H2 sections containing lists of links.

A proposal. Rather than assert that no major provider has adopted it, here is the check, which takes about two minutes: of the four crawler documentation pages cited above, Anthropic’s and Google’s contain the string “llms.txt” zero times. OpenAI’s and Perplexity’s do contain it. In both cases every occurrence is a link to that documentation site’s own /llms.txt index, offered to agents reading the docs. developers.openai.com/llms.txt and docs.perplexity.ai/llms.txt both return 200.

That distinction carries the whole section. Two providers publish an llms.txt for their documentation. Neither one’s crawler documentation says its crawlers read yours. Serving a file and consuming a file are different behaviours, and the claim “OpenAI supports llms.txt” collapses them.

None of which is an argument against publishing one. It costs almost nothing, and a Markdown index is genuinely useful to an agent that fetches it. The argument is against believing that filing it does anything by itself, and against the second failure mode, which is writing claims into a machine-readable file that your pages do not support. Two of mine did that. /.well-known/ai-plugin.json pointed at an openapi.yaml of 189 bytes whose paths object was empty: a plugin manifest advertising an API with no operations in it, filed at the well-known path where machines go looking. Searching OpenAI’s current Plugins documentation on 5 August 2026, the strings ai-plugin.json and well-known appear zero times. The page describes something built on skills and the Model Context Protocol instead. A dead spec, implemented badly. The ai.txt and llms.txt next to it were worse, because they were live and false: they advertised “19 products tracked, 171 comparison pages, updated hourly” and “public review data from 10+ sources,” about a benchmark that fails most of its own published promises.

The direction of travel on access runs the opposite way from manifests. On 1 July 2025 Cloudflare announced it was “changing the default to block AI crawlers unless they pay creators for their content”, citing referral ratios: “With OpenAI, it’s 750 times more difficult to get traffic than it was with the Google of old. With Anthropic, it’s 30,000 times more difficult.” Whatever you make of the policy, “what will read my site” is now answered partly by your CDN and not only by your robots file.

Inference: what actually gets you cited

Argued, not documented. Read it accordingly.

Nobody outside the labs knows how retrieval weights or selects a page. Every provider above documents which agent fetches what and how to block it. None documents what happens next, so anyone offering you the ranking factors for LLM citation is describing a system whose operators have published nothing whatsoever about its ranking factors.

What can be reasoned about is narrower. To repeat a page without hedging, a model needs a specific claim, attached to a named source and a date, that resolves to something a reader can open. A manifest describing you favourably supplies none of that: it is self-assertion, and self-assertion is the cheapest text there is: the exact output a language model generates without limit at no cost, which is why it carries no weight coming from you either.

So the durable move is to be the primary source of a fact somebody needs. Publish the number you measured, the method you used, the date. A fact available only from you can be cited; a restatement of what a hundred other pages already say cannot, and no file in your web root changes that. I am fairly confident about that and I cannot prove it. Inference, labelled as inference. That is the arrangement I should have used the first time.

Page 6 tells the raters their ratings do not move rankings

contact@therezaali.com (Mo RezaAli) — Wed, 05 Aug 2026 00:00:00 +0200

The file is 182 pages, it is called General Guidelines, its header carries the date 11 September 2025, Google links to it from the helpful-content documentation, and it costs nothing to download.

Page 6 tells the raters, before they rate anything, that their work does not move rankings. Which is roughly the opposite of what gets asserted on this document’s behalf.

So: quotes rather than summary. Section numbers refer to that version, and every block quote below is the document’s own wording.

The hierarchy, verbatim

Section 3.4 names the hierarchy before it defines a single term:

Experience, Expertise, Authoritativeness and Trust (E-E-A-T) are all important considerations in PQ rating. The most important member at the center of the E-E-A-T family is Trust.

General Guidelines, 11 September 2025, section 3.4

Trust is defined first, and defined as a property of the page rather than of the author:

Trust: Consider the extent to which the page is accurate, honest, safe, and reliable.

General Guidelines, section 3.4

The other three arrive as supporting evidence for that judgement. The document’s own framing is that they “can support your assessment of Trust”:

Experience: Consider the extent to which the content creator has the necessary first-hand or life experience for the topic.

Expertise: Consider the extent to which the content creator has the necessary knowledge or skill for the topic.

Authoritativeness: Consider the extent to which the content creator or the website is known as a go-to source for the topic.

General Guidelines, section 3.4

Then the sentence that gets left out of every summary, stated later in the same section with an example that is hard to misread:

Trust is the most important member of the E-E-A-T family because untrustworthy pages have low E-E-A-T no matter how Experienced, Expert, or Authoritative they may seem. For example, a financial scam is untrustworthy, even if the content creator is a highly experienced and expert scammer who is considered the go-to on running scams!

General Guidelines, section 3.4

Max out three of the four letters and the page can still score Lowest. The acronym does not describe four boxes of equal weight, whatever the diagrams say. It is one criterion and three kinds of supporting evidence, and Google’s public documentation says as much in a line: “Of these aspects, trust is most important. The others contribute to trust, but content doesn’t necessarily have to demonstrate all of them.”

What page 6 tells the raters about their own work

Section 0.1, before any rating happens:

No single rating can directly impact how a particular webpage, website, or result appears in Google Search, nor can it cause specific webpages, websites, or results to move up or down on the search results page. Using ratings to position results on the search results page would not be feasible, as humans could never individually rate each page on the open web.

Instead, ratings are used to measure how effectively search engines are working to deliver helpful content to people around the world.

General Guidelines, section 0.1

Google repeats it in three places outside the PDF, which I checked one at a time.

The helpful-content page: “Search raters have no control over how pages rank. Rater data is not used directly in our ranking algorithms. Rather, we use them as a restaurant might get feedback cards from diners.” The generative-AI guidance page says the guidelines “are not a guide to ranking first in Google,” and that raters’ “ratings don’t directly influence ranking.” The December 2022 post that added the second E, on the raters: “they don’t directly influence ranking.”

The same helpful-content page also disposes of the phrase everyone uses, in a subordinate clause, which is presumably why nobody quotes it: “While E-E-A-T itself isn’t a specific ranking factor, using a mix of factors that can identify content with good E-E-A-T is useful.” Note the shape of that sentence. Not a factor. A mix of factors that can identify the thing.

Four readings the document does not support

“E-E-A-T is a ranking factor.” Covered above, so only the part that usually gets dropped: that mix of factors is weighted more heavily on topics “that could significantly impact the health, financial stability, or safety of people, or the welfare or well-being of society.” Which is the definition of YMYL, given in section 2.3.

“You need formal credentials.” For a whole class of pages, section 3.4.1 says the opposite: “Pages that share first-hand life experience on clear YMYL topics may be considered to have high E-E-A-T as long as the content is trustworthy, safe, and consistent with well-established expert consensus.” The December 2022 post drew the same line with tax. Filing your return correctly is a question for “an expert in the field of accounting.” Choosing tax software, the post suggests, might send you to “a forum discussion from people who have experience with different services” instead.

“Expertise is the most important letter.” Section 3.4 says Trust is, twice, and gives the expert-scammer example specifically to close this reading.

“A small site with no press coverage looks bad.” Section 3.3.5 addresses this directly:

However, small websites may have little or no reputation information. This is not indicative of high or low quality. […] A lack of reputation about people who post personal content is neither a positive nor a negative sign in your assessment of the page.

General Guidelines, section 3.3.5

Absence of coverage is scored as absence, not as a negative. That is a meaningfully different instruction from the one most SEO advice implies.

What the document asks of a small site

Section 2.5.2 is titled Finding Who is Responsible for the Website and Who Created the Content on the Page, and it sets two separate requirements:

Every page belongs to a website, and it should be clear:

Who (what individuals, company, business, organization, government agency, etc.) is responsible for the website.

Who (what individuals, company, business, organization, government agency, etc.) created the content on the page you are evaluating.

General Guidelines, section 2.5.2

Two questions, asked separately. Who runs the site, and who wrote this page. On a one-person site both answers are the same name, which is the rare structural advantage of being one person: nothing to navigate, nothing to trace up an org chart.

Section 2.5.3 then grants personal sites an allowance that stores and banks do not get:

Occasionally, you may encounter a website or content creator with a legitimate reason for anonymity. For example, personal websites may omit personal contact information such as an individual’s home address or phone number.

General Guidelines, section 2.5.3

No street address required, then. Identifiable and reachable, yes. Those are two different bars, and running them together is how people end up either publishing their home address or hiding behind a contact form with no name on it.

Section 3.4 names three sources of evidence a rater is told to weigh:

What the website or content creators say about themselves

What others say about the website or content creators

What is visible on the page, including the Main Content and sections such as reviews and comments

General Guidelines, section 3.4

Only the first of those is under a publisher’s control. The second belongs to other people, and the only way to control it is to fake it, which section 3.3.5 has already declined to reward, since it scores an absent reputation as neutral. The third is whatever is actually on the page.

The section closes on a warning that is easy to skim past:

Important: The website or content creator may not be a trustworthy source if there is a clear conflict of interest.

General Guidelines, section 3.4

Google’s example is a manufacturer reviewing its own product. Generalise the shape (someone with a stake evaluating the thing they have a stake in) and it covers an affiliate link inside a “best of” list, a review written by the vendor’s own agency, and an undisclosed employment history sitting inside a software comparison. The document does not treat any of those as a footnote. It treats them as grounds for distrusting the source outright.

How to read the thing

Not as a ranking manual. Page 6 rules that out.

Read it instead as the most detailed public account anyone has of what a trained stranger, paid to be sceptical, is told to check when they land on a page cold. Read that way, the instructions for a small site come out short.

Be identifiable: a real name, attached to a real role, at an organisation that exists. Be reachable, which section 2.5.3 is careful to say does not mean publishing a home address. Separate what you have used from what you have only read about, because the document defines Experience and Expertise apart from each other and puts both underneath Trust. Put the conflict on the page before the reader finds it.

And if nobody has ever written about you, section 3.3.5 says that is worth nothing. Not minus something. Nothing. Which is also not an invitation to go and manufacture some.

LCP is four consecutive spans of time, not one number

contact@therezaali.com (Mo RezaAli) — Wed, 05 Aug 2026 00:00:00 +0200

LCP is usually discussed as though it were a single number you make smaller. It is not. It is four consecutive spans of time laid end to end, and until you know which of the four is eating your budget you are guessing at which fix to apply.

Largest Contentful Paint reports the render time of the largest image or text block visible in the viewport. web.dev’s LCP page gives the target: “sites should strive to have Largest Contentful Paint of 2.5 seconds or less,” and it is explicit about where to read that from: “a good threshold to measure is the 75th percentile of page loads, segmented across mobile and desktop devices.”

The percentile is the harder half of that sentence. Your 75th-percentile visitor is not your average one. Old phone, poor connection, cold cache. A comfortable median tells you very little about the number Google is actually scoring, and the gap between the two is where most people’s optimism lives.

So the useful question is never “is my LCP good” but “which of the four spans is spending the budget, on the quarter of visits I have not been looking at”. That distinction is the whole point, because a 3.4-second LCP caused by a font that blocks a headline and a 3.4-second LCP caused by an uncompressed hero on a slow connection are the same number describing two entirely different repairs.

One thing to fix before the model. Text counts. LCP candidates are <img>, <image> inside <svg>, <video>, an element with a background image loaded via url(), and block-level elements containing text nodes or inline text children. So if your hero is a headline in a webfont, the font file is on the LCP path and you should treat it exactly as seriously as an image.

The four sub-parts

web.dev’s optimization guide breaks LCP into four consecutive parts and gives a target share for each.

The four LCP sub-parts and their target share of total LCP, per web.dev's Optimize Largest Contentful Paint guide. The guide notes these are guidelines rather than strict rules.
Sub-part	What it is	Target share of LCP
Time to first byte	until the first byte of the HTML response	~40%
Resource load delay	from TTFB until the LCP resource starts loading	<10%
Resource load duration	transferring the LCP resource	~40%
Element render delay	from resource loaded until the element paints	<10%

The two small numbers are the diagnostic ones. web.dev flags this itself: of the four sub-parts, two have the word “delay” in their names, which “is a clue that you want to get these times as close to zero as possible.” Nothing useful happens during a delay. If either delay is large you have a discovery problem or a blocking problem, and those are usually the cheap fixes. TTFB and load duration are real work, which is why they get 40% each.

For a static site behind a CDN, hosting has already largely solved TTFB. Which leaves load delay plus load duration. On most pages both of those are properties of exactly one image.

The guide adds a warning worth repeating, because the table invites the mistake: “Given the 2.5 second target for LCP, it may be tempting to try to convert these percentages into absolute numbers, but that is not recommended.”

The audit, in the order I run it

Four checks, each one a command you can run against your own repository in a few seconds. The sample outputs below come from this site’s own history at 22b7c22d^ (the commit before the machine-generated corpus was deleted), because that is a codebase whose lines I can quote in full. It is private, so the file facts are mine and the reasoning from them is Google’s, linked throughout.

Check 1: weigh everything the templates can put above the fold

git ls-tree -r --long 22b7c22d^ -- static/images/rezaali-fallback.png
# 100644 blob 5932e974… 2074882 static/images/rezaali-fallback.png

2,074,882 bytes. 1536 × 1024. Non-interlaced 8-bit RGB PNG. Its visible content is the word LOADING.. in white on a near-black field, with the word SIGNALS under it.

A placeholder that never got replaced is an ordinary accident. What turns it structural is where the path is written down, so the second half of this check is to grep your archetypes and partials for it:

git show 22b7c22d^:archetypes/post.md | grep image:
# image: "images/rezaali-fallback.png"

Every post created from that archetype inherited the line, which made two megabytes of PNG the default cover image. A default is the one thing nobody looks at.

git grep -l 'images/rezaali-fallback.png' 22b7c22d^ -- content/post/insights \
 | grep -vE '\.(it|ar|zh-cn)\.md$' | wc -l
# 229

229 English posts, each served through a template that rendered the cover as an <img> carrying loading="lazy". On the branch that fires for a static path rather than a Hugo page resource, it also carried no width and no height at all.

The repair. Not “compress the placeholder.” Do not have a placeholder. Ship no image, let text be the LCP element, and pay a few kilobytes instead of two megabytes. PNG is lossless, and lossless spends its bytes faithfully preserving photographic detail that a flat graphic with three colours does not have, so the format was wrong for the content and the content was wrong for the page. When a real photograph is ready, encode it as WebP or AVIF. Can I use maintains current support tables for both, and WebP is the safer floor if you are shipping one format with no <picture> fallback chain.

Check 2: find out whether your hero is lazy-loaded

git grep -n 'class="portrait"' 22b7c22d^ -- layouts/index.html

<img class="portrait" src="{{ "images/mo-rezaali.jpeg" | relURL }}"
 alt="Portrait of Mo RezaAli" loading="lazy" />

A homepage hero portrait: 311,372 bytes, inside the first screen, carrying an instruction to the browser not to hurry. No width, no height, no srcset, no fetchpriority.

loading="lazy" defers an image until it approaches the viewport. Correct below the fold. Exactly wrong at the top, because browser-level lazy loading has to wait for layout before it can decide whether to fetch, which pushes the request behind CSS. web.dev’s optimization guide states it without hedging: “Never lazy-load your LCP image, as that will always lead to unnecessary resource load delay, and will have a negative impact on LCP.”

Read that against the table. Lazy-loading your hero takes the sub-part with a target under 10% and makes it the largest line in the budget.

The repair. loading="lazy" on every image except the LCP candidate, which gets fetchpriority="high" instead.

Check 3: confirm you have any responsive images at all

git grep -l srcset 22b7c22d^ -- layouts static
# (no output)

No output is the finding. Not a single responsive image anywhere on the site: a phone at 400 CSS pixels wide downloaded the same file as a 5K desktop. Since load duration is around 40% of the budget, and duration is bytes over bandwidth, serving four times the necessary bytes to the slowest connections is the most direct route there is to failing at the 75th percentile.

The repair.

<img src="/img/mo-800.webp"
 srcset="/img/mo-400.webp 400w,
 /img/mo-800.webp 800w,
 /img/mo-1200.webp 1200w"
 sizes="(max-width: 40rem) 90vw, 400px"
 width="800" height="800"
 alt="Mo RezaAli"
 fetchpriority="high"
 decoding="async">

Two details are not optional. width and height must be there so the browser can reserve the box before the bytes land; a missing intrinsic size is a layout shift, and CLS has its own threshold of 0.1 at the 75th percentile. And sizes has to describe your real CSS layout. If sizes lies, the browser picks the wrong candidate and you have made the page worse than a single fixed image would have been.

fetchpriority raises or lowers a resource’s priority within its class. web.dev’s Fetch Priority guide covers both directions: high for the hero, low for images you know are decorative.

When the image is not in the HTML

fetchpriority only helps if the preload scanner can see the image. A CSS background or a JavaScript-injected hero cannot be discovered until the stylesheet or the script has been parsed, and that wait is the definition of a large resource load delay. Preload it, responsively, so you do not undo the srcset work:

<link rel="preload" as="image"
 href="/img/mo-800.webp"
 imagesrcset="/img/mo-400.webp 400w,
 /img/mo-800.webp 800w,
 /img/mo-1200.webp 1200w"
 imagesizes="(max-width: 40rem) 90vw, 400px"
 fetchpriority="high">

imagesrcset and imagesizes mirror the <img> attributes; web.dev’s guide to preloading responsive images covers the pairing. A plain href preload sitting next to a responsive <img> is a classic own goal. It fetches a fifth copy at the wrong size.

Where you can get it, the better answer is to put the hero in an <img> tag in the HTML and delete the preload entirely. Preload is a workaround for resources the parser cannot find.

Check 4: read the first three lines of your head partial

git show 22b7c22d^:layouts/partials/mo/head.html | head -3

{{- $cssBust := now.Unix -}}
<link rel="stylesheet" href="{{ "css/new-face-mo.css" | relURL }}?v={{ $cssBust }}">
<link rel="stylesheet" href="{{ "css/mo-site.css" | relURL }}?v={{ $cssBust }}">

now.Unix is the current Unix time. As a cache key it has one disqualifying property: it is not derived from the file’s contents. Two consequences follow.

Every deploy changes the query string on every stylesheet whether or not a byte of CSS has changed, so returning visitors re-download identical files forever. Cache-busting that fires unconditionally is cache-disabling with extra steps.

Worse, the value is computed while each template renders rather than once per build. Pages rendered on either side of a second boundary get different values, so one deploy can serve three distinct URLs for one physical file. Three URLs, three cache entries, and a visitor moving from page to page cannot reuse the stylesheet they fetched a moment earlier. Cache hits are not unlikely here. They are structurally impossible.

The repair is a content hash. Hugo’s resources.Fingerprint (“Cryptographically hashes the content of the given resource.”) rewrites the filename to include the hash, defaulting to SHA-256:

{{ with resources.Get "css/site.css" }}
 {{ $css := . | minify | fingerprint "sha256" }}
 <link rel="stylesheet"
 href="{{ $css.RelPermalink }}"
 integrity="{{ $css.Data.Integrity }}"
 crossorigin="anonymous">
{{ end }}

The URL now changes if and only if the CSS changes, and every page in a build gets the same URL because the input is the same. Because the URL is content-addressed, the file can then be cached permanently. On Cloudflare Pages, via static/_headers:

/css/*
 Cache-Control: public, max-age=31536000, immutable

immutable tells the browser not to revalidate during the freshness lifetime, which is safe precisely because a changed file gets a different name. MDN’s Cache-Control reference documents the directive. Do not put this on your HTML. HTML has to stay revalidatable or nobody ever sees a new deploy.

Fonts: the text-LCP path

If your LCP element is a headline rather than an image, the webfont is on the critical path and the failure mode is invisible text.

font-display: swap renders the fallback face immediately and swaps when the webfont arrives; MDN’s font-display reference documents the timeline values. swap buys you text that is never invisible at the cost of a flash of fallback. That is the right trade when the text is the LCP element, because under block the browser hides the text during its block period and your LCP sits waiting on a font file.

The cost is layout shift. Fallback and webfont almost never share metrics. Three mitigations, ordered by how much I trust them:

Self-host the font and <link rel="preload" as="font" type="font/woff2" crossorigin> the specific faces used above the fold. The crossorigin attribute is required even same-origin. Omit it and the browser fetches the file twice.
Ship only the faces you actually use. Declaring a family you do not serve guarantees a fallback render for no benefit.
Tune the fallback with size-adjust, ascent-override and descent-override in a @font-face block for the local fallback family. This genuinely reduces shift, but the numbers have to be measured per font pair, I have not measured mine, and copying someone else’s values is worse than skipping the step.

web.dev’s font best-practices guide goes deeper on loading strategy.

What this walkthrough does not give you

A number. There is no field data here: this site has never had the traffic for a Chrome UX Report entry, and I removed its analytics, so there is no measured before-and-after on this page and I am not going to manufacture one.

What the four checks give you is a list of file facts (a byte weight, a loading attribute, an absent srcset, a timestamp in a template), each with the command that produces it. Those are cheap to gather and hard to argue with, which makes them a good place to start. They are not a substitute for measurement. If you want a measured number for your own site, use CrUX for field data and Lighthouse for lab data, and when the two disagree, believe the field.

I removed Google Analytics rather than implement consent mode, and the tag is still live

contact@therezaali.com (Mo RezaAli) — Wed, 05 Aug 2026 00:00:00 +0200

I took the Google Analytics tag out of this build rather than implement consent mode on it. What follows is the whole of that reasoning, in the order I worked through it, ending with the least flattering part: as I write, the tag is still live on the deployed site.

Start from the consequence, because the consequence is the part nobody argues about. Send Google no consent signal for a visitor in the EEA and that visitor stops appearing in the audiences your linked advertising products use. Not fined. Subtracted.

That has been the arrangement since early March 2024, Google documents it in plain words, and it changes what kind of question this is. Consent mode gets written about as a compliance feature. It behaves like a data-loss mechanism with a legal trigger bolted on, which is why the decision a small site actually faces (whether to run the tag at all) is not the same decision as configuring it correctly.

The legal shape, in two sentences

Two instruments apply and people routinely conflate them.

The ePrivacy Directive governs putting things onto, or reading things off, a visitor’s device. Article 5(3) of the consolidated text says that storing information, or gaining access to information already stored, in a user’s terminal equipment “is only allowed on condition that the subscriber or user concerned has given his or her consent”, with a carve-out for whatever is strictly necessary to deliver the service the user asked for.

Analytics is not strictly necessary to deliver a blog post.

The EDPB’s Guidelines 2/2023 on the technical scope of Article 5(3), version 2.0, adopted 16 October 2024, exist largely because people read “cookie law” as “law about cookies.” The article is about storage and access on a device. The mechanism is irrelevant to it.

The GDPR then governs what counts as consent, and what you may do with the personal data afterwards. Article 4(11) defines consent as a “freely given, specific, informed and unambiguous indication of the data subject’s wishes by which he or she, by a statement or by a clear affirmative action, signifies agreement.” Article 7(3) adds the right to withdraw at any time, and requires that the person be told about that right before they give consent in the first place. For what all of that means in practice, the reference text is the EDPB’s Guidelines 05/2020, adopted 4 May 2020.

So: consent before the tag stores anything, and a real withdrawal path afterwards. Consent mode is Google’s mechanism for the first half.

Consent mode is an API in gtag.js and Google Tag Manager. Your consent banner tells Google’s tags what the visitor agreed to, and the tags change behaviour accordingly. Google’s tag platform documentation lists seven parameters. Four matter for advertising and measurement:

ad_storage: “Enables storage, such as cookies (web) or device identifiers (apps), related to advertising.”
analytics_storage: the same, “related to analytics, for example, visit duration.”
ad_user_data: “Sets consent for sending user data to Google for online advertising purposes.”
ad_personalization: “Sets consent for personalized advertising.”

The last two are the “v2” part. Google’s Ads Help page on EEA traffic states the requirement plainly: to keep using the tags and SDKs for measurement, ad personalization and remarketing, you must collect consent from end users based in the EEA and share those signals with Google. The Analytics-side page is blunter about the consequence of doing nothing. As of early March 2024, if you take no action, only end users outside the EEA are included in audiences used by your linked advertising products.

That is the mechanism behind the sentence this article opened with, quoted from the party that operates it.

Basic versus advanced

Google’s documentation distinguishes two implementations, and the difference is the entire compliance question.

Basic mode prevents Google tags from loading at all until the visitor interacts with the banner. Nothing is transmitted to Google before that interaction. Google describes the modelling you get back as the “general model,” which is the less detailed one.

Advanced mode loads Google tags when the page opens, sets default consent states of denied, and sends cookieless pings until and unless the visitor grants consent. Google says this “enables improved modeling compared to the Basic one as it provides an advertiser-specific model.”

Advanced mode is what most agencies install, because it recovers more data. It is also the mode where a tag runs before consent. Whether the cookieless ping is lawful under Article 5(3) is a genuinely contested question, and the honest answer for a one-person site is that you should not be the person testing it. Basic mode is the conservative reading and it is the one I would pick if I ran Google tags at all.

The correct minimal implementation

If you do run it, the ordering matters more than the syntax. The default call has to execute before the Google tag loads, or the tag has already acted.

<!-- 1. Defaults. Must run BEFORE gtag.js is requested. -->
<script>
 window.dataLayer = window.dataLayer || [];
 function gtag(){dataLayer.push(arguments);}

 gtag('consent', 'default', {
 'ad_storage': 'denied',
 'ad_user_data': 'denied',
 'ad_personalization':'denied',
 'analytics_storage': 'denied',
 'wait_for_update': 500
 });
</script>

<!-- 2. The Google tag. -->
<script async src="https://www.googletagmanager.com/gtag/js?id=TAG_ID"></script>
<script>
 gtag('js', new Date());
 gtag('config', 'TAG_ID');
</script>

wait_for_update gives an asynchronous banner a window, in milliseconds, to answer before the tag proceeds on the defaults.

Then, and only from inside the banner’s accept handler:

gtag('consent', 'update', {
 'ad_storage': 'granted',
 'ad_user_data': 'granted',
 'ad_personalization':'granted',
 'analytics_storage': 'granted'
});

The default call also takes a region array of ISO 3166-2 codes, so you can deny by default in the EEA and behave differently elsewhere. I would not bother. One default, denied everywhere, is fewer branches to get wrong.

The part that is not code

Here is why I stopped. Google’s own EU user consent policy (the policy your account is bound by, separate from the tag documentation) requires that you obtain consent for “the use of cookies or other local storage where legally required” and for “the collection, sharing, and use of personal data for personalization of ads.” It then requires you to “retain records of consent given by end users,” to “provide end users with clear instructions for revocation of consent,” and to “clearly identify each party that may collect, receive, or use end users’ personal data.”

Read that as an engineering backlog for a personal site:

A banner with a reject button no harder to press than accept.
A consent record store: what was consented to, when, under which banner version, retained and retrievable.
A withdrawal UI reachable from every page, plus the logic to actually revoke and delete.
A privacy policy naming Google as a recipient, listing the purposes, and stating retention.
Ongoing maintenance of all four as the policies change.

Every item is real work, and every item is a place a solo operator ships a bug that nobody reviews. The value I was getting in exchange was a pageview count. That trade is bad, and it stays bad no matter how well I implement it.

Cookieless alternatives, as documented by their vendors

I checked these against the vendors’ own pages rather than comparison posts.

Cloudflare Web Analytics. Two documents, five and a half years apart. The launch post of 29 September 2020 says “We don’t use any client-side state, like cookies or localStorage, for the purposes of tracking users,” and, immediately after, “we don’t ‘fingerprint’ individuals via their IP address, User Agent string, or any other data for the purpose of displaying analytics.” The current documentation for the beacon that actually executes in your visitor’s browser, last updated 16 April 2026, is narrower and therefore more useful to check against: “The RUM beacon script does not store any data in the browser or access any storage data, such as cookies, localStorage, sessionStorage, IP address, or IndexedDB.”

Two things to know before you reach for it. Cloudflare’s “visit” is defined as “a successful page view that has an HTTP referer that doesn’t match the hostname of the request,” which is not a session and should not be read as one. And on the free plan the beacon is switched on for you with Europe carved out. The docs say “Free customers have RUM enabled automatically, with EU traffic excluded, and can switch it off if they prefer.” For a site whose readers are mostly in the EEA, that is most of the audience missing by default.

Plausible. The data policy states “We don’t use cookies, we don’t generate persistent identifiers and we don’t collect or store personal data that can be used to identify individuals,” and “Raw IP addresses and User-Agent data are never stored.” Daily uniques are counted with hash(daily_salt + website_domain + ip_address + user_agent), with the salt rotating every 24 hours. Plausible states you do not need cookie banners for analytics with it.

Fathom. The data page states “Our technology doesn’t use cookies, so you won’t need an annoying cookie consent banner,” and that raw IP addresses are stored only for security purposes and are not part of customer data exports. Fathom describes EU Isolation, launched in late 2021 following the Schrems II ruling, under which EU visitors’ IP addresses do not touch US-controlled infrastructure.

Some caveats I am not going to paper over. Dropping cookies removes the Article 5(3) storage-and-access trigger, and that is all it does; whether processing an IP address to derive a country needs its own lawful basis under the GDPR is a separate question that none of the paragraphs above answer. Everything in them is also a company describing itself, quoted by me, which is evidence of what a vendor has committed to in public and is not a legal opinion about whether the commitment is sufficient.

And none of the three gives you Google Ads conversion tracking. If you run paid acquisition, consent mode is not optional for you and the conclusion of this article does not transfer.

The decision, and what has not happened yet

I can be specific about what getting this wrong costs because I got it wrong here, at length, and (at the moment I am writing this) am still getting it wrong.

This domain runs a Google Analytics 4 tag. Property G-XMEW7KSTQD, declared in config.yaml, served from a host in Italy to visitors who are mostly in Europe, and it has been running for months with no consent banner, no consent mode implementation and no privacy policy behind it. Checking took one command:

curl -s https://therezaali.com/ | grep -c -E 'consent|denied'
# 0

Zero. Not a misconfigured banner, and not a banner defaulting to granted. Nothing at all. The tag loads, sets its cookie and reports, and has done every day for months. I have spent working hours advising other people on measurement setups while my own site fails at the first line of the rule.

So I took the tag out of the build. G-XMEW7KSTQD is gone from config.yaml and from every template in this rebuild, which ships no analytics of any kind: nothing setting a cookie, nothing writing to localStorage, no third-party script at all. The privacy page states what is collected and by whom.

Now the part it would be easy to leave out.

This article is written on a branch that has not been deployed. At the time of writing, the live therezaali.com still serves the GA4 tag, still with no consent banner, exactly as described in the section above. So do not take my word for either half of that:

curl -s https://therezaali.com/ | grep -c -iE 'gtag|googletagmanager'

Running it while writing this paragraph printed 4. grep -c -E 'consent|denied' against the same page still printed 0. When this rebuild ships, the first command starts printing 0; if it ever goes back to printing anything else, something has regressed and I would like to be told. Either way the date lands on the corrections page, which is where the changes to this domain are logged rather than narrated.

Writing “I removed Google Analytics” in the present tense, on a branch, about a site still serving the tag, is the specific kind of small forward-dated lie that a reader can catch with one command. On a page arguing that claims should be checkable, getting caught on that would be deserved.

The reason to publish the failure instead of quietly fixing it is that the failure is the only part of this I can currently prove.

hreflang for sites with three pages and no translation team

contact@therezaali.com (Mo RezaAli) — Wed, 05 Aug 2026 00:00:00 +0200

Two versions of one Italian headline. This is the one that got published:

Come Ottimizzare Le Campagne Google Ads

This is the one an Italian would have written:

Come ottimizzare le campagne Google Ads

The difference is Le. It is a definite article, capitalised in the middle of a sentence, and Treccani’s grammar entry on capital usage rules it out without qualification: with the titles of a book, a work of art, a film, a song, “la maiuscola si limita alla prima parola del titolo” (the capital is limited to the first word of the title). The Accademia della Crusca’s standing answer on capitals, from Luca Serianni and Giovanni Nencioni in La Crusca per voi n. 2, April 1991, treats the wider question as genuinely unsettled in places, mostly around the boundary between proper and common nouns. There is no boundary here to be near. Nothing in Italian licenses a capital Le mid-title, and an Italian reader clocks it before finishing the line.

I know how many there were, because I counted them.

That headline came out of a machine-translation pass on this domain, since deleted. Of the 728 Italian files it produced, 228 carried at least one Italian article, preposition or conjunction capitalised somewhere other than the first word. Thirty-one per cent. Here is how to get that number, against $B, the commit immediately before the deletion:

B=22b7c22d^
git archive $B content/post/insights/ | tar -x -C /tmp/corpus
cd /tmp/corpus/content/post/insights
for f in *.it.md; do
 awk '/^title:/{sub(/^title: */,""); gsub(/^"|"$/,""); print; exit}' "$f"
done > /tmp/it-titles.txt
wc -l < /tmp/it-titles.txt # 728

python3 -c '
import re, sys
fw = set("""il lo la i gli le un uno una di a da in con su per tra fra
del dello della dei degli delle al allo alla ai agli alle dal dalla dallo dai dagli dalle
nel nello nella nei negli nelle sul sullo sulla sui sugli sulle col coi
e ed o od che ma se non ne si come quando mentre""".split())
t = [l.strip() for l in sys.stdin if l.strip()]
print(sum(1 for x in t if any(w[0].isupper() and w.lower() in fw
 for w in re.findall(r"[A-Za-zÀ-ÿ'"'"']+", x)[1:])), "of", len(t))
' < /tmp/it-titles.txt # 228 of 728

The test is deliberately narrow. It fires only on closed-class words: articles, prepositions, conjunctions, the words whose capitalisation is not a matter of taste in Italian. Nothing a proper noun or a brand name does can trip it, and 228 is therefore a floor rather than an estimate, which is the direction I would rather be wrong in when the number is about my own work.

An earlier draft of this page said 57%. Trying to reproduce it, I ran four different definitions of Title Case over the same 728 titles (every non-initial word capitalised, most of them capitalised, at least two of them capitalised, and the function-word test above), and not one landed near 57. So the 57 is gone. What stands is the number that has a command attached to it.

Onto all 2,183 of those translated pages (728 Italian, 728 Arabic, 727 Chinese) I shipped hreflang, correctly.

What correct looked like

The <link rel="alternate"> sets came out of the template, which is why they were clean. Every version listed every other version and itself. The codes were valid. x-default was present. The return links resolved. Audit the markup and it passes.

The markup was never the problem. The markup worked, and working is what did the damage: hreflang exists to route a reader in a given language to the URL in that language, so it took the headline above and delivered it, efficiently, to the people best equipped to see what was wrong with it.

The mechanics, from Google’s documentation

Google’s page on telling it about localized versions is the primary source and it is short. Five rules carry most of the weight.

Return links are mandatory. “If two pages don’t both point to each other, the tags will be ignored. This is so that someone on another site can’t arbitrarily create a tag naming itself as an alternative version of one of your pages.” One-directional annotations are discarded outright. They do not degrade to a hint.

Every version lists itself. “Each language version must list itself as well as all other language versions.” The set of links is identical on every variant of the page, which is the property that makes the whole thing template-generatable.

x-default is the fallback. “The reserved x-default value is used when no other language/region matches the user’s browser setting.” Google recommends it for language selectors and auto-redirecting home pages.

The code is language first, region optional. First code, the language, ISO 639-1. Optional second code after a hyphen, the region, ISO 3166-1 Alpha 2. Only codes listed in those two standards work; Google’s page names es-419 as one that does not. Then the warning that catches almost everyone: “You can’t specify the country code by itself. The first code stands for the language and Google doesn’t automatically derive the language from a country code.” be is Belarusian. Belgium is de-be, nl-be, fr-be.

Three delivery methods, pick one. Head tags, an HTTP Link: header, or a sitemap.

Head tags

<head>
 <title>Widgets, Inc</title>
 <link rel="alternate" hreflang="en-gb"
 href="https://en-gb.example.com/page.html" />
 <link rel="alternate" hreflang="en-us"
 href="https://en-us.example.com/page.html" />
 <link rel="alternate" hreflang="en"
 href="https://en.example.com/page.html" />
 <link rel="alternate" hreflang="de"
 href="https://de.example.com/page.html" />
 <link rel="alternate" hreflang="x-default"
 href="https://www.example.com/" />
</head>

Cheapest to implement. Most expensive to serve. With n locales you carry n links in the head of every page, so across the site the head grows as the square of your locale count.

HTTP header

Useful for non-HTML files. The header returned is identical for every version:

Link: <https://example.com/file.pdf>; rel="alternate"; hreflang="en",
 <https://de-ch.example.com/file.pdf>; rel="alternate"; hreflang="de-ch",
 <https://de.example.com/file.pdf>; rel="alternate"; hreflang="de"

Sitemap

Moves the weight out of the pages entirely. Declare the XHTML namespace, then give every <url> a full set of <xhtml:link> children including itself:

<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
 xmlns:xhtml="http://www.w3.org/1999/xhtml">
 <url>
 <loc>https://www.example.com/english/page.html</loc>
 <xhtml:link rel="alternate" hreflang="de"
 href="https://www.example.de/deutsch/page.html"/>
 <xhtml:link rel="alternate" hreflang="de-ch"
 href="https://www.example.de/schweiz-deutsch/page.html"/>
 <xhtml:link rel="alternate" hreflang="en"
 href="https://www.example.com/english/page.html"/>
 </url>
 <url>
 <loc>https://www.example.de/schweiz-deutsch/page.html</loc>
 <xhtml:link rel="alternate" hreflang="de"
 href="https://www.example.de/deutsch/page.html"/>
 <xhtml:link rel="alternate" hreflang="de-ch"
 href="https://www.example.de/schweiz-deutsch/page.html"/>
 <xhtml:link rel="alternate" hreflang="en"
 href="https://www.example.com/english/page.html"/>
 </url>
</urlset>

Three versions, three <url> entries, three identical children each. It is n² either way. The sitemap simply moves the entries somewhere a static host does not re-send on every request.

One more line from the documentation that people miss once a site grows past a few locales: “If it becomes difficult to maintain a complete set of bidirectional links for every language, you can omit some languages on some pages; Google will still process the ones that point to each other.” What you should not omit is the link from a newly added language back to your dominant one.

Separately, and unrelated to hreflang: set lang on the <html> element. That is the declaration browsers and assistive technology use, W3C’s internationalization guidance covers it, and the tag syntax is RFC 5646. hreflang tells a search engine which URL to show whom. lang tells a screen reader which voice to use. Different jobs, and you need both.

Underneath the markup

Italian. The 228 titles above came from one mechanical cause: the pipeline ran an English-language title-casing function over output that was no longer English. The transform was locale-blind. It fired on every title in every language, and on Italian it produced a visible error in any title containing an article, a preposition or a conjunction past the first word.

Arabic. Here I have to correct myself, and the correction is large. An earlier version of this page claimed 116 Arabic files shipped a verb cut in half with an English stem left inside it. I went to count it and the count did not come back anywhere near 116. Every token in the 728 Arabic files where a lowercase Latin run is fused directly to Arabic letters, with the number of files each appears in:

cd /tmp/corpus/content/post/insights
python3 -c '
import re, glob, collections
AR = r"ء-يٱ-ۓ"
tok = re.compile(f"[A-Za-z{AR}]*(?:[{AR}][a-z]|[a-z][{AR}])[A-Za-z{AR}]*")
n = collections.Counter()
for f in sorted(glob.glob("*.ar.md")):
 for t in set(tok.findall(open(f, encoding="utf-8").read())):
 n[t] += 1
for t, c in n.most_common():
 print(c, t)
'

23 وtur
2 لتautomate
1 تتblur
1 وb
1 السيمantics
1 يautomates
1 treatingها

That is the whole population. لتautomate, يautomates, تتblur and treatingها are the corruption I described: an Arabic verbal prefix or object suffix welded to a bare English stem, because a substitution pass treated a templatic language as if it were space-delimited Latin words. Arabic builds a verb by interleaving a root with affixes. The pass split one open and left English sitting in the gap. السيمantics is the same fault on a noun. Six distinct files, not 116.

The top row is a different bug and it is the common one. Twenty-three files carry a Markdown link whose label was truncated mid-word and published as a fragment: …وtur](https://blog.hubspot.com/topic/data-analytics). It rendered as a fragment, on twenty-three pages, for months. Nobody read it. وb I cannot classify and am not going to guess at.

Chinese. I have no basis to characterise the Chinese quality in either direction. I do not read Chinese and I never had it reviewed, which is itself the finding.

Why the multiplier goes both ways

Take the general shape. hreflang multiplies whatever your per-locale quality is by the number of locales you serve. Positive quality, more reach. Negative quality, more reach.

Which means that everything I did right (the self-referencing sets, the valid codes, the x-default fallback, the bidirectional return links that a Search Console audit would have waved through without a note) worked precisely as designed, and what it was designed to do was take a headline no Italian would write, put it in front of Italian readers, in Italy, at the moment they were looking for exactly that topic, four locales wide, from 31 October 2025 to 27 April 2026.

Google’s spam policies name the pattern under scaled content abuse: “Scraping feeds, search results, or other content to generate many pages (including through automated transformations like synonymizing, translating, or other obfuscation techniques), where little value is provided to users.” Translating is listed as an automated transformation not because translation is suspect, but because translating at volume without review turns one source into many thin pages. As a description of what I built, that is accurate, and the hreflang was the delivery mechanism.

What I would tell a solo operator

Pick one non-English locale, or zero.

Zero is respectable, and it is what this site’s articles use now. If you pick one, pick the language you can read. I live in Messina. I can check Italian output against how people around me actually write, which is precisely the check I skipped at 2,183-file scale and could have done in an afternoon at three-file scale. Then:

Translate a page only when you have read the translation end to end.
Never run English text transforms on non-English output: title casing, truncation at a character count, possessive handling. Gate every one of them on locale. The 228 titles and the truncated Arabic link labels are the same bug wearing two hats.
Add the hreflang set only after the translation passes review. hreflang last, not first.
Keep the set self-referencing and bidirectional, add x-default, and let the template generate it so it cannot drift.

Three pages in two languages that a human has read will outperform two thousand in four. I am not offering that as a consolation prize. It is the only configuration in which the mechanics above are worth implementing at all.

A gate that reads only the artifact scores the surface, and mine passed 842 articles

contact@therezaali.com (Mo RezaAli) — Wed, 05 Aug 2026 00:00:00 +0200

An automated evaluator that scores a generated artifact using only features it can read inside that artifact does not measure the property it names. It measures the surface of the property, and anything generating against it will produce the surface. Mine required each article to carry at least two named entities and at least one number. It passed 842 of them. 470 of the 739 English articles carried a before-and-after metrics table, and not one figure in any of them describes something that happened.

The URL of this page rounds. The precise figure is 842 commits titled Add post:, and it belongs in the first paragraph rather than a footnote, because a slug that sands 842 up to 900 is exactly the sort of small convenience this whole article is about. 739 of those articles were live in English when I deleted the archive. Most were then machine-translated into Italian, Arabic and Chinese: 728, 728 and 727 files. That is how 739 English articles became 2,922 files on disk.

Nothing about that defect is specific to content bots, and you do not need to have run one to have it in a gate you own. It is the same defect as an LLM-as-judge rubric that rewards well-formed citations, a keyword-count SEO brief, an engineering scorecard that counts test files rather than running them, and a hiring filter that counts years of a word in a document. Every one of those scores a text, and a language model is very cheap at producing text. What this page has that an argument about it would not is the source of a gate that had the defect, quoted from the file, and seven checks that find it in yours.

I wrote that gate. It lives in one file, 2,177 lines of JavaScript, and after the deletion I read all of them again, which is why the references at the foot of this page are line numbers rather than recollection, and why the validator further down is quoted instead of described. That is the qualification behind everything below. Not that I avoided this failure. That I have the source of a system which manufactured citations 842 times, and have been back through it since.

Nobody read any of them before they went out. Me included.

Three dates, kept apart on purpose. The bot published from 31 October 2025 to 27 April 2026. It then went quiet, and the archive sat there, indexed, for another three months. I deleted it on 5 August 2026.

I wrote the bot. I set the cron to hourly and AUTO_PR to "false", which is the setting that made every commit go straight to the published branch. This is not a piece about a model that misbehaved, and “AI hallucinates” explains nothing about why this particular failure took the particular shape it took. The model was handed three instructions: write 900 words, include at least one number, invent no numbers. The only material it was given was 450 characters of RSS description. Only two of the three could be satisfied at once, and the gate I wrote decided which two.

The system, and what I can show you of it

The 2,177 lines are a Cloudflare Worker. The commits and the code sit in a private repository, so on those you have my word; every external source on this page is linked and checkable, which is the split the limits line at the top states outright.

The configuration is four lines long and tells you most of the story:

[triggers]
crons = ["0 */1 * * *"]

[vars]
RSS_URLS = "https://neilpatel.com/blog/feed/, https://blog.hubspot.com/marketing/rss.xml, https://feeds.feedburner.com/mitsmr, https://www.forrester.com/blogs/feed/, https://www.beautyofsaas.com/feed"
OPENAI_MODEL = "gpt-4o-mini"
GITHUB_BRANCH = "main"
GITHUB_PR_BASE = "main"
AUTO_PR = "false"
MAX_ITEMS = "5"
STRICT_MODE = "true"

Once an hour it pulled five marketing feeds (Neil Patel, the HubSpot Marketing Blog, MIT Sloan Management Review, Forrester’s blogs, and Beauty of SaaS), took up to five new items, sent each to a model, and wrote the result into main with a GitHub contents PUT. There was no pull request. There could not have been one: the PR function opens with if (config.branch === config.prBase) return;, and I had set both to main. Every article was committed with draft: false and author: "Mo Reza Ali". That is not even how I spell my name.

The reasoning at the time was ordinary enough. I wanted a publishing habit I could not skip, I wanted the domain to have something on it, and I had read enough about content velocity to talk myself into believing that shipping often mattered more than shipping well, with the quality gate holding the floor underneath. That was the deal I made with myself. Nearly all of my attention after that went into the gate.

Here is what the deal produced. The repository held 2,967 commits when I deleted it. 842 of them said Add post:. 63 said Update post:, 845 auto-assigned a cover image, 220 auto-translated, 160 auto-healed metadata. That is 2,130 commits no human wrote and no human read. I was tuning the instrument and never once reading the dial.

The contradiction inside the gate

This is the part that generalises, so it is worth being precise about.

The only input the model ever received about a story was this, from the prompt builder:

rss_summary: stripHtml(item.description || "").slice(0, 450),

450 characters. That is roughly the length of this paragraph. From that, the prompt asked for a 600–900 word “insider briefing”, and then the validator decided whether the result was allowed to ship. Two of its checks did the damage:

// 2. Named entities — require at least 2 proper nouns (companies, products, people)
if (entities.size < 2) {
 errors.push(`only ${entities.size} named entities found (need >= 2): ...`);
}

// 3. Specific numbers — require at least 1 concrete data point
if (numbers.length < 1) {
 errors.push("no specific numbers or data points found (need >= 1)");
}

Read those two rules next to the 450-character input and the conflict is structural. A 450-character excerpt very often contains no statistic at all. The validator refused to pass any article without one. The prompt, in the same breath, said: “NEVER fabricate statistics, percentages, or study results.” So the model was given a requirement it could satisfy only by breaking a prohibition. It resolved the contradiction the only way the contradiction could be resolved.

And when it could not, the article shipped anyway:

if (config.strictMode && !processed.validation.ok) {
 if (attempts >= AI_REPAIR_ATTEMPTS) {
 console.warn("strict_mode_fallback", { title: item.title, errors: processed.validation.errors });
 processed.validation.warnings = [
 ...(processed.validation.warnings || []),
 ...processed.validation.errors.map(e => `[relaxed] ${e}`)
 ];
 processed.validation.ok = true;
 return processed;
 }
 throw new Error(`Validation failed: ${processed.validation.errors.join("; ")}`);
}

AI_REPAIR_ATTEMPTS was 2. After two failed retries, every error was rewritten as a warning with a [relaxed] prefix, validation.ok was set to true by hand, and the article was published. STRICT_MODE = "true" in the config. It did nothing except delay publication by two API calls.

You can measure how little the gate held. The prompt shipped a kill list of 114 banned phrases and told the model that articles containing them “will be REJECTED by our validator”. 686 of the 739 published articles (92.8 percent) contain at least one phrase from that list. 604 of them, 81.7 percent, contain a word from the hard-reject subset that was supposed to fail an article outright. The word “crucial” is in 486 published articles. “Essential” is in 424. “Foster” is in 368. The list was real, the enforcement was real, and it made no difference to what went out the door, because the escape hatch ran after it.

What a check for the presence of digits produced

Here is a block from one published article, unedited:

## What Good Looks Like in Numbers

| Metric | Before | After | Change |
|-------------------|--------|-------|----------|
| Conversion Rate | 2% | 5% | +150% |
| Retention Rate | 60% | 75% | +25% |
| Time-to-Value | 30 days| 15 days| -50% |

Source: HubSpot Blog

That exact Retention Rate | 60% | 75% | +25% row appears in 29 of the 739 articles, under sixteen different attributions: four crediting Source: HubSpot Blog, two Source: Forrester Research, nine naming HubSpot in some other phrasing, two naming Neil Patel, and ten pointing at an internal source that does not exist: Internal Company Data, Internal Marketing Analysis, Internal Marketing Metrics 2026. Conversion Rate | 2% | 5% | +150% appears 126 times. Time-to-Value | 6 months | 3 months | -50%, 134 times. 470 of the 739 English articles carried a before-and-after table of this kind, and not one figure in any of them describes something that happened.

The attribution lines are worse than the tables. 341 articles carry a line beginning Source:. Six of those 341 contain a URL.

180 of the 739 English posts cited a source called “Internal Analysis” that does not exist. That is the figure, with its denominator. And because the argument of this article is that a number ought to be re-derivable by a reader who does not trust the person who wrote it, here is the command rather than a description of the command. $B is the commit immediately before the deletion:

B=22b7c22d^

# denominator: the English article files
git ls-tree -r --name-only $B content/post/insights/ \
 | grep '\.md$' | grep -v '\.\(it\|ar\|zh-cn\)\.md$' | wc -l
# 739

# numerator: pattern Source:.*Internal, case-insensitive, each file counted once
git grep -lEi 'Source:.*Internal' $B -- 'content/post/insights/*.md' \
 | grep -v '\.\(it\|ar\|zh-cn\)\.md$' | wc -l
# 180

Run it against this repository and you get 180. You do not have this repository. It is private, which the limits line at the top of the page says in as many words. What you actually have is the pattern, the denominator, and the commit, which is more than the old site ever gave anyone.

There is no internal analysis, and no internal company data. I have never run the study. Some of the other attribution lines are worse still, because they put invented figures in the mouths of institutions that never said them: Bank of England Internal Metrics, ABN AMRO Internal Metrics, Anthropic Case Studies, Cloudflare's internal metrics post-rewrite.

Even the fabrications disagree with each other. The same before-and-after pair, 70 percent to 85 percent, is labelled +15% in one article and +21% in another. 2.5% to 4.0% is labelled +60% in one and +1.5% in another. A gate that checked arithmetic would have caught this. Mine checked for the presence of digits.

What it cost other people

The site’s credibility, first, which is why it was deleted rather than corrected. The larger cost is what it did with other people’s work.

31 of the published article titles contain the word “Forrester”. They are not articles about Forrester. They are Forrester’s own headlines, republished under my byline: Announcing The Forrester Wave™: Digital Experience Platforms, Q4 2025. Call For Entries: Forrester B2B Summit North America 2026 Awards. Meet Jess Lloyd: Forrester's New Principal Analyst Covering Consumer. A staff announcement about somebody’s new job at somebody else’s company, with my name in the author field.

52 articles have a description written in the first person by a person who is not me, lifted straight out of the feed excerpt. One of them reads: “In the first episode of my Nudge podcast, I interviewed the fantastic psychologist Dr.” That sentence belongs to Phill Agnew, who writes for the HubSpot Marketing Blog and hosts Nudge. It is the opening line of his post about the psychology of music, and it is truncated mid-name because my code sliced the description at 100 characters. My site published his memory of his own podcast episode as my description, under my byline, above 900 words of invented metrics.

Not one of the 739 articles carried a link back to the item it was derived from. The frontmatter schema the bot wrote has thirteen fields. source_url is not one of them. The pipeline knew the origin URL (it is right there in the prompt payload as source_url) and dropped it before publishing. That was not an oversight in the model. That was a field I did not add.

Google’s spam policy has a name for the shape of this. “Scaled content abuse is when many pages are generated for the primary purpose of manipulating search rankings and not helping users,” and the policy is explicit that it applies no matter how the pages were created. I would have argued at the time that mine was different, because mine had a quality gate. Its two load-bearing checks are quoted above. They read a table of invented metrics and marked it a pass.

What I deleted

On 5 August 2026 I removed all of it in two commits. content/post/insights/: 2,922 files. All Arabic and Chinese translations, including six Arabic files that had shipped a verb split open with a raw English stem left in the gap, and a further twenty-three carrying a link label truncated mid-word. They stayed live for months in a language I do not read. An earlier version of this paragraph put the first figure at 116; I recounted it, and the token-by-token derivation is in the hreflang piece. A comparison page publishing AggregateRating structured data that scored a real product at 1.36 out of 5 while every public review platform put it above 4.4, presented as independent, and ranking a former employer without disclosing the relationship. A 2,074,882 byte “cover image” whose visible content is the word LOADING.., served on 229 posts. Of the 739 English articles, 736 carried a disqualifying defect I could name individually. The other three were entirely derived from someone else’s post. The honest survivor count is zero.

Deleting the source file does not undeploy a Worker. The cron keeps firing and the token keeps working until you remove the Worker in the Cloudflare dashboard and revoke the GitHub personal access token by hand, which is why the commit message for the pipeline removal ends with a line to myself in capital letters: MANUAL STEP REQUIRED. If you are reading this because you built something similar, that is the step people forget.

The same defect in four other systems

The claim at the top of this page is not about content bots. Stated in full: a quality gate which cannot distinguish “sourced” from “present” will manufacture whatever shape it is asked to measure. My validator could see that a paragraph contained a percent sign. It could not see whether the percentage referred to anything. So it did not enforce evidence. It enforced the appearance of evidence. Then I pointed a generative system at it and told it to score well. The output was not a failure of the system. It was the system working, on the objective I actually gave it.

This is Campbell’s law with a shorter feedback loop. The moment a proxy becomes the target, the cheapest way to move the proxy is to fake the thing it stands for, and a language model is very cheap at producing the surface of things. But the point is not confined to content bots. Any automated evaluator that scores a generated artifact on detectable surface features has this defect: keyword-count SEO briefs, LLM-as-judge rubrics that reward citation formatting, engineering scorecards that count tests rather than run them, hiring filters that count years. If your gate measures a shadow, you will get very good shadows.

Volume is the symptom and never the cause. What scales the damage is how long the loop runs unread, and mine ran hourly for six months. Tightening the rules inside that window changed nothing that mattered, because the defect was never how strict the rules were. It was that every one of them could be satisfied by the text under test.

How to detect it in your own pipeline

Seven checks, in the order I would run them, on any pipeline where a generator is scored by an evaluator. Each of them is an afternoon at most, and the first is ten minutes.

1. Unplug the network and run the gate. If every check returns the same verdict offline, nothing in the gate resolves against the world, and every rule in it is a property of the text. Mine passed offline. That one test in November would have told me what I found out in August.

2. Find every write to the pass flag. Search the pipeline for each assignment to whatever variable means “this passed”. Any write outside the function that computes it is an escape hatch, and a gate with an escape hatch is advisory. Mine is quoted above: processed.validation.ok = true, five lines after the errors were renamed warnings.

3. Test the ban list against shipped output, not against the prompt. I had 114 banned phrases and a sentence telling the model that articles containing them “will be REJECTED by our validator”. 686 of the 739 that shipped contained at least one. That measurement is a single grep over the output directory. Any rule you have never run against what actually went out is a rule you are assuming works.

4. Sort the corpus by repeated line and read the top of the list. Fabricated evidence clusters, because a model asked for a plausible number reaches for the same plausible numbers. One row, Retention Rate | 60% | 75% | +25%, appears in 29 of my 739 articles under sixteen different attributions. Independent measurements of different companies do not agree to the digit.

# the most repeated table rows across a generated corpus
grep -rhE '^\|' content/ | sed 's/[[:space:]]\{1,\}/ /g' \
 | sort | uniq -c | sort -rn | head -20

5. Check the corpus against itself before you check it against the world. Internal contradiction costs nothing to detect and needs no external source: the disagreeing percentage labels above were sitting in my own output the whole time. Arithmetic that does not close is fabrication that did not coordinate.

6. Count the outputs carrying a pointer a machine can follow. Not a citation, a resolvable pointer. 341 of my articles carried a line beginning Source:. Six of those 341 contained a URL. The ratio between those two numbers is the entire finding of this article, and getting it takes two greps.

7. Write one deliberately false artifact by hand and run it through the gate. If it passes, you are done and you have your answer in an afternoon rather than six months. Mine would have passed a table of pure invention, because it did, 470 times.

How to build a gate a generator cannot satisfy by writing more

One rule covers all of it. Every check must be able to fail for a reason that does not exist inside the artifact being checked. Something outside has to answer: a server, a file on disk after a build, the exit code of a process that actually ran, a row in a database, a person who picks up the phone. If you cannot name the external thing a check resolves against, you have a text property with a serious-sounding name.

Applied, in four places the same gate shape keeps appearing:

Content. Do not check whether an article cites a source. That is a string test and a generator passes it for free, which is how 341 articles carried a Source: line and six carried a URL. Check that the URL resolves, then that the fetched document contains the figure being claimed. The first needs a regex. The second needs a server.

LLM-as-judge. A rubric that rewards well-formed citations scores citation formatting, and the model under evaluation will supply well-formed citations. Put the retrieved document in front of the judge alongside the candidate answer and score whether the claim is supported by that document. A judge shown only the answer can only score the answer’s surface, which is my defect one layer up and with better vocabulary.

Engineering scorecards. A dashboard that counts test files counts test files. Run them and read the exit code. Line coverage is a milder version of the same mistake, because it records which lines executed rather than which assertions were made, so a suite that asserts nothing can still score well on it.

Hiring screens. A filter counting years of a keyword in a document scores the document, and documents are cheap to write and getting cheaper. Resolve outside it: a work sample scored blind, a reference who answers.

The strongest form of the rule is to delete the thing the gate was guarding. Nothing that publishes. That is the whole design change I made, and it costs me nothing I valued: the useful half of the bot read five feeds and told me what was new, and the half that destroyed the site was one PUT. Drop the five links into a queue I open over coffee and you keep everything worth keeping, minus the GitHub token, the write scope and the byline.

The correction is never a stricter version of the same gate. I tried that. One of my own commits in that window is literally titled fix(insightbot): promote 30+ AI words to hard-reject quality gate, and 686 articles containing banned words shipped anyway. Tightening a surface check buys a better forgery. Check the URL, not the word “Source:”.

What each check on this site resolves against

The checker in this repository resolves every citation on the site against the live internet on every build, and a dead link stops the deployment. Precisely what that means:

Every article under content/writing/ must carry a non-empty sources array and a limits sentence in its frontmatter, and every source entry must have a title, a publisher, a URL and an accessed date. Missing any one of those fails the build.
Each source URL is fetched on every build: HEAD first, GET when a server refuses HEAD, two retries with backoff on transport failures and 5xx, six requests in flight, twenty second timeout. The response must be 2xx or 3xx. The count grows with every article, so it is not written down here: an earlier version of this sentence said 142 URLs across ten articles and was wrong within a fortnight, while the homepage carried the current figure. One site, one fact, two numbers, which is the thing this whole page is about. The checker prints the real count on every run and the homepage reads it from the content rather than from my memory.
It runs before Hugo. If a cited document died since the last deploy, nothing renders at all, and the failure happens before a single page exists.
It runs a second time after the build against public/, checking that every site-root link written in content/ resolves to a file that exists, and that every URL in the sitemap exists and is not caught by a redirect rule. A sitemap advertising a URL the edge redirects away is its own small lie.
It is invoked from scripts/build.sh, which is the command Cloudflare Pages is configured to run, so the checker’s exit code is the deployment’s exit code. Non-zero and nothing ships. That last link is a dashboard setting rather than a file, which makes it the one part of the chain a repository cannot enforce, and it is written down as such at the top of that script.
401, 403 and 429 are downgraded to warnings in the deploy path, because a publisher refusing an unfamiliar user agent from a datacenter address is not a dead link. It cannot hide a fabricated citation: an invented URL fails DNS or returns 404, never 403.
Zero dependencies. A check that guards against unreviewed automation should not itself install several hundred unreviewed packages.

Two further checks exist because of a failure found in the rebuild rather than in the bot: one canonical spelling of my name, and one fixed value per fact, so a figure an earlier draft got wrong becomes unwritable everywhere on the site.

The property that matters is not thoroughness. It is that every one of those checks is satisfied only by something outside the file being checked. There is nothing in it I can argue past by writing more confidently, which is precisely what my validator could be argued past by doing. I have already had to obey it twice while writing these pages.

If I ever did want machine help with a draft again, the gate would run backwards from the one I built: extract every factual claim, demand a URL for each, fetch the URL, and fail the build when it does not answer 200 or does not contain the figure being claimed. Harder to write. Much cheaper to trust, because no amount of fluent prose moves it. Only a server answering does.

I publish no statistics about my own work. I have not collected any, and the previous version of this domain is a 2,922-file demonstration of what happens when I let a machine fill that particular silence. Where I do not know something, the article says so. The limits line at the top of this one admits that the repository I keep quoting is private, so my external citations are checkable and my commits are not.

The full rules are on the editorial policy page, along with what I do when I get something wrong.

One rule, in the same words both times it appears here. Every check must be able to fail for a reason that does not exist inside the artifact being checked. You can settle that about your own gate this afternoon: unplug the network and run it, and see whether a single check changes its verdict. Mine did not change one. The four systems at the top of this page are the ones I would put through it first, because every one of them scores a text, and a text is the cheapest artifact a machine now makes.

Primary evidence

Everything above comes from artifacts in the repository behind this site rather than from memory. For my own record, and so the numbers can be re-derived:

Worker source extracted-projects/rezaali-insightbot/mo-content-worker.js, 2,177 lines by wc -l: AI_REPAIR_ATTEMPTS at line 11, the banned-phrase list at lines 26–103, the [relaxed] fallback at lines 686–697, the frontmatter builder with draft: false and the byline at lines 714–735, the 450-character slice at line 827, the entity and number checks at lines 1440–1454, the unreachable pull-request function at line 1945.
Configuration: extracted-projects/rezaali-insightbot/wrangler.toml.
Commit fa6312b3 removed the pipeline; commit 22b7c22d removed the content, and its message contains the file-by-file accounting I have summarised here.
Corpus counts were taken at 22b7c22d^, the commit immediately before the deletion, over the 739 English files under content/post/insights/. The internal-source figure of 180 is the file count for git grep -lEi 'Source:.*Internal' at that commit, restricted to the English files. The command is printed in full earlier on this page.
The checker described above is scripts/check-sources.mjs in the repository behind this site, and the deploy gate that invokes it is scripts/build.sh. The citation and URL counts are that script’s own output on the build that produced this page.