I’d deploy most of it tomorrow. That’s what made it hard to articulate why it bothered me for days, until I landed on it: it’s a diagram of a request. It follows one call from a user, through the controls, to an answer, and it governs that call well. It shows you the AI, secured at runtime. It doesn’t show you the work.

Here’s the test I’d put to any diagram of this kind: the operational questions should land somewhere on it. Why can’t the agent deploy this fix to production? Who authorized that tool call at 2am — and had the agent earned the right to make it without asking? What re-validates every automated decision when the model version underneath changes? Point to the box where those answers live. On the request diagram you can’t, and it isn’t carelessness: those answers live in layers a runtime flow has no place for, because a request is over in seconds and the answers play out over months.

So I drew the whole thing, for the kind of environment I work in: a cloud-native platform sitting on data that matters. And I drew it for the operations side, deliberately — not the AI in the product, but the AI that does the work. What I want is a model of AI workers: engineers, product and security both, handing real work to agents that hold real access and do it securely, with every layer those workers stand on drawn in — the infrastructure, where the orchestrator takes hold, the prompts, the guardrails, the controls. Not just prompts pointed at a tenant: actual infrastructure. The model itself is one layer of thirteen, near the bottom, and nearly everything that determines whether any of this works sits above it. The irony of answering an architecture diagram with a bigger architecture diagram is not lost on me; the extra layers are the argument.

The full stack

Read it bottom-up. Everything below a layer is what that layer stands on, and the two boxes just under the humans row are the reason the rest exists.

Two rails, the stack’s last two layers, run its full height and touch every row.

Hold the two diagrams next to each other. The request flow covers a genuine chunk of this stack: both rails, the gateway, the data row, orchestration, and the static half of the authorization row — the RBAC, the token validation, the least privilege. What it has nowhere on it is the other half of authorization, the part where what an agent may do changes as it earns it, or the environments ladder, or evaluation, or the work row, or the humans. And it can’t have them, structurally: those are the layers where trust gets built over weeks, and a request diagram lives inside a single call. Everything the flow doesn’t reach is the part you can’t buy, which is also the part that decides whether the agent in the middle of it ever gets to do more than answer questions.

The rows everyone already draws

I’ll be brief about the bottom of the stack, because this is the territory the request diagram already covers well.

Substrate is real engineering but it’s the most commodity layer here, and the interesting property is the boundary work: private endpoints, egress control, tenancy. Data is where the stakes live: the sensitive-data boundary is a property of the data layer, and half the security architecture above it exists to keep that property true. Models, plural, is an inventory for the workers: frontier models rented by API, embedding models, and a long tail of classical ML that was “AI” before the rebrand. The register that matters isn’t which models; it’s that they carry versions and deprecation dates, which means everything above them inherits change it didn’t ask for.

The gateway is a layer the request diagram gets exactly right — an AI gateway with model access control sits dead center of it — and it’s the one that makes the model layer governable at all: one place where every model call gets routed, throttled, content-filtered, cached, and, the part I care about, attributed. A model call is an invoice with a caller on it. Readers of the log-source post can guess how I feel about AI spend that can’t name which workload incurred it.

Where the demo ends

The context layer is where things stop being infrastructure and start being behavior. System prompts, instruction files, skills, retrieval, memory, the MCP tool surface: this is the layer that tells the agent who it is and what the org wants, and I spent an entire post on the fact that it’s executable-as-intent while being reviewed like documentation. Nothing in this post retracts that worry. The diagram makes it worse-looking, actually, because you can now see how much sits on top of that layer.

Orchestration is the layer the demos live in: the loop, the tool calls, the sandbox, the handoffs. It’s also where most organizations currently stop, and the diagram shows why that’s a strange place to stop. An agent with orchestration but nothing above it is a very capable process with nowhere legitimate to act: it can reason about your systems but holds no rung on any environment ladder, no policy that says what it may do without asking, no eval gate that would let anyone trust it further. It’s a parked capability, and the parking is usually an accident of nobody owning the next three layers.

The load-bearing rows

The environments row is the one I’d defend hardest, because it encodes the actual difference between a demo and a colleague, and it isn’t intelligence. It’s access with a shape. An engineer’s agent that can run the test suite but never touch staging can’t validate its own work; an agent that can reach production without having earned it is an incident with a start date. So the ladder: dev, test, staging, prod, each rung a separate credential scope, each held per task class, each revocable. The rung the agent holds is the trust it has earned, and “earned” is doing real work in that sentence, because the next two rows are what earning means.

Evaluation is the proof machinery: replay against historical cases, regression gates on every change to a prompt, a policy, a tool schema, or the model version underneath, adversarial testing for the surfaces where the adversary writes the input. I made this argument for triage in Page on Decisions, Not Alerts and it generalizes cleanly: any material change replays before it ships, and an agent’s track record is a dataset, not a vibe.

Authorization is where the eval results become permission. Policy written in English, versioned in git, precise enough to replay: what the agent may do autonomously, what it does with a human assisting, what it must never do without a page. Autonomy graduates per task class, shadow to assisted to autonomous, and high-impact paths never enter the pipeline at all. The decision rule that runs the whole layer is the one from the tree post: can this be undone in five minutes? Reversible actions are where autonomy starts, and irreversibility is the signal to escalate, for an agent exactly as for a human, except the agent applies the rule the same way at 3am as at 3pm. And none of this is the model grading its own homework. The ladder bounds what an agent can reach; tool-call gates, checked outside the model against the agent’s graduation level, bound what it may do within reach, with the irreversible classes intercepted for approval. The authority comes from the policy and the gates, not from the model’s reading of either.

Two tenants, one stack

The top of the diagram is the claim I actually want to make. In conversations I’ve been calling it the next gen of work, which I realize is exactly the kind of phrase these diagrams ship wrapped in, so here is the mechanical version: the work moves up the stack, and the humans move with it.

The engineering harness is the box I want engineers to live in. The work becomes intent plus review instead of typing: the agent writes the change, opens the pull request, runs it through CI, and holds exactly the environment rung its task class has earned. The part that matters more to me than shipping is maintaining, because shipping was already fun; maintenance is where careers go quiet.

Walk the 2am page through it, the version I’m arguing for. A canary starts failing an hour after a dependency bump lands. The agent holding the page reads the runbook the way the tree post describes an investigation, as hypotheses to rule in or out: recent deploys against this workload first, and there’s the bump, sitting at the top of the deploy history. Rolling back the platform’s own deploy is undoable in five minutes, so the reversibility rule authorizes it without waking anyone; the rollback goes out from the staging-and-rollback rung the agent actually holds, the canary re-runs clean, and by morning there’s a pull request pinning the dependency with the whole decision record attached: what fired, what was checked, what policy authorized the rollback, what evidence closed it. Now re-run the same page, except the failing service is mid-way through a data migration. The next action isn’t reversible, and the agent doesn’t get to be clever about that, because irreversibility is a gate, not a judgment call. So it pages a human, and what the human wakes to is not an alert but the tree already walked: evidence gathered, reversible steps taken, the irreversible branch named as the reason a person is now awake. Every mechanism in that story is three rows down in the diagram; nothing about it requires a smarter model than the ones we already rent.

And the security decision layer is the same box wearing our clothes. When we redesigned triage to remove Tier 1, the write-up I produced wasn’t, on reflection, a SOC document. It was this diagram with different nouns: candidate signals instead of backlog tickets, dispositions instead of merged PRs, policy-gated closure instead of deploy rights, decision records instead of commit history, shadow mode instead of a feature branch. Detection content is code, so the engineering harness applies to it directly, and the detections the agents maintain are the ones watching the telemetry rail of this very stack, which closes a loop: the stack defends itself with the same architecture it works with.

Which is why I keep insisting these aren’t two stacks. An org that builds an agent setup for engineering and an “AI SOC” as a second, separate thing is running the same shared layers twice to differ in the work row alone, and the seam between the copies is a place where an attacker, an outage, or an invoice hides. Call it an organizational bifurcation problem if you want the whitepaper version of the sentence; I’d put it more plainly: you don’t have two AI strategies, you have one harness with two tenants, and the sooner the org chart admits it the fewer layers get built twice.

The humans box stays on top, and not as decoration. Somebody authors the policy, reviews the exceptions, and runs the review of the misses, because the graduation gates are only as honest as the people auditing them, myself included. Accountability never delegates: the agent may hold the pager, but a person owns what it did with it.

The bill

A diagram that only advertises benefits is a sales deck, so here’s what this one costs.

The harness concentrates exactly the surface I spent July worrying about. Instruction files, skills, memory, tool registries: the context row is executable, and the moment agents hold environment rungs, it’s executable with credentials. The payload-is-a-sentence problem doesn’t get better in this architecture; it gets a bigger blast radius, and the telemetry that would catch it (instruction-file lifecycle events, provenance on context writes, drift between agent output and written policy) is telemetry almost nobody collects yet. If any of that agent output feeds a training corpus, the poisoning arithmetic applies to it too. Build the top of this stack before instrumenting the middle and you’ve built the incident first.

The identity rail is a population explosion. Every agent per task class per environment rung is a principal with a lifecycle, which means hundreds of scoped non-human identities that need issuance, rotation, attestation, and retirement. At this volume that’s an operations program of its own, run against identity practices that mostly still assume a principal is a person with a badge photo.

Evaluation is a standing commitment that rots quietly. The replay corpora, the regression gates, the agreement-rate baselines: all of it decays the way reference lists decay, and an autonomy gradient calibrated against last quarter’s model version is a lie with a dashboard. Sensitive data adds its own drag: some task classes will never graduate, because the reversibility rule fails permanently anywhere that data could move, and the decision records do double duty as engineering hygiene and as the thing you hand an auditor.

And someone has to own the harness. Not engineering, not the SOC, the shared rows underneath both tenants, and today that owner doesn’t cleanly exist on most org charts. Unowned shared infrastructure doesn’t stay unowned; it gets owned by whoever’s outage it causes.

The synthesis

The request diagram is a diagram of the part you can buy, and the part you can buy is the part someone else operates: the gateways and guardrails are products, and the models behind them improve on a vendor’s roadmap, get re-priced on a vendor’s schedule, and get deprecated on a vendor’s timeline, and none of that asks your permission. Everything above the model row is the part you operate, and it’s where the actual outcomes live, because those are the layers that either pay down toil and incidents or quietly generate them. That’s the harness. It’s the part that decides whether the work at the top of the diagram becomes real or stays a demo, and it’s built out of the least glamorous rows on the page: credential ladders, replay corpora, policies somebody maintains, telemetry somebody prices. Draw the whole stack, because every layer you leave off the diagram is a layer you’ll meet later in a post-mortem, and it will not be wearing a tidy box.

Disclosure: written with AI assistance, by an agent working inside a small version of exactly the harness this post describes: push rights to a branch, no rights to main, a human on the diff. The diagram, the opinions, and the review are mine.

Severity and willingness to act should move together, but in practice they don’t, and once you notice that, it shows up everywhere: a weird process sits on an endpoint, nothing about it screams malware, and it goes half-investigated for a day because nobody wants to be the one who paged on-call over nothing; a firewall rule breaks silently and log volume climbs for a week before anyone connects the two; someone finds a DNS record pointing at infrastructure that no longer exists, a takeover waiting to happen, and it sits in a backlog because there’s no clean next step, just a vague sense that someone should do something.

I’ve watched some version of this freeze at every place I’ve worked, small teams and large ones alike, and it isn’t a story about one org or one team having a gap so much as it’s a default. What’s changed for me is where I’m standing now. Working this deep in cloud environments, where almost everything is an API call away and the connective tissue between resources is actually queryable, has made the shape of a fix visible in a way it never was when the evidence was scattered across systems that didn’t talk to each other. The pieces below — reversibility as a decision rule, the investigation as a tree, reviewing the misses — feel, for the first time, like things that could be stitched together into something more cohesive and automatic than any one person’s judgment, instead of staying tribal knowledge that lives and dies with whoever happens to hold it.

Nothing, or the war room

My first instinct was to blame documentation — better runbooks, clearer playbooks, a wiki page for every scenario — but I have most of that already, and it isn’t the bottleneck. The actual problem is that most of these situations don’t offer a middle option: you either do nothing, or you declare an incident and put your judgment on record in front of the org, with nothing sized in between, so ambiguous signal gets rounded down to nothing, not because people don’t know it might matter, but because the only escalation path available is loud, and being loud and wrong is what people are actually afraid of.

Compare that to the one case that reliably works: a posture tool flags a resource exposed to the internet, someone kills the exposing rule immediately, and only afterward do they figure out why, because that response is fast when the first move is cheap and fully reversible, and nobody’s getting called out for closing an exposure that turns out to be nothing. The action is small enough that it doesn’t need permission in the moment, because permission was implicitly granted the day someone decided exposed resources get shut off on sight.

That’s a rule of thumb worth having on purpose instead of leaving implicit: can this be undone in five minutes? If so, act now and explain later; if not, that’s the actual signal to escalate, not the ambiguity itself. It turns “is this serious enough to make noise about,” a question people are bad at under pressure, into “can I take this back if I’m wrong,” a question almost anyone can answer fast.

More tools didn’t build the judgment

Reversibility solves the freeze on the easy cases, but it doesn’t solve the harder problem underneath all of this, which is that a lot of people genuinely don’t know what to look at once they’ve decided to look.

I used to build my own dashboards by hand, because that was the only way to see anything, and the forcing function was built into the work: before you could look at something, you had to decide it was worth looking at. Now there’s a posture tool, a SIEM, a dashboard for every team, all surfacing things constantly, and surfacing everything turns out to be functionally close to surfacing nothing, because nobody can hold thirty open questions in their head as things worth idle curiosity, so the default becomes only looking when something pages you, which is the opposite of proactive, even though the tooling is better than it’s ever been.

None of that volume teaches anyone what to do with an alert once it fires; that’s a different skill, and it doesn’t come from having access — it comes from having gone deep enough, often enough, to know where the evidence actually lives for a given kind of question.

What “going deep” actually looks like

Say a detection fires on a shell landing inside a container in a managed Kubernetes cluster; I already know, going in, that the platform has no container-level logging, so I only get the cloud provider’s built-in threat detection and its activity logs, which means the investigation doesn’t start with “what happened” — it starts with a set of hypotheses I can rule in or out against whatever evidence actually exists for each one.

shell_on_container:
  check: recent CI/CD pipeline runs against this workload
    -> if match: probably a deploy, not an attacker. confirm and close.
  check: does the access pattern look interactive
    -> if yes: pull identity sign-in logs, look for the session
       -> got an IP: cross-reference it against sign-in logs org-wide
  check: platform-native device telemetry for the underlying VM nodes
    -> the "pod" is really autoscaled VMs under the hood; check them directly
  check: posture tool's connectivity graph for the node
    -> what is this thing connected to, and how — not just what it's tagged as

Nobody handed me that tree; it’s built from having hit this exact wall — no container logging, here’s what actually exists instead — enough times that the next question is automatic. Someone without it doesn’t lack access: the detection fired, the posture tool’s connectivity graph is sitting right there, all of it available, and what’s missing is knowing which tool answers which sub-question and what the answer tells you to check next, so with seven tools open, one alert on the screen, and no sense of where to start, it either gets escalated immediately or gets picked at without going deep enough to know it was nothing.

That’s the actual gap tool sprawl doesn’t close: more surfaces just means more leaves, and nobody’s handing out the branches to go with them.

Readers who followed the detection altitude post already know the shape of the first branch in that tree: knowing what you don’t have — no container-level logging, in this case — is itself a piece of the map, and it’s the kind of thing that never makes it into a runbook because it’s infrastructure trivia, not procedure.

Reviewing the misses, not just the fires

There’s a second gap hiding behind the first one: post-incident reviews only happen after something got declared, so the DNS record that sat in a backlog for two weeks, the process that got glanced at and dismissed — nobody ever revisits those to check whether the call was right, which means the org never actually learns its own threshold; only the individual who made the call does, silently, and it leaves with them if they ever do.

If the tree above is what expertise looks like in the moment, reviewing the misses is how you’d actually transfer it — not “here’s the runbook for a confirmed incident,” but “here’s the thing three people looked at and closed as nothing, walk through why that was the right call and what would have flipped it.” That’s a different kind of practice than a tabletop on a confirmed breach, and almost nobody runs it, because it requires admitting you have a backlog of things you decided, without much ceremony, weren’t worth escalating.

Concretely, that means two things happening on a real cadence, not sitting as an aspiration in a wiki page nobody opens. First, a standing, low-ceremony review — pull a handful of things that got closed as nothing over the past couple weeks, and have whoever made the call walk the group through the actual reasoning, in order: what they checked first, what each answer ruled in or out, and what single piece of evidence would have flipped it to an escalation. It doesn’t need the weight of a postmortem, because nothing happened; it needs the habit of narrating a tree that normally stays silent inside one person’s head. Second, and this is where the mentorship piece stops being a nice idea and becomes a mechanism: that review is the actual teaching moment, not the live incident. A live incident is close to the worst environment to learn in — the stakes are real, a junior person is watching someone perform under pressure, and there’s no room to stop and ask “why not” at every branch without slowing down something that matters. A closed-out miss has none of that. Sit a T2 down next to the senior responder who made the call, on a ticket that’s already resolved and boring, and have them re-walk the tree together, out loud, with room to interrupt. Do that regularly, on both directions — reviewing misses as a team, and pairing on them specifically for anyone still building the tree — and the expertise stops living in exactly one head, because more than one person has now walked the branches themselves instead of just hearing about them after the fact.

None of that works without leadership treating it as legitimate, on purpose. The instinct that makes people freeze on an ambiguous signal in the first place — fear of being wrong somewhere visible — doesn’t disappear just because the venue changed from a war room to a review meeting. If a miss review turns into a place where someone’s judgment call quietly gets held against them later, in a performance conversation or a promotion that doesn’t happen, people stop bringing their real misses and start bringing only the safe, obvious ones, and the whole exercise collapses back into exactly the theater it was supposed to replace. That’s a leadership problem, not a process problem: someone with actual authority has to show up to these, say out loud that they’d have made the same call, and mean it, because the review only works if being wrong in the room is cheaper than staying quiet about it. The same person has to protect the time itself, too — a recurring thirty minutes is nothing until the week gets busy, and it’s busy every week, so it survives only if someone senior enough treats it as non-negotiable instead of the first thing that gets cut when the calendar fills up.

The bill

Reversibility as a rule of thumb breaks down the moment something is only mostly reversible — a rule you can flip back, but not before something read from the exposed thing while it was open; teaching the tree instead of the runbook is slower and harder to standardize, because it’s closer to mentorship than documentation, and it doesn’t scale the way a wiki page does; and reviewing the things that didn’t become incidents means creating a record of every judgment call that turned out fine, which is exactly the kind of paper trail people get nervous about keeping, for reasons that have nothing to do with security.

The synthesis

The tooling problem and the confidence problem turn out to be the same problem wearing two costumes: more dashboards without more judgment just means more things nobody’s sure whether to look at, and neither one gets fixed by buying something. What actually closes the gap is a habit, not a tool — reviewing what got closed as nothing on a real cadence, and treating that review as the mentorship mechanism itself, instead of hoping the tree transfers by osmosis the one time a year a junior person happens to be on the call during a real incident. It costs nothing to start and doesn’t need a budget or a vendor, but it does need someone with actual authority willing to protect it once it exists, because the only thing that kills a review like this faster than a full calendar is one bad experience where someone’s honest miss got held against them later. Start it with a recurring thirty minutes and someone willing to narrate their own reasoning out loud, including the parts that feel too obvious to say; keep it alive by making sure whoever runs it has enough standing that the room actually believes them when they say nothing said here leaves the room. I’ve believed some version of the diagnosis for a long time; what’s new is that cloud has made the shape of a fuller fix visible enough to look buildable, but the review-and-pair habit is the part that never had to wait for that.

Disclosure: written with AI assistance for formatting and research. This one took quite a few hours.

I set out to write a post that didn’t either. I failed, because the same three habits I spend all week defending showed up at the pit table without being invited. I’ll point at them when they appear, briefly, and then we can all go back to arguing about shock oil.

Pick a car, pick a surface, get a direction

The tuner is one React component, no build step required, that answers the question every heat of every club race starts with: the track changed, what do I change? It covers six buggies across Mugen, Mayako, and Tekno — a nitro and electric pair from each brand — and six surfaces, from blown-out dust bowl to blue groove to mud. Pick the car, pick the condition, and it gives direction across eleven setup categories: diffs, shock oil and pistons, springs, bars, links, ride height, toe, steering geometry, tires, wing, and the nitro-vs-electric stuff at the bottom.

The design decision that makes it work is in the word direction. The app almost never tells you a number. It tells you a move:

bumpy: {
  diff: [
    { s: "Front diff",  d: "HOLD —",    v: "baseline",
      n: "Not the first lever for bumps." },
    { s: "Center diff", d: "THINNER ▼", v: "one step down (or more — see car note)",
      n: "Softer power delivery keeps the rear from skating over chop on throttle." },
    { s: "Rear diff",   d: "THICKER ▲", v: "one step up",
      n: "Keeps both rear wheels driving when one is airborne through holes." },
  ],
  ...
}

That’s not laziness; it’s the only honest way to write a multi-car tuning sheet. The kit baselines these cars ship with are genuinely different philosophies. Mugen sends the MBX8R out with 5k/5k/2k diffs and #550 shock oil, and runs that unusually thin center even on the electric car so the Eco drives like its nitro sibling. Tekno’s 2.2 kits are 7k/5k/5k with dual-piston shock stacks and a 24/26mm ride height — already race-tuned for rough tracks out of the box. Mayako publishes medium-THICK-medium as an explicit design philosophy and means it. An absolute number — “run 450 oil,” “run a 7k center” — is correct for one of those cars and quietly wrong for the other five. The direction survives on all six: one step thinner, because the shocks have to cycle faster than the chop. The number is a property of the car. The reason is a property of the physics.

Readers of the detection-altitude post can see where this is going, so I’ll keep it to one paragraph: a direction relative to a baseline is a behavior, and an absolute setting is an indicator. “Run 450 oil” is a hash — right until the platform underneath it changes, then silently wrong. “One step thinner than your baseline, for this reason” transfers across every car in the pits, the same way a detection written against the skeleton transfers across campaigns. I did not plan for my hobby app to restate the Pyramid of Pain. It did anyway, because relative-to-baseline is just what durable advice looks like.

Two smaller things I want on the record. First, when the factory has spoken, the factory wins: the app carries per-car overrides, so on a rough track the Mayako card tells you to drop the center to 5–7k — a huge move off its own thick baseline — because that’s Mayako’s published rough-track guidance, not my opinion. Second, provenance is marked. The README separates what came from a kit manual, what came from the lube bag in the box, and what’s community practice, and every unconfirmed value renders with a ≈ in front of it. I spent a whole post making a point of stating attribution confidence plainly. Shipping a tuning sheet that laundered forum consensus into factory spec would have been an embarrassing way to end the week.

When’s my race up

The second tool exists because of a failure mode every club racer knows: you’re races 14, 38, and 61 on an eighty-race day, LiveRC tells you what’s completed, and somewhere around race 30 you wander to the pit table, open a diff, and hear your class called to the stand with your car in eight pieces.

rc-timer (formerly titled whenup) started as a Python script polling LiveRC from a terminal and grew into a Cloudflare Worker anyone at the event can open on their phone. Track subdomain, driver name, done — the URL is shareable, it refreshes every 75 seconds, and each of your remaining races shows an ETA and a countdown. At twenty minutes out the row turns red and says GET READY, with an optional push notification.

The part I actually care about is how the ETA is computed, because the obvious approach — multiply races remaining by “about eight minutes” — is a static assumption, and race days do not honor static assumptions. Instead the worker measures the day it’s actually having:

/**
 * Compute race pace from completion timestamps.
 * Uses the 25th percentile of inter-race gaps so ETAs err on the early side.
 */
export function measuredPace(completions) {
  if (completions.length < 4) return { paceMinutes: DEFAULT_PACE_MIN, samples: 0 };

  const times = completions.slice(-(PACE_WINDOW + 1)).map(c => c.completedAt);
  const gaps = [];
  for (let i = 0; i + 1 < times.length; i++) {
    const g = (times[i + 1] - times[i]) / 60000;
    if (g > 0 && g <= BREAK_GAP_MIN) gaps.push(g);
  }
  if (gaps.length < 3) return { paceMinutes: DEFAULT_PACE_MIN, samples: gaps.length };
  return { paceMinutes: percentile(gaps, 25), samples: gaps.length };
}

Pace is the gap between consecutive completed races over the last twelve, with anything longer than twenty minutes discarded as a lunch break rather than cadence. The anchor is the most recent completion, so every finished race re-anchors the whole forecast and delays self-correct on the next poll. When a race is actively running, a socket connection to LiveRC’s live timing tightens the anchor in real time off the race clock.

And it’s the 25th percentile, not the median, which is the one deliberate bias in the whole tool. The two failure modes are not symmetric. An estimate that runs early costs you five minutes of standing at the drivers’ stand watching someone else’s main. An estimate that runs late costs you the race you spent a week preparing for. So the estimator is tilted toward the cheap failure on purpose — which regular readers will recognize as the same argument as fail-open triage in Page on Decisions, Not Alerts: when you must be wrong, choose the direction of wrongness whose cost you can afford. Same logic, better weather.

The other habit that snuck in: the tool names its own evidence gaps. Early in the day, before three clean inter-race gaps exist, the API doesn’t silently pretend it knows the pace — it returns a warning: Pace is the 8-min default — only 2 usable gaps so far. Parse failures on a heat sheet don’t vanish either; they come back as warnings naming the round that couldn’t be read. Missing evidence is itself evidence, even when the case being adjudicated is whether you have time for a sandwich.

Under the hood it’s the unglamorous kind of engineering: regex over LiveRC’s HTML (no DOM in a Worker), unit tests running against fixtures of real captured pages so parser changes can’t silently rot, and a 45-second per-track cache so a whole pit lane of drivers watching the same event costs LiveRC one fetch cycle instead of forty.

The bill

Small tools still get an honest invoice. The timer is a screen scraper, and screen scrapers are a standing bet that someone else’s markup won’t change; the fixtures, the tests, and the warnings channel are the hedge, not a fix. The tuning sheet is direction, not gospel — the ≈ values are typical for the platform, unconfirmed by anyone with a dyno, and the app’s own footer tells you to change one thing at a time. And both tools were built in evenings with a coding agent doing most of the typing, which, three posts after The Payload Is a Sentence, means yes — I read every diff. It would take a certain amount of nerve to publish that post on Wednesday and merge unreviewed agent output on Saturday.

The synthesis

I went to the track to stop thinking about detection engineering and came home having rebuilt it in miniature, twice. Know your baseline and express every change as a direction relative to it, because directions transfer and numbers don’t. Measure the thing instead of assuming it, re-anchor on every new observation, and when you must be wrong, be wrong in the direction you can afford. Say plainly what’s confirmed, what’s typical, and what you simply don’t know yet.

The last line of the tuning sheet reads: change one thing at a time and let tires do the first 70% of the work. Swap two nouns and it could close a detection backlog review. Apparently the discipline doesn’t care whether the thing being tuned is a rule set or a buggy — which is either reassuring or a sign I need a second hobby.

Sources: kit baselines from the Mugen lube bag, the Tekno manuals, and Mayako’s setup wiki, as marked in the tuner’s README; everything with a ≈ is community practice, not factory spec.

Disclosure: written with AI assistance, like the tools themselves. The setups, the scraped heat sheets, and the main event I missed with a diff in pieces are all mine.

The unstated dependency

Deriving telemetry from a threat model only works if your detections are expressible as behaviors. If the thing you actually detect on is indicators, meaning this hash, this IP, this domain, then the backward derivation is impossible, because you cannot know in advance which indicators you’ll need. Indicators are unknowable before the incident and perishable after it. An organization whose detection capability lives at the indicator level has no rational choice but to hoard, because “collect everything” is the only sane response to “we can’t predict what we’ll have to match against.”

So the hoarding reflex I spent a whole post attacking isn’t really a discipline problem at all. It’s a symptom, and specifically it’s what detecting at the wrong altitude looks like from the budget side. You can’t lecture an indicator-driven SOC into collecting deliberately, because deliberate collection isn’t available to them: their detection model makes collection undecidable, and undecidable collection defaults to hoarding every time.

That’s the missing term between the two posts. Behavioral detection is the mechanism that makes deliberate collection possible in the first place.

Why behaviors are derivable and indicators aren’t

The difference comes down to what constrains each one. An indicator is an accident of a particular campaign: the attacker picked that domain, compiled that binary, rented that box. Nothing about your threat model predicts it, so nothing about your threat model tells you what to collect for it. You find out what you needed only after you needed it.

A behavior is constrained by the adversary’s objective and by the physics of your environment, and both of those you can reason about ahead of time. To take over an identity and steal data, an attacker has to authenticate, has to add or alter an authentication method or token to persist, and has to touch the data. Those aren’t stylistic choices; they’re load-bearing steps in the objective. Strip away the specific kit and the invariant remains.

The previous post’s passkey campaign is one skin stretched over that skeleton. Storm-2949’s MFA-fatigue-into-SSPR play is a different skin over the same bones. Whatever runs this pattern next quarter with fresh infrastructure and a new lure is a third. If your detection keys on the skeleton, you catch all three; if it keys on a skin, you catch one and wait to be told about the next.

The Pyramid of Pain, pointed at the wrong axis on purpose

David Bianco drew the Pyramid of Pain in 2013, out of the APT1 period, to make a point about detection: indicators sit in tiers by how much it costs the adversary to change them when you deny them. Hashes and IPs are at the bottom because they’re trivial to swap. Domains cost a little more. Tools cost real effort. Tactics, techniques, and procedures sit at the apex because they’re learned behavior, and forcing an adversary to relearn how they operate is the most expensive thing you can do to them. Detect high on the pyramid and you impose durable cost; detect low and you’re playing whack-a-mole with things the attacker regenerates for free.

That’s the canonical reading, and it’s about detection strategy. The two posts before this one imply a second reading on a different axis: detection altitude also determines whether your collection is decidable.

Here’s the same pyramid, read as a collection argument. Detect at the bottom and you can’t derive your telemetry, because the indicators you’ll need are unknowable in advance, so you hoard. Detect at the top and each log source becomes justifiable from the threat model, because a behavior traces cleanly back to a modeled threat in a way an indicator never can. The unit of justification changes. At the bottom it’s “which detection does this source enable,” and you can’t answer it before the fact. At the top it becomes “which behavior do I need to be able to see,” and that you can answer, because you wrote the threat model that names the behavior.

Move up the pyramid and collection becomes a design problem with a right answer. Stay at the bottom and it stays a hoarding problem with no floor.

The whole argument lives in two queries

I already shipped the worked example in the last post without naming it as one. The passkey post carried two sign-in-correlated queries, and the difference between them is exactly this axis.

The retro-hunt query keyed on the actor’s ASNs, DDoS-Guard and IQWeb, which is the infrastructure Okta published. That’s a skin, near the bottom of the pyramid, it’s disposable by design, and I told you to run it once and throw it away precisely because the moment the actor re-hosts it goes silent and starts lying to you about being clean. It depends on an indicator I can only know after someone else caught the campaign.

The standing detection keyed on the skeleton: an authentication-method registration inside a short window of a sign-in from an ASN that user has never used. No indicator anywhere in it. It survives the actor re-hosting, and it survives the actor switching techniques entirely, because the behavior it watches (persist by adding a credential, from somewhere anomalous) is downstream of the objective, not the tooling. That query is derivable from the threat model. I could have written it before O-UNC-066 existed, and it’ll still be standing after Pink rebrands.

One is a thing you match. The other is a thing you model. That’s the pyramid, and it’s why only one of the two belongs in your rule set.

The bill, because a post that only sells the upside is the thing I warned you about

Behavioral detection is not free and it does not mean “collect less.” It means collect a different shape, and it comes with costs the first post’s own logic demands I put on the table.

Behaviors need baselines, and baselines need retained history. The standing detection fires on “an ASN this user has never used,” which means something has to have kept enough sign-in history to define “never.” That actually amends the three-question test from the invoice post. Some sources earn their keep not by enabling a detection directly but by being the baseline substrate a behavioral detection stands on. So the test grows a fourth question: does this source establish or enrich a baseline that a detection depends on? A log that’s useless in isolation can be load-bearing as history.

A single behavior is noisy; the fidelity is compositional. First-seen-ASN on its own fires on every business trip and every new VPN egress. What makes the standing detection worth an analyst’s minutes is the conjunction: new network and a credential change and inside the same hour. The rarity lives in the sequence, not in any one event. Get the composition wrong and you’ve rebuilt the alert-fatigue problem from the first post one layer up; get it right and the analyst-hour math finally works in your favor. This is the actual craft, and it’s where most behavioral programs either pay off or quietly bankrupt themselves.

It demands fielded, entity-resolved data at ingest, not raw blobs you grep later. To correlate a registration with a sign-in you need the who, the from-where, and the against-what already resolved and sitting next to your reference data. That is the same “right fields at the right fidelity, next to the reference data that makes the alert actionable” that the invoice post asked for, which means behavioral detection is the reason you shape and strip at ingest instead of warehousing raw JSON. Deliberate collection stops being a cost-cutting story and becomes a capability you’re building toward.

MITRE is already scoring this

If the collection reading of the pyramid were only my own extrapolation, I’d flag it as such. It isn’t. The MITRE Center for Threat-Informed Defense has been building the formal version since 2023 under the name Summiting the Pyramid, and its whole purpose is to score how robust a detection is (how hard it is for an adversary to evade) by scrutinizing what the analytic actually depends on. Detections that lean on swappable artifacts score low; detections anchored to behavior score high. The Sigma repository now carries a robustness flag from this work, so you can see the score on open analytics.

The part that matters for the collection argument is where they took it next. The v4.0 release adds a methodology for the minimum telemetry required to detect ambiguous, living-off-the-land techniques, and Telemetry Confidence scores that rate how effective a given log source is against a given technique. Read that back against the invoice post. That is a formal, published answer to “which sources do I actually need, and how much does each one buy me,” computed from the behavior you’re trying to catch, not from a diagnostic default someone left on. Their 2026 roadmap frames it as making collection purposeful and measuring telemetry completeness against ATT&CK. The field is converging on the same place from the research side that the budget forced me to from the operations side: your log sources should be a function of the behaviors you’ve decided to detect, and you can now put numbers on it.

The synthesis

You don’t collect deliberately by being frugal. You collect deliberately by detecting behaviors, because a behavior is the only detection target that derives from a threat model instead of from yesterday’s incident. Frugality is the byproduct; altitude is the cause.

That closes the arc these three posts were tracing: why you should collect deliberately, what a source that earns its keep looks like in the wild, and now the mechanism underneath both. Detect high on the pyramid and the question “why is this log here” finally has an answer you can derive instead of defend. Detect low and you’re back to collecting everything and hoping, paying the invoice on telemetry no behavior ever needed.

Point your detections at the skeleton. The collection strategy falls out of it for free.

Sources: David Bianco, “The Pyramid of Pain” (2013). MITRE Center for Threat-Informed Defense, “Summiting the Pyramid” (project and v4.0 documentation, incl. Telemetry Confidence scoring) and the CTID 2026 roadmap. Actor and query specifics carry over from the prior post and its cited Okta / Unit 42 reporting.

Disclosure: written with AI assistance. The threat model, the argument, and the opinions are mine.

The arc those posts traced was about detection altitude: detect the adversary’s skeleton, not the skin, and derive your telemetry from the behavior you’ve decided to catch — the argument I lay out in full in a companion post. I want to point that exact lens at a surface that barely existed two years ago and now sits inside almost every engineering org I know, mostly unreviewed. The files your coding agent reads to decide how to behave. CLAUDE.md, AGENTS.md, SKILL.md, .cursor/rules, the Copilot instructions file. The uncomfortable thing I keep arriving at is that the worst version of an attack on these files is the one my own framework says is hardest to catch, and the first rungs of it already shipped as real malware while I was writing about log invoices.

The file your agent obeys is not reviewed like it obeys it

Here’s the asymmetry that started this. An instruction file is read by an agent that has commit rights, push rights, sometimes deploy rights, and a shell. The agent treats the file’s contents as intent, as what the human wants, and acts on it with all of that authority. That is the most privileged possible reading of a file. And the file gets reviewed, if it gets reviewed at all, like a README: skimmed, rubber-stamped, exempt from the scrutiny a .github/workflows change would draw, because it’s “just documentation.”

We already learned this lesson once, on the CI side. A workflow file is plain YAML that looks like configuration and is in fact remote code execution with your repo’s secrets attached, and it took a wave of incidents before teams started reviewing workflow changes like the privileged objects they are. The instruction file is the same category error, one layer up and much softer, because at least the workflow file runs through a parser with a schema. The instruction file runs through a language model that will do its sincere best to carry out whatever the file appears to ask for, including the parts a human skimming the diff never registered.

The rungs that already exist

I want to separate what has happened from what I’m extrapolating, because the credibility of the worry depends on the seam between them.

What has happened, in rough order of nastiness:

In March 2025, Pillar Security disclosed what they called the Rules File Backdoor: malicious instructions smuggled into Cursor and Copilot rules files using hidden Unicode, bidirectional-text markers and zero-width joiners, so the payload is invisible in the diff a human reviews and fully legible to the agent parsing the file. GitHub eventually started warning on hidden Unicode in the web UI. The important part isn’t the Unicode trick, which is patchable and now partly mitigated. It’s the proof that the instruction file is a viable injection point at all.

Then the skill ecosystems arrived and made it worse, because a skill is an instruction file that ships like a package. Researchers demonstrated a SKILL.md that read as a plain GitHub integration while carrying a hidden instruction to exfiltrate repository contents; Anthropic patched that specific vector in Claude Code in February 2026. Snyk audited a few thousand skills from a public registry and found roughly a third carried at least one security flaw and dozens carried confirmed malicious payloads. CVE-2025-59536, from Check Point Research, was the blunt version: a .claude/settings.json committed to a repo could specify a session-start hook that ran a shell command the moment you opened the project, before any trust prompt appeared. A companion flaw in the same project-file trust turned it into an API-key exfiltration path.

And then, in May and June of 2026, someone actually built the thing. The npm worm tracked as Miasma, or Mini Shai-Hulud, used a package install as initial access, then scanned the developer’s filesystem for exactly these files, Claude and Cursor and Gemini and VS Code configs, and wrote itself into them, including a session-start hook that re-ran the malware every time a new agent session opened. It self-propagated by stealing npm tokens and poisoning more packages. Snyk separately documented payloads whose instruction was to have the agent write malicious instructions into other context files on the machine. Which is to say: the agent was directed to spread the infection itself.

If you’d asked me to design this attack as a purple-team exercise, that’s most of the design, and I’d have felt clever. It’s already in the wild. So the interesting question isn’t “could this happen.” It’s “what’s the version that’s worse than Miasma, and why haven’t we seen it yet.”

The version I actually worry about

Every rung above has one thing in common that makes it catchable: there is something that runs. A hook, a dropper, a node process, a postinstall script. The entire current detection wave, and it is mobilizing fast, is tuned to that. Watch for the shell command in settings.json. Flag the postinstall. Hash the dropper. Scan the skill for bundled scripts. All of it keys on an executable artifact somewhere in the chain.

Now take the executable away.

The version I keep arriving at carries no hook and no script. The payload is a sentence. It’s a paragraph of plausible-sounding engineering guidance sitting in AGENTS.md that says, in effect: when you touch authentication middleware, prefer this pattern; when you generate infrastructure, this egress range is approved; when you handle these tokens, log them here for debugging. Nothing executes at infection time. There is no sample to hash, because the sample is English. There is no process to catch, because the compromise doesn’t run, it waits, and it fires later, in the agent’s own output, as helpful-looking code the agent sincerely believes it was asked to write.

This is exactly the shape I argue is hardest to catch in that companion post, and that’s why it bothers me. An executable dropper is an indicator: near the bottom of the pyramid, disposable, but real, hashable, catchable. A sentence of malicious intent is not an indicator in any useful sense. You can rephrase it infinitely. There is no byte sequence to match, no ASN, no domain, nothing to put on a blocklist. The malicious instruction and a legitimate one are the same kind of object, natural-language guidance, and telling them apart requires understanding what the agent will do with it, not what it is.

Two properties make the theoretical version genuinely worse than what’s shipped, and both are things the current defenses don’t look at.

The first is that it’s aimed at the human reviewer, not just the agent. Every disclosure so far stops at “the agent emitted bad code.” None of them model an attacker who has also read the target team’s git history and learned how the team reviews: the commit-message voice, the review cadence, who rubber-stamps and who actually reads. The payload doesn’t have to produce obvious badness. It has to produce a change small enough and plausible enough to ride through a real code review inside a large, legitimate-looking refactor, under the diff-size noise floor, ideally late on a Friday. The agent writes commits in your team’s own idiom because it learned that idiom from your repo. The last unguarded step in the whole chain isn’t the agent, it’s the human who approves the pull request, and a semantic payload can be tuned to that human specifically.

The second is the propagation channel. Miasma spread by stealing tokens. The sentence doesn’t need tokens. It rides git. Poison one AGENTS.md in a popular starter template, an internal scaffolding repo, a widely-forked example, and it copies itself into every downstream repo that clones the template, every team that adopts the shared skill, every pipeline that reads the file. It’s a worm whose transport is normal, encouraged developer behavior, reuse the template, adopt the standard, pull the shared config, and whose body is a paragraph nobody re-reads because it came from the blessed source. There’s no anomalous callback to catch it in flight. The spread is just people sharing config the way we keep telling them to.

How bad it gets

Ride this two more hops and you leave code integrity behind entirely.

Hop one is production. The poisoned instruction produces a commit, the commit survives review because it was built to, and now the behavior is in the shipped artifact: a weakened auth path, an approved egress that shouldn’t be, a token quietly logged somewhere the attacker can read. That’s a supply-chain compromise reached without ever delivering a binary to the target. The initial access was a sentence in a config file; the delivered payload is your own engineers’ reviewed, merged, signed code. That laundering is the point. By the time it’s in production it has your provenance on it, not the attacker’s.

Hop two I’ll flag clearly as extrapolation, because I have not seen it demonstrated end to end and I don’t want to launder speculation as reporting. If the agent’s output flows into anything that trains or grounds a customer-facing model, an eval set, a fine-tuning corpus, a RAG knowledge base, few-shot examples pulled from the repo, then the poisoning stops being a code problem and becomes a model problem. And the reason this keeps me up is that the data-poisoning research says the thresholds are horrifyingly low. Anthropic, with the UK AI Security Institute and the Alan Turing Institute, found in late 2025 that backdooring a model takes a near-constant, small number of poisoned documents essentially regardless of how big the clean dataset is: as few as 250, holding roughly flat from a 600-million-parameter model up to a 13-billion-parameter one trained on more than twenty times the clean data. A fixed count, not a percentage. RAG-poisoning studies land in the single digits of malicious documents for high attack success against a targeted question. These are not “boil the ocean” numbers. They’re “get a handful of poisoned artifacts into the corpus” numbers, and an agent that’s been quietly emitting attacker-shaped content into a repo for months is a very efficient way to manufacture exactly that handful, with your provenance on every one.

And notice the shape this takes if the model you’ve poisoned is itself a coding assistant. Its output flows back into repositories, into instruction files, into the next training corpus, which trains the next model, which writes the next batch of code. At that point the attack doesn’t need the original package anymore. It comes full circle: the compromised output becomes the input that sustains the compromise, and the whole thing rides the ordinary machinery of how these models get built and used, which is the one piece of infrastructure nobody is going to turn off.

I’m not claiming anyone has run that full chain end to end, or closed that loop. I’m claiming every individual link in it is already demonstrated, and the seams between the links are made of exactly the trust we currently extend to instruction files by default.

You can’t hash a sentence

Here’s where the last three posts pay off, because the defense falls out of the same argument.

You cannot detect this at the indicator level. There is no hash, no domain, no byte pattern, because the payload is meaning and meaning is infinitely re-expressible. Anyone selling you a scanner that greps instruction files for “malicious phrases” is selling you the blocklist-of-hashes model one abstraction up, and it will lose the same way, for the same reason, the moment the attacker rephrases. If your whole answer is signatures on the file contents, you’ve already lost: you’re detecting the skin.

So you detect the behavior, and the behavior here is not in the file’s contents at all. It’s in the file’s lifecycle and in the divergence between the file and everything around it. A few skeletons, stated as behaviors so they survive the attacker rephrasing:

An instruction file changed, and no human reviewed the diff. That’s the load-bearing one. These files should be the most privileged config in the repo, reviewed like .github/workflows and gated by CODEOWNERS, because they are executed-as-intent by something holding a shell. A change to AGENTS.md that merged without a human on the diff is an event, full stop, independent of what the change says.

An instruction file’s provenance doesn’t match its content. It arrived via a template pull, a dependency update, a package install, a machine wrote it and not a person, and now it carries behavioral guidance. Machine-authored changes to the agent’s own instruction set are a different risk class than a teammate editing coding standards, and they’re distinguishable if you’re watching where the write came from.

The agent’s output drifted from written policy. This is the compositional one, and it’s the expensive one, and it’s where the real program lives. If you have a written policy, auth patterns, approved egress, logging rules, then agent-generated commits can be checked against it as a behavior, and a persistent, low-grade lean toward “technically passes review but always wrong the same way” is a signal no single diff will show you. One suspicious commit is noise. A statistical lean across a body of agent output is the behavior, exactly like the first-seen-ASN-plus-credential-change from the passkey hunt… the rarity lives in the conjunction over time, not in any one event.

None of these are telemetry most shops collect today. Almost nobody diffs and alerts on CLAUDE.md the way they alert on a workflow change. Almost nobody records the provenance of a write to .cursor/rules. Almost nobody baselines agent output against policy to measure drift. Which is the same finding as every post before this one: the log source you need is a function of the behavior you’ve decided to catch, and you haven’t decided to catch this one yet, so you’re not collecting for it. The instruction layer is a new altitude on the pyramid, and right now it’s completely uninstrumented.

The synthesis

The thing I keep coming back to is how ordinary the whole attack looks from every angle we currently watch. No binary, no callback, no anomalous process. A sentence in a file we don’t review, read by an agent we’ve handed a shell, producing code our own engineers approve, possibly feeding a model our customers talk to. Every hop is invisible to a defense that’s looking for something that runs, because nothing runs until it’s already our own trusted output doing it.

So treat the instruction layer as executable, because to your agent it is. Review CLAUDE.md and AGENTS.md and .cursor/rules like the privileged code they functionally are. Gate them with owners. Record who, or what, wrote them. And measure your agent’s output against policy over time, because that’s the only place the sentence eventually has to show its work.

The payload is a sentence. You can’t hash it. You can only model what it makes the agent do, which is the whole argument of this blog, pointed at the newest and least-guarded surface I know of.

Sources: Pillar Security, “New Vulnerability in GitHub Copilot and Cursor” — the Rules File Backdoor (disclosed to Cursor Feb 2025, GitHub Mar 2025; GitHub later added a hidden-Unicode warning). Reporting on the Miasma / Mini Shai-Hulud npm worm and its AI-agent config-file injection (StepSecurity, Snyk, SafeDep; May–June 2026). Cloud Security Alliance, “Agent Context Poisoning: SKILL.md and the New AI Supply Chain Attack Surface” (May 2026). Snyk, “ToxicSkills” skill-registry audit (2026). Check Point Research, CVE-2025-59536 (pre-trust SessionStart-hook RCE, CVSS 8.7, fixed in Claude Code 1.0.111, Oct 2025) and the companion CVE-2026-21852 (API-key exfiltration). Data-poisoning thresholds: Anthropic with the UK AI Security Institute and the Alan Turing Institute, “Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samples” (Oct 2025; ~250 documents), plus RAG-poisoning success-rate research. Pyramid-of-Pain framing carried over from the prior post and its cited sources.

Disclosure: written with AI assistance. The threat model, the argument, and the opinions are mine.

I’ve been working through what it looks like to split those two jobs apart. The short version: treat detections as sensors, and build a separate layer whose only job is turning signals into decisions.

A smoke detector is not a fire department

A detection is evidence. It tells you something happened. Whether that something matters depends on the actor, the asset, data sensitivity, change context, related signals, and history, none of which live inside the rule that fired. A decision is a different object entirely. It says what happens next.

So the design shift is to stop asking every analytic to be page-worthy. Most rules become candidate signals: normalized observations that flow into a triage layer instead of straight into PagerDuty. That layer gathers context, applies written policy, and produces one of a small set of dispositions: close benign, correlate as duplicate, monitor, open a non-paging work item, escalate to Tier 2, or page on-call.

The first stage gets to be noisy because its output is internal. The second stage optimizes for precision because its output reaches a human at 2am. This is basic cascade architecture, and it’s the same economics argument I made in Every Log Source Is an Invoice applied one layer up: you paid to collect the telemetry, so stop paying again in human attention for every observation it generates.

Where the LLM fits, and where it doesn’t

The triage layer I’m describing is LLM-guided, and I want to be careful about what that means, because “let the model decide security risk” is not it.

The model is a reasoning interface over approved tools and approved policies. It receives a candidate signal, normalizes the entities, pulls the evidence a policy requires (identity risk, device compliance, resource ownership, data classification, change records, related alerts, telemetry health), builds an investigation graph, and applies policy gates. Its authority comes from the policy, not from conversational freedom. It can close a case only when a specific policy authorizes closure and every evidence requirement is met. Anything else escalates.

The governing rule is fail open. Missing evidence, degraded telemetry, ambiguous policy, tool failure: all of these escalate rather than close. If the Purview classification lookup fails, the agent does not infer benign. It records the missing evidence and hands the case to a human with the gap named. Missing evidence is itself evidence, and treating it that way is most of what makes autonomy defensible.

Policy as English

This is the part I find most interesting in practice. Every SOC already has triage policy. It lives in response KBs, tribal knowledge, and playbook conditions, and it’s enforced with whatever consistency a tired responder can manage while finding the right Confluence page under time pressure. Two people read the same KB and do different things. That’s not a people problem, it’s a medium problem.

The fix is to make the KB executable without making it code. A production English-language policy has structure: scope, required evidence, escalation criteria, closure criteria, forbidden actions, an owner, and a review cadence. Something like:

If diagnostic settings are changed on a production or sensitive-data-adjacent resource, verify the actor, the change record, the before/after logging state, and related alerts. Escalate if logging was disabled, routed away from approved destinations, or the actor and change are not approved. Close only if the change is approved, expected, and logging remains healthy. If required evidence is missing, escalate as evidence-insufficient.

That’s readable by a detection engineer, an IR lead, and an auditor. It’s also precise enough to replay against historical cases and version in git. The LLM interprets it consistently at 3am the same way it does at 3pm, which is more than I can say for humans, myself included.

Every decision leaves a record

The artifact analysts review changes too. Instead of an alert plus a pile of raw logs to pivot through, the agent hands over an investigation graph: identity, device, resource, classification, change context, and related signals already connected, with a timeline. Attacks rarely arrive as one obvious signal. A risky sign-in, a privilege change, a logging modification, and a first-time access to sensitive data are each individually dismissible, and together they’re the whole story. The graph is the object worth reviewing.

Every graph resolves into a decision record: disposition, severity with mission impact, confidence, evidence used, evidence missing, the policy path that produced the outcome, and the recommended next action. A decision without that record isn’t acceptable for autonomous operation, because you can’t audit it, replay it, or improve it. This also quietly solves the tuning problem. When the agent closes the same benign automation pattern forty times a week, that’s not just relief, it’s a structured tuning backlog with evidence attached.

Autonomy is earned, not enabled

None of this starts with the model closing cases. It starts in shadow mode: the agent evaluates alerts in parallel while current paging behavior stays exactly as it is. Agreement rate with human disposition, evidence completeness, and false-closure risk become the calibration dataset. Disagreements aren’t failures, they’re the interesting part, and each one gets categorized: evidence gap, policy ambiguity, model reasoning error, or human inconsistency. That last category shows up more often than anyone wants to admit.

Only after those metrics clear predefined gates does an alert class graduate to assisted triage, then to autonomous disposition for narrow low-risk categories. High-impact conditions never enter that pipeline at all. Confirmed exfiltration, privileged identity compromise, logging disabled on critical stores: those keep a direct-page path where the agent enriches after the page, not before. And every material change to the model, prompts, policies, or tool schemas has to pass replay against historical incidents, near-misses, and synthetic cases before it ships. Autonomy is a maturity level per alert class, not a switch.

The biggest risk in the whole design is false confidence: believing the agent understood a case when it didn’t. The mitigation isn’t avoiding AI, it’s refusing to accept any decision that doesn’t show its evidence, name what’s missing, and cite the policy it applied. The agent may reduce toil. It may not reduce accountability.

What actually changes for humans

The on-call queue becomes an exception queue. Responders stop reconstructing context from scratch for every raw analytic and start reviewing adjudicated risk: evidence gaps, high-impact uncertainty, and cases where policy deliberately requires judgment. The Tier 1 skill set shifts from memorizing KB edge cases to challenging evidence quality and correcting policy outcomes, which is a better use of experienced people anyway.

There’s a role hiding in this model that I think detection engineering grows into: someone who owns the conversion of evidence into action. Call it a decision engineer. They own policy quality, evaluation cases, routing behavior, and evidence completeness, because in this model the work product is no longer the alert. It’s the decision, and the counterintuitive result is that detection count should go up, not down. You can afford broad behavioral analytics when a noisy observation costs a policy evaluation instead of a page.

The source is the Entra authentication-methods audit stream, and the threat that makes it worth the money is a group that turns your passkey rollout, the thing your identity team has been begging users to adopt, into the phishing lure itself.

The actor

Okta Threat Intelligence published on this cluster on July 6, 2026. They track it as O-UNC-066. Palo Alto Networks Unit 42 tracks the same activity as CL-CRI-1147 and reports it under the extortion brand Pink, which stood up a data leak site on May 31, 2026. Unit 42 places Pink inside The Com, the same decentralized cybercrime milieu that produced Scattered Spider, ShinyHunters, and LAPSUS$. The motivation is data extortion: get in, pull data out of SharePoint and OneDrive, then name-and-shame on the DLS to force payment.

Victimology, per Okta, spans food and beverage, technology, healthcare, automotive, construction, and aviation. That spread tells you what this is: not a targeted intrusion set picking a single vertical, but a repeatable social-engineering script run at scale against whichever help desk or user picks up the phone.

Attribution confidence here is reasonable but worth stating plainly: Okta observed the phishing kit and the infrastructure directly; the Pink/The Com linkage comes from Unit 42; and Okta notes it did not directly confirm a Microsoft account compromise from its own vantage point, so it’s inferring the account-takeover objective from the kit’s design. If you have Recorded Future, this is the first place to spend your pivots: confirm the DLS victim list against the sectors above, and validate the Com affiliation against RF’s own clustering rather than taking the vendor label at face value.

The technique: weaponizing the security upgrade

As of May 2026, Microsoft admins can run passkey registration campaigns, the sign-in “nudges” that prompt users to enroll a passkey, and in some configurations those nudges are on by default, which means the pretext already exists inside the tenant. The actor just has to phone a user and say the quiet part your own IT team has been saying for months: you need to set up a passkey.

The delivery is voice phishing (vishing). The actor registers domains with the word passkey baked in (Okta listed assignpasskey[.]com, deploypasskey[.]com, passkeydeploy[.]com, passkeyadd[.]com, setpasskey[.]com) and stands up per-target subdomains that mimic the victim org’s Entra sign-in page. Generic Microsoft chrome loads from Microsoft’s real CDN; the victim-specific logo and background are pre-staged in the kit backend. Infrastructure has been hosted on DDoS-Guard (AS57724) and IQWeb FZ-LLC (AS59692).

The kit is where it gets unusual, because it isn’t a transparent adversary-in-the-middle proxy, which is the pattern most of your AiTM detections are tuned for, but an operator-controlled PHP panel. A human operator steers the victim through the auth stages in near real time using a one-second heartbeat polling loop, adapting the page flow to whatever MFA the victim actually gets challenged for: TOTP, push with number matching, SMS OTP. The documented kit stages: /gate (anti-analysis checks), /identify (username), /password (credentials POSTed to /backend.php), then /processing (a stall screen while the operator replays the creds against the real Microsoft login). There’s even a decoy step asking the user to save and confirm a word from a BIP-39 seed phrase — which has no role in real Entra passkey enrollment and appears to exist purely to make the ceremony feel legitimate to a user who’s never done this before.

The endgame: while the victim thinks they’re enrolling a passkey, the operator enrolls their own passkey against the victim’s account, and can name it something benign so the legitimate Microsoft “new passkey registered” email doesn’t raise alarm.

Why “we deployed passkeys” doesn’t save you here

Passkeys are phishing-resistant as an authentication method, because the private key never leaves the authenticator and there’s nothing to replay, but the enrollment ceremony is a different surface entirely. Phishing resistance protects the sign-in; it does nothing to stop a socially-engineered user from adding an attacker’s credential to their own account. The moment you make self-service passkey setup easy, which is exactly what a registration campaign does, you’ve created a workflow that a caller can walk a user through.

So the control that matters isn’t whether you have passkeys, it’s who is allowed to enroll one, from where, and how you find out when it happens.

Deriving the detection

Same discipline as always: threat model, then detection requirement, then telemetry requirement, then the query. Don’t start from “what can I write in KQL.”

Threat: an attacker socially engineers a user into enrolling an attacker-controlled passkey (or any auth method), then uses the account to exfiltrate from SharePoint/OneDrive.

Detection requirement: alert when a security-info / passkey registration happens in a context that doesn’t look like a normal user self-service enrollment. Specifically, shortly after a sign-in from infrastructure or a network the user has never used, with enough context that an analyst can act without rebuilding the case.

Telemetry requirement: the Entra authentication-methods audit events (in AuditLogs), sign-in context (SigninLogs), and file-operation logs for the exfil stage (OfficeActivity). All three are first-party and, if you’re already ingesting Entra and M365 into Sentinel, already paid for, which is what a source earning its keep actually looks like.

A note on grounding, because it’s the whole point of the last post: I verified every table and column below against the Microsoft Azure Monitor table references before publishing. One thing I deliberately did not do is hardcode a nested JSON path for the method type (e.g. digging into TargetResources[].modifiedProperties to isolate “FIDO2” specifically). That structure varies and I couldn’t verify a stable path, so keying a detection on it would be exactly the kind of assumed-field-name mistake that breaks silently. Instead the queries key on the verified operation names and correlate with sign-in context, which is the stronger signal anyway. If you want method-level granularity, sample your own tenant’s records first and confirm the path before you trust it.

Four queries follow, and they are not four options to choose between. The first is orientation, the second is a one-time retro-hunt you run and then throw away, the third is the analytic rule you actually deploy, and the fourth catches the exfil if the first three miss. Only #3 belongs in your rule set.

1. Baseline hunt: security-info and passkey registrations

Start by just seeing the events. These OperationName values and the LoggedByService value are from the Entra audit activity reference; the TargetResources[0].userPrincipalName extraction is Microsoft’s own documented pattern.

AuditLogs
| where TimeGenerated > ago(7d)
| where LoggedByService == "Authentication Methods"
| where OperationName in ("User registered security info",
                          "User registered all required security info",
                          "Get passkey creation options")
| where Result == "success"
| extend UPN = tostring(TargetResources[0].userPrincipalName)
| project RegTime = TimeGenerated, UPN, RegOperation = OperationName,
          RegCorrelationId = CorrelationId, InitiatedBy
| order by RegTime desc

2. Retro-hunt: were we already hit?

This one is not a detection, and you should not deploy it as a scheduled analytic rule. It answers a single, backward-looking question you have exactly once (did this already happen to us before we knew the campaign existed?) and then it stops being useful.

The reason it works at all is a quirk of the kit’s design. Because the panel operator replays the harvested credentials against the real Microsoft sign-in page from their own infrastructure, the sign-in Entra records carries the operator’s IP and ASN, not the victim’s. The victim’s browser only ever talks to the phishing domain, which never appears in your tenant’s logs. So the operator’s hosting is the thing you can actually see, and SigninLogs.AutonomousSystemNumber is where you see it.

Run it once over as long a lookback as your retention allows. The campaign has been active since April, so 30 days is the floor rather than the target. Triage every hit by hand, then retire it.

// Retro-hunt only. Do NOT schedule this as an analytic rule.
// Widen the 30d windows to match your retention; campaign dates to April 2026.
let actorAsns = dynamic(["57724", "59692"]);  // DDoS-Guard, IQWeb: pull current infra from RF first
let regs =
    AuditLogs
    | where TimeGenerated > ago(30d)
    | where LoggedByService == "Authentication Methods"
    | where OperationName in ("User registered security info",
                              "User registered all required security info")
    | where Result == "success"
    | extend UPN = tolower(tostring(TargetResources[0].userPrincipalName))
    | project RegTime = TimeGenerated, UPN, RegOperation = OperationName;
SigninLogs
| where TimeGenerated > ago(30d)
| where AutonomousSystemNumber in (actorAsns)
| where ResultType == "0"
| extend UPN = tolower(UserPrincipalName)
| project SignInTime = TimeGenerated, UPN, IPAddress, ASN = AutonomousSystemNumber,
          Location, AppDisplayName, UserAgent
| join kind=inner regs on UPN
| where RegTime between (SignInTime .. (SignInTime + 1h))
| project UPN, SignInTime, RegTime, IPAddress, ASN, Location, AppDisplayName, UserAgent, RegOperation
| order by RegTime desc

Two caveats that matter. DDoS-Guard and IQWeb are shared hosters, so an ASN match is a reason to look, not proof of compromise, so triage the hits rather than paging on them. And pull the current infrastructure from Recorded Future before you run this, because the ASNs Okta published in July will not be the ASNs in use by the time you read this.

The reason to keep this query firmly in the retro-hunt bucket is the failure mode if you don’t. An IOC-anchored rule that runs on a schedule keeps returning zero results long after the actor has re-hosted, and a rule returning zero looks exactly like “we’re clean” when it actually means “my indicators are stale.” That’s worse than having no rule, because it buys false confidence. Which is why the standing detection has to be the next one.

3. The standing detection

This is the one you deploy. It flags a security-info registration within 60 minutes of a sign-in from an ASN the user hasn’t touched in the prior two weeks. There’s no indicator dependency anywhere in it, so it survives the actor re-hosting, and it will catch the next group that runs this play with entirely different infrastructure.

let lookback = 14d;
let userAsnBaseline =
    SigninLogs
    | where TimeGenerated between (ago(lookback + 1d) .. ago(1d))
    | where ResultType == "0"
    | summarize KnownAsns = make_set(AutonomousSystemNumber)
        by UPN = tolower(UserPrincipalName);
let recentSignins =
    SigninLogs
    | where TimeGenerated > ago(1d)
    | where ResultType == "0"
    | extend UPN = tolower(UserPrincipalName)
    | project SignInTime = TimeGenerated, UPN, IPAddress,
              ASN = AutonomousSystemNumber, Location, UserAgent;
let recentRegs =
    AuditLogs
    | where TimeGenerated > ago(1d)
    | where LoggedByService == "Authentication Methods"
    | where OperationName in ("User registered security info",
                              "User registered all required security info")
    | where Result == "success"
    | extend UPN = tolower(tostring(TargetResources[0].userPrincipalName))
    | project RegTime = TimeGenerated, UPN, RegOperation = OperationName;
recentRegs
| join kind=inner recentSignins on UPN
| where SignInTime between (RegTime - 60m .. RegTime)
| join kind=leftouter userAsnBaseline on UPN
| extend FirstSeenAsn = isnull(KnownAsns) or not(set_has_element(KnownAsns, ASN))
| where FirstSeenAsn
| project UPN, SignInTime, RegTime, IPAddress, ASN, Location, UserAgent, RegOperation
| order by RegTime desc

Tune the window and the baseline period to your environment. A legitimate first-time passkey enrollment on a new corporate egress will trip this, which is fine, because a human should glance at a passkey enrollment tied to a never-before-seen network anyway. That’s the point of a source that earns its keep: the false positives are still worth a look.

4. The exfil stage

Account takeover is only the means; SharePoint and OneDrive theft is the end, so watch for the mass-download pattern. OfficeWorkload, Operation, UserId, ClientIP, and SourceFileName are all verified OfficeActivity columns; FileDownloaded and FileSyncDownloadedFull are the canonical download operations.

OfficeActivity
| where TimeGenerated > ago(24h)
| where OfficeWorkload in ("OneDrive", "SharePoint")
| where Operation in ("FileDownloaded", "FileSyncDownloadedFull")
| summarize FileCount = count(),
            DistinctFiles = dcount(SourceFileName),
            SampleFiles = make_set(SourceFileName, 25),
            IPs = make_set(ClientIP, 10),
            FirstSeen = min(TimeGenerated),
            LastSeen = max(TimeGenerated)
    by UserId
| where DistinctFiles > 200   // tune to your environment's normal
| order by DistinctFiles desc

Correlate a hit here back to Query 3 on UserId / UPN. A first-seen-ASN sign-in, a passkey registration, and a download spike on the same account inside a few hours is the whole kill chain in three tables.

Hardening

Detection is the backstop; the cheaper wins are upstream, and they’re all in the vendor guidance:

• Constrain who can self-enroll. Restrict passkey (FIDO2) self-service setup, and where feasible require enrollment from a managed/compliant device via Conditional Access. If a registration campaign is driving your rollout, scope it rather than leaving the nudge on by default tenant-wide.
• Treat inbound “enroll a passkey” calls as hostile by default. Any passkey/MFA enrollment request that originates from a phone call the user didn’t initiate should be verified out-of-band through a known channel before anyone touches a URL.
• Verify the help desk, both directions. Establish a scripted way for users to confirm they’re actually talking to IT, and for IT to confirm the user, before any credential change.
• Audit existing passkeys on privileged accounts. Don’t assume “MFA is on” means the account is clean. Review which passkeys are enrolled, and flag names that look off or enrollments you can’t tie to a known device.

Where this leaves the invoice argument

The Entra authentication-methods audit stream is not a glamorous source: it isn’t big, it doesn’t light up dashboards, and if you asked most SOCs to justify its retention they’d shrug. But run the three-question test on it and it goes three for three, because it enables a detection (attacker passkey enrollment), it speeds an investigation (the full ATO-to-exfil chain lives across it and two sibling tables), and it satisfies a compliance obligation (credential-change auditing). That’s the profile of a source worth paying for, and it’s the same source most orgs already ingest without ever pointing a detection at it.

Which is the actual argument. Collecting deliberately only pays off if you then point something at what you collected.

Sources: Okta Threat Intelligence (July 6, 2026); Palo Alto Networks Unit 42; corroborated across BleepingComputer, SecurityWeek, The Hacker News, and TechNadu reporting. Table and field references verified against Microsoft’s Azure Monitor AuditLogs, SigninLogs, and OfficeActivity schema documentation prior to publication.

Disclosure: written with AI assistance. The threat model, the KQL, and the opinions are mine.

I want to be fair to that lesson, because it used to be true. On-prem enterprises were genuinely starved for telemetry. Coverage was accidental. What got logged depended on which sysadmin built which server in which year. Turning on one more source really did buy you visibility you didn’t have, and “collect everything” was the right answer to the world we were operating in.

It’s not the world we’re in anymore, and the reflex it trained is now wrecking budgets and burning out analysts.

Some context for why this is on my mind. I’ve been reading through the detection engineering books Packt has put out over the last couple of years, and they’re good. The field finally gets treated like engineering: rules in version control, tests, validation, named owners, an actual lifecycle instead of folklore. But read them with your cloud bill open in the other window and something jumps out. All that discipline lands on the rules. The telemetry underneath the rules is still treated as a given. You acquire it on reflex and you keep it on faith.

The constraint flipped

A cloud-native environment does not have a telemetry scarcity problem. The control plane logs every operation in every subscription whether you asked for it or not. The identity provider emits four log families on its own (sign-in, audit, provisioning, risk). Kubernetes brings its own four. Every API gateway logs every call, every key vault logs every key operation, every storage account can log every object touch. Then the security stack itself: endpoint, identity, posture, app sec, each with an alert stream plus raw events. And now AI workloads are showing up with model invocation traces and tool-call logs on top of everything else. No single source is unreasonable. Add them up and you’re staring at petabytes a year, growing every quarter.

Here’s the arithmetic problem, and it’s the same one that runs the alert queue. My team works through a few hundred thousand alerts a year, so none of this is abstract for me. Take one rule producing 500 alerts a day at 70% precision. On paper that’s a strong detection. In practice it’s 150 false positives a day, and if each one takes an analyst a couple of hours to run down, that single rule eats 300 analyst-hours a day before anyone has touched a true positive. The rule isn’t wrong. It’s unaffordable. And unaffordable rules all die the same death: tuned down until the recall is gone, or switched off outright.

Every SOC lead I know has lived that story at the alert layer. Almost nobody prices it one layer down, at ingestion, where the same economics run with bigger numbers and nobody watching. The constraint on a detection program today isn’t the data you don’t have. It’s the consequences of the data you do have. A saturated queue. Enrichment nobody has capacity to build. A storage bill quietly competing with headcount. Enough moving parts that the whole program gets brittle.

A liability with a recurring invoice

Try this exercise on your own environment. For each log source, ask three questions:

• Which detection does this enable?
• Which investigation does this speed up?
• Which compliance obligation does this satisfy?

A source that goes zero for three is not an asset waiting for its moment. It’s a liability with a recurring invoice, and the ingestion line you can see is the smallest charge on it. Storage tiers get assigned once and never revisited. Parsers break when a vendor changes their schema and someone has to notice, then fix them. Retention runs years past any detection value on the strength of a compliance assumption nobody ever validated. And every source you add is one more table an analyst has to consider at 3am. A source that’s present but poorly understood doesn’t speed investigations up. It slows them down.

I’ll say the uncomfortable version out loud: an organization that collects everything is not safer than one that collects deliberately. It’s just poorer. Every dollar going to telemetry nobody uses is a dollar not going to the detections, the enrichment, and the analysts that actually turn data into protection.

Derive it backward

There are two ways most logging architectures actually get built, and both run backward. The first is reactive: a threat shows up, the team realizes it doesn’t have the logs, and everyone scrambles to turn on a connector. Threat, then logs, then detection, which guarantees every new capability lags the threat by weeks. The second is what platform teams tend to do: turn on every diagnostic setting, ship it all to the SIEM, sort it out later. Collect, then store, then build. That fixes the lag, at a cost that scales with the size of your environment and bears no relationship to the value of anything collected. Stay on that path long enough and you’ll eventually discover that a serious chunk of your analytics spend covers telemetry no detection references and no investigation has ever pulled up.

The sequence that works runs the other direction: threat model, then detection requirements, then telemetry requirements, then logging architecture.

Worked example. Take a threat your model actually cares about, say identity compromise leading to sensitive data access. The detection requirement falls out of it: alert when a principal touches a sensitive store it has never accessed before, outside the known ETL windows, with enough confidence that an analyst can act without rebuilding the case from raw logs, and within minutes, because exfil finishes inside a single session. From there the telemetry requirement falls out too. Storage access logs on the sensitive accounts specifically, with the caller, auth type, and status fields intact. Sign-in logs for session context. A maintained list of which stores are sensitive and who’s supposed to be in them. Only then do you architect the logging. Diagnostics go on for the accounts that hold sensitive data, not for the hundreds of accounts holding build artifacts and Terraform state, where the volume dilutes the signal and pads the bill. Analytics tier for the ninety days the detection actually needs, archive after that for the compliance tail. Automation keeps the sensitive-store list current, so a newly classified store enters scope without a human remembering to add it.

Run it that way and volume drops while coverage goes up, because the data that matters arrives with the right fields at the right fidelity, sitting next to the reference data that makes the alert actionable. Your logging becomes a function of your threat model instead of a byproduct of whatever the platform’s diagnostic defaults happen to be.

What it costs

I don’t trust arguments that only list benefits. So, the bill.

The first objection is always “what if we need it later,” and it’s legitimate. That’s the forensic argument. The answer isn’t deletion, it’s tiering. Data with plausible investigation value but no detection on it goes to cheap storage, or gets trimmed at ingestion (strip the unused fields, drop the routine events) into something that keeps its forensic value without paying hot-tier rates for years. In my experience that kind of transform cuts a source’s volume roughly in half while preserving every field anyone actually queries. You pay for the transformation once. The savings recur for the life of the data. Which is why “collect less” is the wrong slogan for all of this. The slogan is collect deliberately.

The second cost is the real one. This approach turns logging from a default into a discipline somebody has to own. The threat model has to be genuinely maintained or everything derived from it rots at the same rate. Those reference lists are standing operational commitments, not one-time configs. If nobody owns the chain, it decays quietly, and it decays in exactly the places your coverage map claims you’re fine.

Where the noise starts

The reason I care about this isn’t really the bill, though the bill is real. It’s the queue.

We talk about alert fatigue like it starts at the analytics rule, like the fix is one more round of tuning on whatever rule is screaming this week. But by the time a badly chosen log source has become an alert in front of a human, every option left is a bad one. Tune the rule down and give back recall. Eat the noise and burn analyst trust you won’t easily get back. The noise your team is drowning in was purchased months earlier, at ingestion, when a source got switched on because it was available rather than because anything required it. Signal-to-noise isn’t a property you add at the rule layer. You either engineer it at the ingestion layer or you apologize for it at the queue.

Fifteen years ago the mark of a good detection program was that the data was there when you needed it. Now the mark is being able to say why the data is there at all. Every source traces to a detection or an investigation. Every detection traces to a threat somebody actually modeled. Logging stops being an inventory and starts being a consequence of the threats you’ve chosen to defend against.

That old post-mortem line, “we didn’t have the logs,” deserves an update for the world we operate in now. The failure mode didn’t go away. It flipped. We had the logs. We had all of them. That was the problem.

Disclosure: this post was written with the assistance of AI, after only a few thousand input prompts to make it sound like me. The opinions, the scars, and the alert queue are all mine.