Inference Loops

Writing

Notes on building with AI: agents, systems, evaluation, and what it all means for Cambodia and the region.

The Privacy Leak Is Between the Agents: Why Output Audits Miss Most of It

Splitting one agent into a coordinator and workers is supposed to be the safer architecture. A 2026 benchmark shows it quietly moves the privacy risk somewhere your audits can't see. AgentLeak (arXiv 2602.11510) instruments seven communication pathways across 1,000 scenarios, five production models, and 4,979 execution traces — and finds that multi-agent setups do lower final-output leakage (27.2% vs 43.2% single-agent) but push total system exposure to 68.9%, because inter-agent messages leak at 68.8%. The blunt consequence: an output-only audit misses 41.7% of violations. Here's why the leak lives in the coordination channels, why you have to instrument them, and why that's an opportunity for teams close to regulated, local data.

Read
  1. On-Device Agents and the Privacy Dividend: Why Local Isn't Yet Private Running an agent on your own hardware is sold as a privacy win — the data never leaves the building. But 'local' and 'private' are not the same claim, and 2026 made the gap precise. AgenTEE (arXiv 2604.18231) places an agent's runtime, inference engine, and tools into independently attested confidential VMs on Arm's Confidential Compute Architecture, enforcing hardware isolation even on a device its own user might tamper with — at under 5.15% overhead. Amulet (arXiv 2512.07495) does the matching job for the model itself, shielding weights with TEE-backed obfuscation at 2.8–4.8x the latency of an unprotected model while running 8–9x faster than naive full-TEE inference. Here's why the real privacy dividend isn't 'it runs locally' — it's confidential, attestable execution — and why that mechanism favors Southeast Asia.
  2. We Named Our Company After This: Loop Engineering Is Eating Software Development The man who built Claude Code says he doesn't prompt it anymore — he writes loops that prompt it. That single line named the paradigm shift the whole agentic era has been building toward: from one-shot prompting to persistent, feedback-driven loops. Here's why the prompt was always a placeholder, why the loop is becoming the unit of software work, and why a company called Inference Loops bet its name on exactly this.
  3. The Environment Is the Product: How RL Gyms Shape the Next Agents If reinforcement learning in verifiable environments is the lever that improves agents, then the environment itself — not the model — is the asset worth owning. 2026 brought the OpenAI-Gym moment for agents: GEM, an open-source simulator that standardizes the environment-agent interface across 24 environments and 5 RL frameworks, with asynchronous vectorized execution and dense per-turn rewards. Standardization is what turned RL from a research curiosity into a field, and it's about to do the same for agents. Here's why the gym — the task suite, the reward shape, the definition of 'correct' — is becoming the defensible product, and why building one is a judgment game, not a capital game.
  4. Why Multi-Agent Systems Fail: It's Not the Model, It's the Org Chart We've written about what makes a multi-agent orchestra work. This is the honest companion: what makes it fail. Two 2026 studies took the question seriously — one hand-annotating 1,600+ traces across 7 frameworks into 14 distinct failure modes, the other mining 13,602 issues from 40 real agentic projects into a taxonomy of 5 fault dimensions, 13 symptoms, and 12 root causes. The verdict is uncomfortable: most multi-agent failures aren't the model being dumb. They're specification gaps, agents talking past each other, and no one checking the work — organizational failures, not capability failures. And the tactical fixes barely move the needle.
  5. Observability for Agents: You Can't Debug What You Can't Trace A non-deterministic agent fails once and never the same way twice — and a log file can't explain a failure you can't reproduce. The 2026 answer is observability built for agents: capture the full execution trace, correlate what the agent intended (its prompts) with what it actually did (its system calls), and represent that trace as structured data rather than plain text. One framework bridges the intent-vs-action gap with eBPF boundary tracing at under 3% overhead; another shows that how you format a trace can cut analysis cost by half to two-thirds. Observability is becoming the floor under everything else you do with agents — evaluation, debugging, and security all sit on top of it.
  6. The Leaderboard Lies About Your Agent Computer-use agents have made a stunning leap: on the OSWorld benchmark, where humans score 72.36% and the best agent managed just 12.24% at its 2024 launch, the top systems are now reported to be closing in on the human baseline. A real milestone — and exactly the kind of single number that misleads. A 2026 framework, AgentAtlas, argues that for agents, final task success is the wrong unit of measurement: it hides whether the agent asked before the irreversible step, recovered from its mistakes, or took a safe path to the goal. Worse, which axis you score on can change the rankings outright. Here's why the headline score is the least useful thing you know about an agent.
  7. DEFA and the ASEAN AI Safety Network: When Soft Coordination Gets Teeth ASEAN's first binding digital economy agreement, DEFA, is up for signature in November 2026, embedding AI governance and data-flow rules. What it means for Cambodian builders.
  8. The Agent Economy Needs Rails: Agentic Payments and Verify-then-Pay MCP let agents reach tools; A2A let them reach each other. The next question is how they pay — and 2025–26 answered it with a sudden land-grab of open payment standards. Stripe and OpenAI shipped the Agentic Commerce Protocol on September 29, 2025; Google announced the Agent Payments Protocol on September 16, 2025 as an extension of A2A. But moving money is the easy part. The hard part is trust: did the user really authorize this purchase, is the agent hallucinating the order, and — when one agent hires another — how do you pay for work that may not get done? Here's the emerging answer: pay on cryptographic proof, not on promise.
  9. Oversight Has a Capacity: Why More Human Review Can Make Agents Less Safe The default safety pattern for agents that take real actions — shell commands, file edits, deploys — is a human-in-the-loop approval gate: risky actions pause and wait for a person. The intuition is that more human review means more safety. A 2026 study shows that's wrong past a point: because human attention is finite and fatigues, realized safety follows an inverted-U, and the safety-optimal guard escalates below 'everything.' Oversight isn't a checkbox; it's a finite attention budget the escalation policy spends. Here's why routing every risky action to a human can make your system less safe — and how to design the gate instead.
  10. Cambodia's AI Policy Is a Regional Game Now Cambodia's AI strategy is usually discussed as a national project. In 2026 it stopped being one. At January's ASEAN Digital Ministers' Meeting in Hanoi, the region committed to coordinated AI policy — cross-border data flows, a future-ready workforce, shared digital infrastructure — backed by an ASEAN digital upside estimated near $1 trillion by the end of the decade. Cambodia climbed 27 places on the Oxford AI readiness index in a year, launched 5G, and broke ground on a national data center. But it also faces an automation cliff: 57% of jobs at high risk. Here's why the regional frame changes the stakes — and the strategy.
  11. Cambodia's Sovereign Cloud: ByteDC on Huawei, Not NVIDIA ByteDC launched Cambodia's first high-performance sovereign cloud — built on Huawei Cloud Stack, not NVIDIA or a Western hyperscaler. What does 'sovereign' actually mean when the stack is a Chinese vendor's hardware?
  12. Agent Memory: The Problem Isn't Recall, It's Forgetting Everyone asks whether an agent can remember. The 2026 benchmarks say that's the wrong worry. The real failures sit elsewhere: the architecture (teams treat the context window as memory when it's really just L1 cache — one study cut live context up to 93% by paging information in on demand), the retrieval (the best memory system scores only 57% because similarity-based recall misses what's causally relevant), and hardest of all, forgetting (agents keep acting on facts that are no longer true, and dedicated memory systems barely help). Agent memory is an engineering discipline, not a model feature — here's what the research actually shows.
  13. MCP's 'By Design' RCE: When the Vendor Says the Vulnerability Is a Feature OX Security disclosed a systemic RCE in Anthropic's MCP SDKs affecting 150M+ downloads. Anthropic says it's by design. Here's what operators must do.
  14. Who Judges the Judge? The Hidden Bias in LLM-as-a-Judge More and more agent loops put an LLM in the grader's seat — scoring outputs, gating merges, ranking candidate patches. But the judge has systematic biases, and they're not the ones you're guarding against. A 2026 study found style bias dominates at 0.76–0.92 across every model tested, an order of magnitude larger than the position bias everyone worries about (≤0.04). In code evaluation specifically, a judge's verdict can flip on prompt cues even when the code is unchanged — enough to alter model rankings. If your verification gate is an uncalibrated LLM judge, you may be scoring polish and prompt artifacts, not correctness. Here's how to calibrate it.
  15. Coding Agents Don't Just Fail — They Rot: The SlopCodeBench Finding A deep dive into SlopCodeBench's finding that agent code structurally deteriorates with each iteration — and what loop design counteracts it.
  16. How Agents Learn to Code: RL, Self-Play, and the Open Catch-Up The 2026 jump in coding-agent capability didn't come from bigger pretraining. It came from changing how agents practice. An open 32-billion-parameter model went from 23% to 42% on SWE-bench Verified in just 200 steps of reinforcement learning — no new pretraining, no human-labeled data. Self-play methods now let agents improve by generating their own training signal, beating human-data baselines, and newer techniques match end-to-end RL accuracy at a quarter of the compute. This is a fundamentally different economics from pretraining — and it's the one that favors teams without a frontier lab's budget.
  17. The June 2026 Billing Reset: When Agentic Coding Got Repriced Four coding-agent vendors changed billing within 18 days — OpenAI, GitHub, Anthropic, and Google all moved to decouple programmatic token burn from flat subscriptions. Here's what it means for your team's budget.
  18. The Review Bottleneck: AI Writes More Code, and We Approve the Worse Stuff Generation got cheap; review didn't. As agents flood pipelines with pull requests — GitHub Copilot alone has processed over 60 million code reviews, growing 10x in under a year, with more than one in five reviews now involving an agent — the binding constraint has moved to human approval. And a 2026 peer-reviewed study found the most dangerous part: agent-generated code carries more redundancy and technical debt than human code, yet reviewers feel more positive about it. Surface plausibility masks the slop, so we approve worse code with better feelings. Here's why the review gate is where the loop actually breaks — and how to fix it.
  19. Sandboxing the Agent: Isolation Is the New Runtime Security Layer Prompt injection is the number-one vulnerability in deployed AI agents — and a 2026 result argues it can't be fully solved at the model layer at all. A meta-analysis of 78 studies found attack success rates above 85% against state-of-the-art defenses under adaptive attacks, with most defenses achieving under 50% mitigation; a separate paper proves an impossibility result for instruction-based defenses. If you can't reliably stop the agent from being hijacked, the load-bearing security decision becomes the runtime: how much can it wreck when it is? This is the case for treating isolation — ephemeral microVMs, least privilege, bounded blast radius — as the real agent security layer.
  20. The Other Half of the Interop Stack: A2A and the Agent Internet MCP gave agents a universal way to reach tools. The Agent2Agent protocol (A2A) gives them a universal way to reach each other — and together they're forming the interoperability substrate of a coming agent internet. A2A is built on boring, proven standards (HTTP, JSON-RPC 2.0, Server-Sent Events), uses an Agent Card for discovery, and is now governed vendor-neutrally by the Linux Foundation, with the supporting organizations grown from more than 50 to over 150 in its first year. Here's what A2A is, how it pairs with MCP, the new trust surface it opens, and why an open agent-to-agent standard matters most to the teams no platform owns.
  21. Agents on One Workstation: The On-Device Open-Weight Inflection The assumption that a capable coding agent needs a cloud API is quietly breaking. Open-weight models have crossed two thresholds at once: good enough — Qwen3-Coder-Next scores over 70% on SWE-bench Verified while activating just 3 billion of its 80 billion parameters, and Mistral's Devstral Small 2 hits 68% — and small enough that Devstral runs on a single RTX 4090 or a 32GB Mac. That combination puts a frontier-adjacent agent on your own hardware, where the data never leaves the building, there's no per-token bill, and it works offline. Here's why on-device is suddenly real, what it buys you, and why it's the most portable advantage in the agentic stack.
  22. Your Agent Has Three Attack Surfaces You're Not Monitoring Indirect prompt injection, tool poisoning, and permission escalation — the three attack surfaces every agent deployment inherits. Here's what to watch for.
  23. You Can't Improve What You Can't Trace — Agent Observability for Teams Agent observability is a distinct discipline from traditional APM. Here's what to track, why, and how to do it without enterprise budgets.
  24. The Open-Source Agent Stack: Cline, OpenCode, OpenHands in 2026 Open-source coding agents exploded in 2025-2026. Here's when they beat proprietary tools — and when they don't.
  25. Agentic System Design: A Complete Guide Designing an agentic system is different from designing a normal one: you're not specifying functionality, you're specifying behavior. The hard questions are about autonomy, feedback loops, failure modes, and guardrails — not endpoints and schemas. Here's a practical guide to designing agents that stay safe as they scale: start small, log every decision, make observability a first-class feature, and wire the circuit breakers before you need them.
  26. Cambodia's AI Opportunity: Why Now Is the Moment A June 2026 data refresh on Cambodia's AI moment. A median age of 26, mobile connections at 121% of the population, a $150M AI market spanning 1,200 AI-powered enterprises and 5,400 AI workers, and a draft National AI Strategy aiming for $3.35–6.7B in GDP by 2030. Here's the real state of play — the momentum, the honest gaps, and why the agentic shift tilts the field toward exactly the kind of engineering Cambodia can do.
  27. Context Engineering: The Context Window Is Your Agent's Real Bottleneck As agents run longer, the binding constraint stops being model intelligence and becomes the context window. A 2026 benchmark found an agent given only its last five tool calls plus a running summary hit 91.6% task completion while a full-history agent managed just 71% — using 64% fewer tokens and finishing in 40% of the time. More context made it worse. Here's why 'context rot' happens, why less context produces better agents, and the practical toolkit for engineering what the model sees — a high-leverage, GPU-free discipline.
  28. Dual-Agent Development: Running Claude Code and Codex on the Same Project The 2026 question isn't 'which coding agent should I pick?' — it's 'how do I run both?' The strongest teams have stopped choosing between Claude Code and Codex and started routing work to each for what it does best: Claude Code for architecture and multi-file reasoning, Codex for fast parallel implementation. Here's the hybrid workflow, the tooling that makes two agents manageable, the cost math, and why this asymmetric play favors small Southeast Asian teams more than anyone.
  29. The Bill Is the New Bottleneck: The Economics of Agentic AI Generation got cheap. Running the loop did not. A five-step agent costs roughly 3.2x a single model call for the same work — and by 200 steps the bill is over 100x, with 62% of it spent re-reading context the agent already paid for. Here's where the money actually goes in an agentic system, the four cost levers that work, the open-weight escape hatch at a tenth of frontier prices, and why cost discipline is the sharpest competitive lever a small Southeast Asian team has.
  30. The Reliability Gap: Why Your Best Agent Melts Down First The leaderboard measures the wrong thing. Single-shot benchmarks like SWE-bench score an agent's capability — whether it can succeed once — but production needs reliability, consistent success across long, repeated attempts. A 2026 study of 23,392 episodes found the two diverge so hard that rankings invert at long horizons, and that frontier models have the highest meltdown rates — up to 19% — because they reach for ambitious plans that spiral. Here's the difference between capability and reliability, why the most capable model is often the least dependable, and how to measure and engineer for the metric that actually ships.
  31. When Agents Get Worse the Longer They Run Coding agents don't just plateau on long tasks — they actively degrade the code they're working on, turn after turn. A 2026 benchmark built to measure this found the best agent passed only 14.8% of checkpoints, with code structure eroding in 77% of trajectories and agent output running 2.3x more verbose than human repositories. Here's what the long-horizon benchmarks actually measured, why quality decays as iterations pile up, and the harness discipline that keeps a long run from rotting.
  32. A Bad Loop Ships Bad Code Faster: Evals Are the Real Discipline Everyone is racing to make agents act autonomously. Almost nobody is as eager to make them act correctly. But once generation is cheap and parallel, the only constraint left is trust — and a loop you can't trust just produces wrong work at scale. This is the case for evals as the new test suite: the verification gate, the agent-as-judge, the golden dataset of real failures, and why building one is the most defensible engineering a Southeast Asian team can do.
  33. The Agentic AI Foundation: Why Open Standards Matter for Southeast Asian Developers OpenAI and Anthropic — fierce rivals — just co-founded the Agentic AI Foundation under the Linux Foundation. With AGENTS.md in 60,000+ repos and MCP nearing 9,400 servers, agentic AI is getting its open-standards moment. Here's why that's especially good news for developers in Southeast Asia.
  34. AI Regulation in Southeast Asia: A Comparative Overview How ASEAN nations are approaching AI governance and what Cambodia can learn from regional leaders.
  35. Inside the Agent Loop: The Pattern Behind Reliable AI Agents Swapping in a smarter model rarely fixes a flaky agent. The fix lives in the loop — the phases, the context discipline, and the verification step that turn a model's intelligence into something that reliably ships. Here's the pattern, dissected.
  36. Designing Agent Loops That Run While You Sleep In June 2026, Peter Steinberger told developers to stop prompting agents and start designing the loops that prompt them. Addy Osmani gave that instinct a name — loop engineering. Here's the anatomy of an autonomous loop, the guardrails that keep it from burning your budget overnight, and why this is the highest-leverage skill a Southeast Asian developer can pick up right now.
  37. AI Coding Tools in 2026: Claude Code vs Cursor vs Copilot Three tools dominate agentic coding in 2026 — Claude Code, Cursor, and GitHub Copilot. A practitioner's comparison of how they actually differ, who each one fits, and why the tool you pick matters less than what you build on top of it.
  38. AI Safety & Governance in Southeast Asia: What Developers Need to Know Governance isn't just legal's problem — in Southeast Asia's fragmented regulatory landscape, it lands in your code. A practical guide for the engineers building and shipping AI across the region.
  39. The Evolving Role of Developers in an AI-Powered World: Perspectives from Southeast Asia The developer's job didn't disappear — it moved up the stack. From writer to orchestrator, the skills that appreciate, the brutal junior-developer squeeze, and why the same shift that threatens one model of work opens a real door for Southeast Asia.
  40. How Coding Agents Actually Reason If the harness is 98.4% of a coding agent, what is the other 1.6% — the part that 'thinks' — actually doing? This is a field guide to how agents reason: the Thought-Action-Observation loop, why observation beats raw cleverness, and a Meta result showing that forcing an agent to argue like a logician — not a chatbot — lifts its code judgment by double digits. For builders who want to shape the reasoning, not just rent it.
  41. The Inference Loop: Why Coding Is Becoming a Loop, Not a Keystroke The most important software primitive of the decade fits in ten lines: call the model, run a tool, feed back the result, repeat. Here's why that loop is eating software development — and what it means for your team.
  42. Loop Engineering: What It Means for Southeast Asian Developers In June 2026 the agentic-coding world got a new name for its most important skill: loop engineering. Here's what it is, the three-layer model behind it (Harness → Loop → Orchestration), and why it's the single best bet Southeast Asia's developers can make right now.
  43. MCP in 2026: The Universal Standard — and Its Security Problem The Model Context Protocol won the integration war. By mid-2026 it spans ~9,650 registered servers, 97M+ monthly SDK downloads, and production use at 41% of software organizations. But the same ubiquity created a vast attack surface: a study of 67,057 servers found hundreds vulnerable, leaked credentials, and hijackable maintainers. The standard plug is settled. The security of what you plug in is the new frontier — and that's a guardrail-layer opportunity for the region's builders.
  44. Models vs Agents: The Shifting Boundary Every few months a new model release deletes someone's clever agent scaffolding. So is the scaffolding a dead end — or the part that actually compounds? A clear-eyed look at the boundary between model and agent, and how to build on the right side of it.
  45. Spec-Driven Development: When You Edit the Spec, Not the Code As agents write more of the code, the human's durable artifact shifts up a level — from the code to the specification. Martin Fowler maps three maturity stages, ending in spec-as-source, where humans only ever edit the spec and code is a compiled output. Here's why the shift is happening, the honest critique that it might divest from design, and why turning local domain rules into living specs is exactly the leverage Southeast Asia's developers should grab.
  46. The Code Agent Orchestra: What Makes Multi-Agent Coding Actually Work 2026's real shift isn't smarter agents — it's more of them, working in parallel. Anthropic's data shows multi-agent workflows delivering 2–4x acceleration, with one enterprise compressing a 24-day project into 5. But orchestrating a squad of agents is a different skill from directing one, and most teams fail at it the same way. Here's what makes the orchestra work, where it collapses, and why it's a force multiplier built for small teams.
  47. The Harness Is the Product: Why 98.4% of an AI Agent Isn't the Model A study of Claude Code's source found that only about 1.6% of a production coding agent is AI decision logic — the other 98.4% is harness: the scaffolding that feeds the model, runs its tools, checks its work, and decides when to stop. Here's what that number means, and why it's the most important thing for Southeast Asia's builders to understand right now.
  48. The Harness That Rewrites Itself We've argued all year that the harness — the scaffolding around the model — is where the real work lives. In mid-2026 a new class of research closed the loop on that idea: agents that improve their own harness, with no human engineer and no smarter model to copy from. A June arXiv paper lifted three different models by 33–60% on a hard benchmark just by letting each one rewrite its own scaffolding. Here's what self-harness is, why it doesn't make engineers obsolete, and why it tilts the field toward exactly the work Southeast Asia can own.
  49. The $100 Billion Guardrail: Why AI's Safety Layer Is Becoming the Product The market for AI guardrails is projected to grow from $0.7B in 2024 to $109.9B by 2034 — one of the fastest-growing categories in software. As agents stop suggesting and start acting, the defensible spend moves off the model and onto the layer between the agent and your business. Here's what a guardrail actually is as a buildable product, why ASEAN's soft-law governance turns that layer into your compliance boundary, and why that's an opening for the region's engineers.