Issue #17: Self-Improving Agent Loops Ship Real Results Over a Weekend

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

The Hook

A developer ran a self-improving agent loop on real research code over a weekend — and it worked. Meanwhile, an iPhone 17 Pro ran a 400B parameter LLM on-device, and a mechanic shop got an AI receptionist that pays for itself with a single salvaged job. The signal this week isn’t theoretical: operators are shipping revenue-positive agents right now.

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

This Week’s Signal

Self-Improving Agents Are Real — Here’s the Exact Loop

A developer ran Karpathy’s “autoresearch” framework on their own eCLIP medical imaging research codebase, achieving measurable ML metric improvements while doing chores. 316 HN points, 71 comments — this week’s top AI discussion. The architecture is deceptively clean:

• Claude Code as the reasoning engine
• Sandboxed Docker for execution (no direct Python, no pip install, no network by default)
• scratchpad.md as working memory — the agent’s internal journal
• A tight loop: hypothesis → edit → train → evaluate → commit
• ~5 minutes per experiment run; final phase unlocked web access to read papers

What makes this significant for operators: it’s not a demo. It ran on production research code with real metrics, isolated via containers for safety, and the human stayed out of the loop except to watch. The agent found improvements a human researcher hadn’t.

Your move: Steal this architecture for any iterative optimization task — A/B test generation, prompt tuning, model eval loops. The key primitives: a scratchpad for agent memory, a sandboxed executor, and a measurable objective function. If you have those three, you can run this loop today.

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

3 Operator Playbooks

1. On-Device Agents Are No Longer a Future Bet

A viral demo showed an iPhone 17 Pro running a 400B parameter LLM fully on-device — 531 HN points, 251 comments, the week’s highest-scoring story. Six months ago, 400B parameters meant a data center rack. Now it fits in a pocket. The immediate implication: fully offline, air-gapped agent pipelines are becoming viable on consumer hardware. Enterprise deployments that blocked cloud AI on privacy/compliance grounds are running out of objections. The latency and cost math changes entirely when inference is local.

Your move: If you’re building agents for regulated industries (healthcare, finance, legal), start prototyping offline-first architectures now. The hardware is arriving faster than the governance frameworks. Being early means owning the enterprise customer who can’t touch cloud.

2. SMB Voice Agents Have Immediate, Measurable ROI

A developer built “Axle” — a custom RAG + voice AI receptionist for her brother’s luxury mechanic shop. The business case is simple: each missed call costs $450–$2,000 in lost jobs. The stack: MongoDB Atlas Vector Search, Voyage AI embeddings (1024-dim, 21+ documents), and a voice agent layer that answers calls, quotes prices, and books callbacks. 238 HN points, 261 comments — the community recognized this immediately as a template, not just a story.

Your move: The SMB voice agent playbook is proven. Pick a vertical with high missed-call costs (contractors, clinics, auto shops, legal), build a RAG layer on their existing knowledge base, and wrap it in a voice agent. The pricing conversation is trivial when the ROI is one prevented missed call per month. Package this as a productized service.

3. Your CI/CD Pipeline Has a Supply Chain Problem

The Trivy vulnerability scanner’s GitHub Actions were compromised — attackers force-updated version tags v0.69.4–0.69.6 to deliver infostealer malware. Docker Hub images with the same tags were also poisoned. 178 HN points, 63 comments. This is the second Trivy ecosystem compromise in March 2026. Any pipeline pinned to Trivy Actions by tag (not commit SHA) was exposed — and that’s most of them.

Your move: Audit every uses: reference in your GitHub Actions workflows right now. Replace tag-based pins with commit SHA pins — the only format that can’t be silently overwritten. actions/checkout@v4 becomes actions/checkout@<full-sha>. Automate this with pin-github-action or Dependabot. If you run agentic CI pipelines with secret access, this isn’t optional hygiene — it’s the attack surface your adversaries are actively targeting.

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Steal This

The Coding Agent Knowledge Layer (Mozilla.ai cq)

Mozilla.ai shipped cq — an open-source MCP server that gives coding agents a persistent knowledge base of “gotchas.” Agents propose and query Knowledge Units (KUs): structured lessons learned that survive across sessions and models. Think Stack Overflow for your agent, scoped to your codebase.

Install as a Claude Code plugin or OpenCode MCP server. Local SQLite by default; team-syncable via Docker Compose. Human-in-the-loop UI for reviewing KUs before they’re committed.

Template — Seed your cq instance with high-value KUs:

KU: GitHub Actions pin format
Problem: Tag-based pins (e.g., v4) can be silently overwritten by attackers.
Solution: Always pin by commit SHA. Use `pin-github-action` to automate.
Applies to: Any workflow using third-party Actions.

KU: MongoDB Atlas Vector Search — embedding dimensions
Problem: Mismatched embedding dimensions cause silent retrieval failures.
Solution: Set index dimension to match your model output (e.g., 1024 for Voyage AI).
Applies to: Any RAG pipeline using Atlas Vector Search.

KU: Agent scratchpad pattern
Problem: Agents lose context between experiment runs in iterative loops.
Solution: Mount a persistent `scratchpad.md` as working memory; agent reads/writes it each step.
Applies to: Any self-improving or long-horizon agent loop.

Your move: Install cq, seed it with your team’s top 10 gotchas, and let every future agent session inherit that knowledge automatically.

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

The Bottom Line

This week confirmed what operators have been building toward: self-improving agent loops work on real codebases, on-device 400B inference is here, and the SMB revenue case for voice agents is closed. The infrastructure layer is maturing fast — Dapr Agents hit GA for production Kubernetes deployments, and the security surface is expanding just as fast as the tooling. The gap between operators who ship and operators who watch is widening. The playbooks above are not hypothetical — they’re proven patterns from this week’s community. Pick one and run it.

AI Agent Insider is published by Digital Forge Studios.