Editorial note
Today’s picks orbit a single idea: agentic systems don’t behave like interchangeable SDKs. Small differences in model, repo context, or prompt nudge agents toward very different external tools — with practical consequences for product positioning, security controls, and developer expectations.
In Brief
Which tools do Claude, Codex and Cursor choose? We measured 17k runs to find out
Why this matters now: Armature’s study of agent behavior shows Anthropic’s Claude Code, OpenAI’s Codex, and Cursor make materially different choices about external tools and web search — information that matters for anyone building integrations, SDKs, or security controls around coding agents.
A team from Armature ran nearly 17,000 agent sessions across real-world repositories to see what third‑party tools each agent actually selects during coding tasks, then released the traces for public analysis. According to the Armature post, one headline result is that “Claude Code rarely searches the web while Codex almost always does it and Cursor sits in the middle.” The experiment also found that library or service mentions in conversation don’t reliably translate into the agent actually choosing those services — tools like LangChain, Supabase, Netlify, Paypal and Adyen were “almost always mentioned … but never chosen.”
“Claude Code rarely searches the web while Codex almost always does it and Cursor sits in the middle.”
This is a solid, pragmatic data point for teams that build integrations or track agent telemetry: agents have distinct behavior profiles, and those profiles change how you should document, secure, and market to them.
Working to Make Python Lazy
Why this matters now: Python’s evolving lazy-imports feature gives CLI tools, serverless functions, and REPL-driven workflows a practical route to faster startup times with minimal code changes.
Python is moving toward configurable lazy imports (for example, via [mechanisms introduced in 3.15 and environment toggles]), which defer module loading until the module is used. The pitch is simple: reduce cold-start time without rewriting import graphs. Proponents suggest there's little downside when implemented carefully — one write-up claims “there’s no drawback to making imports lazy: they otherwise behave exactly as intended.” That’s attractive for CLIs and microservices, but it also shifts when import-time side effects occur and can hide errors that used to show up during startup.
“there’s no drawback to making imports lazy: they otherwise behave exactly as intended”
Expect library maintainers and observability tools to debate guardrails: should lazy imports be opt-in, globally toggleable, or signaled in package metadata? The trade-offs are tangible — lower latency versus more subtle runtime behavior — and teams will need to test for assumptions around import-order side effects.
Deep Dive
Note on selection: none of today’s items crossed our usual Deep Dive threshold (Quality Score ≥ 7.5). That said, the Armature 17k-run dataset is timely and actionable enough to unpack further, because it reveals patterns that influence product design, security posture, and go‑to‑market strategies for developer tooling.
Which tools do Claude, Codex and Cursor choose? We measured 17k runs to find out
Why this matters now: Armature’s dataset shows concrete, reproducible differences between major coding agents — information product and security teams can act on immediately to change how they instrument, document, and protect integrations.
The core finding — different agents prefer different evidence sources and different actions — reframes several assumptions. Many vendors treat “AI integration” as a single channel to reach developers: ship an API key or an SDK and expect the model to call it. The Armature traces suggest that won’t be enough. If Claude prefers repo context and Codex prefers web search, then your docs, example snippets, and default prompts all need to be tailored to the agent’s behavioral tendencies. A one-size-fits-all integration guide risks low adoption because the agent simply won’t choose your tool in the wild.
There’s also a security and compliance angle that’s easy to miss. Agents that default to web search create more egress exposure and a higher likelihood of pulling in unvetted code or secrets leaked in web content. Conversely, agents that lean on local repo context might exfiltrate internal artifacts if not properly sandboxed. Armature’s observation that mentions don’t equal selections — popular services being talked about but not actually used — matters here: an agent can discuss a payment provider in prose but still take a different integration path when executing, so monitoring conversational traces isn’t the same as monitoring runtime calls.
Practically, three immediate actions follow from the study:
- Instrument agent runs the way you would instrument user traffic. Track not just prompts and responses, but the agent’s chosen tools and network calls so you can measure real-world selections.
- Design integration UX for the agent’s decision mode. For agents that prefer repo context, provide code snippets, path-based hints, or repo-level config files that make the right choice obvious. For agents that search the web, prioritize canonical documentation, quickstart pages, and robust public API docs.
- Harden egress and secrets. Treat agent execution like any other runtime: restrict outbound network calls, enable allow-lists for domains or APIs, and ensure agent credentials are scoped and rotateable.
Finally, a methodological note: Armature made the traces public and invited others to dig in, which lets vendors and security teams reproduce results against their own repos and prompts. That’s important — these are not black‑box claims. If you’re responsible for an integration or security posture, run similar experiments on representative repos and prompts, then bake the findings into policy and product copy.
Closing Thought
Agents are not monolithic endpoints you can plug into once and forget. They make decisions — about web search, repo context, and which third‑party tools to touch — and those decisions are the practical interface your product, security, and docs teams must design for. Treat agent behavior as telemetry: measure it, influence it, and build controls around it.