Editorial note
Small shifts in infrastructure and workflow are quietly reshaping what AI costs and how teams use models. Today I’m flagging two engineering‑forward pieces: one is a big architectural bet on inference efficiency, the other is a pragmatic, repo-level hack that reduces token spend dramatically.
In Brief
Can guitar frets perform multiplication?
Why this matters now: Charles Petzold’s exploration shows that a guitar neck can visually mimic a slide‑rule for single‑octave multiplications, which is a neat intersection of music, geometry, and computation that will interest makers and educators.
Charles Petzold chased a striking claim about fret spacing and found a playful — and mathematically honest — answer in his writeup. Fret placement follows the twelfth root of two, so frets approximate a geometric scale; that makes certain one‑octave visual multiplications work, but the trick breaks across octaves. The piece is a tidy reminder that intuitive analogies can mislead: frets mimic the multiplicative nature of pitch shifts but aren’t a true logarithmic slide rule.
Gimlet's Series B
Why this matters now: Gimlet Labs’ $300M Series B and multisilicon pitch could reshape inference economics for labs buying managed capacity — if their multisilicon approach delivers the promised tokens‑per‑watt gains.
Gimlet announced a $300M round and a bold product claim: a “first multisilicon cloud, built from the ground up for inference performance,” disaggregating work across GPUs, near‑memory compute, dataflow chips, and CPUs to squeeze more throughput at lower power. They claim potential 5–10X speedups for the same power footprint or better latency/throughput tradeoffs, which would matter to any shop paying inference bills. The writeup is both a fundraising pitch and a technical manifesto; independent benchmarks and production case studies will determine how much of that promise is practical at scale.
Portal by Spotify cut my Claude Code token usage by 90%
Why this matters now: Spotify’s Portal plus a simple plugin shows teams can cut AI assistant token bills dramatically by delegating nonreasonsing I/O to cheaper modes without rebuilding the stack.
An engineer at Spotify published a practical pattern for cutting token spend: a Claude Code plugin called shunt routes bulk file reads and boilerplate to Portal by Spotify’s AiKA modes, keeping the expensive model focused on reasoning and edits. The author reports mean bulk‑read savings around 90%, while noting delegation misses can happen (a thread‑safety bug slipped past the worker). For teams wrestling with runaway bills, this is a concrete tactic to try.
Deep Dive
Gimlet's Series B
Why this matters now: Gimlet Labs’ multi‑chip inference cloud is pitching a model of inference that, if validated, changes the tradeoffs for labs and cloud customers deciding where to run latency‑sensitive models.
Gimlet’s post describes a software‑led approach to heterogeneous disaggregation: split model work—memory‑heavy layers, attention patterns, activation processing—across the silicon best suited for each phase. That’s a sensible engineering direction. We’ve seen the same thesis in smaller pockets (custom inference runtimes, memory‑near compute), but Gimlet is scaling it into a managed product with capital behind it.
“the first multisilicon cloud, built from the ground up for inference performance.”
What to watch for next: reproducible benchmarks and transparent workloads. Claims of 5–10× need context: which models, what batch sizes, what latency targets, and what baseline hardware? Performance can blow out on small‑batch, low‑latency workloads where movement between chips creates overhead. On the other hand, if Gimlet’s runtime can keep pipeline stalls low and scheduling predictable, heterogeneity can win substantial tokens‑per‑watt gains for batchy production workloads like recommendation or retrieval‑augmented inference.
Operationally, the real work is not just the hardware but the scheduler and telemetry: mapping model stages to chips, tracing cross‑chip communication, and handling fallbacks when a chip is busy or fails. Gimlet’s pitch is software as the differentiator; that’s where battle scars will show up. If they can deliver meaningful cost reductions without dramatically increasing developer friction, hyperscalers and big labs will pay attention. If not, the promise remains an expensive experiment.
Portal by Spotify cut my Claude Code token usage by 90%
Why this matters now: Spotify’s Portal+shunt pattern offers an immediately actionable way for teams to reduce model token consumption and costs without rebuilding agents or giving up on high‑quality reasoning from frontier models.
The Spotify engineer’s central observation is worth repeating: “Most of what an AI coding agent does for me isn't thinking. It's I/O.” They built a plugin (shunt) that intercepts Claude Code’s pre‑tool hooks and redirects heavy file reads, test generation scaffolding, and routine output to Portal’s specialized AiKA modes. Those modes are cheaper to run, and the expensive model only receives distilled, relevant context.
“Most of what an AI coding agent does for me isn't thinking. It's I/O.”
The results in the post are striking: about 90% savings on bulk reads in their tests. That’s not magic — it’s engineering discipline: replace repeated, token‑heavy dumps of repository files with a deterministic worker that extracts and returns only the lines the expensive model truly needs. The post is candid about limits: delegation latency (10–30s) and a 30s Portal cap mean this pattern favors larger, non‑interactive tasks. Also, the worker missed a subtle concurrency bug in one example, underscoring that cheaper workers need decent correctness checks.
From a team perspective, the pattern generalizes: enforce thresholds for what gets delegated, bake in verification steps, and treat the worker as part of the trust boundary. It’s a low‑inertia play compared with building remote execution infra or training a smaller internal model. If your team is getting slammed by tokens while the assistant spends cycles reading a monolithic repo, this approach is worth a prototype.
Closing Thought
Big bets on infrastructure (Gimlet) and small, pragmatic workflows (Spotify’s shunt) are both ways of attacking the same problem: inference is now the recurring cost, and efficiency wins on either the hardware or process side. The most consequential wins this year won’t come from a single breakthrough chip or model, but from combining smarter runtimes with smarter workflows — and from those who actually show transparent, repeatable numbers.