Swarm Signal - AI Research for People Who Build
Articles about Swarm Signal - AI Research for People Who Build
ComplexMCP Finds Tool-State Gaps
ComplexMCP is a useful benchmark because it tests the part of tool use that simple function-calling tasks avoid: one action changes the state that the
Android Bench Tests Long Coding Work
Android Bench turns mobile coding evaluation from small patch repair into long-horizon Android work. Google says the updated benchmark adds multi-day
Can Memory Systems Track a Group?
GroupMemBench reframes agent memory as a group problem, not a private chat log. The paper argues that assistants used inside teams need to remember who
KNOWS Tests Browser Work After Search
The KNOWS paper turns web-agent evaluation from finding information into producing a usable document, spreadsheet or slide deck
OpenShell Moves Agent Safety Below the Model
NVIDIA’s Open Agent Safety Platform is worth reading as an infrastructure claim, not as a new prompt rule [NVIDIA
When Tools Get in the Way Cuts Accuracy
The paper *When Tools Get in the Way* isolates a failure mode that is easy to miss in agent design: a model can answer worse when an unnecessary but
ToolGrad Turns Tool Failures Into Data
Google’s ToolGrad reframes tool-use training data as an execution problem before it is a prompt-writing problem. Instead of asking a model to invent a
OSWorld-Pro Exposes Click Failures
OSWorld-Pro turns desktop automation into a step-by-step audit. The paper extends OSWorld-style computer-use evaluation with process checks, so a model
Progress Hints Cut Tool-Call Waits
Tool-calling systems already know more about long-running work than the model server can see. A new paper, *Ask the Tool, Don’t Guess*, argues that the
BenchAgent Tests Multi-Agent Lift
BenchAgent gives multi-agent workflow design a useful test: hold the benchmark loader, tool access, answer contract, usage accounting and trajectory
Can Next-Turn Metrics Test Workflows?
The paper *When Better Turns Do Not Make Better Agents* tests a quiet failure in agent evaluation: a model can improve at predicting the next local action
Cross-Substrate Authority Moves to Runtime
The *Beyond Agent Harnesses* paper names a production failure that ordinary workspace inspection misses: an agent can see the same files and memory while
Hyper-Tau Bench Tests Agent Building
Hyper-Tau Bench changes the unit of evaluation from an agent using tools to a coding system building a deployable customer-service agent [Hyper-Tau
Can SentinelBench Measure Waiting?
SentinelBench turns waiting into an agent benchmark [SentinelBench](https://arxiv.org/abs/2606.05342). The Microsoft Research paper introduces 100
Mid-Range Tasks Cut Agent Eval Cost
Efficient Benchmarking of AI Agents asks a practical release question: can teams compare agent systems without rerunning every expensive interactive task