Apple Launches 'Apple Upgrade' Lease-to-Own Program With Klarna on July 28Global AI Experts Push Back on US Distillation Claims Against Moonshot's Kimi K3OpenAI Launches Presence, an Enterprise Agent Platform Deployed by Consultants, Not APIsThe Distillation Wars: Why 'Model Theft' Is the New Front in the US-China AI FightAPIs Out, Consultants In: The Enterprise Agent Platform War Has StartedThe Thailand Problem: How Southeast Asia Became the Hole in America's Chip WallAsia Startup Funding Hits Multiyear Peak: $42.8B in Q2, Led by China and AICorgi Reportedly Raises Again at $4B — Its Third Round in Eight WeeksApple Launches 'Apple Upgrade' Lease-to-Own Program With Klarna on July 28Global AI Experts Push Back on US Distillation Claims Against Moonshot's Kimi K3OpenAI Launches Presence, an Enterprise Agent Platform Deployed by Consultants, Not APIsThe Distillation Wars: Why 'Model Theft' Is the New Front in the US-China AI FightAPIs Out, Consultants In: The Enterprise Agent Platform War Has StartedThe Thailand Problem: How Southeast Asia Became the Hole in America's Chip WallAsia Startup Funding Hits Multiyear Peak: $42.8B in Q2, Led by China and AICorgi Reportedly Raises Again at $4B — Its Third Round in Eight WeeksApple Launches 'Apple Upgrade' Lease-to-Own Program With Klarna on July 28Global AI Experts Push Back on US Distillation Claims Against Moonshot's Kimi K3OpenAI Launches Presence, an Enterprise Agent Platform Deployed by Consultants, Not APIsThe Distillation Wars: Why 'Model Theft' Is the New Front in the US-China AI FightAPIs Out, Consultants In: The Enterprise Agent Platform War Has StartedThe Thailand Problem: How Southeast Asia Became the Hole in America's Chip WallAsia Startup Funding Hits Multiyear Peak: $42.8B in Q2, Led by China and AICorgi Reportedly Raises Again at $4B — Its Third Round in Eight Weeks
A developer working with AI coding tools
Faros AI
Brief

New SWE-Together Benchmark Stress-Tests Multi-Turn Coding Agents

A new benchmark evaluates coding agents across interactive, multi-turn sessions rather than single-shot tasks. Claude Opus 4.8 leads at about 63% pass@1 with the lowest steering burden.

D
Daniel ParkAI Correspondent
2 min read

A new benchmark called SWE-Together is pushing coding-agent evaluation past the single-shot paradigm that has dominated the field, testing how agents perform when users clarify goals, add constraints, and correct mistakes across multiple turns.

Reconstructed from real user-agent sessions, SWE-Together curates 109 repository-level tasks drawn from more than 11,000 recorded interactions. To make those sessions replayable across different models, the researchers built a reactive, LLM-based user simulator that preserves each original user's intent and supplies feedback when an agent's progress calls for it. Crucially, the benchmark scores not just final repository correctness but the number of corrective feedback turns an agent needs — a proxy for how much "steering" a human has to do.

On the leaderboard, Claude Opus 4.8 leads with roughly 63 percent pass@1, ahead of GPT-5.5 and Claude Opus 4.6 (both around 58 percent) and GLM-5.2. It also required the fewest corrective interventions, at about 1.38 user-correction turns per trial. The paper reports a strong inverse relationship between capability and hand-holding — stronger agents reach the top of the board while asking less of their users.

The shift matters because real coding assistance is interactive, not one-and-done. As the field moves beyond static SWE-bench-style tasks toward agentic, multi-turn evaluation, SWE-Together offers a sharper picture of which models are genuinely good collaborators.

Newsletter

Get Lanceum in your inbox

Weekly insights on AI and technology in Asia.

Share

More in Brief

Lanceum

Independent coverage of AI and technology across Asia. We go beyond headlines to explain what matters.

Colophon

Typeset in Space Grotesk & DM Serif Display. Built with Nuxt & Tailwind. Powered by curiosity.

© 2026 Lanceum. All rights reserved.

Independent • Rigorous • Asia-Focused