Officials say:
- Claude Code — What’s new: Week 37 (Sep 7–11) —
claude plugin eval scores a plugin against a test suite and reruns each case without the plugin as a baseline (claude plugin eval init drafts cases/graders); also added Desktop pane pop-out windows, a maxEffortLevel cap on every provider, and WebFetch now fails after 5 minutes instead of hanging (v2.1.263 → v2.1.269).
AI
- FrontierPhysics: Evaluating Agents for End-to-End Physics Research — a 56-task benchmark finds none of 26 agent configurations (22 models) pass more than 46 of 168 rollouts, with failures traced to weak rigor and expert intuition, not missing physics knowledge.
- LLMs Don’t Know What They Know: Benchmarking Temporal Self-Knowledge for Financial Backtesting — GPT-5 answers 76% of controlled facts correctly but claims to know only 38% of them, and gets 61% of “don’t know” facts right, showing self-reports understate actual knowledge.
- Transforming Integer Grids to Spatial Intuition: Perception, Not Planning, Governs ARC-AGI-3 — adding a perception engine (SARBLOH) lifts rubric score from 3/28 to 25/28 while using 89.4% fewer tokens, reaching 16.53 on the hidden Kaggle eval (13.7x the first Milestone winner).
- CompressionBench: How can your coding agents compress LLMs? — coding agents autonomously compress pretrained LLMs, with 29 of 36 submissions passing verification and the best design averaging +3.4% perplexity vs BF16 at 77.33% downstream accuracy (vs 79.56% uncompressed).
- SpecCompact: Accelerating LLM Agent Context Compaction — slot-parallel speculative verification speeds agent context compaction 2.04–5.63x over Hermes-style summarization on AppWorld/τ³, with a median 8.68x speedup in a Hermes user study.
- When Do Summaries Become Instructions? — agent-written handoff summaries add restrictions the underlying task record doesn’t support in 1.2–2.4% of readable summaries across two open-weight models; rewording the approval instruction cuts the rate from 5.0% to 1.8%.
- Dull, Dirty, Dangerous? Working Conditions and the Limits of Occupational AI Exposure — across 754 US occupations, higher AI exposure correlates with worse working conditions, but after controlling for physical demands, education, and wages, the effect becomes statistically indistinguishable from zero.
- Reinforcement Learning Enables Scaling Generalist Robot Policy Improvement — online reinforcement learning across many tasks at once improves generalist robot policies, including on tasks the policy never saw during online training.
- From Pixels to Actions: Towards Native Vision-Language-Action Policies — an encoder-free vision-language-action model (NEO-VLA) reading intermediate backbone layers hits 90.9% average LIBERO success with a frozen backbone (vs. 79.5% for Qwen3-VL-2B, 84.5% for Gemma-3-4B), rising to 95.3% after full fine-tuning.