https://derekwang2002.github.io/ai-intelligence/en/projects/fde-gym/ Material changes in AI models, agents, engineering and infrastructure, with primary sources and recommendations. en Mon, 02 Nov 2026 16:56:00 GMT https://derekwang2002.github.io/ai-intelligence/en/events/ev-20261015-06/ ev-20261015-06-published Sat, 17 Oct 2026 15:59:50 GMT New TRIAL <p>FDE-Gym contains 95 end-to-end delivery tasks across 20 customer scenarios and 18 sectors. Agents must uncover unstated constraints, modify running systems and are scored by improvement in the customer business metric. The authors evaluated 13 model configurations from seven providers; the best resolved only 14 tasks. The harness and three example tasks are public, while the full task set requires access approval.</p><p><strong>Why read: </strong>The benchmark extends from fixing code to discovering requirements and improving business outcomes, exposing a coding-benchmark blind spot.</p><p>Recommendation: TRIAL</p><p>Sources: <a href="https://arxiv.org/abs/2610.22350">FDE-Gym paper</a> · <a href="https://github.com/deeplethe/fde-gym">FDE-Gym code and harness</a> · <a href="https://huggingface.co/datasets/DeepLethe/FDE-Gym">FDE-Gym dataset</a></p>