https://derekwang2002.github.io/ai-intelligence/en/projects/dust/ Material changes in AI models, agents, engineering and infrastructure, with primary sources and recommendations. en Mon, 02 Nov 2026 16:56:00 GMT https://derekwang2002.github.io/ai-intelligence/en/events/ev-20261005-09/ ev-20261005-09-published Tue, 06 Oct 2026 16:01:24 GMT New TRIAL <p>Dust estimates transformer gradients using token-wise activation perturbations and only forward evaluations. The authors report performance competitive with backpropagation in small pretraining experiments and much better efficiency than weight-space evolution strategies. Public MIT-licensed code reproduces the estimator up to a 37.7M-parameter setup, but the authors say the method still needs orders-of-magnitude efficiency gains before becoming practical.</p><p><strong>Why read: </strong>Dust provides a public minimal implementation for training transformers without backpropagation.</p><p>Recommendation: TRIAL</p><p>Sources: <a href="https://qlabs.sh/research/dust/">Dust: Pretraining Transformers Without Backpropagation</a> · <a href="https://github.com/qlabs-eng/dust">qlabs-eng/dust</a></p>