OpenAI has published the MultiLangBench tasks and tests; the 87.4% versus 26.8% comparison is OpenAI-run.
OpenAI publishes MultiLangBench for low-resource coding
2026-10-19Verified announcementTRIAL
OpenAI has published MultiLangBench, an Apache-2.0 benchmark covering 20 programming languages with less public training data, including COBOL, Fortran and Visual Basic. OpenAI reports GPT-6 Astra averaging an 87.4% test pass rate versus 26.8% for GPT-5.6 Sol. The repository contains public tasks and tests, but the headline model results are vendor-run and do not by themselves show production migration quality.
Official · ReferenceAdvancing coding in low-resource programming languagesOfficial · ReferenceOpenAI MultiLangBench repository
Availability: Apache-2.0 tasks and tests; repository created 2026-10-19T20:58:00Z
What this means for you
For teams maintaining legacy systems, public tests make it possible to compare models on languages that mainstream coding benchmarks mostly ignore. Treat test pass rate as a narrow code-correctness signal, then add repository conventions, integration tests and human review before using it to plan a migration.
Evidence & related material
Technical details & scores
| Scope | 20 programming languages with lower public data availability |
|---|---|
| Availability | Apache-2.0 tasks and tests; repository created 2026-10-19T20:58:00Z |
| Vendor Result | GPT-6 Astra 87.4% average test pass rate; GPT-5.6 Sol 26.8% |
| Evidence Limit | vendor-run model evaluation; benchmark pass rate is not production migration success |
| Time Basis | official article published 2026-10-19T12:00:00Z; public repository created 2026-10-19T20:58:00Z |
Technical impact 3
Engineering value 4
Adoption signal 1
Maturity 3
Verification cost 3
Risk 2