LIFT-Bench publishes 784 test tasks and LIFT-Train publishes 34,258 training tasks for constraints that apply only to selected sections of long structured outputs. Apache-2.0 code includes synthesis and evaluation pipelines. Scores depend on a pinned Gemma judge, and AI21 reports only small out-of-benchmark gains.
AI21 opens LIFT-Bench for scoped long-form instruction following
2026-10-25Frontier watchTRIAL
LIFT-Bench publishes 784 test tasks and LIFT-Train publishes 34,258 training tasks for constraints that apply only to selected sections of long structured outputs. Apache-2.0 code includes synthesis and evaluation pipelines. Scores depend on a pinned Gemma judge, and AI21 reports only small out-of-benchmark gains.
What this means for you
For teams training models to write reports, plans or documents, section-scoped constraints are closer to production than whole-answer style checks. Judge dependence and first-party results make this a reproducible trial, not a general capability claim.
FRONTIER WATCH
Public artifacts and validated results are different.
The recorded fact is the public work and its artifacts. Author-reported outcomes do not establish independent reproduction. Next checks include usable code, task conditions and third-party validation.
Evidence & related material
Technical details & scores
| Tasks | 784 test tasks across 10 splits; 34,258 training tasks |
|---|---|
| License | Apache-2.0 code and pipeline |
| Judge | google/gemma-4-31B-it; comparison requires the same endpoint |
| Human Check | Scoped judging improved macro-F1 by 11-19 points on 305 human-labelled examples, reported by AI21 |
| Time Basis | AI21 page datePublished 2026-10-25T08:52:27Z; before overlap and recovered as late discovery |
Technical impact 3
Engineering value 4
Adoption signal 2
Maturity 3
Verification cost 2
Risk 2