FlashVector tunes production serving across kernels, servers and features

Unity reports deploying FlashVector across its advertising model-serving stack. The agent optimizes CUDA kernels, PyTorch graphs, NVIDIA Triton C++ code, Python feature transforms and serving parameters in one closed loop. Reported production gains reach 2x throughput and 1.98x lower latency on the model server, plus 1.6x throughput on the feature store. The paper does not release the system or an independent reproduction.

For inference teams, the useful unit of optimization is the full serving path, not only a GPU kernel. This is a production signal that agents can search across languages and layers while checking service-level constraints. The closed implementation and vendor-reported measurements keep it at WATCH until another team can reproduce the workflow.