ML-Safe Testing Series, Part 6: Our second Cosmos Curator study, on a much broader set of autonomous vehicle video, shows significantly smaller file size and preserved caption meaning, while the added shift from Beamr’s optimization is practically unnoticeable. The right operating point is specific to each pipeline, model, and content
Ronen Nissim, AI lead at Beamr
Autonomous-vehicle data programs cannot economically retain, move, and repeatedly process raw video at full scale. Compression is operationally necessary. However, most compression settings in autonomous vehicle pipelines were never chosen for the models consuming the video. They arrive as defaults rather than from a structured evaluation of how a vision-language model responds to compression.
The gap matters because the impact of compression may change with footage, encoder configuration, and model architecture – so a level that’s conservative for one pipeline can be wasteful for another. The practical question is how far each program can compress before the outputs used to index, search, filter, and curate its footage change in a way that matters.
In this research we answer this question across a broad set of video data, different model architectures, and an extensive compression range, measured under NVIDIA Cosmos Curator’s production-like generation settings. NVIDIA Cosmos Curator is a framework for filtering, annotating, and deduplicating data for physical AI development.
Broader evidence at the production operating point
The current research builds on our previous study on the NVIDIA Cosmos Curator AV pipeline. Then, we evaluated nine videos, one vision-language model (Qwen2.5-VL), and one compression point. We found then that compression-induced embedding shifts were smaller than the model’s ordinary run-to-run variation and that video identity remained strongly recoverable.
This time we significantly expanded the research scope to see what it takes to gain compression savings while preserving the fidelity of ML captioning model outputs, what we call ML-Safe compression.
We broadened the dataset from 9 to 96 clips, representing 46 drive or sequence clusters across four data sources (PandaSet, ROAD, A2D2, and Beamr/internal material). We ran three models Qwen2.5-VL (7B), LLaVA-OneVision (7B), and InternVL3.5 (8B) over NVIDIA’s standard constant-quality hardware encoder (NVENC) and Beamr’s Content-Adaptive Bitrate compression (CABR). More details on the models and the encoders below.
In the previous research we used a stochastic temperature of T=0.7. This time we used the NVIDIA Cosmos Curator (up to v2.8.0) production policy: T=0.1, top_p=0.001, and 2 fps frame sampling.
One of our key findings: No statistically meaningful degradation in the NVIDIA Cosmos Curator model: Qwen2.5 VL showed only about 0.003 additional semantic shift, within the uncertainty of the measurement.
(*) CABR adds 0.003 for Qwen, 0.019 for LLaVA, and 0.022 for InternVL on top of NVENC-CQ. Qwen’s addition sits inside the uncertainty of that comparison.
Wording shifts while meaning is retained
Across the three VLM architectures using the same near-deterministic text-generation policy, the study shows that HEVC compression changed how the models described each scene: exact matches became less common, and the first wording difference arrived earlier, as compression increased. This establishes compression sensitivity, but not by itself a loss of scene meaning.
At the same time, we found that a different wording can still describe the same scene. While the words shifted, the broad meaning stayed close and results consistently preserved the descriptions’ core meaning.
We found that a small embedding movement can still conceal a changed object or count. We accounted for that ambiguity by weighing the measured distances against supporting evidence – and found that broad caption meaning stayed close, at a cosine similarity of 0.96 or higher.
The Operating Point Depends on the Task
The study doesn’t prescribe one quality setting for everyone – there is no universal “safe compression level.” Our research shows the necessity of mapping how captions respond to compression, what we call the rate–caption-response curve. The curve allows each team to pick the best performance–savings point for its own model, content, and downstream task.
The right point depends on where the compressed file is used, and also on what the pipeline must preserve. For example, a semantic-search or broad scene-curation workflow can tolerate natural paraphrase and may choose a lower-rate point.
Beamr Compression Improves the Savings
At the most aggressive compression level we tested, the mean chord distance – how far a caption’s embedding moves from the uncompressed original – corresponded to cosine similarities of 0.962–0.970, where 1.0 means an identical embedding.
Compressed captions remain, on average, semantically closer to their uncompressed counterparts, compared to captions between different clips.
At mid-range levels, Beamr’s Content-Adaptive Bitrate compression (CABR) cut file size by 45.9% compared to standard constant-quality encoding, while adding only about 0.003 chord distance for Qwen, within the uncertainty range of the comparison, and about 0.02 for the other two models. When those same mid-range CABR files are compared with the stored uncompressed source, they are 97.1% smaller. Their mean cosine similarity to the source captions is 96.9–97.4%.
The compression map in deployment
To conclude, a clear map for evaluating the right compression point is possible because the measurement is clean: repeat generation across the large dataset of 96 clips and three models reproduced every source caption exactly, so a change in the caption traces to the pixels the encoder changed.
In deployment, measure file size and the downstream task for the specific model and content, and choose the lowest rate that still holds the required task performance.
-> Run Beamr’s ML-Safe compression on your own data: beamr.com/autonomous
ML-Safe AV Video Data Testing series:
Part 1: Beamr is Pushing the Boundaries of AV Data Efficiency, Accelerated by NVIDIA
Part 2: ML-Safe AV Video Data Processing Achieves Up to 50% Storage Reduction
Part 3: Deep Dive: Managing the Petabyte-Scale AV Video Data Bottlenecks
Part 4: Beamr’s ML-Safe Video Compression Validated on NVIDIA Cosmos Curator
Part 5: Stress-Testing ML-Safe Compression on 3D Object Detection
Which encoders did we test?
NVENC-CQ is the baseline configuration of the NVIDIA NVENC hardware video encoder, using constant-quality encoding.
NVENC-CABR starts from the same CQ target and adds Beamr’s Content-Adaptive Bitrate (CABR) optimization, which spends fewer bits while preserving the quality target. We include it because it is the savings option a customer would deploy, and we keep NVENC-CQ beside it so the extra savings can be judged at the same quality target and, separately, at the same file size.
Which models did we test?
Qwen2.5-VL (7B) is the vision-language model used in the NVIDIA Cosmos Curator AV captioning workflow evaluated in this study. It uses Qwen’s own vision transformer; the tested workflow generates each caption in two stages, a draft and then a refinement. We included it to evaluate compression in that configuration.
LLaVA-OneVision (7B) pairs a SigLIP vision encoder with a Qwen2 language model. The language side is related to Qwen; the vision front end is not. It tests whether the compression response is specific to Qwen’s vision tower.
InternVL3.5 (8B) pairs an InternViT vision encoder with a Qwen2.5 language model, a third visual architecture at a similar scale. A result that holds for all three is much less likely to be an accident of one vision encoder.
