Treble Technologies and Hugging Face Launch FFASR Leaderboard for Far-Field Speech Recognition

One of the more persistent frustrations in automatic speech recognition development is the gap between benchmark performance and real-world deployment. Models that ace standard evaluations often behave entirely differently once real room acoustics enter the picture, reverberation, background noise, and microphone distance all come into play. The complex interactions between these factors affect performance in ways that clean-speech benchmarks simply fail to capture. The FFASR Leaderboard is an attempt to quantify that gap.

Treble Technologies and Hugging Face are rolling out the Far-Field ASR (FFASR) Leaderboard, the first open, community-driven benchmark built to evaluate ASR models under realistic far-field acoustic conditions. It's live now, and the community can submit models, dive into results, and help shape what comes next.

The far-field challenge

Voice interfaces have moved well beyond the headset and the smartphone. AI voice agents, conference room transcription, in-car assistants, humanoid robots, smart glasses, and hands-free tools are all seeing rapid adoption. What they share is that they operate in acoustically complex environments: reverberation, background noise, overlapping sounds, and a microphone that might be anywhere from a few feet to several meters away from the speaker.

The dominant ASR evaluation paradigm hasn't caught up with that reality. Clean, close-microphone benchmarks remain the standard, and while they're useful for measuring core recognition quality, they don't predict far-field performance. A model that shines on LibriSpeech or other near-field sets can degrade substantially once real room acoustics come into play. Several research efforts have tackled far-field and noisy speech evaluation, including CHiME, URGENT, and NOIZEUS, but the community hasn't had a standardized, open way to measure that degradation consistently across models in a continuously updated leaderboard format. That's exactly what FFASR is built for.

How FFASR works

A major hurdle for far-field evaluation is data availability. Collecting far-field recordings across a representative range of room types, microphone distances, and noise conditions at scale is prohibitively expensive with physical measurements alone. Simulation makes it possible to cover that space systematically and extend coverage over time without a corresponding jump in measurement cost.

Another goal of FFASR is to spur development of models that are explicitly robust to these conditions. Leaderboards have historically been effective at directing research effort. By making far-field performance visible and comparable, the hope is to raise the priority of real-world acoustic robustness across the field.

The FFASR Leaderboard evaluates models across nine conditions. The four that determine the primary ranking score, as of June 22, 2026, are clean far-field, noisy far-field at three SNR tiers, and reverberant far-field. To give a sense of what these conditions actually sound like, the leaderboard provides sample audio: the same speech utterance as dry anechoic audio, then convolved with a room impulse response, and finally with noise added at each SNR tier.

Two additional columns, Lab Measured and Lab Simulated, serve as a sim-to-real validation track. The leaderboard also includes moving-source splits, currently in beta, which evaluate models against audio where the speaker is moving rather than stationary. This condition reflects use cases such as humanoid robots, in-car speech, and mobile voice assistants.

The acoustic data is generated using Treble's hybrid simulation engine, which combines a wave-based solver at low to mid frequencies with geometrical-acoustics modeling at higher frequencies. This approach captures physical phenomena that simpler simulation methods tend to miss: diffraction, scattering, interference, and modal behavior. The result is simulated data that closely mirrors measured acoustic conditions.

Fourteen fully furnished rooms are included in the benchmark, ranging from 20 to 470 cubic meters and covering bathrooms, living rooms with hallways, offices, classrooms, and restaurant spaces. Each acoustic scene contains one target speaker, recorded in an anechoic chamber, and up to three noise sources. Every scene includes both a transient noise source, like coughing, and a continuous noise source such as HVAC, at three SNR levels.

Alongside word error rate (WER), the leaderboard reports RTFx, audio seconds per inference second, for every submission, evaluated on an NVIDIA L4 GPU under identical conditions. Accuracy and latency together are what matter in real deployments, and the Pareto front view in the Analysis tab makes that tradeoff explicit.

Early results and insights

With the leaderboard live, a consistent pattern is emerging across all submitted models: the gap between near-field and far-field performance is large, and it widens significantly as SNR drops. Near-field WER values on clean dry speech look comparable to what the same models achieve on established benchmarks. Far-field WER at low SNR tells a very different story, often several times higher. The benchmark makes this degradation visible and comparable in a way that was previously difficult to do outside proprietary evaluation pipelines.

The Pareto front of average WER against RTFx is also revealing. There's a genuine spectrum of approaches represented in the current submissions: models that prioritize speed at the cost of some accuracy, models that push accuracy at the cost of throughput, and a smaller handful that manage a competitive position on both axes. Visualizing these tradeoffs against far-field accuracy rather than clean-speech accuracy produces a materially different picture of where the real differences between systems lie.

Submitting models and the road ahead

To submit, head to the Submit tab on the FFASR Leaderboard, paste a Hugging Face model ID, and evaluation runs server-side against the held-out dataset. The pipeline supports Whisper variants, IBM Granite Speech, Cohere Transcribe, Wav2Vec2 and HuBERT CTC heads, SpeechBrain ASR, and most other ASR architectures on the Hub without any custom configuration.

For teams using more complex inference stacks, including systems that combine speech enhancement with ASR, a custom evaluator option allows you to define your own evaluate() function. Custom evaluators run on Hub Jobs after moderator review. The held-out evaluation set uses 2,000 anechoic speech samples across 14 rooms at three SNR tiers, roughly 8 hours of audio per condition, with Whisper-style text normalization applied consistently.

Future tracks being explored include multi-talker scenarios, microphone array evaluation covering beamforming and spatial filtering approaches, and echo cancellation. The direction of development will depend on community feedback. The FFASR Leaderboard is designed to grow, and its evolution should reflect real deployment needs.