Hugging Face and VoiceArena have partnered to introduce Hindi and Indian English evaluation sets to the Open ASR Leaderboard, marking the first Indic language on the leaderboard. The new sets, Monsoon en-IN and Monsoon hi-IN, are designed to address biases in automatic speech recognition (ASR) by varying along nine axes including geography, age, gender, and more. Each set includes 4,888 speakers with 12 speaker attributes recorded for each, ensuring a comprehensive evaluation of ASR models. The datasets are available for self-scoring and are speaker-disjoint to prevent benchmark-specific optimization.
Source: Hugging Face
