Abstract
<title>Abstract</title> <p>Whisper exhibits strong robustness and cross-lingual generalization in open-domain ASR, but its performance remains limited in low-resource domain-specific speech. We propose a parameter-efficient adaptation framework for Whisper-based ASR that is applicable across model scales and target languages. The framework combines LoRA-based adaptation in key attention modules, on-the-fly SpecAugment for low-resource training, and a two-stage decoding strategy with N-best beam search followed by frozen language model rescoring. This design enables efficient adaptation with minimal trainable parameters while retaining the generalization ability of the pretrained model. The framework is readily extensible to different Whisper backbones and diverse low-resource professional speech settings.</p>