Summary on the Chat-Scenario Chinese Lipreading (ChatCLR) Challenge

Abstract

Lipreading which infers spoken content based solely on visual information such as lip movements is crucial in multi-modal research medicine and human-computer interaction. We organized the Chat-scenario Chinese Lipreading (Chat-CLR) challenge focusing on unscripted chat scenarios among native Chinese speakers. We placed emphasis on two tasks wake word lipreading (WWLR) and target speaker lipreading (TSLR). We are dedicated to fulfilling the requirements of waking up smart home devices within household settings and utilizing video for speech recognition with these smart home devices. For the WWLR task we received submissions from 5 teams with the top-performing system showing a 71.4% improvement over the baseline system. In the TSLR task we received submissions from 6 teams and the best system achieved a 22.1% improvement compared to the baseline system.