Game Changers

Competition Details

SPRING: Speech and Pronunciation Improvement via Games

by Anuj Tewari
Co-authors: Nitesh Goyal, Matthew Chan

Lack of proper English pronunciations is a major problem for immigrant population in developed countries like U.S. This poses various problems, including a barrier to entry into mainstream society. We have explored the use of speech technologies merged with activity-based and arcade-based games to do pronunciation feedback for Hispanic children within the U.S. We have also conducted a 3-month long study with immigrant population in California to investigate and analyze the effectiveness of computer aided pronunciation feedback through games.
The two games we developed followed the principles of teaching, drill, immediate feedback, scores, and repetition. Both games feature the word, associated playable American accented female voice, and spelled-out-pronunciation to aid the users. The spelled-out-pronunciations were obtained from the online dictionaries [online dictionary] and then modified accordingly by a trained linguist with five years of experience.
Zorro is a character based arcade game, which involves moving Zorro, the main character, of the game from left to the right of the scene using arrow keys until he reaches the castle. On the way, he encounters five closed chests, dangerous animals, tricky terrain, and obstacles, which must be overcome. The obstacles can only be overcome by opening up the chests. Each chest contains a word, associated pronunciation, and the associated audio pronunciation coupled animated spelled-out-pronunciation. The word is pronounced three times every time it is played. Next, the user gets an opportunity to record their pronunciation of the word by the click of a button. A feedback screen showing the correct and wrong parts of the pronunciation, and the associated score follows this. She also hears her own pronunciation and the intended pronunciation. After crossing the five obstacles by practicing the five words and avoiding the deadly animals, the user gets to win the game.
Voz.Guitar is an activity-based game that displays the word, associated spelled-out pronunciation, and plays the associated pronunciation. Next it allows the users to hit the falling alphabets of the words at the right time. Next, the user is obliged to pronounce the word. The feedback screen displays the hits and misses in the spelled-out-pronunciation and corresponding errors. The user hears her pronunciation followed by the intended pronunciation. The game is adaptive and hence, tends to automatically repeat the words, which have not received a satisfactory pronunciation response from the players. Each positive utterance increases the score of the users.