Source code samples

		source code VIEW SLIDE SHOW DOWNLOAD ALL

How to do model imitation recording exercises to improve language learner pronunciation in the LRC and beyond

1. Sometimes teachers ask about support for voice recognition in the LRC. The term voice recognition or speech recognition (the former appears to be analogous to face recognition in authentication and other security contexts?) is usually reserved for software that can transcribe your voice into text – still no free option for this, AFAIK. Dragon naturally speaking is the oft recommended market leader outside of education (and within, Auralog Tell me more, see below). Update summer 2012: We are working on enabling the Speech recognition built into Windows 7 Enterprise for English, Chinese (Simplified and Traditional), French, Spanish, German, and Japanese.
2. Often times, what is actually desired is a digital audio recorder with voice graph, ideally a dual track recorder.
   1. In the LRC student computers, we have for exactly this purpose a digital audio recorder as part of the SANAKO Study 1200language learning system
      1. It features a dual track recorder (allows to listen to teacher track which can be a prerecorded model to imitate on the left channel while recording the student track on the right channel of a stereo track) with a voice graph: . See this dual-track-voice-graph screencast demo from the vendor and also our student cheat sheet from the vendor documentation.
      2. The Sanako is available in the LRC, as well as in many other educational institutions around the world, but neither free nor web-based (although a web-based version seems to be in the works). It currently requires MS-Windows to run.
   2. A popular and free audio editor (but not an SLA – specific application, let alone geared towards model imitation; also, for all practical ends and purposes,  requires an extra download and installations of an MP3 encoder to be able to save recordings as compressed MP3) is Audacity. To use for model imitation exercises,
      1. the student can open a model track (mp3 recommended)
      2. and manage within the program the imitation portion, using the voice graph:
      3. then export  back out as mp3,
         1. either her responses individually (see my demo screencast, requires Windows Media Player on Windows, which actually shows a question/response rather than a model imitation, but same principle),
         2. or, by deleting the model track, the response parts mixed down to one track,
         3. or also, if, like in my demo screencast, the timeline sequence of model (with pauses) and responses is carefully managed (so that model and imitation do not overlap), mixed down to one track.
   3. In one language program, I have worked extensively with Auralog Tell me more
      1. which was (not exclusively, but arguably too much) based on this pedagogic concept of having students compare the voice graph of their imitation with the model voice graph (while it do did not allow for teachers to upload their own content, and was certainly not free).
      2. To my knowledge, Auralog Tell me more does not allow for adding teacher-produced content as models.
      3. I did like the self-reflective and repetitive practice element. However, I found  that students – apart from intonation and (not useful for not pitch based languages) pitch -, did not benefit as much as one might have expected from viewing the voice graph, indeed tended to get overwhelmed, even confused by the raw voice information in  such a voice graph.
      4. And automated scoring of pronunciation (or speech recognition” – not free form, but on a level that has been commoditized in operating systems like Windows 7, the level of voice-directed selection between a limited set of different options, like menu options, and in the case of Auralog, choosing between different response options) seemed iffy and less than transparent in Auralog Tell me more, even though this is  their primary selling point. E.g. when I made deliberate gross mistakes, the program seemed to change its standards and wave me through ( English pronunciation example; also observed by me when testing Auralog with East Asian speakers of English).
3. A voice graph  is not the same as a more abstract phonetic transcription (although I do not know whether language learners can be trained in phonetic symbol sets like the IPA).  There are now experimental  programs that can automate the transcription of text into phonetic symbol sets for e.g. Portuguese or Spanish. Maybe you will find that practice with recording and a phonetic transcription of the recorded text is more useful for your students’ pronunciation practice than a fancy voice graph.

Learning materials management: Textbook exercises (2000-2008)

Textbook exercise management is a rapidly evolving field, with more textbook becoming digital and online resources and more metadata getting added and AI getting implemented to enable personalized (data-driven, feedback-based) learning paths.

German.xls was an attempt to be able to sort, search, filter the exercises of some bigger textbooks in the American college market, each containing thousands of exercises (how many? why does it take a sumif() to find out?):

Subtitles.xls converted text files with movie subtitles which can be extracted from DVDs or found into spreadsheet for post-processing (search, filter, sort – and assign different show times, for DVD editions differ). online,
Auralog Tell me more 7 is a language program that allegedly comes with “more than five times the amount of content than other language programs” – but strangely not with a table of contents of its exercises. Automation extracted the exercises first into the file system for full text search with Windows Desktop Search, then converted the extracted files into links in the Auralog Content XLS.

Auralog Tell-me-more Demo Screencasts

An overview, mostly narrated:

Introduction	http://plagwitz.blob.core.windows.net/content/Auralog_Basics.wmv
Introduction	http://plagwitz.blob.core.windows.net/content/auralog_basics_with_writing_speech.wmv
Introduction	http://plagwitz.blob.core.windows.net/content/auralog_test_english_speech_recognition-farm-visit.wmv
Introduction	http://plagwitz.blob.core.windows.net/content/auralog_tmm_1learningpath_all_excercisetypes__not_linked_to_lesson_speech_errors.wmv
Introduction	http://plagwitz.blob.core.windows.net/content/auralog_tmm_1learningpath_all_excercisetypes_linked_to_lesson_speech_errors.wmv
Introduction	http://plagwitz.blob.core.windows.net/content/auralog_tutortools_studentrecordings.wmv
Content-Searching	http://plagwitz.blob.core.windows.net/content/auralog_TutorTools_Tools_Content_Printout.wmv
Content-Searching	http://plagwitz.blob.core.windows.net/content/auralog_autoit_contentprinting.wmv
Content-Searching	http://plagwitz.blob.core.windows.net/content/auralog_content_printout_search_with_windows_desktop_search.wmv
Content-Searching	http://plagwitz.blob.core.windows.net/content/auralog_content_xls.wmv
Content-Linking	http://plagwitz.blob.core.windows.net/content/auralog_autoit_tutortools_learningpaths_getting.wmv
Content-Linking	http://plagwitz.blob.core.windows.net/content/auralog_learningpathcreating_testrun.wmv
Account-Creation	http://plagwitz.blob.core.windows.net/content/auralog_autoit_studentgroups+studentgrouptypes.wmv

Example 8: Auralog Tell-Me-More Speech Recognition Test

How usable is the Auralog Speech Recognition for language learning? This test, by a non-native speaker of English, gives some authentic data points.

The test shows: Auralog Speech Recognition

1. can be easily tripped up; however, by errors that  a non-native language learner would not normally make
2. more concerning is that the built-in AI, instead of e.g. escalating to additional feedback or help, like the pronunciation waveforms (which in itself seem to encourage only repeated attempts to mimic a given intonation, while not being fine-grained enough to spot mispronunciations on a word, let alone letter level) – lowers the requirements when a speaker repeatedly fails (which in extreme seems to amount to “waving through” any utterance).
3. the preset dialogue – only few exercises including wrong answer options, most exercises testing only a comprehensible pronunciation of a given reading text which makes the exercise much easier for the built-in speech recognition, but also much less realistic and useful for a language learner (or more of a reading exercise).

Automating Auralog Tell-Me-More with AutoIt. Presentation at EUROCALL 2008

Auralog Tell-Me-More is a leading language learning software system which provides a vast amount of content in an advanced technical infrastructure that we found lacking in usability within an higher education language learning environment.

AutoIt is a programming language for GUI automation which I used to better integrate the Auralog software into the higher education language learning process, including

1. programmatic creation of courses and accounts
2. programmatic extraction and digital repository management for over 30.000 learning units.
3. programmatic creation of 10,000s of learning paths,

Results were presented (screencast) at EUROCALL 2008: “Automating Auralog (pdf)”:

1. cpurse and account creation

creates 100s of courses , creates and enrols up to 2,000 student accounts every term,

1. content extraction produces files for adding search and spreadsheet for sort/filter functionality:
2. learning path creation.

More detailed background information here: plagwitz_auralog_accounts_project_pub.pdf, plagwitz_auralog_project_pub.pdf

Auralog Tell Me More Language Learning Software : Making Content Searchable & Automating Customization