Some educational resources related to automatic speech recognition (ASR)

Note: The latest version of this page is now at a new location. The copy you are currently viewing is not as up-to-date.

Online resources:

• Slides from the ASR class taught at Columbia in 2005 by IBM staff
• Slides from ASR course taught at Stanford by Dan Jurafsky: here and here
• Dan Ellis' slides from his Spring 2004 audio processing class
• Dan Ellis' slides from his Fall 2006 DSP class
• Slides from Jeff Bilmes' winter 2005 speech processing class
• Steve Renals' n-gram language modeling lecture from a computational linguistics class
• Joshua Goodman's publications. There is useful tutorial material in The State of the Art in Language Modeling and A Bit of Progress in Language Modeling
• Slides from Bryan Pellom's 2004 speech recognition class
• Slides from Automatic Speech Recognition, Spring 2003, MIT
• Hidden Markov Models for Speech Recognition, Spring 2004, OGI (PowerPoint ONLY)
• "A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition": this classic article by Rabiner can be found with Google or Google Scholar
• "Automatic speech recognition: History, methods and challenges" is a 2008 article by Douglas O'Shaughnessy in the journal Pattern Recognition (volume 41, issue 10)
• Morgan and Bourlard intro to hybrid (Neural Net/HMM) speech recognition systems
• Eric Fosler's HMM Tutorial
• There is a nice little overview of the EM algorithm, with references, in the paper Convergence Results for the EM Approach to Mixtures of Experts Architectures by Jordan and Xu
• Jeff Bilmes' Tutorial on the EM algorithm.
• Essex phonetics articles
• Jason Eisner's Interactive Spreadsheet for Teaching the Forward-Backward Algorithm
• Yaroslav Bulatov's recommendations of online machine learning materials: here and here.
• Some intro material on dynamic programming: here and here (includes Diff.java which implements the diff command). If you are having trouble learning the Viterbi algorithm, it may help to start by learning dynamic programming and DTW -- see (e.g.) the Columbia/IBM slides for information about the connection.
• Matlab code to train a Gaussian mixture model with EM
• Adaptation techniques in ASR
• The Scientist and Engineer's Guide to Digital Signal Processing free online DSP book

Books:

Here is a short list of books. This is not a complete list of popular or recommended books! The books I list here vary in focus so I've mentioned how to find the table of contents online. I also recommend checking out the reviews on amazon.com.

• Spoken Language Processing: A Guide to Theory, Algorithm and System Development by Xuedong Huang, Alex Acero, Hsiao-Wuen Hon. The table of contents can be viewed here.
• Speech and Audio Signal Processing: Processing and Perception of Speech and Music by Ben Gold and Nelson Morgan. The table of contents can be viewed on amazon.com.
• Speech Processing -- A Dynamic and Optimization-Oriented Approach by Li Deng and D. O'Shaughnessy. The table of contents can be viewed on amazon.com.
• Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition by Daniel Jurafsky and James H. Martin. The table of contents (and long excerpts) can be viewed here.

Gunnar Evermann's book recommendations can be found here. I'm not sure if that page will stay up now that he's leaving the HTK project, so here is a summary:

• Pattern Classification by Duda, Hart, and Stork (this is one of my favorites too)
• Introduction to Statistical Pattern Recognition by Fukunaga
• Automatische Spracherkennung -- Grundlagen, statistische Modelle und effiziente Algorithmen by Schukat-Talamazzini
• The above-mentioned book by Xuang, Acero and Hon.
• Automatic Speech Recognition -- The Development of the SPHINX Recognition System by Lee.
• Statistical Methods for Speech Recognition by Jelinek.
• Corpus-Based Methods in Language and Speech Processing, edited by Young and Bloothooft

Applications of ASR:

Speech Technology Magazine is a good source for information about applications of speech technology.

Current ASR research:

If you want to know what techniques are currently attracting attention at the cutting edge of ASR research, papers that describe speech recognition systems that were built for official project benchmarks can be a good source of information. These systems tend to use a lot of different, carefully chosen techniques. Many of these papers have a year (the year of the benchmark) and the word "system" in the title, which makes them easy to find. For example, many such papers can be found with a Google Scholar search for:
intitle:2007 intitle:system speech recognition Some of these papers use the word "recent" in the title instead of the year. So in that case the above search would change to
intitle:recent intitle:system speech recognition

NIST organizes many of these benchmarks and they have information on their web site.