Updated: October 2026
Speech recognition—often called automatic speech recognition or ASR—is technology that converts spoken audio into text or another machine-readable representation.
Modern ASR systems can transcribe meetings, generate captions, support voice interfaces, and make recorded information searchable. They do not automatically understand meaning, identify the speaker, or generate spoken responses; those are related but separate capabilities.
Key takeaways
- Automatic speech recognition converts spoken audio into text.
- ASR is different from speaker recognition, text-to-speech, NLP, and conversational AI.
- Word error rate measures substitutions, deletions, and insertions, but it does not capture every real-world consequence.
- Performance varies with language, accent, noise, vocabulary, recording conditions, and deployment context.
What is speech recognition?
Speech recognition is the process of converting a speech signal into text or linguistic units a computer system can use. The term automatic speech recognition emphasizes that software performs the transcription rather than a human transcriber.
ASR begins with audio. A model analyzes patterns in the signal and estimates the most likely sequence of words or tokens. Other systems may then add punctuation, identify speaker turns, summarize the transcript, search it, or use it to trigger an action.
Speech recognition is one branch of the broader field of machine perception.
Speech recognition vs. related technologies
| Technology | Primary purpose | Typical output |
|---|---|---|
| Automatic speech recognition | Determine what was spoken | Transcript or linguistic tokens |
| Speaker recognition | Identify or verify who is speaking | Speaker identity or match score |
| Text-to-speech | Generate spoken audio from text | Synthesized speech |
| Natural language processing | Process or generate language | Labels, extracted information, summaries, responses |
| Conversational AI | Support an interactive exchange | Actions or generated responses |
How modern speech recognition works
System designs vary, and modern end-to-end models can combine stages that older pipelines treated separately. A simplified teaching model includes:
- Capture and digitize audio. A microphone or recording supplies the speech signal.
- Preprocess the signal. The system may normalize volume, reduce noise, divide audio into segments, or synchronize channels.
- Create representations. The audio becomes engineered features, a spectrogram, or learned representations.
- Estimate linguistic output. An acoustic or end-to-end model maps the representations toward speech units, tokens, or text.
- Decode and format. The system selects a likely sequence and may apply punctuation, capitalization, timestamps, or language-model context.
- Evaluate the transcript. Test outputs are compared with reference transcripts under defined conditions.
From classical ASR to deep learning
Early systems used pattern- or template-matching approaches. Statistical speech-recognition systems later combined acoustic models, pronunciation lexicons, hidden Markov models, and separate language models. Deep learning allowed neural networks to learn richer audio representations and sequence mappings, while end-to-end models learned to map speech more directly to text or tokens. Transformer and transformer-derived architectures later improved long-range context modeling and large-scale pretraining, including self-supervised learning from audio. These approaches overlap rather than forming a strict replacement sequence. Jinyu Li’s review of end-to-end ASR explains the modern transition and practical trade-offs.
Historical milestones
Bell Labs’ AUDREY system in the 1950s could recognize spoken digits. IBM’s experimental Shoebox system, developed in 1961 and demonstrated publicly around the 1962 World’s Fair, recognized a small vocabulary of numbers and command words. Later decades moved toward statistical approaches, including hidden Markov model–based systems, before deep neural networks and end-to-end architectures changed the field again.
IBM’s history of speech recognition provides institutional context for Shoebox and the transition toward statistical methods.
For the underlying model families, read What Is Deep Learning?, Transformers in NLP, How Deep Learning Works, and Neural Networks Explained.
Foundation models for speech
Large pretrained speech models can learn from broad audio datasets and later be adapted to transcription, translation, language identification, or related tasks. Pretraining may improve transfer to new domains and languages, but it does not guarantee uniform performance. With self-supervised speech pretraining, a model learns useful patterns from audio without requiring a transcript for every pretraining example, then can be fine-tuned using transcribed speech.
Results still depend on the represented languages, accents, vocabulary, recording devices, background noise, domain data, and evaluation conditions.
What is word error rate?
Word error rate (WER) compares a predicted transcript with a reference transcript. It counts three types of error:
- Substitution: one reference word is replaced with another.
- Deletion: a reference word is missing from the prediction.
- Insertion: the prediction contains an extra word.
WER = (substitutions + deletions + insertions) / number of words in the reference transcript. Multiply the result by 100 to express it as a percentage. The denominator is the reference word count, not the predicted transcript’s length.
Lower WER usually means the transcript is closer to the reference. However, WER does not describe every practical consequence. A mistaken name, medication, number, or command can matter more than several low-impact errors.
Character error rate (CER) applies the same idea to characters and can be useful depending on the language, writing system, and word-segmentation conventions. Deployment checks may also track task-specific outcomes, such as correct names, numbers, or commands, alongside transcription errors. Choose measures that reflect the use case; see Model Evaluation Metrics Explained.
Google Cloud’s speech-accuracy guidance explains the standard WER components. NIST’s OpenASR evaluation work demonstrates why results must be interpreted by language, dataset, and evaluation conditions rather than treated as a universal accuracy score.
Real-world applications
- Meeting and call transcription
- Live and recorded captioning
- Dictation and documentation support
- Voice interfaces and hands-free controls
- Search across audio and video
- Contact-center analytics
- Translation and multilingual processing pipelines
The appropriateness of ASR depends on the task, expected error types, privacy requirements, and whether users have an alternative when recognition fails.
Speech recognition and NLP
ASR converts speech into text or linguistic representations. Natural language processing can then classify, summarize, search, translate, or generate responses from that content.
The fields increasingly overlap in multimodal and speech-language models, but they remain conceptually distinct. Continue with How NLP Works.
Common challenges
Noise and recording quality
Background noise, reverberation, microphone differences, channel quality, and compression can reduce performance.
Accents, dialects, and languages
Performance can vary across accents, dialects, languages, and speaker groups when training and evaluation data are not sufficiently representative.
Domain vocabulary
Names, technical terminology, abbreviations, and specialized language can create consequential transcription errors.
Multiple speakers
Overlapping speech makes transcription harder. Speaker diarization—estimating who spoke when—is an additional task and is not the same as ASR.
Real-time latency
Interactive applications may need to balance accuracy, available context, computational cost, and response speed.
Privacy and security
Voice recordings and transcripts can contain sensitive personal, business, or health information. Some systems can also derive biometric characteristics from a voice recording for speaker identification or verification. Organizations should understand whether audio is retained, where processing occurs, who can access the data, whether it is used for model training, how consent laws apply, and how recordings and transcripts are protected and deleted. For related privacy, fairness, and accessibility considerations, see AI Ethics.
Accessibility
Speech recognition can improve accessibility through live captions, transcription, and voice control. Those benefits depend on actual performance for the people using the system. Uneven recognition can create new barriers if alternative interaction methods are unavailable.
Frequently asked questions
Is speech recognition the same as voice recognition?
The terms are sometimes used interchangeably, but speech recognition usually means recognizing what was said. Speaker or voice recognition can refer to identifying or verifying who said it.
Is speech recognition part of NLP?
The fields overlap. ASR focuses on converting speech into linguistic output, while NLP focuses more broadly on processing and generating language.
What is WER?
Word error rate measures substitutions, deletions, and insertions in a predicted transcript relative to a reference transcript.
Are modern ASR systems perfect?
No. Accuracy varies by language, accent, background noise, domain vocabulary, recording conditions, and system design.
Where to learn next
Continue with How NLP Works to understand what can happen after speech becomes text, or return to Machine Perception for the broader sensory-data context.