Named Entity Recognition in Simple Terms
Named Entity Recognition (NER) identifies entity mentions as exact spans of text and assigns labels from a chosen set of categories. A span is a continuous stretch of text, such as “Microsoft” or “New York.” Labels might include organization, person, location, and date.
Think of NER as a digital highlighter with written labels. It marks the relevant words and records their type. The result can help an application search documents or fill structured fields, but extracting a name does not verify a claim about that name.
NER is a task within Natural Language Processing (NLP). It can be implemented with rules or learned models. It is not, by itself, a type of machine learning or a complete account of how a system processes language.
Entity Spans: Where a Mention Starts and Ends
A useful prediction needs both the right boundary and the right label. In “Maya visited New York,” the place mention is the full span “New York.” Returning only “York” misses part of the name. Including “visited” includes text that does not belong.
These boundaries are part of the task, not a formatting detail. A downstream search or database lookup may fail when a name is clipped. The spaCy EntityRecognizer documentation describes this as identifying labeled spans.
Diagram 1: Exact text + a label
Input: Maya visited New York.
- Span: “Maya”Label: PERSONOne complete name
- Span: “New York”Label: LOCATIONOne entity spanning two words
Keep the original text alongside predictions. Applications often store character positions as well as the extracted text and label, so each mention can be traced back to its source.
A Worked NER Example
Consider this illustrative input, used to demonstrate labeling rather than report an event: “Microsoft acquired LinkedIn in California in 2016.” With an organization/location/date schema, the expected mentions are:
- “Microsoft” → ORGANIZATION
- “LinkedIn” → ORGANIZATION
- “California” → LOCATION
- “2016” → DATE
The system has located four spans and assigned four labels. That output does not yet say which company acquired the other, resolve the companies to database records, or verify where a transaction happened. Those are separate jobs.
It also does not label every noun. In “Hotels near Disney World,” a suitable schema might label “Disney World” as a facility or location. The generic word “Hotels” is not a named entity in this example.
How an NER System Works
Using an existing model is called inference. A practical workflow is:
- Choose a model or rule set whose language and labels fit the task.
- Supply text using the preprocessing the system expects.
- Predict entity spans and types using rules, context, or both.
- Check the output against the original text and permitted labels, then pass it to the application.
These are conceptual steps, not a requirement that detection and classification happen in separate model stages. Some systems predict both together.
Many learned systems use tokenization to split or encode text. Neural models then use numerical representations, introduced in Word Embeddings Explained, to process the input. A token boundary is not necessarily an entity boundary: one name can cover several words and many subword tokens.
Training or fine-tuning is different. It uses labeled examples to adjust a model. Keep separate development and test examples for choosing settings and measuring performance. For the broader pipeline, see How NLP Works.
Entity Labels Depend on the Schema
A schema is the set of labels and annotation rules for a task. There is no single universal list. A news model may recognize people and organizations, while a specialist model may extract drug names or legal references.
| Category | Example mention |
|---|---|
| Person | Maya Patel |
| Organization | Microsoft |
| Location | New York |
| Product | iPhone |
| Date or time | March 15, 2025 |
| Money or quantity | $50; 20 kilograms |
| Domain-specific | ibuprofen; Section 5 |
Dates, monetary amounts, and measurements show why the task can include more than literally named objects. A model only returns the types its schema supports; adding a new label requires more than changing the display text.
Document boundary rules too. Does a person span include “Dr.”? Are locations inside organization names labeled separately? Some tools support only non-overlapping entities; others support nested or overlapping spans. Compare systems only after aligning these choices.
Main Approaches to NER
- Rules and dictionaries: match known names or patterns. They are useful for controlled vocabularies, but need maintenance and can miss unfamiliar names. The spaCy EntityRuler guide shows how rules can complement learned models.
- Statistical sequence labeling: methods such as conditional random fields learn patterns across a sequence of labels, often using hand-designed features.
- Neural methods: learn text representations rather than relying entirely on manually selected features. Recurrent models are one example.
- Transformer-based methods: use contextual representations for token classification or span prediction. This is a family of neural approaches, rather than a separate task.
- Generative or LLM extraction: produces entity lists or structured output from instructions. Check that each generated mention occurs in the input and every type is allowed.
No approach is automatically best. Compare accuracy on your schema, language, and document type, alongside speed, cost, and maintenance needs. A domain-specific model can help, but the improvement must be measured. For background on contextual models, see Transformers in NLP.
Where NER Can Help
NER is useful when applications need repeatable fields from prose. Examples include:
- Document search: filter articles by mentioned companies, people, or places.
- Customer support: extract a product name from a request before routing it.
- Research and document review: find organization names, dates, or specialist terms for a person to check.
- Knowledge organization: supply candidate mentions to later linking and relationship-extraction stages.
These are possible uses, not claims that every search engine or chatbot includes a separate NER module. Extracting a company name alone does not detect fraud, establish compliance, or make a clinical decision.
For sensitive documents, choose appropriate data access and retention controls and assess errors with domain experts. A useful starting point is one narrow field in one document type, with human review before acting on the result.
Common Challenges and Mistakes
Ambiguity: “Jordan” might refer to a person or a country. Surrounding words help select the type, but a short fragment may not provide enough evidence. Recognizing PERSON still does not identify which person named Jordan is meant.
New names and domain shift: a model tested on news may perform differently on customer messages, scientific papers, or scanned forms. Misspellings, OCR errors, abbreviations, and changing product names all affect extraction.
Language and writing conventions: multilingual text, scripts, capitalization, and word boundaries differ. Performance on one language does not establish performance on another.
Inconsistent annotations: people may disagree about boundaries or labels. Resolve those rules before treating disagreements as model failures. Check performance across the names, languages, and document sources relevant to the application; a single overall score can hide uneven results.
Keep punctuation and source text unless the chosen system specifies otherwise. Aggressive cleaning can erase useful cues or break the positions that connect predictions to the original document.
NER, Entity Linking, and Relation Extraction
These tasks answer different questions. NER asks which spans have which labels. Entity linking resolves a mention to a particular entry in a knowledge base, as described in spaCy’s entity-linking documentation. Relation extraction identifies a relationship between entities; the Stanford CoreNLP relation extractor is one implementation.
Diagram 2: Three different questions
Illustrative input: Maya works for Acme.
- NER: Which spans?“Maya” → PERSON“Acme” → ORGANIZATION
- Entity linking: Which identity?Match “Acme” to the correct company record, if sufficient evidence is available.
- Relation extraction: Which connection?Maya → works for → Acme
Event extraction goes further by identifying an event and its participants or details, such as an acquisition, buyer, target, and date. These tasks may be combined in an application, but their outputs need separate checks.
Text classification instead assigns categories to a larger unit such as a message. Sentiment analysis estimates expressed opinion or attitude, often using positive, negative, or neutral labels. Neither task is a substitute for finding exact entity spans.
How to Evaluate NER
Evaluate predictions against human-checked mentions in a held-out, representative test set. Under strict entity-level evaluation, a prediction is correct only when both its exact span and its type match the reference. The CoNLL-2003 NER shared-task paper uses complete entity matches when calculating precision, recall, and F1.
Diagram 3: Boundary and type both matter
Reference mention: “New York” → LOCATION
- Correct match“New York” → LOCATIONExact span and correct type
- Boundary error“York” → LOCATIONPart of the name is missing
- Type error“New York” → PERSONSpan is right; label is wrong
| Metric | What it measures |
|---|---|
| Precision | Correct predicted entities ÷ all predicted entities |
| Recall | Correct predicted entities ÷ all reference entities |
| F1 | The harmonic mean of precision and recall |
For example, if a system predicts 10 entities, 8 match exactly, and the reference contains 12, precision is 80%, recall is about 67%, and F1 is about 73%. A boundary or type error in place of an expected entity adds a false positive and a false negative.
Inspect per-label scores and actual errors. Token accuracy alone can look high when most tokens are not entities. Record the evaluation rules, test data, and sample size so the score has context; a confidence score on one prediction is not an accuracy guarantee.
Quick Questions About NER
What do BIO labels mean?
BIO is one way to encode spans over tokens: B begins an entity, I continues it, and O is outside. With word-level tokens, “New York” can receive B-LOCATION and I-LOCATION. The Hugging Face token-classification guide explains this labeling pattern and the extra alignment needed when a tokenizer splits words into subwords.
Can an LLM extract entities?
Yes. An LLM can be prompted to return spans and labels. That capability does not establish that a particular product, including ChatGPT, uses a separate traditional NER component internally. Test the output rather than infer the architecture from a successful example.
Can NER find multiple mentions of the same name?
Yes. Each occurrence has its own position in the text. Recognizing repeated mentions does not automatically determine whether they refer to the same real-world entity; that may require linking or coreference resolution.
Key Takeaways and Your Next Lesson
NER turns selected text spans into labeled mentions. Start with clear categories and boundary rules, choose an approach that fits the documents, and evaluate whole entities. Keep extraction separate from identity resolution, relationships, and factual verification.
Continue to Text Classification Explained to see how assigning a label to a whole message differs from labeling spans inside it.