A dataset is a collection of examples used to develop, evaluate, or operate a machine-learning system. Depending on the task, each example may contain features, labels, text, images, audio, sensor readings, transactions, or other information. The Google Machine Learning Glossary provides useful reference definitions for examples, features, labels, training sets, validation sets, and test sets.
Datasets are central to machine learning because models learn patterns from the information they are given. But bigger is not automatically better. A useful dataset needs to be relevant, representative, appropriately labeled, well structured for the task, and evaluated for quality problems such as missing values, leakage, imbalance, or distribution shift.
What Is a Dataset?
A dataset is an organized collection of data points, often arranged so each row or example represents one observation.
For a house-price model, one example might include square footage, location, number of bedrooms, age of the property, and the final sale price.
For an image-classification model, one example might be an image paired with a label such as “cat,” “dog,” or “car.”
For language models, datasets may contain very large collections of text, code, conversations, documents, or other sequences.
The structure depends on the machine-learning problem.
Features and Labels
Features
Features are the inputs a model uses to make predictions.
Examples include:
- Age, income, and account history.
- Pixel values in an image.
- Words or tokens in text.
- Sensor measurements.
- Product attributes.
Labels
A label is a target value associated with an example in supervised learning.
Examples include:
- Fraudulent vs legitimate.
- Customer churned vs did not churn.
- Sale price.
- Product category.
Not every dataset has labels. Unsupervised and self-supervised methods can learn from data without manually assigned task labels.
Read Supervised Learning Explained and Unsupervised Learning Explained.
Training, Validation, and Test Data
A machine-learning dataset is often divided into separate subsets.
Training set
The training set is used to fit model parameters.
Validation set
Validation data, or cross-validation folds, is used during development to compare models, tune hyperparameters, choose thresholds, and make other design decisions.
Test set
A final held-out test set is reserved for estimating how the selected system performs on unseen data after development decisions have been made.
Using the test set repeatedly during tuning weakens its value as an independent evaluation.
Learn more in Training vs Testing Data and Cross-Validation Explained.
Structured vs Unstructured Data
Structured data
Structured data follows a defined schema, such as rows and columns in a table.
Examples include:
- Financial transactions.
- Customer records.
- Inventory tables.
- Sensor measurements.
Unstructured data
Unstructured data does not naturally fit into a fixed table format.
Examples include:
- Text documents.
- Images.
- Audio.
- Video.
Modern machine-learning systems often convert unstructured data into numerical representations before modeling.
See Structured vs Unstructured Data.
What Makes a Dataset High Quality?
Dataset quality is multidimensional.
Relevance
The examples should reflect the actual task the model will perform.
Representativeness
The dataset should reasonably reflect the populations, conditions, environments, or operating scenarios where the model will be used.
Accuracy
Labels, measurements, and recorded values should be as correct as the application requires.
Coverage
Important cases should not be systematically missing.
Consistency
Definitions, units, labels, and collection procedures should be reasonably consistent.
Timeliness
Older data may become less useful if user behavior, markets, products, regulations, sensors, or other real-world conditions change.
Common Dataset Problems
Missing values
Some observations may lack fields or measurements. The right treatment depends on why data is missing and how important that field is.
Duplicate examples
Duplicates can distort training and evaluation, especially if nearly identical examples appear across training and test sets.
Label noise
Incorrect or inconsistent labels can reduce performance and make evaluation unreliable.
Class imbalance
Some categories may be much rarer than others.
For example, fraud may represent only a small fraction of all transactions. In these cases, accuracy alone can be misleading.
Read Imbalanced Datasets Explained.
Data leakage
Leakage occurs when information unavailable at real prediction time enters the training process or evaluation.
Examples include:
- Using future data to predict the past.
- Including a target-derived field as a feature.
- Splitting records from the same user across training and test sets when the real task requires generalization to new users.
Leakage can make a model appear much stronger than it really is.
Distribution shift
The data seen after deployment may differ from the data used for training.
Changes can occur in customer behavior, sensors, products, language, fraud tactics, medical populations, or other conditions.
A model can therefore degrade even if the original dataset was high quality.
Does More Data Always Improve a Model?
No.
More data can help when the additional examples are relevant, representative, accurately measured or labeled, and useful for the model.
More data may not help when it is:
- Duplicated.
- Biased.
- Outdated.
- Incorrectly labeled.
- Irrelevant to the target problem.
- Drawn from a different distribution than deployment data.
Data quality, task definition, model design, training procedure, and evaluation strategy all matter.
Data Preprocessing
Raw datasets usually require preparation before training.
Common steps include:
- Cleaning missing or invalid values.
- Encoding categorical variables.
- Scaling numerical features when appropriate.
- Removing or investigating duplicates.
- Creating features.
- Splitting data safely.
- Preventing leakage.
See Data Preprocessing Explained and Feature Engineering Explained.
Dataset Bias and Fairness
A dataset can reflect historical, sampling, labeling, or measurement biases.
Bias can enter when:
- Some groups are underrepresented.
- Outcomes reflect historical inequalities.
- Labels encode subjective judgments.
- Collection methods favor certain users or environments.
Dataset review should therefore consider not only model accuracy but also who and what the data represents. The NIST AI Risk Management Framework Playbook specifically emphasizes evaluating dataset representativeness in the intended context of use and examining data quality and sourcing.
Dataset Size and Model Complexity
More complex models often require more data to estimate reliably, but there is no universal minimum dataset size.
The amount of data needed depends on:
- Task difficulty.
- Number and quality of features.
- Noise level.
- Model complexity.
- Availability of pretrained representations.
- Desired confidence in evaluation.
Transfer learning and pretrained models can sometimes reduce the amount of task-specific labeled data required.
Frequently Asked Questions
What is a dataset in simple terms?
It is a collection of examples used to train, validate, test, or operate a machine-learning system.
What is the difference between features and labels?
Features are model inputs. Labels are target values used in supervised learning.
Why split a dataset?
Separate subsets help fit the model, make development decisions, and evaluate generalization without reusing the same examples for every purpose.
Is a bigger dataset always better?
No. Relevance, quality, representativeness, labeling, and freshness matter as much as size.
What is data leakage?
Leakage is information entering training or evaluation that would not legitimately be available when the model makes real predictions.