Why is an independent evaluation dataset used to test a classifier?

An independent evaluation dataset—often called a testing set—is essential for assessing a classifier because it provides an objective measure of how well a model generalizes to new, unseen data.1

When a model is trained, it learns patterns from a training set. If the model is evaluated using the same data it was trained on, the performance estimates become overinflated because the model has already "seen" those specific examples. Using an independent dataset prevents this bias and helps identify two common issues:1

• Overfitting: Occurs when a model is too closely adapted to the training data, capturing noise rather than true patterns, which leads to poor performance on new data.1
• Underfitting: Occurs when a model is too simple to capture the underlying relationships in the data, causing it to miss important trends.1

By testing on data that the model did not use during the training phase, researchers can accurately estimate the model's predictive capacity and ensure it will perform reliably in real-world applications.1