cross validation

Cross-validation is a technique used in machine learning to assess how well a predictive model will perform on new, unseen data. It's a method of resampling data to test a model's generalizability.1

The core idea is to avoid "overfitting," where a model learns the training data too well and fails to predict future outcomes accurately. Cross-validation involves splitting the data into training and testing sets. The training set is further divided into an analysis set (to train models) and an assessment set (to test them).1

Common types include k-fold cross-validation, where data is split into k subsets, and each subset is used as an assessment set once. Monte Carlo cross-validation randomly assigns data to analysis and assessment sets across iterations. Leave-one-out cross-validation uses a single data point as the assessment set in each iteration.1

In psychology, cross-validation is a crucial step in establishing the validity of personality tests. It helps ensure that a test's results are stable and reliable when applied to new individuals, preventing overinflated estimates of a test's predictive power.2