Quick answer
Both preserve class proportions, but they partition differently. StratifiedKFold splits the data into k non-overlapping folds and uses each fold as the test set exactly once, so every sample is tested exactly once. StratifiedShuffleSplit generates independent random train/test splits, so samples may repeat across splits or never appear at all. Use StratifiedKFold for cross-validation, StratifiedShuffleSplit when you want a specific test size or many random repeats.
StratifiedKFold and StratifiedShuffleSplit are two types of stratified sampling techniques that can be used to split a dataset into train and test sets in scikit-learn.
The main difference between StratifiedKFold and StratifiedShuffleSplit is in the way the data is split. StratifiedKFold performs k-fold cross-validation, where the data is split into k folds, and the model is trained and evaluated k times, each time using a different fold as the test set and the remaining folds as the training set.
On the other hand, StratifiedShuffleSplit randomly shuffles the data and splits it into train and test sets. The number of splits is specified by the user, and the data is shuffled and split into train and test sets multiple times, according to the number of splits specified.
Both StratifiedKFold and StratifiedShuffleSplit aim to maintain the class balance in the train and test sets, meaning that the proportion of samples from each class is approximately the same in the train and test sets as it is in the original dataset. This is particularly important when the classes in the dataset are imbalanced (i.e., there is a disproportionate number of samples from one class compared to the other).
StratifiedKFold and StratifiedShuffleSplit are two types of stratified sampling techniques that can be used to split a dataset into train and test sets in scikit-learn.
Here is an example of how to use StratifiedKFold for k-fold cross-validation:
from sklearn.model_selection import StratifiedKFold
# Load data and labels into X and y
X = ...
y = ...
# Set the number of folds for cross-validation
n_folds = 5
# Create the StratifiedKFold object
skf = StratifiedKFold(n_splits=n_folds, shuffle=True)
# Iterate through the folds
for train_index, test_index in skf.split(X, y):
# Split the data into train and test sets
X_train, X_test = X[train_index], X[test_index]
y_train, y_test = y[train_index], y[test_index]
# Train and evaluate the model on the train and test sets
model.fit(X_train, y_train)
y_pred = model.predict(X_test)
accuracy = accuracy_score(y_test, y_pred)
print(f"Accuracy: {accuracy:.2f}")
In this example, the data is split into 5 folds, and the model is trained and evaluated 5 times, each time using a different fold as the test set and the remaining folds as the training set.
Here is an example of how to use StratifiedShuffleSplit to split the data into train and test sets multiple times:
from sklearn.model_selection import StratifiedShuffleSplit
# Load data and labels into X and y
X = ...
y = ...
# Set the number of splits
n_splits = 10
# Create the StratifiedShuffleSplit object
sss = StratifiedShuffleSplit(n_splits=n_splits, test_size=0.2, random_state=42)
# Iterate through the splits
for train_index, test_index in sss.split(X, y):
# Split the data into train and test sets
X_train, X_test = X[train_index], X[test_index]
y_train, y_test = y[train_index], y[test_index]
# Train and evaluate the model on the train and test sets
model.fit(X_train, y_train)
y_pred = model.predict(X_test)
accuracy = accuracy_score(y_test, y_pred)
print(f"Accuracy: {accuracy:.2f}")In this example, the data is shuffled and split into train and test sets 10 times, according to the number of splits specified.
Both StratifiedKFold and StratifiedShuffleSplit aim to maintain the class balance in the train and test sets, meaning that the proportion of samples from each class is approximately the same in the train and test sets as it is in the original dataset. This is particularly important when the classes in the dataset are imbalanced.
In summary, StratifiedKFold is used for k-fold cross-validation, and StratifiedShuffleSplit is used to split the data into train and test sets multiple times, with a random shuffle of the data in each split. Both techniques are used to maintain class balance in the train and test sets.
Key takeaways
- •StratifiedKFold guarantees every sample is in the test set exactly once; StratifiedShuffleSplit gives no such guarantee.
- •StratifiedShuffleSplit's splits are independent and may overlap, so a sample can appear in several test sets or none.
- •Use StratifiedKFold for standard k-fold cross-validation and model comparison — it uses all the data with no wasted samples.
- •Use StratifiedShuffleSplit when you need an exact test proportion (for example test_size=0.2) independent of the number of splits.
- •With StratifiedKFold the test size is fixed at roughly 1/k; with StratifiedShuffleSplit you set n_splits and test_size independently.
- •Both require enough members in each class — stratification fails if any class has fewer samples than the number of folds.
Frequently asked questions
What is the main difference between StratifiedKFold and StratifiedShuffleSplit?
StratifiedKFold partitions the dataset into k mutually exclusive folds, so the test sets never overlap and every sample is tested exactly once. StratifiedShuffleSplit creates each split by independently shuffling and sampling, so test sets across splits can overlap and coverage is not guaranteed.
Which should I use for cross-validation?
StratifiedKFold. Because the folds are non-overlapping and cover the whole dataset, the k scores can be averaged into an unbiased estimate of model performance. StratifiedShuffleSplit is better suited to repeated random validation or when you need a specific train/test ratio.
Do both keep the class distribution balanced?
Yes. Both are stratified, so the proportion of each class in every train and test split approximately matches the original dataset. This matters most with imbalanced classes, where an unstratified split can produce folds missing a minority class entirely.
Can I control the test set size with StratifiedKFold?
Only indirectly — the test size is approximately 1/k of the dataset, so n_splits=5 gives a 20% test set. If you need an exact test proportion decoupled from the number of splits, use StratifiedShuffleSplit with its test_size parameter.
Why do I get 'The least populated class has too few members' error?
Stratification needs at least one sample per class per fold, so the smallest class must have at least n_splits members. Either reduce n_splits, merge or drop very rare classes, or gather more samples for the minority class.
Software Engineering Leader & Technical Author · Updated July 21, 2026