Train-Test vs Train-Validation-Test: Choosing the Right Dataset Split

Jaspinder kaur Walia

3 min read

·

Feb 13, 2026

--

Listen

Share

How data is split decides how well a model will perform in the real world.

Press enter or click to view image in full size

In this blog, we will learn when to use a Train-Test split and when to use a Train-Validation-Test split.

Train-Test Split:

In a Train- Test split, generally we divide the dataset into an 80–20 ratio in which the model learns from 80% data and 20% data is used for final accuracy check.

Train-Test Split is used when:

• The dataset size is small or medium
• We only need a quick evaluation
• When we are not doing a heavy model tuning

Example: Train–Test Split (Simple Case)

Example 1: Train–Test Split (Simple Case)

from sklearn.model_selection import train_test_split
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score

# Sample text data
texts = [
    "I love this product",
    "This is the worst experience",
    "Amazing quality and service",
    "Not good, very disappointing",
    "I am very happy",
    "Terrible support"
]

labels = [1, 0, 1, 0, 1, 0]  # 1 = Positive, 0 = Negative

# Step 1: Train-Test Split
X_train, X_test, y_train, y_test = train_test_split(
    texts, labels, test_size=0.2, random_state=42
)

# Step 2: Convert text into TF-IDF features
vectorizer = TfidfVectorizer()
X_train_vec = vectorizer.fit_transform(X_train)
X_test_vec = vectorizer.transform(X_test)

# Step 3: Train model
model = LogisticRegression()
model.fit(X_train_vec, y_train)

# Step 4: Evaluate on Test set
y_pred = model.predict(X_test_vec)
print("Test Accuracy:", accuracy_score(y_test, y_pred))

Train-Validation-Test Split:

In a Train- Validation-Test split, generally we divide the dataset into a 70–15–15 ratio in which the model learns from 70% data,15% data is used for model tuning and to check the performance during training and the remaining 15% is used for final evaluation.

Train-Validation-Test split is used when:

• We are training a proper ML model
• We want to tune hyperparameters(like max_depth, C, learning rate)
• We want to avoid overfitting on the test set

Example : Train–Validation–Test Split (Proper ML Case)

Example 2: Train–Validation–Test Split (Proper ML Case)

from sklearn.model_selection import train_test_split
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score

texts = [
    "I love this product",
    "This is the worst experience",
    "Amazing quality and service",
    "Not good, very disappointing",
    "I am very happy",
    "Terrible support",
    "Excellent work",
    "Bad quality",
    "Loved it",
    "Hated it"
]

labels = [1, 0, 1, 0, 1, 0, 1, 0, 1, 0]

# Step 1: Split into Train+Temp and Test
X_train, X_temp, y_train, y_temp = train_test_split(
    texts, labels, test_size=0.3, random_state=42
)

# Step 2: Split Temp into Validation and Test
X_val, X_test, y_val, y_test = train_test_split(
    X_temp, y_temp, test_size=0.5, random_state=42
)

print("Train size:", len(X_train))
print("Validation size:", len(X_val))
print("Test size:", len(X_test))

# Step 3: TF-IDF Conversion
vectorizer = TfidfVectorizer()
X_train_vec = vectorizer.fit_transform(X_train)
X_val_vec = vectorizer.transform(X_val)
X_test_vec = vectorizer.transform(X_test)

# Step 4: Train model
model = LogisticRegression()
model.fit(X_train_vec, y_train)

# Step 5: Validation Accuracy (for tuning)
val_pred = model.predict(X_val_vec)
print("Validation Accuracy:", accuracy_score(y_val, val_pred))

# Step 6: Final Test Accuracy (only once)
test_pred = model.predict(X_test_vec)
print("Final Test Accuracy:", accuracy_score(y_test, test_pred))