ML Interview Cheatsheet

Deepak Mishra

9 min read

·

Aug 18, 2026

--

Listen

Share

Part 2 — End 2 End Machine Learning Project →

Press enter or click to view image in full size

1. What is Machine Learning?

The most important definition:

Machine Learning is programming computers so they can learn from data.

The engineering definition :

A system learns from experience E with respect to task T and performance measure P if its performance on T, measured by P, improves with E.

Interview example: Spam detection

Concept:- Spam Filter

Task T :- Classify email as spam/ham

Experience E:- Historical labeled emails

Performance P:- Accuracy / other classification metric

Training set:- Examples used for learning

Training instance:- Individual email

Interview answer

“Machine learning is a paradigm where a system learns a mapping from data rather than relying entirely on manually programmed rules. Formally, a system learns from experience E for task T if its performance P on T improves with E.”

2. Why do we use Machine Learning?

The gives four major reasons.

1. Replace large numbers of hand-written rules

Traditional spam filtering:

if email contains "free":
    spam
if email contains "credit card":
    spam
...

ML instead learns patterns automatically.

2. Solve problems where no explicit algorithm exists

Example:

Speech recognition

It is extremely difficult to manually encode rules for thousands of words, speakers, accents and noisy environments.

ML learns from examples.

3. Adapt to changing environments

Suppose spammers change:

"4U" → "For U"

A traditional system requires a new rule.

An ML system can learn the new pattern from updated data.

4. Discover hidden patterns

ML can analyze huge datasets and reveal patterns humans did not explicitly know.

This is referred to as data mining.

Interview answer

“I would use ML when the problem has complex or changing patterns, when rule-based systems become difficult to maintain, or when we want to discover patterns from large datasets.”

3. Three major ways to classify ML systems

This is extremely important for interviews.

This classifies ML systems along three dimensions:

Machine Learning
                       |
        +--------------+--------------+
        |              |              |
   Supervision      Incremental    Generalization
        |              |              |
  Supervised       Batch/Online   Instance/Model
  Unsupervised
  Semi-supervised
  Reinforcement

4. Supervised Learning

Training data contains:

Input X → Label y

The model learns:

f(X) → y

Examples:

• Spam classification
• House-price prediction
• Fraud classification
• Image classification

Two major supervised problems:

Classification

Predict a discrete class.

Email → Spam / Ham
Image → Cat / Dog
Transaction → Fraud / Not Fraud

Regression

Predict a continuous numeric value.

House features → $750,000
Car features → $25,000

This specifically identifies classification and regression as the two common supervised tasks.

Algorithms mentioned

• k-NN
• Linear Regression
• Logistic Regression
• SVM
• Decision Trees
• Random Forests
• Neural Networks

5. Unsupervised Learning

There are no labels.

X1
X2
X3
...
Xn
 ↓
ML algorithm
 ↓
Discover structure

The algorithm tries to find hidden structure in the data.

Major tasks

TaskExample:- Clustering

Customer segmentation

Visualizationt:- SNE

Dimensionality reduction:- PCA

Anomaly detection:- Fraud detection

Association rules:- Market-basket analysis

Interview question

Q: Customer segmentation — supervised or unsupervised?

Answer:

If I don’t know the customer groups beforehand, I use clustering → unsupervised learning. If I already have predefined customer classes and labeled examples, I use classification → supervised learning.

Exercise explicitly makes this distinction.

6. Semi-Supervised Learning

You have:

Small labeled dataset
        +
Large unlabeled dataset
        ↓
Semi-supervised learning

Example:

You have thousands of photos but only a few photos labeled with people’s names.

The system can use the unlabeled data plus limited labels.

Interview answer

“Semi-supervised learning is useful when obtaining labels is expensive but unlabeled data is abundant.”

7. Reinforcement Learning

This is different from supervised learning.

There is an:

Agent
  ↓ action
Environment
  ↓
Reward / Penalty
  ↓
Agent learns policy

The objective is to maximize cumulative reward over time.

Important terms:

• Agent
• Environment
• Action
• Reward
• Policy
• State

Examples:

• Robot learning to walk
• AlphaGo

Interview answer

“In reinforcement learning, the agent interacts with an environment, takes actions and receives rewards or penalties, and learns a policy that maximizes long-term reward.”

8. Batch vs Online Learning

This is another very common interview question.

Batch Learning

Train using the complete available dataset.

Dataset
   ↓
Train
   ↓
Model
   ↓
Production

If new data arrives, generally retrain using the full dataset.

Advantages:

• Simple
• Stable
• Good when data changes slowly

Disadvantages:

• Expensive retraining
• Slow adaptation
• Difficult with huge datasets

Online Learning

Learn incrementally.

data → update model
data → update model
data → update model
...

Training can happen one instance at a time or using mini-batches.

Advantages:

• Adapts quickly
• Suitable for streaming data
• Can process huge datasets
• Lower incremental computation

Interview example

For a real-time fraud detection system:

“I might consider online learning if the fraud distribution changes rapidly and I need the model to adapt continuously.”

9. What is Out-of-Core Learning?

Important for large datasets.

Suppose:

Dataset = 10 TB
RAM     = 64 GB

You cannot load everything into memory.

So:

10 TB dataset
      ↓
mini-batch
      ↓
update model
      ↓
next mini-batch
      ↓
update model
      ↓
...

This describes out-of-core algorithms as processing data in mini-batches, using online-learning techniques.

10. Instance-Based vs Model-Based Learning

This distinction is particularly important.

Instance-Based

The system essentially memorizes examples.

For a new input:

New instance
     ↓
Similarity calculation
     ↓
Find similar training instances
     ↓
Prediction

Example:

k-Nearest Neighbors

Model-Based

The algorithm learns a mathematical model from the training data.

Training data
     ↓
Learning algorithm
     ↓
Model parameters
     ↓
Prediction function

Example:

Linear Regression

The model learns parameters such as:

y = θ₀ + θ₁x

and uses those parameters to predict unseen examples.

11. Model Parameters vs Hyperparameters

Very common interview question.

Parameter

Learned from training data.

Example:

Linear Regression:

y = θ₀ + θ₁xθ₀ and θ₁ → parameters

Hyperparameter

Set before/during training, but not learned as model parameters.

Examples:

• Learning rate
• Regularization strength
• Number of trees
• Tree depth
• Number of neighbors k

The explicitly defines a hyper parameter as a parameter of the learning algorithm, rather than the model itself.

One-line answer

Parameters are learned by the model; hyperparameters control how the learning algorithm learns.

12. Main ML Challenges

Remember:

D-N-P-I-O-U

You can use this mnemonic:

Data quantity

Nonrepresentative data

Poor-quality data

Irrelevant features

Overfitting

Underfitting

summary emphasizes exactly these issues.

12.1 Too Little Data

More complex models generally require more data.

Too little data
      ↓
Poor generalization

12.2 Nonrepresentative Data

Your training data doesn’t represent the real population.

Example:

Production users:
50% mobile
50% desktop

Training:
95% desktop
5% mobile

Model may perform poorly in production.

12.3 Poor-Quality Data

Examples:

• Missing values
• Incorrect labels
• Noise
• Outliers
• Corrupted records

Classic principle:

Garbage in → garbage out.

This explicitly highlights noisy and poor-quality data as major causes of poor performance.

13. Irrelevant Features

Suppose you’re predicting house prices.

Useful:

location
area
bedrooms
age

Potentially irrelevant/noisy:

random_customer_id

Feature engineering and feature selection can improve the model.

14. Overfitting

Definition

The model performs extremely well on training data but poorly on unseen data.

Training error     ↓
Test error         ↑
        =
   OVERFITTING

This describes overfitting as occurring when the model is too complex relative to the amount and noisiness of training data.

Causes

• Too complex model
• Too many features
• Too little training data
• Noisy training data

Solutions

1. More training data
2. Simpler model
3. Fewer features
4. Regularization
5. Remove noise/outliers

15. Underfitting

Opposite of overfitting.

Model too simple
       ↓
Cannot capture underlying pattern
       ↓
Poor training performance
       +
Poor test performance

Solutions:

• More powerful model
• Better features
• Reduce regularization

16. Regularization

One of the most important concepts.

Regularization constrains model complexity to reduce overfitting.

Complex model
      ↓
Regularization
      ↓
Simpler model
      ↓
Better generalization

This also points out an important tradeoff:

Too much regularization → underfitting.

The amount of regularization is controlled by a hyperparameter.

17. Generalization

This is the real objective of ML.

Not:

“How well does my model perform on training data?”

But:

“How well does it perform on unseen data?”

This repeatedly emphasizes generalization as the true goal of ML.

18. Training Set vs Test Set

Basic workflow:

Dataset
                    |
             +------+------+
             |             |
         Training        Test
             |             |
          Train          Evaluate
             |
           Model

The test set estimates generalization error.

A common split described:

80% → Training
20% → Test

19. Validation Set

Suppose you try:

Model A
Model B
Model C

and different hyperparameters.

You shouldn’t repeatedly optimize against the test set because you will effectively overfit to the test set.

Instead:

Training set
     ↓
Train multiple models
     ↓
Validation set
     ↓
Select model + hyperparameters
     ↓
Final test set
     ↓
Estimate generalization

Key interview distinction

Dataset Purpose

Training:- Learn model parameters

Validation:- Select model / tune hyperparameters

Test :- Final unbiased-ish estimate of generalization

20. Cross-Validation

Instead of maintaining a separate validation set, divide training data into folds.

Example 5-fold CV:

Fold 1 → Validation
Fold 2-5 → Training

Fold 2 → Validation
Fold 1,3-5 → Training...Fold 5 → Validation
Fold 1-4 → Training

Then average the validation performance.

Benefits:

• Better use of limited data
• Model selection
• Hyperparameter tuning

This explicitly presents cross-validation as a way to compare models and tune hyper parameters without requiring a separate validation set.

21. No Free Lunch Theorem

Interview favorite.

There is no universally best ML algorithm.

Dataset A → Linear model may win

Dataset B → Random Forest may winDataset C → Neural Network may win

Without assumptions about the data, there is no reason to prefer one algorithm over another.

Therefore:

Choose algorithms based on the problem/data and validate empirically.

22. The Most Important Interview Diagram

You should be able to reproduce this mentally:

MACHINE LEARNING
                                |
             +------------------+------------------+
             |                  |                  |
        Supervision        Incremental        Generalization
             |                  |                  |
       +-----+-----+        +----+----+        +----+----+
       |     |     |        |         |        |         |
      Sup   Semi   Un     Batch     Online   Instance   Model
             |             |         |
             RL        Offline    Streaming

Then:

TRAINING DATA
                  |
                  v
          Learning Algorithm
                  |
                  v
                Model
                  |
          +-------+-------+
          |               |
      Validation        Test
          |               |
  Tune/select model    Final evaluation
          |
          v
     Generalization

23. Top 15 Interview Questions

You should be able to answer these without hesitation:

Q1. What is machine learning?

Learning from data to improve performance on a task.

Q2. When should you use ML instead of rules?

Complex problems, changing environments, huge rule sets, or unknown solution patterns.

Q3. Supervised vs unsupervised?

Labels vs no labels.

Q4. Classification vs regression?

Discrete output vs continuous numeric output.

Q5. What is semi-supervised learning?

Small labeled dataset + large unlabeled dataset.

Q6. What is reinforcement learning?

Agent learns actions/policy through rewards and penalties.

Q7. Batch vs online learning?

Train all data at once vs incremental learning.

Q8. What is out-of-core learning?

Processing datasets too large to fit in memory using chunks/mini-batches.

Q9. Instance-based vs model-based?

Similarity to stored examples vs learned predictive model.

Q10. Parameter vs hyperparameter?

Learned from data vs configured for the learning algorithm.

Q11. What is overfitting?

Excellent training performance but poor generalization.

Q12. How do you reduce overfitting?

More data, simpler model, fewer features, regularization, less noise.

Q13. What is underfitting?

Model is too simple to capture the underlying pattern.

Q14. Why do we need a validation set?

To select models and tune hyperparameters without contaminating the final test evaluation.

Q15. Why shouldn’t we tune hyperparameters on the test set?

Because we can overfit to the test set and obtain an overly optimistic estimate of generalization.

24. ML Engineer — How I Would Answer

For a ML/AI interview, don’t just give textbook definitions. Connect them to production.

For example:

“I think about an ML system along three dimensions: supervision, learning cadence, and generalization strategy. First, is the problem supervised, unsupervised, semi-supervised, or reinforcement learning? Second, do we train offline in batches or incrementally online? Third, are we generalizing through similarity to stored instances or through a learned model? Once those decisions are made, the biggest production risks are data quality and representativeness, leakage, overfitting/underfitting, and incorrect evaluation. I would therefore separate training, validation and final test data, use cross-validation where appropriate, and optimize for generalization rather than training performance.”

That is a much stronger answer than simply defining supervised learning.

🔥 One-page memory map

ML
│
├── Why?
│   ├── Replace rules
│   ├── Complex problems
│   ├── Changing environments
│   └── Discover patterns
│
├── Supervision
│   ├── Supervised
│   │   ├── Classification
│   │   └── Regression
│   ├── Unsupervised
│   │   ├── Clustering
│   │   ├── Dimensionality Reduction
│   │   ├── Anomaly Detection
│   │   └── Association Rules
│   ├── Semi-supervised
│   └── Reinforcement Learning
│
├── Learning Mode
│   ├── Batch
│   └── Online
│
├── Generalization
│   ├── Instance-based
│   └── Model-based
│
├── Challenges
│   ├── Too little data
│   ├── Nonrepresentative data
│   ├── Poor-quality data
│   ├── Irrelevant features
│   ├── Overfitting
│   └── Underfitting
│
└── Evaluation
    ├── Training set
    ├── Validation set
    ├── Cross-validation
    └── Test set

This is essentially the conceptual map wants you to internalize; its own end-of-exercises test these exact distinctions.

For your interview preparation

Given that you’re preparing for ML/AI-level interviews, I would study in this order:

ML definition → supervised/unsupervised → batch/online → instance/model → overfitting/underfitting → regularization → train/validation/test → cross-validation → No Free Lunch.

Those are the concepts most likely to become follow-up questions in system design and ML fundamentals rounds.

Continue the Series:

Next → Part 2 — End 2 End Machine Learning Project