Understanding Linear Regression in Machine Learning with Real Examples
Imagine you want to predict the price of a house based on its size, number of bedrooms, and location. How can a machine learn the relationship between these factors and accurately estimate the price of a completely new house?
This is exactly where Linear Regression comes into the picture.
Linear Regression is one of the most fundamental and widely used algorithms in Machine Learning. Even though it is mathematically simple, it introduces some of the most important concepts in ML such as prediction, relationships between variables, model training, and optimization.
In this article, we’ll understand Linear Regression from scratch using simple real-world examples.
What You’ll Learn in This Article
By the end of this article, you’ll understand:
- What Linear Regression is
- Difference between Regression and Classification
- How Linear Regression works
- The concept of the best-fit line
- Single Variable vs Multiple Linear Regression
- What features are
- How regression works in higher dimensions
Supervised Learning
Machine Learning is broadly divided into two major categories:
- Supervised Learning
- Unsupervised Learning
In Supervised Learning, the model learns from labeled data. This means the dataset already contains both:
- Input values
- Correct output values
The goal of the model is to learn the relationship between inputs and outputs so that it can make predictions on new unseen data.
Supervised Learning is further divided into two types:
1. Regression
Regression is used when the output is a continuous numerical value.
Examples:
- Predicting house prices
- Predicting temperature
- Predicting salary
- Predicting stock prices
2. Classification
Classification is used when the output belongs to categories or classes.
Examples:
- Spam detection
- Disease prediction
- Image classification
- Fake news detection
What is Linear Regression?
Linear Regression is a supervised Machine Learning algorithm used to predict continuous numerical values by finding the relationship between input variables and output variables.
The core idea is very simple:
The algorithm tries to draw the best possible straight line through the dataset in order to make accurate predictions.
This straight line is called the:
- Best-fit line
- Regression line
Understanding Linear Regression with a Real Example
Imagine we have a dataset of house prices where:
- The X-axis represents the square foot area of a house
- The Y-axis represents the price of that house
Now suppose we want to predict the price of a new house whose square foot area is not present in the dataset.
For example:
- Existing houses may have areas like 1000 sq ft, 1200 sq ft, and 1500 sq ft
- But we want to predict the price of a new 1300 sq ft house
So how can we do this?
One simple and powerful approach is to draw an approximate straight line that best fits the data points.
Once this line is created, we can provide the square foot area of a new house as input, project it onto the line, and obtain an estimated house price as output.
This is the fundamental idea behind Linear Regression.
Equation of Linear Regression
The equation of a straight line is:
y = mx + b
Where:
- y = predicted output
- x = input feature
- m = slope of the line
- b = intercept or starting point
In the context of house price prediction:
- x could represent the square foot area
- y would represent the predicted house price
The slope m determines how strongly the house price changes when the square foot area changes.
The intercept b represents the baseline value when the input becomes zero.
What is the Best-Fit Line?
The best-fit line is the line that minimizes prediction error across all data points.
In simple terms:
Linear Regression tries to draw a line that stays as close as possible to all the points in the dataset.
- Actual Price: 50 Lakhs
Predicted Price: 48 Lakhs
Error: 2 Lakhs - Actual Price: 70 Lakhs
Predicted Price: 72 Lakhs
Error: 2 Lakhs
The model continuously adjusts the line to reduce these errors.
What Are Features in Machine Learning?
In Machine Learning, variables such as:
x1,x2,x3,…x1, x2, x3, ….x1,x2,x3,…
are called features.
Features are the input variables that help the model make predictions.
Get Gilnapradeep’s stories in your inbox
Join Medium for free to get updates from this writer.
For example, in a house price prediction problem:
- Square foot area
- Number of bedrooms
- Number of bathrooms
- Parking space
- Location score
all of these are considered features.
Single Variable Linear Regression
If the model uses only one feature, it is called:
- Single Variable Linear Regression
- Single Feature Linear Regression
For example:
- Predicting house price using only square foot area
The equation remains:
y = mx + b
Here, the model learns the relationship between one input feature and the output.
Graphically, the model learns a straight line in 2D space.
Multiple Linear Regression
Real-world problems are usually more complex.
House prices do not depend only on square foot area. They may also depend on:
- Number of bedrooms
- Number of bathrooms
- Location
- Age of the house
- Distance from the city
- Nearby facilities
Now the model uses multiple features together to predict the output.
This is called Multiple Linear Regression.
The equation becomes:
y=b0+b1x1+b2x2+⋯+bnxn
Where:
- y= predicted output
- x1,x2,x3,…,xnx1, x2, x3, ….., xnx1,x2,x3,…,xn = input features
- b1,b2,…,bnb1, b2, ….., bnb1,b2,…,bn = coefficients
- b0 = intercept
Each coefficient represents how strongly a feature influences the prediction.
For example:
- Larger house area may increase the price significantly
- More bedrooms may slightly increase the price
- Greater distance from the city may reduce the price
From Line to Plane to Hyperplane
When we use:
- One feature → the model learns a line
- Two features → the model learns a plane
- Multiple features → the model learns a hyperplane
For example:
If we use:
- Square foot area
- Number of bedrooms
- House price
Then:
- X-axis → square foot area
- Z-axis → number of bedrooms
- Y-axis → house price
Instead of fitting a line, the algorithm now tries to fit a plane in 3D space.
As dimensions increase further, the model learns a mathematical structure called a hyperplane.
Even though humans cannot easily visualize dimensions beyond 3D, mathematically the idea remains exactly the same.
How Linear Regression Learns
Linear Regression learns by minimizing prediction error.
The algorithm:
- Draws an initial line
- Makes predictions
- Measures errors
- Adjusts the line
- Repeats the process until the error becomes minimum
The goal is to find the best relationship between input features and output values.
Why Linear Regression is Important
Linear Regression is important because it teaches the foundational concepts of Machine Learning such as:
- Prediction
- Feature relationships
- Optimization
- Model training
- Error minimization
Many advanced Machine Learning algorithms build upon these same ideas.
Real-World Applications of Linear Regression
Linear Regression is widely used in many industries.
Some common applications include:
- Real Estate → House price prediction
- Finance → Stock trend analysis
- Healthcare → Disease progression prediction
- Marketing → Sales forecasting
- Education → Student performance prediction
- Weather Forecasting → Temperature prediction
Final Thoughts
Linear Regression may appear simple at first, but it forms the foundation of many advanced Machine Learning algorithms.
By understanding how models learn relationships between variables, you develop the intuition required for deeper topics such as:
- Gradient Descent
- Neural Networks
- Deep Learning
- Optimization Algorithms
Mastering Linear Regression is one of the first major steps toward becoming confident in Machine Learning.
That’s it for this part. More coming soon.
