OpenLearn | The Purpose of Machine Learning

11 min readJun 14, 2025

“Taking the right decision is less important than making the decision right.”
— Ajit Doval, NSA of India

Welcome To Open Learn : Our Learning Pioneers !

Hey Pioneers!
A Heartfelt welcome to The OpenLearn from the Pathfinders! Being the first blog in the series, we’ll cover today, the most interesting parts of Machine Learning, which will establish the vibe, the intuition and the philosophy of this whole process of Artificial Intelligence and its Development which stands on the foundations of Machine Learning.

Before we begin, here’s a small request: please maintain consistency throughout this cohort. Since the results are going to be outstanding if you guys just make sure of being active at least everyday in some or the other way. This entire summer season could be a game-changer for the rest of your college journey.

And another request is to stay curious, and don’t give up, If a tricky concept arrives, stay strong and smile, take the challenge and win over it!
We are all rooting for you! :)

Contents Walk-through

Before we start, Here’s a quick overview of today’s agenda :-

Why Should You Even Care About Machine Learning?
What exactly is Machine Learning ?
An End-to-End Machine Learning Project
Your Tasks for Today
BONUS Resources !

Why Should you even care about Machine Learning ?

On 30th November 2022, OpenAI dropped one of the most revolutionary tech updates in the world, which changed the way we see the tech world, the way we interact with machines, the way we imagine the AI forever by releasing the ChatGPT. Since then, we have seen the tech world evolve so rapidly that we hardly see a day when there are no talks about AI. We all use it in our day to day lives, Some of us use it to complete our assignments, write blocks of code, ask for personal help, or even generate Ghibli-style art.

An artificially generated Ghibli art collection of famous Indian Movie Scenes

All this change, brought in by AI, is essentially powered by the most widely used paradigm today: Machine Learning. Machine Learning is the backbone or foundation of modern day AI, with an overlapping collection of Statistics, Mathematics and Programming. And it literally means to make the “Machines learn”. But again, a pioneer might wonder, But why should I care about Machine Learning, Vatsal ?

JP Morgan Chase & Co. handles assets of worth exceeding $4 trillion (as of 2024*)

Well, Here are a few reasons to answer this question -

Wide-scale Applications of ML : Have you ever wondered how Netflix knows what you’ll binge next? How do banks detect fraud? What powers Google Search? The answer is - ML. As of 2025 one of The Most high paying jobs in the industry since it’s wide range of applications and safest in terms of long term market demand.
Relevance in the Field of Finance : The Machine Learning is the basis of Financial markets’ forecasting, trading, and quantitative research for global banking firms such as JP Morgan & Chase, HSBC, Goldman Sachs. Many of the High Frequency Trading Firms such as Jane Street, Tower Research, Jump Trade etc. hire Quantitative traders at record packages from top Indian Institutes.
The Rise of AI : As AI is booming, there is a surge in hiring professionals in the field of AI. Even the Software Developers are learning Gen-AI and Agentic AI skills since the high demand in the industry. But a good Programmer cannot be a good AI Developer without having strong foundation of Machine Learning.
Higher Education/Research : A lot of ML Researchers/AI scientists are hired in the industry with minimum qualification being a PhD in the field from a reputed institutes. In the institutions of high reputation in India or even globally offer PhD programmes in Machine Learning and related fields. But without a good ML Research experience in U.G., it becomes quite tough to get approved for such roles.
Philosophical Reasons : This is one of my personal reasons to carry forward the learning in AI and ML. This field takes me back to my childhood habit of being curious all the time, of questioning why and how things work as they do. And the origin of Intelligence is one such thing for me. Thus by not just having a monetary Incentives but having an inner reason to move ahead in this field, to find reasons and question even the origin of life, intelligence and consciousness.

What exactly is Machine Learning ?

Let’s understand the whole concept of Machine Learning in simpler terms. Learning is the process of acquiring new understanding, knowledge, behaviours, skills, values, attitudes, and preferences. Let’s see the process of learning in humans, for example. A newborn baby is born with no “learning” whatsoever, just a genetic human preconditioning exists, but a child is naive in the sense it doesn’t know any “Information” about the world and the environment, and neither does the child know the counter response for the various environmental conditions.

A Newborn’s Learning curve from age 0 to 12 months

How does a child learn to do anything ? Well it turns out to be simple 4 step mechanism :-
1. Perceive Information from senses.
2. Take an action corresponding to information.
3. Learn from the consequences of the actions.
4. Behave according to “Learnt” information.

Interestingly, machines can also be made to learn from the external world through a similar process.

Architecture of Supervised Machine Learning Model

A Machine Learning Task consists of three main components :

Training Dataset : This is the “information source” for the machine to learn from. Based on its nature, a machine learning technique can be Supervised, Unsupervised or Semi-Supervised in nature.
Learning Algorithm : A Learning algorithm is chosen as per the task of ML Model. For example for continuous numerical predictions we use Regression Algorithms, for Single/Multi-class Classification we use Classification Algorithms.
Hypothesis : This is the most important aspect of a Complete ML project. Hypothesis is the final model which can make predictions on real world data. It is the outcome of the whole process, and the goal is to achieve the best performing hypothesis.

So, enough of buzzwords now, let’s take you on a journey in next step, a day in the life of a Machine Learning Engineer to understand what exactly machine learning does, and how is it done!

An End-to-End Machine Learning Project

So this is the story of a machine learning engineer’s daily life, let’s take a sneak peek into what is his job and what does he do.

Important Note : To visit the entire End-to-End Machine Learning Project’s Jupyter Notebook visit this Kaggle link.

The Problem : Uber Fare Prediction Model

So our ML guy is working in this company called Uber, which is famous for cab booking services in North America.

He attends a meet, where the marketing guys are yapping about not making effective and targeted ads for users, the sales guys yap about effective discounting for the customers to be done, and our ML guy sits and hears them carefully. After some thinking he suggests, “What if we could predict how much does it cost for a user to book a cab based on different parameters, would it help you guys make some decisions?”

Everyone immediately agrees ! They tell him how it could help them do selective discounting and route based traffic allocation for cabs etc. So he asks for time and gets to his desk and starts to work on this Machine Learning Model which Predict Fare for a user based on different input parameters.

And this is how he starts :

Step 1 : Data Gathering

Before starting the work, an ML model needs a lot of data to work on. Sufficient amount of data (more is better) lays the basis for making a good hypothesis. So he goes to fetch the Uber archives and asks the team to provide all the data they have for the users, the parameters of journey and the fare. He also finds some data on a platform called Kaggle.

Quick Check : Ponder over this, What kind of learning technique is this ?
Hint : Look at the dataset to determine if model is Supervised, Unsupervised or Semi-Supervised in nature.

Note : The Amount of data directly impacts the accuracy of model. A research work done by Peter Norvig at Google named “The Unreasonable Effectiveness Data” proved that by increasing the amount of data, beyond a certain threshold, it doesn’t matter which Algorithm is chosen. It all reaches similar accuracy.

Step 2 : Setting up environment/Importing data

Our ML Guy primarily works on Python, due to powerful libraries coming handy and the opensource community of Python. He sets up environment by checking his requirements of libraries and imports data as well.
He usually works in Jupyter Notebooks, to analyse different checkpoints and document his work rather than getting in trouble of running different scripts.

import matplotlib.pyplot as plt
import seaborn as sns
import folium
from folium.plugins import HeatMap
import datetime as dt

from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler, PolynomialFeatures
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_absolute_error, mean_squared_error, r2_score

import xgboost as xgb
import lightgbm as lgb

import kagglehub
uber_fares_dataset_path = kagglehub.dataset_download('yasserh/uber-fares-dataset')

# Find the CSV file within the downloaded directory
csv_file_path = None
for filename in os.listdir(uber_fares_dataset_path):
    if filename.endswith('.csv'):
        csv_file_path = os.path.join(uber_fares_dataset_path, filename)
        break #stop after finding the first one.

if csv_file_path:
    df = pd.read_csv(csv_file_path)
    print(df.head()) #or any other operation on the df
else:
    print("CSV file not found in the downloaded directory.")

Step 3: Data Preprocessing

Before proceeding with analysis, he clean the data by removing incorrect values, handling null values, and filtering out unrealistic fare amounts.

Get Vatsal Khanna’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

✔ Actions Performed:

Drop unnamed index column.
Remove negative or zero fare amounts (invalid).
Drop rows where drop-off locations are missing.
Filter NYC geographical boundaries for valid coordinates.

And here he notices that now the size of the dataset is →

Step 4 : Feature Engineering

Our ML guy, notices that the dataset contains the following fields:

key — a unique identifier for each trip
fare_amount — the cost of each trip in $
pickup_datetime — date and time when the meter was engaged
passenger_count — the number of passengers in the vehicle (driver entered value)
pickup_longitude — the longitude where the meter was engaged
pickup_latitude — the latitude where the meter was engaged
dropoff_longitude — the longitude where the meter was disengaged
dropoff_latitude — the latitude where the meter was disengaged

Now here he notices, that some “additional information” which can directly affect the fare prices can be derived using given parameters such as using haversine formula we can derive Distance between Pickup and Drop Locations. Similarly one can categorise Time of the day into rush hour etc.
So after adding such “engineered” features to dataset, his new dataset is ready.

Step 5 : Data Analysis

Now that the dataset is ready, our ML Guy starts with some analysis of dataset. He needs to know important characteristics and inter-relations of the dataset to choose the correct model for the best accuracy.
So in this step he does some visualisations, calculates some statistical aspects of dataset.

A Histogram of Uber Fare Amount (Left) and a Correlation Matrix for the new Dataset (Right)

A Heat-map showing distribution of cab bookings in New York city

Step 6 : Model Training

The dataset is split into train and test dataset and then a series of models are chosen for training. His given hypothesis has to perform Regression task i.e. predicting continuous variable therefore he chooses different types of Regression Models. He used following models →

Linear Regression
Multiple Linear Regression
Polynomial Regression
XGBoost & LightGBM

# Selecting features and target
features = ["pickup_longitude", "pickup_latitude", "dropoff_longitude", "dropoff_latitude", 
            "passenger_count", "year", "month", "day", "day_of_week", "hour",
            "is_rush_hour", "is_night", "is_weekend", "distance_km"]

X = df[features]
y = df["fare_amount"]

# Splitting dataset
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Standardization
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)

poly = PolynomialFeatures(degree=2)
X_train_poly = poly.fit_transform(X_train_scaled)
X_test_poly = poly.transform(X_test_scaled)

poly_lr = LinearRegression()
poly_lr.fit(X_train_poly, y_train)
y_pred_poly = poly_lr.predict(X_test_poly)

mae_poly = mean_absolute_error(y_test, y_pred_poly)
r2_poly = r2_score(y_test, y_pred_poly)
print(f"Polynomial Regression - MAE: {mae_poly:.2f}, R²: {r2_poly:.2f}")

Don’t worry, we’ll cover them in detail later in cohort :)

Step 7 : Model Evaluation

Now that our ML Guy has trained the dataset on multiple Algorithms, he’s gotta choose the best model for the job. We do this by evaluating the model score on test data set in terms of Cost and Accuracy usually depicted by MAE and R² Scores respectively. (We’ll cover this in great detail in Cohort)

The above observations show that the best Model that should chosen is on with Higher accuracy (Proportional to R² Score) i.e. Ensemble Model.

Step 8 : Model Exportation and Deployment

So now that our ML guy has completed task of predicting Fare prices. He has to make it usable so that non-tech savvy guys such as Marketing team are able to use it, he needs to make it export this model.

He exports the model into a special file called Pickle file which contains all the model parameters which define the model, and it can behave in input-output manner.

import joblib

# Define the best model (Replace 'best_model' with the actual best model variable)
best_model = xgb_model  # Example: Replace this with the final best-performing model

# Save the trained model
joblib.dump(best_model, "uber_fare_prediction_model.pkl")

print("✅ Model successfully saved as 'uber_fare_prediction_model.pkl'. Ready for deployment! 🚀")

Press enter or click to view image in full size

Now that the model is exported, it can easily be deployed using an API (Application Programming Interface) and be used anywhere by the ML guy, in following format.

// Load the saved model¶
loaded_model = joblib.load("uber_fare_prediction_model.pkl")

// Example prediction (Replace with actual feature values)
sample_input = np.array([[2017, 5, 20, 14, -73.982, 40.767, -73.964, 40.753, 2]])

predicted_fare = loaded_model.predict(sample_input)
print(f"🚕 Predicted Fare Amount: ${predicted_fare[0]:.2f}")

Your Task for Today

Share 1 ML problem you’ve seen in real life that excites you on X.
Set up Python and Anaconda or configure your Kaggle environment (if not already).
Join the community group and introduce yourself with:
1. Name, Branch, and a fun fact about you.
2. ML Learning/Post learning thoughts.