Guide to Distance Metrics in Machine Learning Algorithms
Distance Metrics used in machine learning algorithms
Distance metrics play a very important role in machine learning. A number of machine learning algorithms, both supervised and unsupervised, use distance metrics to calculate the similarity between two data points.
These distance metrics calculate mathematics-driven distance between various data points available in the dataset. If the distance between two data points is less that means they are more likely to be similar and vice-versa.
These distance metrics provide a solid foundation for several machine learning algorithms such as k-nearest neighbours (supervised learning) and k-means clustering (unsupervised learning) etc. An effective distance metric can also improve the performance of our machine learning model.
In this article, we will go through various distance metrics that are used in machine learning algorithms, their working, and where we use these metrics. Let’s get started.
Why do we need Distance Metrics?
The distance metrics are used in various machine learning algorithms such as K-Nearest Neighbors algorithm for classification problems, the K-Means Clustering algorithm for clustering and Self-organizing maps(SOMs) etc.
Therefore, it becomes important to know about the intuition and working of these distance metrics. This helps us to know which distance metric we should use over another metric.
There are several distance metrics that can be used, depending on the type of data that we have and the algorithm that we are using for our problem statement.
Let’s go through them one by one.
Distance Metrics
1. Euclidean Distance
Euclidean Distance is one of the most popular distance metrics. It represents the shortest distance between two data points/vectors in an N-dimensional space. This distance metric is by default used by most of the machine learning algorithms (such as KNN and K-Means clustering) to measure the similarity between observations and also provides good results.
This distance metric should be used whenever we are dealing with features which are continuous, numeric variables(such as integer values and floating points). Euclidean Distance is calculated as the square root of the sum of differences between two data points.
Where N is the number of dimensions.
Example:
from scipy.spatial import distancea = (1, 2)
b = (4, 5)d = distance.euclidean(a, b)
print(d)Output:
4.24
This is how we can calculate the Euclidean Distance between two data points in Python.
Use Cases:
- Euclidean distance metric is a good fit when you are working with low-dimensional data.
- Machine learning algorithms like k-Nearest Neighbors show great result with this distance metric when our dataset is low-dimensional.
2. Manhattan Distance
The Manhattan distance, also known as L1 Distance or City Block Distance, is used to calculate the distance between two data points across all dimensions, given that those two data points are in a grid-like path. There is no diagonal movement/path involved considered when calculating the Manhattan distance.
Let’s say, we want to calculate the Manhattan Distance, d, between two data points A, and B.
The distance(d) can be calculated by the sum of the absolute values of the difference between the coordinates of the two data points as below:
The generalized formula for Manhattan Distance between two data points, A, and B, for N dimensions can be represented as:
Where N is the number of dimensions.
Example:
def manhattan_distance(point1, point2):
distance = 0
for x1, x2 in zip(point1, point2):
difference = x2 - x1
absolute_difference = abs(difference)
distance += absolute_difference return distancex1 = (1,2,3,4,5,6)
x2 = (10,20,30,40,50,60) print(manhattan_distance(x1, x2))
In Python, this is how we can calculate the Manhattan Distance between two data points.
Get Pralabh Saxena’s stories in your inbox
Join Medium for free to get updates from this writer.
Use Cases:
- Manhattan distance metric is a good fit when our dataset contains binary or discreate columns.
- This distance metric is preferred over Euclidean distance and shows great results when there is high-dimensionalilty in the dataset.
What kind of challenges arise when building ML models at scale? Chris Brossman from The RealReal discusses his team’s playbook for big tasks.
3. Minkowski Distance
Minkowski distance is used to calculate the similarity of the distance between the vectors given that there are two or more vectors. Minkowski distance is a generalized distance metric. By generalized it means that the formula of Minkowski distance can be used to calculate the distance between two points in different ways.
Here’s the formula for Minkowski distance to calculate the distance between two data points x and y:
Where p is the order parameter.
Here, we can change the value of p and measure the distance in different ways, such as:
p = 1, Manhattan Distance
p = 2, Euclidean Distance
This Minkowski distance formula is a generalization form of both Manhattan distance and Euclidean distance.
Any intermediate value between 1 and 2(such as p=1.5) provides a controlled balance between the Euclidean and Manhattan distance.
Use Cases:
- In machine learning, the Minkowski distance is widely used to find the optimal correlation or classification of the data.
- It is used in the machine learning algorithms such as k-NN, K-means clustering, and self-organizing map.
4. Hamming Distance
Hamming Distance is used to measure the similarity between two strings/vectors of equal length.
The Hamming distance between two strings of equal length is the number of positions at which the corresponding characters are different in both strings. It includes the count of that which set of the positions are the same and which are different.
Hamming distance is preferred when the features of our data consist of binary values such as 0 and 1. If our dataset consists of dummy boolean values then this distance is best to compute the similarity between two data points.
Example:
from scipy.spatial.distance import hamming# sample data
row1 = [0, 0, 0, 0, 0]
row2 = [0, 0, 0, 0, 1]# Hamming distance
dist = hamming(row1, row2)
print(dist)
Output:
0.2
Here, we can see that 1 out of 5 positions is different, therefore the hamming distance is 1/5 = 0.2
Use Cases:
- Hamming distance metrics is used to calculate the distance between two categorical variables in the dataset.
- It is also used to calculate the error by the counting the number of distorted bits in a binary string.
Final Note
Manhattan distance works pretty well and is preferred over Euclidean distance when working with high-dimensional data. Euclidean distance works great when we have low-dimensional data.
And the hamming distance is used when we need to measure the difference between two features with categorical variables.
Conclusion
In this article, we have learned about different distance measures/metrics. We learned about Euclidean, Manhattan, Minkowski, and Hamming distance and their uses. These metrics are intensively used in machine learning algorithms to find the distance between two columns.
Editor’s Note: Heartbeat is a contributor-driven online publication and community dedicated to providing premier educational resources for data science, machine learning, and deep learning practitioners. We’re committed to supporting and inspiring developers and engineers from all walks of life.
Editorially independent, Heartbeat is sponsored and published by Comet, an MLOps platform that enables data scientists & ML teams to track, compare, explain, & optimize their experiments. We pay our contributors, and we don’t sell ads.
If you’d like to contribute, head on over to our call for contributors. You can also sign up to receive our weekly newsletter (Deep Learning Weekly), check out the Comet blog, join us on Slack, and follow Comet on Twitter and LinkedIn for resources, events, and much more that will help you build better ML models, faster.
