Understanding Machine Learning Models with Python: A Deep Dive into Scikit-Learn

Someone

4 min read

·

Nov 26, 2024

--

Listen

Share

Linear Regression: Predicting House Prices Based on Square Footage

In this section, we’ll explore Linear Regression, one of the most fundamental algorithms in machine learning, which is commonly used for predictive modeling. We’ll use a simple dataset where the goal is to predict the price of a house based on its size in square feet.

from sklearn.linear_model import LinearRegression
from sklearn.model_selection import train_test_split
from sklearn.metrics import mean_squared_error

X = [[1500], [1800], [2400], [3000], [3500], [4000]]
y = [400000, 500000, 600000, 650000, 700000, 750000]

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

model = LinearRegression()
model.fit(X_train, y_train)

y_pred = model.predict(X_test)

print("Test Data Output:", y_test)
print("Test Data Predicted Output:", y_pred)
print("Mean Squared Error:", mean_squared_error(y_test, y_pred))

Step-by-Step Explanation

Import Libraries:

• LinearRegression is the model that we’ll use to make predictions based on the input data.
• train_test_split helps divide the dataset into training and testing subsets.
• mean_squared_error is used to calculate how well our model's predictions match the actual data.

Prepare the Data:

• X represents the features (input), in this case, the square footage of a house.
• y represents the target variable (output), which is the price of the house.

Split the Data:

• We split the data into training and testing sets using train_test_split. The test_size=0.2 means 20% of the data will be used for testing, and random_state=42 ensures reproducibility.

Train the Model:

• model.fit(X_train, y_train) trains the linear regression model using the training data.

Make Predictions:

• model.predict(X_test) generates predictions using the test data.

Evaluate the Model:

• Finally, the actual test values (y_test), predicted values (y_pred), and the Mean Squared Error are printed to assess the model’s accuracy.

Output

Test Data Output: [400000, 500000]
Test Data Predicted Output: [512611.01243339 540852.57548845]
Mean Squared Error: 7175086522.656796

Logistic Regression: Classifying Customers Based on Age and Salary

Logistic Regression is widely used for classification problems. In this section, we’ll use Logistic Regression to predict whether a customer will purchase a product based on their age and salary.

from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score

X = [[25, 40000], [30, 50000], [35, 60000], [40, 70000], [45, 80000]]
y = [0, 0, 1, 1, 1]

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

model = LogisticRegression()
model.fit(X_train, y_train)

y_pred = model.predict(X_test)

print(f"Test Data Output: {y_test}")
print(f"Test Data Predicted Output: {y_pred}")
print(f"Accuracy: {accuracy_score(y_test, y_pred)}")

Step-by-Step Explanation

Import Libraries:

• LogisticRegression is the model for binary classification (0 or 1).
• accuracy_score is used to evaluate the model’s prediction accuracy.

Prepare the Data:

• X contains features, which are the customer’s age and salary.
• y contains the target labels: 0 (no purchase) and 1 (purchase).

Split the Data:

• The dataset is split into training and testing sets.

Train the Model:

• The model.fit(X_train, y_train) trains the logistic regression model on the training data.

Make Predictions:

• model.predict(X_test) generates predictions for the test data.

Evaluate the Model:

• We evaluate the model using the accuracy score by comparing the predicted values (y_pred) with the actual values (y_test).

Output

Test Data Output: [1]
Test Data Predicted Output: [1]
Accuracy: 1.0

Support Vector Classifier (SVC): Classifying Points in a 2D Space

Support Vector Machines (SVM) are powerful models used for both classification and regression tasks. In this section, we’ll use a Support Vector Classifier (SVC) to classify data points in a 2D space.

from sklearn.svm import SVC

X = [[2, 3], [4, 6], [6, 8], [1, 1], [2, 2], [7, 7]]
y = [0, 0, 0, 1, 1, 1]

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

model = SVC(kernel='linear')
model.fit(X_train, y_train)

y_pred = model.predict(X_test)

print(f"Test Data Output: {y_test}")
print(f"Test Data Predicted Output: {y_pred}")
print(f"Accuracy: {accuracy_score(y_test, y_pred)}")

Step-by-Step Explanation

Import Libraries:

• SVC is used to implement Support Vector Classifiers. Here, we use a linear kernel.

Prepare the Data:

• X contains the features, which are coordinates in a 2D space.
• y contains the class labels: 0 and 1.

Split the Data:

• The data is split into training and testing sets.

Train the Model:

• The SVC(kernel='linear') creates the support vector machine with a linear kernel. model.fit(X_train, y_train) trains the model.

Make Predictions:

• Predictions are made using model.predict(X_test).

Evaluate the Model:

• The accuracy is calculated using accuracy_score, comparing the predicted labels with the actual ones.

Output

Test Data Output: [0, 0]
Test Data Predicted Output: [1 0]
Accuracy: 0.5

Decision Tree Classifier: Making Classifications with Decision Trees

Decision Trees are a great tool for classification problems. In this section, we’ll use a Decision Tree Classifier to classify data points based on their features.

from sklearn.tree import DecisionTreeClassifier

X = [[1, 2], [2, 3], [3, 4], [4, 5], [5, 6]]
y = [0, 0, 1, 1, 1]

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

model = DecisionTreeClassifier()
model.fit(X_train, y_train)

y_pred = model.predict(X_test)

print(f"Test Data Output: {y_test}")
print(f"Test Data Predicted Output: {y_pred}")
print(f"Accuracy: {accuracy_score(y_test, y_pred)}")

Step-by-Step Explanation

Import Libraries:

• DecisionTreeClassifier is the model used for classification using decision trees.

Prepare the Data:

• X contains the input features.
• y contains the output labels.

Split the Data:

• We split the dataset into training and testing sets.

Train the Model:

• model.fit(X_train, y_train) trains the decision tree model.

Make Predictions:

• We use model.predict(X_test) to make predictions.

Evaluate the Model:

• Accuracy is calculated using accuracy_score.

Output

Test Data Output: [0]
Test Data Predicted Output: [0]
Accuracy: 1.0

Conclusion

We’ve explored multiple machine learning models using Python’s Scikit-Learn library, each tailored for different types of problems, including regression, classification, and decision making.

Whether you’re predicting house prices, classifying customer behaviors, or plotting decision boundaries, these models provide a solid foundation for machine learning applications. With these tools, you can start tackling real-world problems with data-driven solutions.

You can check out my Github for others codes and follow me on Twitter to follow my journey.