Bagging
this blog is the continuation of the last blog.
Today we will learn about bagging which is the powerful ensemble technique used for both classification and regression problem.
starting with bagging classifier
lets understand the meaning of bagging , it is made up of two word
Bootstrapping and Aggregating : Bootstrapping means creating the subset of the data and feed to the model and make classification , Aggregating means evaluating the output by different models , don’t worry we will discuss the technical language too .
Bagging or Bootstrap Aggregating, works by training multiple base models independently and in parallel on different random subsets of the training data. These subsets are created using bootstrap sampling, where data points are randomly selected with replacement, allowing some samples to appear multiple times while others may be excluded.
- In classification tasks, the final prediction is decided by majority voting, the class chosen by most base models.
- For regression tasks, predictions are averaged across all base models, known as bagging regression.
- Bagging is versatile and can be applied with various base learners such as decision trees, support vector machines or neural networks.
- Ensemble learning broadly combines multiple models to create stronger predictive systems by leveraging their collective strengths.
For each of the bootstrapped subsets, a separate classifier (e.g., decision tree, logistic regression) is trained.
Get Mayank Gariya’s stories in your inbox
Join Medium for free to get updates from this writer.
The predictions from all the individual classifiers are combined to form a final prediction. This is often done through a majority vote (for classification) or averaging (for regression).
Bagging the technique is best for creating the optimum model with low variance and low bias , using estimators with high variance and less bias random sampling nullifies the effect of outlier and make the reliable model .
OOB concept .
Out-of-Bag (OOB) Evaluation: Samples excluded from a particular bootstrap subset (called out-of-bag samples) provide a natural validation set for that base model. OOB evaluation offers an unbiased performance estimate without additional cross-validation.
Why Use Bagging?
- Reduces Variance: By combining multiple models, bagging smooths out predictions, making the overall model less sensitive to specific noise in the training data.
- Prevents Overfitting: Individual models (like deep decision trees) are prone to overfitting, but the aggregation of many different models helps cancel out these errors.
- Parallel Processing: Because the base models are trained independently of one another, they can be processed in parallel, making training computationally efficient
