Hook
More breakout videos from this creator.
Linear regression. That's D tier. Each coefficient tells you how much the prediction moves when one input grows by one. But almost nobody uses plain linear regression in production. It only fits straight lines and outliers pull it hard. XGBoost. S tier. Each new tree fixes the errors of the trees before it. In 2015, 17 of 29 winning Kaggle solutions used it, but without early stopping it overfits. CatBoost. S tier. Same idea, but handles categories like city names directly without one-hot encoding. And even with its default settings it often beats tuned models. But on data that is all numbers the category handling no longer helps. Naive Bayes. C tier. It trains by counting and early spam filters ran on it. But it treats every word as independent so its probabilities are overconfident. Random forest. A tier. Hundreds of trees vote, each trained on a random slice of the data. And the defaults usually work, but it cannot predict outside the range it saw in training. Support vector machines. C tier. It finds the widest possible gap between the classes, but training time grows at least with the square of the rows, so large data sets are out. PCA. B tier. It squeezes many columns into a few, keeping the directions where the data varies most. But it only sees straight line structure and the new axes are hard to explain. Isolation forest. B tier. It finds anomalies without labels because random splits cut an odd point off in far fewer steps. But you still have to guess how much of your data is anomalous. Logistic regression. A tier. It turns a weighted sum into a probability and banks build credit scorecards with it because every decision can be explained. But its boundary is a straight line.