Same data.

Same data. Eight algorithms. Same data. Eight algorithms. Eight completely different ideas of what learning means. learning means. learning means. The supervised ML map. LINE Linear regression. PROBABILITY Logistic regression. NEIGHBORHOOD KNN. MARGIN SVM. TREE Decision tree. FOREST Random forest. CORRECTIONS Gradient boosting. NETWORK Neural network. Neural networks are one part of the toolbox. If machine learning still just means neural networks in your head, you're missing most of the map. AI WITH Professor Whitaker. A map of machine learning. TODAY Same examples. Different ways to learn. Welcome to AI with Professor Whitaker. LEARN FROM LABELED EXAMPLES Features. Known answers. MOVIE 041 Budget $90M Marketing $41M Genre Action Director score 8.1 MOVIE 127 Budget $45M Marketing $12M Genre Drama Director score 5.7 THE ESSENTIAL PART Give examples. Include the known answer. Let's say I give a computer data from hundreds of movies: their budgets, marketing, genre, director history, and whether they succeeded. HIT FLOP FEATURES + LABELS The answer is attached to each example. I also give it the correct answer for every example. That last part is important. SUPERVISED LEARNING Learn a pattern from labeled examples. When we train on examples that already have known answers, we call it supervised learning. CHOOSE THE TARGET Revenue -> number Hit / flop -> class And depending on what we want to predict, we can attack the problem in very different ways. PREDICT A NUMBER Linear regression. BOX OFFICE ($M) BUDGET REGRESSION Fit a relationship. Predict a number. Start with linear regression. Suppose we only want to predict box-office revenue from a movie's budget. Plot the old movies, fit a line through them, and use that line to estimate where a new movie should land. ILLUSTRATIVE PREDICTION $110M budget -> $287M box office Obviously the real mathematics can involve many features at once, but the basic idea remains surprisingly civilized. Find a relationship. Use it to predict a number. PREDICT A CLASS Logistic regression. P(HIT) WEIGHTED FEATURE SCORE CLASSIFICATION Features -> score -> probability -> class We only want to know: hit or flop? That brings us to logistic regression. Despite the name, logistic regression is usually used for classification. It takes a weighted combination of the features and turns it into a probability, often using a sigmoid function. So instead of saying, 'This movie makes exactly 312 million dollars,' it can say, 'There is an 82 percent chance this P(HIT) = 0.82 belongs in the hit class.' YES, THE NAME... Regression that classifies things. Wonderful. Regression that classifies things. Very considerate naming. ASK THE NEIGHBORS K-nearest neighbors. MARKETING BUDGET LOCAL SIMILARITY Find the five nearest examples. But linear models assume the boundary can be described fairly simply. What if successful movies form little clusters instead? K-nearest neighbors takes a wonderfully lazy approach. K = 5 NEW MOVIE For a new movie, find the K most similar examples nearby and ask what happened to them. NEIGHBOR VOTE 4 HIT / 1 FLOP -> Predict HIT If five nearby movies were mostly hits, predict hit. WHEN THE WORK HAPPENS Store examples. At prediction: measure distances. There is almost no traditional training step. The work happens when you ask for a prediction. LEAVE A MARGIN Support vector machine. MARKETING BUDGET MAXIMIZE THE MARGIN Choose a boundary with breathing room. Next, support vector machines. An SVM tries to separate classes with a boundary, but it does not merely want any boundary. It prefers the one with the largest possible margin between the classes. SUPPORT VECTORS The closest points set the margin. The points sitting closest to that boundary are the support vectors. Separate nonlinear example KERNELS Can represent curved boundaries. And with kernels, an SVM can represent boundaries that are not simply straight lines. Because apparently drawing one line was becoming too relaxing. ASK QUESTIONS Decision tree. MOVIE D41 FOLLOW THE BRANCHES Question. Answer. Prediction. Now let's stop drawing smooth boundaries entirely. A decision tree repeatedly asks questions. Was the budget > $80M? NO FLOP YES MARKETING > $30M? NO FLOP YES DIRECTOR SCORE > 7? NO FLOP YES HIT Was the budget above this amount? Was marketing above that amount? Was the director's previous score high enough? Each answer sends the movie down another branch until it reaches a prediction. MARKETING BUDGET Questions create rectangular regions. OVERFITTING Too many splits can memorize training quirks. Trees are intuitive, but one tree can become overly attached to the peculiarities of its training data. ONE TREE -> MANY Vary the trees. Combine results. So we build many of them. COMBINE MANY TREES Random forest. TREE 1 budget genre TREE 2 marketing director TREE 3 budget marketing TREE 4 genre director TREE 5 budget director VARIETY MATTERS Different samples. Random feature subsets at splits. That gives us a random forest. Each tree sees a slightly different sample of the training data and considers random subsets of the features. HIT HIT FLOP HIT HIT MAJORITY VOTE -> HIT COMBINE PREDICTIONS Many varied trees. One shared vote. Then the forest combines their predictions. One questionable tree is a problem. Five hundred slightly different questionable trees voting together can be surprisingly useful. CORRECT THE ERRORS Gradient boosting. BOX OFFICE ($M) BUDGET ROUGH ESTIMATE SEQUENTIAL LEARNING Model 1 + correction + correction Then we have gradient boosting. Instead of building independent trees and voting, boosting builds small models sequentially. FIT THE REMAINING ERROR Each new model focuses on correcting errors left by the models before it. ADD THE CORRECTIONS and a weak set of learners can become a very strong predictor. XGBoost Optimized gradient-boosted trees Regularization Efficient tree building Engineering optimizations AN IMPLEMENTATION Gradient boosting, carefully engineered. XGBoost is a popular optimized implementation of gradient-boosted trees, with techniques such as regularization and efficient tree building. LEARN TRANSFORMATIONS Neural network. BUDGET MARKETING GENRE DIRECTOR LEARNED CONNECTIONS Inputs move through layers. A neural network passes the input through layers of artificial neurons. Each connection has a learned weight. weighted sum -> nonlinear activation WEIGHTS + NONLINEARITY Combine. Transform. Predict. The network repeatedly combines inputs, applies nonlinear activation functions, and transforms the information until the final layer produces a prediction. 82% HIT With enough data, computation, and the right architecture, neural networks can learn extraordinarily complicated relationships. THE SUPERVISED ML MAP One goal. Eight approaches. LINE Linear regression. PROBABILITY Logistic regression. NEIGHBORHOOD KNN. MARGIN SVM. TREE Decision tree. FOREST Random forest. CORRECTIONS Gradient boosting. NETWORK Neural network. MACHINE LEARNING A toolbox. Not one algorithm. But notice what happened. We reached neural networks at the end of the map, not the beginning. Machine learning is not one algorithm. It is a toolbox. NO UNIVERSAL BEST Choose for the data, the task, and the constraints. A line. A probability. A neighborhood. A margin. A tree. A forest. A sequence of corrections. Or a network of learned transformations. CLASS DISMISSED Supervised machine learning. LABELED EXAMPLES LEARN A PATTERN PREDICT A NEW EXAMPLE SAME BASIC GOAL Learn from answers. Generalize to new examples. Same basic goal. Very different ways of learning how to get there. And that is supervised machine learning. Professor Whitaker. AI, EXPLAINED. THE SUPERVISED ML MAP Eight models. One dataset. I'm Professor Whitaker. Class dismissed. Class dismissed.