I broke into professional industry machine learning at 18.

@lachinemearning.com

20.9K followers · usually 8.3K views a post

Views

80.5K

Likes

8.8K

Comments

119

386

Why it worked

The video explains a complex machine learning concept (symbolic regression) in an accessible way, using a clear demonstration and comparison to standard models. The presenter's personal experience and enthusiasm also contribute to engagement.

Summary

The video explains symbolic regression, a machine learning model that discovers mathematical equations from data, contrasting it with traditional models that learn patterns. It highlights symbolic regression's ability to find underlying rules and its application in fields like physics and finance, while also discussing its computational cost and potential for hybrid approaches.

Structure

1Introduction to symbolic regression
2Comparison with traditional machine learning
3Explanation of the evolutionary process
4Demonstration of symbolic regression finding an equation
5Comparison of symbolic regression with a standard model
6Discussion of symbolic regression's limitations and hybrid approaches

Product placement

The video discusses and demonstrates symbolic regression, a machine learning technique. The presenter shows code and output related to this technique, specifically using a Python library for symbolic regression.

On-screen text

I broke into professional industry machine learning at 18. I'm gonna be talking to you about a model that you probably have not heard of called symbolic regression. And I know many industry professionals and data scientists that do not know about it either. So don't be scared if you don't. This is a model that's mainly used in, like, physics simulations and biology and quantitative finance. So here's the thing, right? Most machine learning works like this. You tell the computer what shape the answer should be, and it fills in the numbers. Like with linear regression, you already have the equation. You're just trying to find Constants to multiply your data points with to fit that line. Your computer is only working out which line out of all the lines you decided, right? So basically, every model you've heard of is like that. You pick the shape, and it fills in the blanks. Symbolic progression doesn't get told the shape. It works that out, too. And what it hands back is a proper equation, the kind that you see in a textbook. So it's not looking for just Constants It's actually looking for a math equation, too. Here's how. And it's honestly just evolution. So it starts by writing a few hundred completely random equations. They're all crap and rubbish, and it scores each one on how close it gets and deletes the worst and breeds the survivors. It's called, um, genetic programming. Like survival of the Fittest, it takes a chunk of one equation and splits it into another and randomly swaps a plus for a times. Then it does it again and again and again, thousands of times. Now, bad equations die out. The bits that work get passed on. Same as evolution, except it takes a minute instead of a billion years. So I tested it properly. I made up an equation, first number squared minus the second number plus 1, and generated 500 data points from it. Then I deleted the equation and handed over nothing but the points. Now you're about to see the equation that it actually found in row three there. That's mine. Exactly. It worked it out just from the data alone. And here's a bit that actually matters. I trained a normal machine learning model on that same data. Then I tested both of them on numbers neither had ever seen. So the normal one falls apart. Like, look at this. 300 different inputs, and it gives the same answer to all of them because all it ever Learned was how to copy patterns it had already been shown. But the equation just keeps working because it's an equation. It doesn't have an edge. This was the whole appeal. Most models learn to imitate your data, but this one actually learns the rule underneath it. And you're probably like, Iris, why don't we just use symbolic regression then? The thing is, it's a really a costly Approach* because you're trying to basically make it shape up all of the possibilities out of this like huge space of different mathematical expressions. So actually it suffer from combinatorial explosion. But the trade off is that training is slow, but actually using the model to predict something is fast by an inference. Essentially we'd use symbolic regression. For smaller data sets that are very, very niche. However, there are hybrid approaches that come out. So for example, a transformer like looks at the dataset and predicts the most likely equation skeleton in a single pass. So there are workarounds around it. And I think this is a pretty cool model, to be honest. But yeah, I think this is really cool. And I think if you are really into more of the mathematical side, not saying machine learning isn't, but this is actually quite cool. And it's quite a niche thing if you wanna spice up your portfolio.

Original caption

symbolic regression is such a fascinating model, and it quite literally is able to find newtond laws on its own, and I think I read somewhere that these people were able to make the model predict the swing of a pendulum. I might be wrong, Please correct me if I am, but yeah, I think this model is so cool and it was so fun to see it in action and really do hope that somebody out there thags fascinated with it as much as me. It might actually be my new favourite algorithm #programming #computerscience #machinelearning #maths #engineering

More from @lachinemearning.com

Their other posts in the index, biggest breakout first.

Transcript

I broke into professional industry machine learning at 18. I'm gonna be talking to you about a model that you probably have not heard of called symbolic regression. And I know many industry professionals and data scientists that do not know about it either. So don't be scared if you don't. This is a model that's mainly used in, like, physics simulations and biology and quantitative finance. So here's the thing, right? Most machine learning works like this. You tell the computer what shape the answer should be, and it fills in the numbers. Like with linear regression, you already have the equation. You're just trying to find constants to multiply your data points with to fit that line. Your computer is only working out which line out of all the lines you decided, right? So basically, every model you've heard of is like that. You pick the shape, and it fills in the blanks. Symbolic progression doesn't get told the shape. It works that out, too. And what it hands back is a proper equation, the kind that you see in a textbook. So it's not looking for just constants. It's actually looking for a math equation, too. Here's how. And it's honestly just evolution. So it starts by writing a few hundred completely random equations. They're all crap and rubbish, and it scores each one on how close it gets and deletes the worst and breeds the survivors. It's called, um, genetic programming. Like survival of the fittest, it takes a chunk of one equation and splits it into another and randomly swaps a plus for a times. Then it does it again and again and again, thousands of times. Now, bad equations die out. The bits that work get passed on. Same as evolution, except it takes a minute instead of a billion years. So I tested it properly. I made up an equation, first number squared minus the second number plus 1, and generated 500 data points from it. Then I deleted the equation and handed over nothing but the points. Now you're about to see the equation that it actually found in row three there. That's mine. Exactly. It worked it out just from the data alone. And here's a bit that actually matters. I trained a normal machine learning model on that same data. Then I tested both of them on numbers neither had ever seen. So the normal one falls apart. Like, look at this. 300 different inputs, and it gives the same answer to all of them because all it ever learned was how to copy patterns it had already been shown. But the equation just keeps working because it's an equation. It doesn't have an edge. This was the whole appeal. Most models learn to imitate your data, but this one actually learns the rule underneath it. And you're probably like, Iris, why don't we just use symbolic regression then? The thing is, it's a really costly approach because you're trying to basically make it shape up all of the possibilities out of this like huge space of different mathematical expressions. So actually it suffer from combinatorial explosion. But the trade off is that training is slow, but actually using the model to predict something is fast by an inference. Essentially we'd use symbolic regression for smaller data sets that are very, very niche. However, there are hybrid approaches that come out. So for example, a transformer like looks at the dataset and predicts the most likely equation skeleton in a single pass. So there are workarounds around it. And I think this is a pretty cool model, to be honest. But yeah, I think this is really cool. And I think if you are really into more of the mathematical side, not saying machine learning isn't, but this is actually quite cool. And it's quite a niche thing if you wanna spice up your portfolio.