I'm a 19 year old AI engineer, we're going to be building a reinforcement learning model from scratch

I'm a 19 year old AI engineer, and today we're going to be building a reinforcement learning model from scratch for your portfolio together. So here's the project. We're building an AI that has to cross a frozen lake. So picture a 4 by 4 grid. It starts in the top left and the goals in the bottom right and scattered around our holes. If the model falls in a hole, it's going to be game over. And the whole challenge is we're going to tell it absolutely nothing. We're not gonna tell it where the holes are, over the goals are, or not even which way is good. It has to teach that itself using the entire map. And it's going to learn this through trial and error. And this is what reinforcement learning is for. For any logic, We're gonna set our dials, which are the numbers that control how the AI learns, and i'm gonna walk you through them because every single one matters. learning rate is just how big a lesson it takes from each move. And if it's too high, it will overreact to an experience, and if it's too low, it will learn painfully slow. Your gamma is just how much the AI cares about future rewards versus right now. A high gamma means it's playing a long game and thinking about the goals in several steps ahead. The most important one I would say is Epsilon, which is a curiosity dial. And Epsilon is just a chance it does something totally random instead of trusting what it knows. This is exploitation versus exploration, and we started at one, which means 100% random, because at the start it knows absolutely nothing, so it should just experiment. Epsilon decay is just shrinking that curiosity a little after every game, so it slowly shifts from experimenting to trusting itself once we fill that table. And epsilon min just a floor, so that it never stops fully exploring later. And I forgot to set that up, is the actual AI's brain, which is just a table with 16 rows, one per square on the grid, and four columns, which is one per move left, right, down and up. Each cell is its guest. At how good is this move from the square? And when we initiate, it's going to be full of zeros, because the AI knows pretty much nothing. I'm just setting up the environment right now for where our reinforcement learning model is going to play around and live. And gymnasium is really good for that, so I would suggest trying it out. Our first function is the choose action function, and this is how it actually decides what to do. And it's beautifully simple. It rolls a random number, and if that number is below epsilon, it picks a totally random move, which is how it explores. Otherwise, it looks at its brain table from the current square and picks the move with the Highest number, which is exploiting what it learned. This is the entire exploration versus trust balance in three lines. And because Epsilon starts at 1 early on, it's almost always exploring, which is exactly what we want when the table still empty. This next function is the heart of the whole thing, and it's where learning actually happens. So after every move, the AI works out what move was really worth it. And that's two parts added together, which is the reward it just got, plus the best it could possibly do from the squares it landed on. Slightly shrunk down by gamma, because feature reward counts a little less than right now. And that total is the target, which is what the value should be. Then this line here does the learning, where it nudges the old guess towards that target. So the gap between what it expected and what actually happened, which is that's pretty much the error. And the AI shrinks that error every little time. That's why squares next to the goal end up worth almost a full point, and the squares next to holes stay near zero. Now, we tied all together in the training loop. So for thousands of games, we're gonna reset to the start and then keep going. so we're gonna choose a move, take it, and then see what happens. update the brain, move to the new square until it reaches either the goal or falls in a hole. And after each full game, that's where Epsilon's gonna decay once per game and not per move, so that it gets a little less random and a little more confident. Every then we're just gonna print the win rate, so as we go, we can watch it learn. And 0% for thousands of games, there was absolutely nothing. So first time this happened, I genuinely thought my code was broken, but it wasn't. And this is the most important lesson in the video. Looking back at those dials, my epsilon was decaying way too fast, and the AI was getting confident and trusting. It's a brain table way too early, before it had ever actually reached the goal even once. So it locked onto one move and just walked into the same wall forever, totally convinced with an empty brain. And it committed before it had lived long enough to know anything. So the fix is right back in those parameters. I slow the decay down so it stays curious much longer, and I give it way more games to learn from. The whole change is basically explore for longer before you start trusting yourself. And that's it. And when we run it again, we get decent, better numbers. So you can't see the full numbers, but it reached 96% And the only thing I really had to change was how long it stayed curious. And there you go, that's a complete reinforcement learning project for your portfolio. An AI that taught itself to solve a problem with zero training data. Have fun.