4 must know skills

If you want to become an AI product manager, these are the four areas you need to know inside and out. Number one is evals and this is the most underrated and I think most sought-after skill right now for a PM to have. I have a series on my page because I just don't think there's enough education on it and this is exactly what I was doing at Microsoft, but how do you measure the quality of your AI agent? You cannot ship an AI feature or product without understanding what the quality is, what the metrics are and how to actually iterate on that agent or on that feature. Then is model fundamentals. Not just ML fundamentals, but actually things like latency, throughput, um, it's just going to help you better understand and talk to the engineers about what's cheaper, what's most reliable, and what's actually possible for you when you're building with AI. This also makes you really efficient and effective in communicating when you understand the differences between one to use RAG or one to fine-tune. The third is agent architectures and harnesses. Obviously we have the brain, which is the model, but everything that sits around the brain is the harness. One of the biggest jobs a PM has is actually to design the harness that the model is going to sit inside. These are things like tool calling, orchestration, and when to actually hint off a task to a human. In the last one is UX trust and responsible AI. I think it can go both ways. This is where the product manager defines how is the product going to fail gracefully, for example, and really try to build an experience where a user may be willing to wait a little longer, but the user isn't going to come back and trust the product if it gives an overconfident and an incorrect answer. And that's the line the product manager really has to pay attention to and clearly define.