scaling your claude app from 100 users to 500,000+ users

I'm a software engineer, I've been in the industry for about six years and today we're going to talk about what to do when things go wrong with your app. Specifically, what to do when 5,000 users, 10,000 users, 100,000 users hit an error. How do you detect that? How do you know that it's happening in the first place? And two, how do you recover from that? I post readable versions of all my content on my newsletter as well, so you can just head to the link in my bio if you prefer to read the blog post version of everything I'm going to discuss. So this concept in software engineering is broadly referred to as observability. And typically when we talk about observability, we're referring to both telemetry and logging. Both of these concepts help you understand what is going on with your app when it's running in real time. So let's say you're the creator of Enmo. You have millions of people that use your app, it's a big hit and you get a ton of traffic every day. How do you know if something's not working properly in your app? The most basic example would be anytime a user hits an issue, they file a customer support ticket and email it to you. That might work if you only have 10 users, you check your email every day and if you see one email, you can look into it, you can investigate the problem based on what they've said in the email and you can try to reproduce the issue yourself. On the scale of 10 users, that's not a big deal because the max volume of problems you're going to be getting are on the order of 10 people. Even if each person is hitting 10 different issues, that would still only be 100 different customer support tickets, which is pretty doable for a single person. However, if you have millions of users, you have tens of thousands of users, 100,000 users, if not more. So this is where observability comes into play. And the concept is that instead of manually waiting for customers to submit support tickets, which is unreliable because one, not every customer is going to do that, and two, like we discussed earlier, the load that you would be having to process is too high for you to be able to handle. What you do is you emit telemetry events and you emit logs. What is a telemetry event? A telemetry event is basically a checkpoint where you insert a different point into your code. It can take a lot of different forms. There's usually business insights telemetry and then also latency metric telemetry. So we'll start with business insights telemetry. Let's say you launch a new feature and you're trying to figure out if people like this, like is this button that you added actually successfully converting users into doing the action that you desired? To measure that, you could manually send out a survey and ask people, did you use this button or not? But there's a lot of reasons why that wouldn't work. People might not open the email, people might not complete the survey. What you could do instead is you could actually add a telemetry event alongside the button click. So every time a user clicks that button on their individual laptop, you would have an event that gets emitted from your app to your backend. There are a number of different services that exist that collect these events for you. But at the end of the day, you know that anytime a user clicks that button, you're going to get an event that tells you that action happened. This becomes really meaningful because then you can do analysis and comparisons to understand if what you built is actually successful. For example, let's say you have a simple telemetry event that gets sent out anytime a user hits your homepage. What you would then do in the button click telemetry example is say, okay, I have 200,000 events that are associated with a homepage click. I know that 200,000 people hit my homepage today. Let's say it's a Wednesday. Then you can say, in comparison to the number of people that hit my homepage today, how many events did I get for the button click? Is the button click not discoverable enough? Is this just an action that people don't want to take? Or conversely, let's say 150,000 people are clicking this button. That's great. That's a good signal that means people are able to get into the entry point that you have for them and take the desired action that you have for them. So that was a very, very quick primer on telemetry and how they work. Typically what you do is you have a concept of a user session. You typically have a user session that starts when the user accesses your app and then any checkpoint event that you emit, you associate it to that same session. And the reason this is helpful is because the same user could have multiple different sessions in a day. For example, with a Venmo case, I might send telemetry at 8 AM in the morning and then later at 4 PM and then again later at 6 PM. And so you could filter by my user ID. But it's more helpful to actually see the user journey per session. Because it's possible that in session A, I hit a case that I didn't hit in session B or C. So that's my very, very quick primer on telemetry and how they work. I'm not always sure the backgrounds of the videos, some people are more technical than others. So if this is something that you haven't dealt with before, let me know and I can do a separate deep dive that just covers best practices when it comes to telemetry.