How to Design an AI Agent System for E-Commerce DTC Brands to Manage Customer Support

everyone this is Sean let's think about how to design an AI agent system like a pro it doesn't matter if you're technical or not if you're a product manager data scientist or computer scientist does not matter the goal here really is to understand how do we think holistically about you know starting from the front end all the way to the back end to database and setting up AI agents so that your app will be smarter than just a traditional Sass product Problem to Solve Today How to Design an AI Agent System for E-Commerce DTC Brands to Manage Customer Support - Goals: >=70% Automation, CSAT >4.5/5, p50<1s, p95<2.5s - Scope: Returns, Exchanges, WISMO - Rules: - Return window (<30d only), must return in good conditions - Refund Policy (<$50 auto, >=$50 manager) - Exchange Policy: good conditions, categories of products - Route to human: asked by user, emotion detection - Backend/DB: - Peak RPS & Seasonality - Data Storage to Relational DB or Vector DB today's example is about how do we design this agent system for the e-commerce for the e-commerce direct to customer brands DTC brands to manage their customer support DTC brands to manage their customer support so as we know that customer support is a field that traditionally has a lot of human involvement when it comes to things like return products exchange products or you know asking about the status of where my product is when it's being shipped and um for a lot of e-commerce company they probably cannot afford to hire a bunch of like call centers or message replying agents so this would be a perfect use case for them to you know increase their response rate as well as improve the satisfaction for their customers Goals so now let's jump in and think about how do we think through this step by step okay so the first thing is that our goal today is that we want to make sure that we will have a over 70% automation in this customer support flow which means that the other 30% probably be like we're gonna loop the humans back into the conversation when a e-commerce site is talking to a customer right and also we wanna make sure that the customer satisfaction rate CSAT is greater than 4.5 out of 5 right and last but not least we wanna make sure that the response with the 50% of the percentage of the customers is below 1 second and then 95% of your customers will get a response under 2.5 seconds why is this important this is important because if the response time is too long it really affects your satisfaction from our customers okay so this is our goal so let's also think about what is our scope today Scope: Returns, Exchanges, WISMO so let me just delete this flow and let's walk through it together so firstly let's think about the rules of this scope Rules: - Return window (<30d only), must return in good conditions - Refund Policy (<$50 auto, >=$50 manager) - Exchange Policy: good conditions, categories of products - Route to human: asked by user, emotion detection - Backend/DB: - Peak RPS & Seasonality - Data Storage to Relational DB or Vector DB in order to return to the customer we need to make sure that the products for example will only be returned if it's bought over under 30 days before and it must be returned in good conditions right otherwise the airlines probably will just allow any products to be returned at any time which is not what we want right the second rule we think about is um how do we think about the refund policy right in this case most companies probably have a refund policies for example if it's under $50 and if they're using a human call center as a service then they will automatically let it to um return the product but if it's over or equal to $50 then they probably need to loop in their manager we want to mimic the exact user behavior here the third one is called exchange policy so if a customer wanna exchange the product they also need to make sure a few things um it needs to be in good conditions and certain categories of products cannot be returned because for example if it's food and you already open it you cannot really return it right and lastly that um how do we route back to a human right there could be all sorts of scenarios here um it could be because the user directly asked that they want to talk to a human or we have detected that there's some very negative of emotions in the chat so uh we must involve a human being to basically provide emotional support for the customer okay um from a back end and database perspective there are a few things that we should consider in this case let me just paste it right in the first thing is that when and how much is the peak for request per second and about the seasonality as well imagine this is Black Friday or Christmas time then probably there will be a lot more people doing online shopping versus the other period of time of the year and also we need to think about how we're gonna store all the chat history um information about the transactions about return and logistics do we store it in a relational database or do we store it in a vector database so here's a concept we need to introduce for AI agents which is vector database what it really means is that traditionally we basically define data tables in imagine you have an Excel sheet there are rows and columns every column means a different feature every row means like a different entry right but what it really means is that traditionally we basically define data tables in imagine you have an Excel sheet there are rows and columns every column means a different feature every row means like a different entry right but in order to search some of the say unstructured data like the return policy or refund policy these kind of things are not gonna be stored in traditional like table format they're probably a paragraph or PDF right so the way to do it is that we can store this information in a vector database in which we will embed each word or each you know paragraphs into uh vectors right so what we mean by that is that we're gonna turn these words into numbers that are in high dimensions if you're really not from a technical background just think of it as like we gotta digitize something like if you're looking at a photo on your iPhone it will be stored in digits and then when they're stored in digits you'll be able to search them by calculating some similarity between photos similarly here we're allowing it to calculate similarities between vectors right so given the current scope the goal the rules the back end database let's jump in to start designing the agent system here the first thing we're gonna do is that we need to think about when will a user start to interact with our system right so in this case our user channels are gonna be website chat and emails okay so User Channels - Web Chat, Email Gateway - Auth/SSO, - PII, - RateLimit, - dedupe Auth User Auth, Fetch Chats/Emails Relational DB Auth Ask a question: I want to return my product X. Respond to the user real-time Human Approval Router Agent - understands user intent - decides agent routing Q&A Agent - speak to user all the time Decide to take the Return Route Action Return the latest status of return back to Q&A Agent to talk to the user Return Planner Agent - decides functional calling - Or, a deterministic workflow Vector DB (FAQs, Policies) RAG: Check Policy Via VectorDB Align with Policy Inform the agent Policy/Keywords: - Capability Tokens, RMAs, human approach, read- back(confirmation) IF it fits the policy, call Shopify API for return Shopify API - track orders status, Return Merchandise Auth (RMA), exchanges Stripe API - Payments/Refund CRM - write/fetch deal pipeline, create/update tickets How The Router Agent Proceeds So here we're gonna introduce a gateway a gateway is going to deal with the authentication single sign on or PII which is for privacy of the user or you're gonna think through the rate limit how many times people can actually use this product request for this product and checking for duplication so all these kind of things are sort of we need to double check with the gateway let's add a bit of arrow to confirm this relationship before we continue to process any information we need to finish authentication first CRUD with Relational DB and also in order to load the previous chat and emails we need to query our data from a relational database remember we talked about the rectangle data tables so what happens here is basically user auth fetch the chats and emails okay How Would AI Agents Interact With User Chats we're gonna start think about how would the agent help with talking to these conversations from customers okay imagine a customer has just asked a question ask a question they said I want to return my product X. Respond to the user real-time Router Agent - understands user intent - decides agent routing Q&A Agent - speak to user all the time Decide to take the Return Route Action Return the latest status of return back to Q&A Agent to talk to the user Return Planner Agent - decides functional calling - Or, a deterministic workflow Vector DB (FAQs, Policies) RAG: Check Policy Via VectorDB Align with Policy Inform the agent Policy/Keywords: - Capability Tokens, RMAs, human approach, read- back(confirmation) IF it fits the policy, call Shopify API for return Shopify API - track orders status, Return Merchandise Auth (RMA), exchanges Stripe API - Payments/Refund CRM - write/fetch deal pipeline, create/update tickets so here for the router agent let's think about the edge case first right if the router agent realized that this customer is very angry like the customers say that I have I must deal with like or they ask a very very complex question and we think that our agent decides that it's way beyond the confidence that I have to deal with myself we need to have a mechanism to allow it to trigger looping the human back into this agentic system so that the humans will take over so that our overall customer satisfaction Ray will not be affected right so I'm just gonna add in this human in the loop Human Approval so now in this case the human will start to approve or disapprove any request from the users How The Router Agent Proceeds if the router agent decides now that this is a valid request and let me just double check if it fits with our policy so that I will process the rest of the steps for you then we will allow you to move on to the next step so the next step we're calling it a planner agent let me just move it back here Decide to take the Return Route Action Planner Agent - decides functional calling - Or, a deterministic workflow if it fits the policy call Shopify API for return Shopify API - track orders status, Return Merchandise Auth (RMA), exchanges Stripe API - Payments/Refund Payment Refund done after this the return planner agent should be updating all the context back to this Q&A agent because this Q&A agent is really doing the conversation with the original user channels which includes the web chat or the emails right The Q&A Agent is the CEO so imagine this QA agent is just like the central brain or the CEO of the company who needs to do all the communication with their customers and then the rest of these people are just part of the organization who's doing their task right just that the router agent is sort of at a higher level return planner is the one that the router agent decides to route to and then you could also route to Exchange Planner or you know where is my order planner all these kind of tools right but after this planner has done its job what you should do is that you should always update the information or the context back to the Q&A agent right so the Q&A agent is like OK so um return the latest updates return the latest status of return back to Q&A agent to talk to the user so this can also include situations when the question or the product doesn't fit into the policy for return in this case we will also update the latest information back to the Q&A agent so they will be able to talk to um the user right respond to the user real-time AI Agent System Design Overview Relational DB User Auth, Fetch Chats/Emails Gateway - Auth/SSO, - PII, - RateLimit, - dedupe Auth User Channels - Web Chat, Email Human Approval Router Agent - understands user intent - decides agent routing Q&A Agent - speak to user all the time Decide to take the Return Route Action Return the latest status of return back to Q&A Agent to talk to the user Return Planner Agent - decides functional calling - Or, a deterministic workflow Vector DB (FAQs, Policies) RAG: Check Policy Via VectorDB Align with Policy Inform the agent Policy/Keywords: - Capability Tokens, RMAs, human approach, read- back(confirmation) IF it fits the policy, call Shopify API for return Shopify API - track orders status, Return Merchandise Auth (RMA), exchanges Stripe API - Payments/Refund CRM - write/fetch deal pipeline, create/update tickets so we got a pretty solid system right here for the goal we have right so what do we still need to do Observability, Metrics, Evals few things No. 1 is that as an agentic system we should always be thinking about observability metrics and evaluations okay remember our goal is that we need 70% are automation 30% are looping back to human right and then the satisfaction rate should be over 4.5 and the response time should be um this much right so what we do is that we gotta track the relevant metrics across the entire system to make sure that these metrics are being met these goals are being met okay so that's the first thing the second thing is that it depends on um this current website or this current client is already dealing with some external like say CRM systems perhaps there should be some automatic triggers as well for us to write back to the CRMs okay so we could add something like this here a CRM system that will fetch write/fetch deal pipeline, create/update tickets CRM Integration Observability, Metrics, Evals User Auth, Fetch Chats/Emails Relational DB Gateway - Auth/SSO, - PII, - RateLimit, - dedupe Auth User Channels - Web Chat, Email Human Approval Router Agent - understands user intent - decides agent routing Q&A Agent - speak to user all the time Decide to take the Return Route Action Return the latest status of return back to Q&A Agent to talk to the user Return Planner Agent - decides functional calling - Or, a deterministic workflow Vector DB (FAQs, Policies) RAG: Check Policy Via VectorDB Align with Policy Inform the agent Policy/Keywords: - Capability Tokens, RMAs, human approach, read- back(confirmation) IF it fits the policy, call Shopify API for return Shopify API - track orders status, Return Merchandise Auth (RMA), exchanges Stripe API - Payments/Refund CRM - write/fetch deal pipeline, create/update tickets so again this is a very simple overview on how do you think about the agentic system um from a non technical background from a non technical perspective but also I know that I have introduced quite a lot of technical concepts here too but um if you are technical you realize that there are a lot of things that we're missing out right there are a lot of things that we didn't mention for example about scalability about things that are related to like the request per second seasonality all these server side of things I think at the end of the day if you're a product manager you would need to discuss with your engineering leader anyways to figure out these things out this basic concept of what we need in order to set up this product ready for production I hope this is helpful I hope you have learned something from this and let me know if you like this kind of format of video I'm happy to make more and I'll probably also be making some videos to explain how do you actually build a system like this with code so stay tuned if you like this video like and subscribe and make a comment down below thanks very much cheers