- Angie Lu
SPEAKERS
Will Bachman, Angie Lu
Will Bachman 00:02
Hello and welcome to Unleashed. I’m your host. Will Bachman, and I am delighted to be here today with McKinsey alum, Angie Lu, who is, among other things, the founder of case with AI, and she’s going to talk about that. Angie, welcome to the show.
Angie Lu 00:22
Thanks so much. Well, super happy to be here.
Will Bachman 00:25
All right, so we can, we’ll loop in your background. I know you worked as a kind of product manager at a few fintechs, and you were involved in doing recruiting when you’re at McKinsey, so you did plenty of case interviews, and you have, you’re the founder of case with AI. Found at case with aidotcom tell us about what this what this tool is all about.
Angie Lu 00:53
Yeah, for sure, it is a very, I guess, simple, straightforward tool. It’s a tool that basically help you with case interview. So the way that it works is that we have an AI powered platform. If you have a case interview coming up, for example, for consulting, you can come to our platform, practice cases, practice drills. We use AI basically to guide you through that that process, and we also would provide you with a very objective, very straight up feedback that you can use to improve your case performance.
Will Bachman 01:32
So and this didn’t just arrive out of nothing. I understand that you had during covid. I think you started a YouTube channel and doing some other content creation. Tell us. Tell us about that.
Angie Lu 01:45
Happy to talk. Talk through that. So it all started during covid. If we all remember what those days feel like, I wasn’t doing much, spending a lot of time watching YouTube videos. And I thought, hey, what about I try to start my own YouTube channel. It’s a new skill and it’s new experience. So I tested a few different topics. It turned out that consulting topics were really popular. There are a lot of demand for me to basically share my experiences at the firm, to share how I got into consulting. So I decided to then make a lot more videos about, you know, practical tips that people can use to improve on their interviews. And as a result, I got more, a lot more people coming in to ask for coaching and mentorship. Given that it’s just me, it’s not possible for me to actually help out a lot of people. So I just kind of kept producing videos as a way to scale my ability to help with more people, and then coming to 2024 23 that’s when the world got amazed by the ability of AI, what AI is able to do, and that’s an area that I’m really interested in. So one day, I was listening to a podcast about someone who basically built this tool called Chat PRD. It’s an AI tool that helps product managers build, you know, produce product briefs. But that person was able to build the platform in basically a weekend, I believe, and I was shocked at the speed of building, and I thought, hey, I already have the platform and case interview seems to be an amazing use case for an AI assisted, I guess, use case. And why don’t I build this as well as a way to both help a lot more people out there, but also as a way for myself to learn more about AI and experimenting with different AI tools. So that’s, that’s how I started. I also drive a lot of joys from building products. That’s why I also, you know, chose a career as a product manager after after McKinsey. So it’s, it’s basically, it’s a it’s a perfect world of combining my passion for building and my curiosity on my eye.
Will Bachman 04:05
And so just to put some numbers on it, I’m looking at your YouTube channel right now. You have my secret to creating a custom case interview, and that was 30,000 views. You have four step to market size anything in the world, best market sizing techniques with 86,000 views. Case, interview prep for dummies. There’s 140,000 dummies who watch that one. And there’s this method improved my case, interview success rate, 210,000 views. So, yeah, that’s performing one. Yeah, that was, that was the best performing so you had all this success, and you’re like, oh, okay, there seems to be some interest around that. Let me build this product. Okay, so now listeners, this is, of course, audio only, but what we’re going to do is Angie, I’m going to ask you to give me a drive through, sort of a demo of the product. I will narrate what I see. But also I’m going to ask you to I’m. You some screenshots and listeners, we’re going to post these screenshots on the permalink for this episode. If you go to umbrexdotcom/unleashed you’ll find the permalink for this episode, and we’re going to put the screenshots there so you can see what I’m seeing. Okay, so we got the homepage of case with aidotcom It says, AI powered case, interview, prep and walk, walk us through what we’re seeing, sure.
Angie Lu 05:27
So this is just a very simple landing page. Basically tells you what it is. And then we have, you know, partnership with top universities around the world. And we do have users who have found our product really, really useful. So we do have some testimonials from there and just couple quick things right, like of what the product is solving for. The key challenges, I think top one is finding the right case partners. Again, like when you do a case interview, you do need two people, and you need basically someone to give you the case and also provide feedback. But sometimes that could be challenging, and also because you have to coordinate with another person, then different times, different schedules can also be challenging. So then, with the platform, you can basically practice at any time, anywhere. I think the most important part is navigating consistent feedback when you have a peer who is acting as your case partner. A lot of times it might be the blind leading the blind. You might not know. Actually, both of you might not know what the consulting companies are looking for, where the bars are. So with our platform, you then get that very objective feedback, and you would know exactly what your strengths are and where you can improve, and obviously affordable quality coaching, it is going to be way cheaper than hiring a private coach. That’s going to cost a couple $100 per session. So let’s get into this. So if you just click on Start Free Practice, we have a very simple UI where you can either go for a full case practice, or you can pick specific drills. We’re still young. We only have two drills at the moment, but we are planning on launch a lot more drills that is targeted for more, you know, more specific case interview skills, okay?
Will Bachman 07:18
And you have market sizing drill. And I’ve, I’ve heard that that’s almost becoming the dominant thing right now is asking people do market sizing exercises and also a framework drill.
Angie Lu 07:31
Yeah, for sure. I actually love market sizing drill because I, to me, it feels more like a mini case. You do have to create your approach, you have do some calculation, you have to come up with insight. It’s basically, you know, a quick exercise, bit of a mini case, right? That that would test a few different dimensions
Will Bachman 07:50
that’s cool and, and can we just mention real quick, the pricing that you have right now, at least with the current pricing that you have on the website, yeah, for sure, a few different versions.
Angie Lu 08:04
We have a lot of options for you to choose from. If you just want, if you practice, we have pay as you go. You can buy credits. Or if you do want a monthly subscription, we have $50 per month, or 150 for six months, which is a 50% discount. It’s an amazing deal to be honest. Yeah.
Will Bachman 08:25
So really, if you just wanted to test out the tool and say, Is this for four credits for $4 you can do one case. So that’s that’s a pretty low cost entry point to test it out.
Angie Lu 08:36
That’s exactly correct. And then the other thing actually are the two things, is, when you do first come visit our platform, you can, like, we have three case that you can experience, right? So you fully try out experience before you even commit it to pay. Because I believe, you know, people should fully know what they’re purchasing before they make the purchase decision. And then secondly, if you’re not happy after you purchase a monthly subscription. You’re not happy with it. We’re happy to return your money within seven days again, just giving you that comfort of, hey, I know you know, like you’re spending your money on the right thing, on the right tool, that can actually make a difference for you.
Will Bachman 09:15
Awesome, all right, cool. Let’s, let’s jump into it. Walk me through. Maybe walk me through. You want to start with the drill, just to kind of show me what the shorter
Angie Lu 09:25
All right, maybe I’ll do a market sizing drill. Just click into it. We have a lot of different market sizing drills. If you’re new, we have a little tip about what the best best practices are, and then it’s, you know, sort of an industry difficulty level, and also whether you have done it or not, if we just click into, let’s say the first one, NYC
Will Bachman 09:47
food delivery. And let me just describe what I was seeing in that previous screen. You have. So these are not just made up by the AI on the fly. It you have some pre. To find ones that you know, presumably have been kind of vetted and looked at and checked out, and there must be at least 30 or 40 of these, and they’re, it shows you the name of it, like streaming for kids, and that’s an entertainment and lifestyle and that’s a medium difficulty, and we have one that’s outsourced HR Payroll, market sizing, that’s in financial services high difficulty. So you have, what is this? About 30 or 40. It looks like
Angie Lu 10:26
so, around 30 or so, and
Will Bachman 10:28
you can see if you’ve done them or not. Okay, so you have a bunch to choose from, so you could work your way through these. All right, okay. So now, now we can keep going. I just wanted to give listeners a maybe we take a screenshot so we can show that what what, what you’d get there, all right,
Angie Lu 10:43
yeah, for sure. And then one thing I also wanted to add is, you’re correct, right? They’re not, you know, generated by AI on the fly. Because one thing we did, that’s one approach that we did consider. But when we did that, we realized that the case quality, or the question quality was actually not that great. You do need a lot of, a lot more kind of a human guidance to create the right level of, you know, giving you the right level of information, as well as the difficulty level to make it a quality drill practice. All right. So, yeah, so we go into this. We have NYC food delivery as an example. So the case question is presented in front of you.
Will Bachman 11:24
I’ll just, I’ll just read it out. Okay? Your client is a venture capital firm considering an investment in a food delivery startup that specializes in healthy, organic meal delivery in New York City. Okay? The startups founders claim that they can capture 2% of the healthy food delivery market within two years, sounds like pretty high for New York before proceeding with the investment decision, the managing partner wants you to estimate the total addressable market for healthy food delivery services in New York City. Okay, ready to begin? You can either ask questions to clarify it, or you can start your response, okay, so keep now, keep going, yeah.
Angie Lu 12:07
So if you have any questions, right, you can just type here. And let’s say, if you’re new, you have no idea where to start, and you’re like, Okay, if I wanted to do this, maybe I wanted to understand what is the population of New York City. So you submit that to here, and we give you a bit of a freebie here, information as then NYC have approximately $8 million sorry, 8 million residents. They’re like, okay, maybe that’s enough, you know, for me to start. And then you can start your response basically typing in what you’re planning on saying. And if you have more question, you can basically toggle back and forth between response and asking questions. We also have a little timer on the top right corner. If you do feel, you know, want to do a little challenge, you can, you know, limit the total amount of time spent to, like, five or 10 minutes. And if you don’t want to see that, you can just simply hide it. So let’s say, in this case, because we have limited time, I will just say that, hey, I think, by the way, can either type or, you know, voice over, the response, which is really easy, but it’s going to be hard because we’re on the Zoom call right now. So I’m just going to type, let’s say, I think we will go with, I don’t know. Let’s say 10% 10% people of NYC is health conscious, so that’s a million times 10% Gosh, haven’t done math in front of someone else in time, and let’s say, assume like it’s, I don’t know health delivery meal is, I don’t know, $10 this is a, by the way, a terrible response. Do not copy this in real life. So let’s say 10 is just 8 million times 10 is, let’s just say a million, which is completely wrong, but I’m going to submit that, and then this is where we’re going to be preparing your feedback.
Will Bachman 14:25
Okay, so you’re gonna get feedback, all right? Exactly,
Angie Lu 14:29
feedback. It’s the best part of this entire tool.
Will Bachman 14:32
Probably gonna tear that tear that response, okay, strengths,
Angie Lu 14:36
of course, my personalized feedback, my overall rating, is, I need improvement. That is very obvious, because a lot of it was wrong. But let me just run you through this very quickly. We have kept it very simple. We have a three tier rating level, we have need improvement, and that means you need a lot of work, right? And then we have middle tier, which is meet expectation. That means, hey, you’re basically you. You know, performing at an average kind of candidate. And we also have outstanding, which means, basically, you have achieved about about like above the bar, and that’s what the top consulting firms are looking for. So what you’re really looking for is to get the right into outstanding, and then for this market size, and specifically, we break it down to 123455, different criteria. Each one, we give you a bit of a it will give you an idea of what your strengths are, where you’re lacking, and what you can do in terms of recommendation. So we have structured approach, which basically means you know how you are approaching the problem. I didn’t really have an approach. So obviously I had need improvement. We have reasonableness, which explains how reasonable your assumptions are. My assumption was not very reasonable, saying that 10% of NYC residents are health conscious, that’s just a very out of nowhere, assumption, obviously you need improvement, and then we have math calculation. Yes, my math was very simple, but it’s oversimplified assumption. So it’s the simple meet expectation. My communication is need improvement. It was very straightforward, but it was too brief to the point where you don’t really know where I’m heading. So again, something that we can do there. And last but not least, actionability and strategic business acumen. This is where you know you’re supposed to talk about the next steps, right? Like, what is the key insights that you would be able to get after coming up with this number, I did not provide any at all. So again, need improvement. So lots of improvement work needs here, but it gives you an idea of, if you’re just a candidate looking to improve, right, like the level of feedback that you will be able to get,
Will Bachman 16:56
yeah, and then, I mean, this is so much more objective and detailed than you’re going to get from almost anybody, but like a top 1% coach. I mean, maybe there’d be, I guess there’s probably human coaches out there who could do better than this, but they’re going to be very expensive and pretty rare. And I mean, because this gets really detailed about what the what, what the recommendation would look for. And then it looks like there’s an example answer that you could pop open and see what really, what does outstanding look like? And much more detailed than yours. Angie, so this really gets into a much more detailed answer. And then your friend, okay, and then as your framework, and then some context, all right, so which we had already seen? All right, so that exactly, that’s an example of just the market sizing, so you could prepare on that. All right, maybe, maybe, I don’t know if we can go through a full case, but, but, but, why don’t we start looking at one of those to see, see what that
Angie Lu 18:00
looks like? Sure, yeah, I can give you an example of what the full case look like. When you do a full case, we have two modes, because I believe, you know, when you’re at different experience level, the kind of the your experience of practice should be very different. We have a progressive feedback mode. It’s perfect for beginners. We basically hand hold you a little bit going through each one of those questions. And we also have an interview prep mode, which is more of a real life interview simulation, so you will go through, you know, kind of verbally communicate your answers responses with an AI interviewer. And this is the interview prep mode. Is a very popular one. I can prop this open and then, to give you an idea, it might be hard for you to hear what the actual prompt, sorry, what the actual case interviews are, because it’s all given, you know, voice, voice over by the interviewer. But this is as much as of a real interview is. Let’s see if we can hear that.
Will Bachman 19:04
Okay, not sure if it’ll come through. Okay. Speaking, you’re hearing it. I’m not hearing it.
Angie Lu 19:09
No, it’s, it’s really tricky when, when I when you have also a zoom, all right, okay, here, right now,
Will Bachman 19:15
now you can pause it, but we’ll just say, I can see on your screen, there’s an interviewer speaking. So that’s kind of interesting. So it’s not just a written like the market sizing. This is really role playing real life, where it’s oral, right? That’s quite interesting. Okay, so you actually, you know, have this discussion back and forth. It looks like you can ask a question ready to proceed. And that’s really neat, that you have this progressive mode where you can kind of get help along the way, instead of just going to the full end and then evaluating you so you get hints and tips and and recommendations along the way. That is exactly correct. This is I haven’t actually seen an AI learning tool like this yet. Should. Is how behind the times I am. But this is so amazing to think about just the future of all sorts of professional development. You can imagine applying this to a wide range of any kind of professional skill, where in this kind of safe, controlled environment, you’re getting to test your interaction. You can imagine doing this, of something like this, to practice giving a board presentation, right, and have it role play like here’s different questions the board might ask you, have you react to them and then have it evaluate your responses. It’s really quite incredible thing. I’d love to hear so I can imagine that the case interview would proceed sort of similar. And is this a text version of it I’m looking at
Angie Lu 20:47
That’s correct. So the progressive feedback is more of a text version, because our assumption is, when you’re a beginner, right, it’s going to be overwhelming to have to listen to the audio and then taking notes the same time. So that’s why we’re saying that this is a more of a guided experience of like hey, you can see everything, and then with each question, proceed with each question, you can also basically see what the answer is before we move on to the next one. And then in the end, you get this overall feedback. Actually can show you probably a what a sample overall feedback would look like for a whole case. So yes, let’s say in this case, right? I have another case called soyco, and you can see that very similarly, we have, you know, for your framework questions, we have a lot of the different ratings, and then we have, then your second, which is quality of analysis, we have various ratings. So, yeah, you get just, I guess, a lot more feedback based on each individual questions response.
Will Bachman 22:02
So it’s really a learn by doing approach here you just, sort of just is. It’s kind of an interesting idea. It’s not even any explanatory Metro just like, just jump in and start and you start getting feedback, and then you can use that on the next one and the next one. What was it like building this thing. Tell us about how you actually created it and how much kind of iteration you had to do to tweak it in,
Angie Lu 22:30
yeah, for sure, really good question. I’m gonna stop sharing for a moment. Yeah. So we started off with a very simple MVP. Basically only had 10 cases on the website and the UI and everything, looking back today, wasn’t just not as smooth, not as good, but we wanted to, you know, launch something very quickly and then really test out if there’s a market, if there’s an appetite for this. And it turned out that there was so have that understanding, we decided to then kind of experimenting with, you know, different pricing point, with different products, with different UI experiences. So it is a very quick kind of iteration, iteration process. It’s this, I think this is the nice part about building a, what I want to say, like a small product with a very nimble team, right? It’s just basically me doing it part time while raising my seven month old, and then two other person also working, also working part time. But we’re able to move quite quickly. I believe, because of one we’re very small team, so we’re able to move very quickly, kind of implement things and test and test things really quickly, but also with a lot of the AI tools that’s made available, I use a lot of AI vibe tooling, basically to do prototype. I’m able to make like, really nice prototype within just a couple hours in an afternoon. I also use a lot of I use Cloud and chat GPT to basically create a lot of like, marketing materials for, you know, for social media, support, email communications. So it really, like AI has improved our productivity, like, so much to the case where, you know, like this is so much easier to build than you would have been. You know, even couple of years ago, a couple years ago, would have probably taken like six months to build the whole thing. But now it’s shortened by like 50% 60% of time.
Will Bachman 24:31
How did you What did it take to train the AI kind of, what to look for, and what good looks like? Did it already have enough stuff in its brain about case interviews, from everything published on the web, that it was already pretty good, or did you have to do a lot of refinement, training, of saying, you know, of actually, I don’t know, getting volunteers to go through the tool, and then you had to give the feedback and train the AI, what is outstanding? What is meets? The bar, what is below average? Like, how did you get that trained up?
Angie Lu 25:04
Yeah, for sure, actually getting that training right is where we spend majority of our time. Because I believe that is where, like, the key value add is for this platform. So we spend tons and tons and tons of time. Excuse me, using like prompt engineering, like, also using like our past User Responses to really train the AI granted that with the general LM model, it’s getting better and better and smarter and smarter, right? Like, so it already have, like, a lot of really good base in terms of, like, what you know good answer could be or like, what you know how you should solve a case interview. However, the problem, a couple problems, that we found is one, LLM model still tend to be very agreeable a lot of times. It will just make you feel really good about where you are, right, like with your with your skills. And the second piece is that it’s not very targeted, because, again, like a lot of the information out there about case interview evaluation is still quite general, like no one has ever published. This is how McKinsey Bain, BCG rate you in, you know, in whatever criteria, right? Or like, what this is, what kind of outstanding basically looks like for them. So a lot of those knowledge are being held by, you know, people like us, people who have been interviewers right for those firms. And this is where we can apply our knowledge to train the model to be a lot better. And I think also the other part is, you know, really train it so that you can not you don’t overwhelm people with, like, a ton of feedback, but you also are giving them enough where they know exactly how they can improve. Is also something that we are very mindful during our training.
Will Bachman 26:57
Wow, that’s very cool. Seems to me that this is the future of education and professional development. Is tools like this that encode that tacit knowledge, and then can can give people feedback so you’re learning by doing, and not by just sort of sitting and having someone talk at you. It sounds like about bringing this to beyond consulting to other industries, right, where they also do case interviews?
Angie Lu 27:29
Yeah, for sure. I just personally found that case interview is such a popular and, by the way, a really great way, great interview format. So beyond consulting tech, also use it quite often for roles like operations, product management. All use, you know, some different format of case interview. The general framework is pretty similar. Where you’re presented with a problem, the specific questions might be different depending on the type of roles that you would apply for. So I do see that there are, you know, a lot more than just consulting that you can cover with our tool. Yeah.
Will Bachman 28:09
I mean, even training frontline managers, let’s say, and you know how to be a manager, you just promote someone who’s an associate to manager. They’ve never managed someone before. This tool, you know, like a version of this tool, could help people, okay, an employee comes up and they need the time off on Friday, whatever you know. How do you deal with those sorts of situations? It’s a you know, can even expand situations like that. But I guess there’s other industries like product development, even right. Product Management, probably investment banking, other professional services also use case interviews. So a popular method, cool and tell us a bit more about you know, you already had your you talked about your YouTube channel. How else have you been getting the word out?
Angie Lu 28:59
Yeah, for sure, YouTube definitely helped quite a lot, just because I have already a baseline. By the way, one interesting insight that I didn’t really realize before I started building this on, kind of on my own, is that marketing matters as much as product, if not more, as a product manager before I previously, I thought, hey, if you just have a great product, right, people will naturally, somehow learn about it, and they will float over and, you know, come and use your product. But that’s just not the case these days, because there’s so much information out there, right? You need to make sure that you identify the right user, and then you inform those users that you have this great product so they can come and try. So anyways, we spend kind of 50% time on product and 50% time on Marketing to just really make sure that, hey, people can hear about this tool and they understand that this is something that they can use during the earlier during the early days. What we have tried use, obviously, social media. So we have YouTube, we have like Tiktok. Talk, we have Instagram, and because I’m Chinese, I’m also, I’m also on the other Chinese social media sites as well. So those contents are really great at generating leads. And Reddit is another really good, great platform. A lot of people do post questions about case interviews, about consulting, and then we go in there and we provide answers. And we also have tried a lot of university partnership just because that’s, you know, students are aware consulting firms recruit quite a lot, so we reach out to a lot of consulting clubs to provide free workshops, give them an opportunity to try out the tool. What’s also really interesting was that in the early days, when I was talking to students, a lot of them don’t even realize that you can use AI to do this. It’s they’re still relying on, I guess, the more traditional method of either getting another friend right to practice case with or they going to spend a couple $1,000 to hire private coach? So this is really a new method of actually preparing, and I think it’s definitely gaining popularity. One final one that’s getting also, that’s also contributing quite a lot of leads for us, is AEO answer engine optimization. What actually means is just when you you know search on Google or chat, GBT or perplexity these days, right? Those search engines use our models to actually provide you with a list of summaries. So if you go in, you know, what’s the best what are some of the best case interview tools powered by AI? Kids with AI would hopefully pop up as one of the the suggestions, and, yeah, the reason why we’re able to, you know, show up in those answers was also because of a lot of the foundations that we have built in producing materials, in being really active on Reddit, being really active on on Social media, and really kind of being out there.
Will Bachman 32:02
Amazing Angie. All right, once again, the website is case with AI, one word that’s one worddotcom case with the aidotcom We’ll include link in the show notes, and if you go to umbrexdotcom/unleashed find the permalink for this episode, and there will be some screenshots so you can see what I was looking at. All right? Angie, thank you so much for joining today.
Angie Lu 32:29
Thanks a lot for having me. You.
