How Duolingo Built Duolingo Max

We started working on these AI features in late 2022. That was the time where we got early access to chat GPT which was significantly better than the previous LLM and we asked ourselves what are some language learning features that we had always wanted to build but we couldn't and in our case that was giving personalized feedback on your mistakes and also having conversations for video call in particular once we launched it as part of the Mac subscription. That was huge for us in terms of monetization. It was huge. The combo of making the features better, scaling them, and also adding video call. That's when we really started seeing the the bigger impact of Dualingo Max. Welcome to Subversive, a podcast dedicated to sharing stories from the best consumer subscription apps in the world. We'll bring you lessons for how to grow your consumer subscription business, including insights and inflection points that led to

exponential growth from leaders at category defining companies and innovative startups. Let's get into the show. My guest today is Natalyia Castjo, who has spent the last 9 years in product roles at Dualingo and who is now a director of product leading monetization. In this episode, I talked to Natalia about her team's work on AI powered core product features like video chat, interactive role plays, and explain my answer. We also discuss her team's more recent work on packaging and monetizing these features under Dualingo Max, the company's new higher price subscription tier that it launched in March 2023. All right, welcome Natalia to the Subversive podcast. It's such a pleasure having you here and talking Dolingo today. In many ways, I probably should have started with Dolingo. I feel like you guys are one of the most successful consumer subscription apps in recent memory, but I'm glad we're

talking about it now. Uh, it's a product that's loved by many, including my own family. Uh, my wife and daughters and I have been studying Spanish and Portuguese on Dolingo for an upcoming trip to Brazil this summer. So, thrilled to have you on the podcast and excited to talk more about your tenure at Dolingo. Um, let's just get started with a little bit more about you and what brought you to Duolingo in the first place. Yeah, so uh, again, my name is Natalia. I'm a director of product at Dualingo. I've been at Dualingo for about 9 years. I actually started out as an intern here. And originally what brought me to Dualingo um is I just I I love the product. Uh and I also I was very much a fan of the mission which is to make the best education in the world and making it universally available. My native language is Spanish. I learned English in school, so I learned firsthand how learning a language can

really transform your your life and your opportunities and I I just loved uh the idea of being able to contribute to that mission and doing it at scale. So that's uh what originally brought me to Dualingo. Uh again, I've been here for about nine years and during my whole time here I've been a a product manager. I started out as a as an intern actually and that was actually 10 years ago and uh now I'm a director of product. I lead the monetization pillar at Dualingo.

Wonderful. Yeah. So you've been at Dolingo for almost a decade if you account count your internship uh which is unusual in tech. And so you've worn a lot of different hats and you've seen Dolingo grow and evolve. I guess paint the picture for me as an intern. You went to Yale for undergrad. I think you studied math and philosophy, but it sounds like language has been a very important part of your journey. What was Dolingo like when you joined as an intern? Like how many people were at the company and and what did it look like?

Yes. Yes. When I joined as an intern, so Dualingo uh they launched in 2012. So I joined as an intern in 2015. Uh so back then we were a small startup. Uh we were about 40 people based in Pittsburgh, Pennsylvania. Um that's uh where the company was founded. Yeah, we were just about 40 people in a small building. It was just a single floor. I remember it was you know we had barely over 1 million DAUs. Whenever I would tell my friends, so I I work at Dualingo. Most people would be like what is that? I've never heard of it. And I guess uh things have changed a lot since. I was the second PM uh to be hired in the company.

And now uh we're about 900 employees, we're a publicly traded company. So uh now we not only just teach languages, we've also started teaching math and music. So a lot has changed uh since I joined. Awesome. Tell me about a few of the different product roles you've gotten to play at Duolingo and maybe one or two of your favorite challenges that you've gotten to tackle as a product manager and now product leader there.

Yeah. So, uh, when I first joined Dualingo, I actually worked on our Dualingo English test, which is, um, you know, it's, uh, you can take a test online and, uh, get certified, uh, you know, you get a Dualingo score. So, so that's, uh, the first project that I ever worked on. Uh, for most of my time here, I actually worked on the core learning experience. So I worked on our home screen uh re redesigning our courses. I worked on content creation making it more efficient also pivoting to um AI a couple years ago u meaning uh AI uhdriven content creation as opposed to human-driven content creation. I've also worked on uh our lessons and creating new language learning features.

That that's been uh the bulk of my time here. Um, as of the last six or nine months, I've been I've been working on monetization. Mh. And uh, one of the projects that I've found more interesting to work on recently has been video call, which actually started off when I was in learning leading the learning pillar and then ended up being our kind of our hero feature for a max subscription, which is our top tier. And we essentially we launched it just as I was moving to monetization. So I kind of managed to see the development of that feature from like the 0ero to one aspect you know before it was launched before it was monetized all the way to you know launching it as part of our max subscription and you know monetizing it.

So so that's been uh pretty interesting. Got it. So you you've had a few stops and the English test is a particularly interesting one because I know that's another revenue stream that Duolingo added many years ago, but I think there's a great segue into the primary topic of the call today, which is that having been at Dolingo for many years, you've gotten to be both on the core product side and more recently on the monetization side and specifically on Duolingo Max, which is Dualingo's second and more expensive subscription tier.

You had Dolingo plus which is now Dolingo super which is sort of your core tier but now you've added a number of features particularly your AI power features to Dolingo max. And so for those who don't know Dolingo max let's

just provide a little bit of context to start like what is Dolingo Max and what are the features that really define that higher price subscription tier. Yes. So essentially the way the main way that we monetize Dualingo it's it's two parts. So, first, our core app is free for the most part. Uh, you do have a set number of hearts. So, every time that you make a mistake, you lose a heart. And you start off every day with about five hearts. Um, you know, if you wait 24 hours or so, then you can get all those hearts back. But you can keep using the app um indefinitely until you run out of hearts. uh at which point you can choose to either watch rewarded videos or you can subscribe uh to uh Dualingo Super or Dualingo Max. Um and then the other part uh we also just have ads at the end of every session. So that's another part uh another way we monetize our free users.

Dualingo super it essentially gives you unlimited hearts and no ads. Max also gives you unlimited hearts and no ads, but it also offers a couple of additional learning features which are powered by AI and those are explain my answer which are is essentially personalized feedback on the mistakes that you're making in your lessons and also we have a couple of uh conversation features. We call them video call and role play. video call. It's a feature where you can essentially have a conversation with a friend. In our case, uh her name is Lily. She's one of our characters and you can kind of have a conversation with her at, you know, at your level. She adapts to the particular level of your course. So, if you're a beginner, the conversations are going to be pretty simple. If you're more advanced, she's going to be using, you know, more difficult vocabulary, more complex sentences. And so, it adapts to

your level. So that's video call. And then role play, it's more around, it's also about conversations, but it's more about, you know, the types of, it's less free form. It's more the type of conversation that you would have when you're traveling and you go to like a coffee shop, for example, or you're at the airport. So role play helps you prepare for those types of conversations. Got it. So you've got three features.

you said, "Explain my answer." That sort of goes through your response and you can sort of get additional clarification on why it's correct or incorrect and how to improve it. There's video call, which is currently video call with Lily, your sort of emo teenager character that a lot of people love or love to not like so much. And then you've got roleplay, which are these sort of dynamic context based experiences like I'm going to a restaurant or I'm at the airport and and it helps you in a more practical way.

Um, let's we'll get into more details on these specific features in a bit, but I'm curious to step back first and ask

what was sort of the origin story behind some of these features? Like how long ago did Dolingo start developing some of these features and what was the impetus for those investments? Uh, great question. So, essentially we started working on these AI features in late 2022. That was essentially the time where we got early access to essentially chat GPT which was significantly better than the previous LLMs. Uh so we got uh early access to it and uh we asked ourselves you know what are some language learning features that we had always wanted to build but we couldn't because either you know it was too complicated to do it you know with static content or you know it would have just required uh humans uh to do it right. Mhm. And in our case, uh, that was essentially kind of what I just mentioned, giving personalized feedback on your mistakes and also having conversations.

I mean, we could we did have a couple features in the past uh that tried to do this without the same level of personalization, but then what we found is that when when the feedback maybe wasn't specific enough or when the conversations were too structured, it it just was it was just not working. It was a little bit too constrained. Mhm. So we had tried to solve these issues in the past without AI but it just became a lot better uh once we had access to to Chad GPT. So that's kind of how it started.

We just realized look with LMS with Chad GPT we can just build much better uh language learning features. And so so so we did and so one of the key points I heard there was personalization. Personalization is really important when you're trying to replicate the experience of having a conversation in another language either with a human tutor or with just another individual um in a country you're visiting. And so previously, prior to 2022, when chat GPT made itself available to third party apps like Dualingo, the personalization just wasn't there. And so that unlocked this opportunity to create features like roleplay and video chat. Where do you

see personalization evolving with AI? Because I see so many consumer apps playing around with personalization right now. It's really powerful, right? Like we're all different people with different wants and needs. Uh where do you see it evolving and and maybe specifically in the context of Duolingo to the extent you can share. Uh what sort of other opportunities do you think it might unlock? maybe kind of starting with the first starting with the user need like what is what is it that users want rather than kind of starting with personalization or starting with AI uh we've we've found before that if we just start with like the technology and then try to build a feature around it that kind of doesn't work so well because then you're building something that's maybe a little bit too sophisticated or too advanced for what the act user actually wants to do. I think in our case it's just the the better um the

conversational AI gets and the more intelligent these models get the better conversations we're going to get to build. So, in our case, at least right now, one of the things that we're trying to get better at is making sure that uh Lily uses the language in the level that, you know, uses languages and words and grammar that you can actually understand that's like appropriate for your level. So, right now, I think she does this pretty well, but not at the point where we want. So that's uh that's one thing that we hope uh to improve. Um and then the other part honestly for us it's um it's memory. Our idea behind video call with Lily is that the conversations that you're having with her are so realistic, so engaging that actually she feels like a real friend.

And so she's going to be asking you about the story you you told her the last week or she's going to remember, you know, what your favorite hobbies is. And she does remember to some extent right now, but it's not it's not there. It doesn't quite feel like talking to a human. I think for us, it's it's how can we make those conversations with Lily feel at the right level and also make her feel like a like a real friend. I think that's that's what we're focused on right now.

Yeah, that's so powerful. We just started working with a client with my advising business called Tolen, and it's an AI companion. It's like this little alien character that gets to know you and adapts to you and sort of helps you solve various problems in life or just somebody you can open up to. And I've only been using it for a few weeks now, but you know, the character has started to understand who I am, what I do for work, uh a little bit more about my wife and my daughters, Kinley and Lyanna. And when it recalls these things, it's so powerful. Like it has a pretty transformative effect on my feelings toward the product. And so that's something I hadn't thought about in the context of language learning, but it makes a lot of sense. Like streaks, badges, leaderboards, those are motivating. But if you have a character you're speaking to who actually knows a little bit about who you are and can

adapt to your needs, that's really powerful. Yeah. No, 100%. And that's I guess that's part of, you know, the way that we try to build language learning features at Dualingo. It's not just about the the function. It's not just about okay, I'm learning these words. I'm learning this grammar. You know, the hardest part about learning a language is actually keeping yourself motivated over a very long period of time because language learning takes years. You know, it doesn't happen overnight. And so for us, it's it, you know, it's it's about teaching effectively, but it's even more important to actually keep you motivated. And so for a video call with Lily, we're really obsessing over how can we make this very engaging. Like you actually genuinely want to talk to Lily.

You're not just coming to the app because you want to learn a language. You're actually just having fun. Yeah, it makes sense. Well, and motivation has been one of Duolingo's superpowers over the last decade. All of these tactics to keep users engaged and retained and AI feels like it opens up a lot of opportunities for that. But I want to return to one thing you said there, which is I think where a lot of consumer apps in particular have gotten in trouble is they start with the question of how can I use AI to build cool stuff.

And what ends up happening is technology in search of a problem that the customer doesn't actually have. And so something that Duolingo has consistently been very good at is starting with the user problem and working backwards from there to what the ideal product solution looks like. So, walk me through as you and your team were working on these various AI powered features, role play, video chat with Lily, and explain my answer.

What were some of the techniques you used to validate that you were working on the right things? I think as I said before, we knew there were certain parts of language learning that we were just not we just had these big language learning gaps in the app. So one of the one of those was grammar, you know, giving you personalized feedback on on your mistakes. Another one was conversation, you know, being we have speaking we had speaking exercises in the app, but they were very, you know, about repeating something that you just heard or reading something out loud. We did not have that many opportunities for you to uh do more of like this free form uh speaking, free form conversation. And so we already knew we had these gaps in the app both from the kind of from the learning perspective and also you know through user interviews through service we knew that people wanted more of that.

So so we knew we had those gaps and we started there. Now also you know when we started developing some of these features we sometimes did fall into some of these like traps where for example when we were building the first version of EMA explain my answer. So the the feedback personalized feedback feature, the first iteration of it was actually a chatbot. So you could kind of chat with this like duo coach. Um, and that version was it it wasn't that good because you essentially would have to put so much effort into asking questions. when you're like using Dualingo, you don't want to be starting, you don't want to start to type and like ask this like very specific questions to a chatbot. You just want to be like, why do I get this wrong? And so, we actually ended up simplifying that part of the product a lot. So, instead of being like a chatbot, you're asking, you know, this AI tutor questions, we would just give

you the answer like, look, this is what you got wrong. We already know your answer. We're just going to tell you exactly what you got wrong. There's no no need for interaction at that point. You don't need that. Got it. And how did that surface in the user research you were doing? Did you have users telling you in interviews that it was too much work? Did you use user surveys to unlock that insight? Or was it more quantitative? Like you looked at usage data and you said, "Well, people who are using the explain my answer feature are ending up churning at higher rates because the amount of effort they're putting in relative to the value they're getting back just doesn't add up." Yeah, we use all of those. So we do user research. So we interview users. Um we also uh we run a so everything that we do at Dualingo, every product change, we run it as an AB test everything. So even if it's a bug fix, we will run it as an

AB test first. A to like understand the impact of the bug, but also you know to to just like learn for everything that that we do. And also on a a final thing is a nice thing about working at Dualingo is that we're all users of the product. We are everyone here if you're a product manager if you work on the product you are using the app every day and you're trying to learn a language yourself. So you end up also developing an opinion just as a user. We essentially look at all all those things like external feedback AB tests. So metrics are people in this case, you know, are they using EMA more once we switched uh to this, you know, different model I just talked about versus when it was just a an AI coach. Um then we we look at UXR as well. Got it. One

question I'm getting asked by a lot of product leaders and product managers right now is what sort of AI tools they can be using to improve the efficiency of being a product manager. And so to the extent you have thoughts on this, are there particular tools or processes that your team has adopted that is making their workflows more efficient because of AI? Yeah, this is a great question and we're actually talking a lot about this internally. How can we get PMs to adopt AI even more in their day-to-day? There's like different ways that we're looking into this. So an example is we're we're trying to see so for every AV test I I just mentioned everything that we change in the app we run as an AB test and as part of that AB test at the end of that AB test we do an experiment analysis so we look at the key metrics and then we create a report with learnings you know what are what were our hypothesis when we initially

run the test what do we actually see in the data what are our learnings and next steps and so one of the things that we're trying to do is is there a way to use AI such that we can uh when we're we can pitch an idea uh to to the AI bot and then it will look at all of our uh existing experiment reports and then provide you know some suggestion or even just link to relevant AB tests that we run in the past because we have you know over a decade of AB tests at this point and reports that uh we can leverage so that's one part how can we use AI in experiment ment analysis and and looking back at experiments we've run before essentially using GPT uh on our day-to-day like for for Slack for writing specs um internally we've also used it for prototyping so there's actually a PM on the team who uh was uh working on on a new initiative but uh at the time because of resourcing uh he didn't have an engineer on on the team

he just had a designer It was him and a designer. And initially the designer was trying to uh develop this specific feature on Figma and creating a prototype on Figma. But it over time it became limited. And so what this uh PM ended up doing is I I think he used Replet to just really quickly do a prototype like a real prototype of uh the app and that essentially accelerated uh the prototyping or the ideation stage without the need of an engineer. So um we're also trying to think how can we get more PMS uh leveraging you know resources like replet to accelerate like ideiation uh for especially for 0ero to1 features or more complicated products.

Yeah. Yeah. That's amazing. I mean I've used replet and lovable a bit and then you have tools like versel or vzero for design prototyping and even the base LLMs like chatbt is getting more and more advanced when it comes to some of this functionality. So, it's pretty incredible how fast the field is advancing these days. Um, well, let's go a little bit deeper on each of these three AI features. So, video call, explain my answer, and role play. I guess maybe one question I'll ask is can

you give me an example of a specific moment and it could be something you saw in the data or it could be something a user said in an interview but what was a moment where you knew it was working across one or all of these features where you knew that the investments you were making in AI were really starting to pay off. Yeah, I mean I would say for us for video call in particular once we launched it as part of the Mac subscription that was huge for us in terms of monetization. It was huge. Um so there was something that was resonated about video call again. Uh we at the at that time when we launched video call we actually we already had roleplay and EMA as part of our max tier. So that was already there. Um, and role play and video call are very similar in that they're both targeting speaking and conversation. Yeah. So they're all kind of very similar features, but there was something about

video call in particular that was resonating with learners a lot more than roleplay. So, there's something about calling Lily, calling a friend that feels very different to users than having maybe a more like generic conversation to prepare you for, you know, your trip to, let's say, Brazil. And so, I think for us that was probably a pivotal moment once we saw that people were really responding well to video call uh and we essentially we were looking at it in the data. Yeah. Yeah.

Well, we had uh Ravi Meta, who was the former chief product officer of Tender on the podcast a couple months ago. He's actually one of the first guests. And one of the things he talked about was the importance if you're going to have multiple subscription tiers, the importance of ensuring that each subscription tier has a very clearly defined and differentiated premium value promise, right? Because if they all blend together, then it's it makes the purchase decision more complex for the consumer and in some ways it can actually hinder conversion. And so what

I hear you saying, but keep me honest here, is you already had Dolingo Super, which had a compelling set of features around more traditional language learning, but Duolingo Max has really become the AI plan that offers specifically these conversational features and they start to mimic immersion in the real world. One of the things you hear from any language teacher professor is it's one thing to sit in the classroom and learn, but you really need to get into the country or the region of the language you're studying and immerse yourself with other people speak the language. And so is that kind of how you guys have been thinking about it? Is is maths really a better way of moving beyond classroom learning into more of these immersive experiences? Yeah, I think that's what it ended up turning into. I will say that I don't think it started that way.

I think it originally probably started more as like, hey, we have AI, let's create a tier around that. Uh, but I think over time we're just seeing more and more that people really react well to this immersion uh, value prop. And so we're trying to double down on that. So now a lot of our Max ads are actually around speaking and conversation and focus more on video calls than say like our explain my answer feature. That makes a lot of sense. I guess another question this raises is are you seeing differences in terms of the user segments who gravitate towards Duolingo Max versus your free users and your Dualingo super users? Yes. Yes. So um I think in particular it seems like it's resonating better for English learners and then more generally for more serious learners. So I think if if you're like a casual dualingo user, maybe super is enough for you because you're you know you're kind of doing it every now and

then. You're not like super serious about learning this language. But we do see that English learners who may be more interested in learning English for like you know their their jobs for for their careers tend to react more uh positively to Max than uh maybe more more casual learners. That makes a lot of sense. And I've worked with some language learning clients and so I've seen this before where obviously it's not a perfect overlap, but a lot of international individuals who are learning English are doing so for their work or for their careers. And so the stakes are just higher. And it sounds like that's what you're seeing is people are willing to pay the higher price for Dualingo Max if the return on investment, which in this case is career outcomes and earning potential are there. Um, so I think this is a good

segue from the core product work you did with your team to build explain my answer video chat roleplay into the monetization work you've now started to embark on over the last 6 to9 months. Let's start with your own personal journey and then we'll get into um more broadly how Dolingo is approaching monetization of Dualingo Max. So what drove your personal transition from a core product leader to a monetization leader and what changes have you had to make as a product leader as you've made that shift? Yeah, so um yeah, I joined monetization in August of last year. At the time essentially the pillar lead of monetization uh went on parental leave was going on parental leave for a few months and I was asked to cover for him while he was gone. So kind of just happened by uh I wasn't actively trying to uh move pillars. Um eventually you know it was decided that I would that we would switch roles. Uh we felt like it's it's

good uh for uh pillar leads to be experiencing other parts of the company like kind of cross-pollinating ideas. Mhm. Um but I think joining monetization for me has really exposed me to a very different type of the business and a different type of product work than I was used to. So my background I guess again again was like on the core product more like 0ero to1 types of features. Um and while there is a metric component to learning that you know there's some proxy metrics that we've developed um to you know for AB test to measure how well we're teaching for the most part um you know when you're working on these like large learning features success is not as clearly defined as in monetization and so that's been that's been interesting like that's been a part of the big shift you know monetization we have very clear success criteria decision making is a little bit more objective and also I found that the pace

of experimentation is much faster because you can dedicate a big part of your product road map to this like small optimizations. So at Dualingo we can see that you know a small copy change on on a call to action can actually lead to a big increase in bookings in learning you're not going to see that type of change. So, you're not going to invest on these like super micro changes. And so, that's that's been new. Um, and I think it's a lot more fast-paced. You know, you you you have metrics, you have goals that you have to hit each quarter. Uh, learning was a little bit more more flexible in that in, you know, it's it's zero to one, you know, it's it's more like zigzag. you know, we we're not uh necessarily expecting specific result within a very tight uh time frame.

That's been also interesting. There's pros and cons uh for each Yeah, I'm not going to put you on the spot and ask you which one you like better, but can you go through how some of the pros and cons that you've experienced like like what Let me ask it this way. Which aspects do you like most of each role? Core product and monetization. Yes. So I would say for core product it's um I loved being able to kind of tune into my product sense my you know my my more creative side. you have a lot more time and a lot more space to just like really really think about what is it that you're trying to build you know do UXR uh take your time in developing 0ero to1 features which is also which is very very exciting and also you're working on at least in the dual lingo case you're working on the core the core lessons the core learning experience for monetization um what I like is uh you are doing some of that so you're doing

some of the zero 0 to one work. But then most of your most of our time at least is spent more on like the optimization world. And what I do like about monetization is the success criteria is a lot clearer. So there's a lot less time spent uh getting alignment. It's if something is making money, y we can all point to the metrics dashboard and be like, okay, this we should launch this experiment because it's moving bookings.

In learning, the answer wasn't always that clear. And so I I I like being able to I like the quick iteration cycles and the quick learning cycles from monetization. We didn't have as much of that in in learning. Yeah, that's something that I can certainly relate to. So I led the consumer core product team at Ibata early on in my product career and was doing a lot more zero to one feature development and then when I moved to Quizlet I helped build the growth product team and then ultimately the subscriptions team and it really is two very different disciplines right one is much more of an art and the other one is a little bit more of a science and there are pros and cons there I'm curious what are to the extent you can share what are some of the most important KPIs or metrics you're looking at now on the monetization team where it is so explicit I could guess but I'm curious what you all look at on your

I mean the main thing so we mostly try to make experiment decisions based on for the most part based on bookings. So is this moving is this change moving subscription bookings? Is it moving ad revenue? And of course, we take into consideration like the long-term effects of that. And we also look at our guard growth, right? Like uh making sure we're not hurting um engagement or or learning. But for the most part, we just look at subscription bookings and ad revenue. Makes sense. I mean, that's what's driving the bottom line. So, that makes a lot of sense. Okay. So, now we've talked about your journey and some of the pros and cons of being a core product versus a monetization leader.

Let's talk about what this has looked like for Dolingo as a company. So, when

did the company officially launch Dolingo Max and how did the initial launch go when it first came out? Yes, we launched Dualingo Max actually in 2023 and uh yeah, May 2023 and it wasn't necessarily an overnight success. Um we we had seen some positive you know indicators. However, um we at the time again we only had explain my answer and also we had uh roleplay and I think our main blocker back then is that we had only launched it in a small number of uh countries and courses. And so while we had we were seeing some positive results, we still hadn't quite seen like the full impact of that tier that uh it took us a little bit of time. It was actually last year when I started joining uh monetization that we started scaling max to more and more tiers. And so I think we we there there there was a non-trivial amount of time dedicated to improving the features and also making them scalable before we

actually started seeing more more of the impact of Dualingo Max and then uh together with video call launching video call last year I think that the combo of making the features better scaling them and also adding video call that's when we really started seeing the the bigger impact of Dualingo Max. Got it. And I feel like that's the typical story, right? So it sounds like you launched Duolingo Max for the first time in early 2023. There were some early signs that you were moving in the right direction, but it wasn't an overnight success. And so it took adding video chat with Lily and then sort of refining some of the other components of the subscription on the margins for it to really take off.

Can you say a little bit more tactically how things like pricing payw wall uh and maybe some of the uh premium feature merchandising for Dualingo Max has changed over the last year or so? Yeah, I think one of the things that we were struggling with a bit is how so again so we have super and we have max. it it was not always very clear to the learner what was off offered on super versus max like our tier differentiation was not very clear.

So that was something that we invested uh quite a bit in and and we also tried kind of rebranding uh both uh both tiers at some point that also quite wasn't really landing and I think uh for us the big unlock was when we started pivoting away from branding Dualingo Max as a this AI tier towards more like it's like conversation tier. I think that was very uh successful for us. But even even then, you know, I think we still have room for uh room for improvement because it's it's still not super clear for learners that actually no dualingo max uh you also get access to all of the super benefits that you also get, you know, video call with Lily and and these other AI features on top of that. So it's something that we still are trying to iterate on and you know deciding which type of learner should get more of like the the max uh ads subscription ads versus the super subscription ads.

That's something we're we're still trying to figure out. Got it. Well, and what I heard there goes back to one of the things we were talking about earlier, which is making sure you're not building technology in search of a problem. And it sounds like you didn't build the features as technology in a search of a problem. You you figured out what the customers really wanted and build it from there. But when Duolingo Max first launched, you were positioning it as the AI subscription, which is compelling. It's a logical appeal, but the emotional appeal that ultimately resonated more was use Dualingo Math to have immersive conversations. And that really gets to the deeper why behind why somebody would pay more money, right?

the AI is just the means to the end. Ultimately, what matters is those immersive conversations. Um, so that that makes a lot of sense. Last question here before we get to our lightning

round. Where does Dolingo go from here? Obviously, AI is having a tremendous impact across all sorts of industries right now, particularly in the technology industry. How do you see language learning evolving over the next few years? And to the extent you can share um what are some things Dolingo is exploring to take advantage of that uh AI wave? So number one is creating these new features we were talking about that previously we just couldn't create. So more personalization more conversation more immersive uh fe language learning features. So that's one piece of it. The other piece is um making content creation much easier and cheaper. So there are certain features that today they're not AI driven. So the content is is pretty much static. An example is a feature that we call dual radio which are these like short you know 2 minute podcast type of uh sessions. They they're for listening comprehension. We

create those with AI and we can create like hundreds and hundreds of these in a matter of like a couple days. Previously before AI, we would have never worked on dual radio because it would have taken us years to create that same amount of content. And so it's both allowing us to create content a lot faster and it's also just allowing us to create new features that previously we would have never worked on because it would have been too expensive to scale.

Particularly for a product like Dualingo because we have all these different languages that we're teaching, you know, we have all these different directions. So you can be learning Spanish from English or Spanish from Portuguese. So there's all these per permutations of uh languages. And so that's that's one thing. um AI uh is helping us with uh content creation. Yeah. And I think for us I I guess like other companies probably can also take advantage of the same benefits that AI is bringing. But again, Dualingo uh I think one way one of the things that we focus on is really creating the most engaging language learning platform and the most engaging features. So that's a mode for us. It's uh you know it's not just about the content. It's about getting you to come back every day. We're the experts of that. And then the other part is uh we're just making our uh app more and more social. So you're learning with

your friends. You have leaderboards. Um you you can have a a super plan with your family or a max plan with your family. So the social aspect also kind of help us. Uh it's another one of our modes. Yeah, that makes sense. Well, and if you look at some of the most successful consumer businesses of the last decade, they have been defined by network effects between people, right? The more people who use the product, the more powerful the product becomes, the more organic growth that the company gets without having to spend money on paid ads. And so that's been a real advantage that has helped to crown category kings. But I see this new moat that's emerging among AI startups or incumbent companies that are leveraging AI, which is this data network effect.

And one of the things I think you alluded to there was while many other language learning apps can apply some of the same AI technologies, they don't have the usage data that Duolingo has that you can now use to power the best language learning experiences and personalize them for each individual user because you have all of this language learning session data that other companies don't have. That's that's right. That's true. So we also iterate even on our AI products based on you know using our millions and millions of learners. Awesome. Well,

this has been great. Let's jump to our lightning round. I'm just going to ask a series of rapidfire questions and you can give me some quick responses. Um first one is a bit of a softball. What is the best part about working at Duolingo and what's one thing you would change? One of the best parts I well I mentioned the mission but I think for me again I've been working on Dualingo for 10 years so it's being able to work with people that I've known for 10 years that I really get along with and who know me and know how how to work together. And maybe one thing that I would change is being able to focus a little bit more on um teaching uh or building features for more serious learners. That's something that I would be very excited to to to work on in the future. Got it. And I assume when you say that you mean a lot of Dolingo's users are users like me and my family, right? are traveling and we

want to learn enough to go on this once in a-lifetime trip and that's that's compelling but uh helping serious learners for whom learning a language is going to fundamentally transform their careers or their earning potential or the trajectory of their life I imagine that's even more rewarding on another level. Yeah. And I was also thinking more like for example if we were able uh to teach like calculus for example. That's probably not something that you would do like you know when you're in the subway or waiting for you know waiting at the doctor's office. Probably something that you would need to like sit down and like stare at a screen for a while. I would love for us to work on those types of subjects at some point.

Makes sense. Well and I know Dualingo has started to invest in math and music over the last couple years. And so now I understand what you're saying. Even moving beyond language learning, there's a whole new frontier of other academic subjects that Duolingo can help unlock for people. You mentioned there you joined Duolingo because of the people and that's what you love most about it. You are a bit unusual in that you've only worked at one company since graduating from college. Uh you've been at Duolingo for a decade. So what is one of the advantages of sticking with one company for so long? Cuz that's something you don't see a lot in tech these days. I think there's a lot of advantages. So working with people for a long time, you get to really understand them, understand what how they like to work. You start kind of predicting, okay, I know this person's going to say this if I do this or they're going to

say this other thing. So that's one part. And then the other part is um kind of having all the history of all the decisions that we've made in the past and what we've learned. It really helps you make much better product decisions today because you're like, "Okay, you know, I know that we tried working on this idea back in the day. This is what went wrong. So now that we're going to try it again, we should make sure to avoid this specific, you know, decision, let's say. Yeah, that makes a lot of sense. I mean, a lot's been written on Malcolm Gladwell's 10,000 hours and how long it takes to develop mastery. But there's also this sort of dynamic version of that, which is when you work with other people long enough, you start to develop kind of a six sense of how to best work together. And so that's really cool that you've gotten to experience that at Duolingo. What's your favorite consumer or proumer subscription app?

And what do you love about it so much? Lately, for me, it's uh I I really like my YouTube premium subscription. Mhm. Uh, I love the no ads. Um, I accidentally stopped my subscription for a little bit and I was like, "Oh no, this is YouTube without ads is so much better." Um, and then I love that as part of the subscription, you also get YouTube music. I kind of like how the both the regular YouTube app and the YouTube music apps, they're kind of bundled together because I know sometimes I want to watch music videos, sometimes I want to just watch a video and I think they work pretty well together.

It is amazing how much just removing ads from a product enhances the experience especially for a content product like YouTube. So that makes a lot of sense. A couple more here. How many languages do you speak and what's your favorite idiom in another language? I I should have mentioned this before. I actually majored in economics and Chinese in college and so I studied Mandarin and there all sorts of these funny idioms and so I'd love to hear if you if you have a favorite one what what it is and in what language. Yeah. Um okay. So, I know Spanish is my native language. Of course, I know English as well, and I know French. Uh, and my favorite idiom, I I really like the phrase human monk in French. Mhm. So, it it means I miss you, but it the literal translation is actually you are missing from me. Huh.

And I think that's like a very poetic uh phrase. Yeah, that's beautiful. It's almost like that person is a part of you. And this is maybe sharing a little too much, but my wife and I just celebrated our 10y year wedding anniversary. And I'm at the point now where if I travel for work or, you know, if I'm just away for a few days, I sleep so much worse cuz I'm just so used to being at home um with my wife and uh so it's amazing how the people you love most and spend the most time with, they really become a part of you. Okay, last question. How do you think the online education landscape will evolve in the next two to three years? We talked about Duolingo specifically and some of the bets Duolingo is making, but you know, I worked at Quizlet for a few years. I know the edtech uh industry pretty well, and it just seems like there's so much change coming right now. So, so what are what are a couple of the biggest changes

that you see coming? Yeah, I think probably content creation is going to unlock a lot of in the education landscape. My hope is that we are just going to have a lot more engaging content across a variety of uh subjects. So again, you know, going back to like the Dualingo example, we teach languages, math, and music. Uh I very excited uh for like very engaging very high quality content starting to appear in like you know physics and calculus and biology uh some of these like uh subjects that where the content available right now it's pretty dry not very engaging. Hopefully it it just continues to increase in quality.

Yeah I love that. It's going to be really exciting to see if Dolingo can replicate its tremendous success in language learning across some of these other subjects starting with math and music. And I'm certainly a believer. You guys have had an incredible story and I'm excited to see what you can do across a broader academic spectrum. So, thanks so much for coming on to the podcast today, Natalyia. It's been a real pleasure. Before we wrap up, anything else uh you want to say uh before we wind down here? No, no. Thank you so much, Phil, for for taking the time. really enjoyed our conversation.

Likewise, it was my pleasure. Thanks, Natalia.