How Rise Turned “Sleep Debt” Into a 10M+ User Sleep App

Sleep debt is something that I think a lot of people still don't understand. Why did you guys decide to focus on sleep debt as the nucleus of Rise? >> There were no wearables back then. Fitbit had just launched. Maybe Jawbone was about to launch soon. Like you couldn't record your sleep. Every sleep researcher agrees there's two systems that control your sleep. One is called the homeostatic system. The longer you're awake, the more you get tired.

And then the other big science angle is just circadian rhythm. We basically base all of what we do in those two main kind of sleep systems that researchers study deeply. >> I work with consumer and prosumer subscription apps all the time. I've seen thousands of onboarding flows. And yours is one of the best I've seen. >> It's not about being efficient. It's about gathering the right information that you can then help someone with whatever data they already have. And once you get clicks for people, then you almost earn the right to be like, all right, do you want to sign up for a trial? There's a lot more here, but at least you gave something back, which in our case it's like your sleep [music] need.

>> Oura and Whoop are two very popular wearable devices that have gotten a lot of momentum over the last few years. On one hand, they could be viewed as partners and [music] very complimentary to your product. On the other hand, they're potentially competitors as well. So how do you see that evolving? >> The way I've always thought about this competitive dynamic is >> Welcome to Subversive, [music] a podcast dedicated to sharing stories from the best consumer subscription apps in the world. We'll bring you lessons for how to grow your consumer subscription business, including insights and inflection points that led to exponential growth from leaders at category-defining companies and innovative startups.

Let's get into the show. This episode is brought to you by RevenueCat. The world's best apps use RevenueCat to power their subscription infrastructure, manage customer data, and grow revenue across iOS, Android, and the web. From any developers launching their first apps to category leaders like OpenAI, Notion, and Ladder scaling to millions of users, RevenueCat handles the complexity of monetization, so you don't have to. Learn more at revenuecat.com/subversive.

My guest today is Leon Sasson, the co-founder and CTO of Rise Science, a sleep and energy app with over 10 million users that helps people understand, track, and pay down their sleep debt over time. In this episode, I talked to Leon about why Rise chose to focus on sleep debt from the very beginning and how it's turned it into the critical aha moment in its new user onboarding flow. About how the company built a sophisticated experimentation culture that allows it to run multiple AB tests per week even with a small team. And about how the company is continuing to evolve its product and core value promise as Aura, Whoop, and other wearables encroach on its space.

All right, welcome Leon to the Subversive podcast. Excited to have you on today, especially because I've been using Rise now for years. It's one one of the first apps that I open every morning when I get out of bed to track my sleep debt. And you guys have come a long way since you first founded this company at Northwestern. So, to get

started, let's just jump into what is the origin story behind Rise and how did you and your co-founder Jeff take what was initially a school project at Northwestern and turn it into what's now one of the top sleep tracking apps in the world? >> Phil, thanks so much for having us. Oh, that's straight to the to the story. I love it. So, uh happy to be here and I'll go back. It's been It's been a while since we started. Actually, we started doing research. Uh funny enough, we were I've always been the opposite of what you call a good sleeper. When I was kind of like through college, I was my whole life even before college, like sleep-deprived as, you know, that was my MO, being like, "Hey, I'll sleep when I'm dead. It sleep is a waste of time. There's so much interesting things to do during the day. You shouldn't sleep." And then my, you know, back then my best friend, now my my co-founder Jeff, was like trying to dig

in into like, "Why are we tired all the time?" And he got into sleep research and he kept telling me like, "Oh, there's all these interesting things on how sleep affects your cognition and your performance and this and that." And he he wanted help uh doing some research on it. Uh so, long story short, we started digging in and I started getting con- convincing myself that this stuff matters, uh which sounds ridiculous now that to feel less tired you should sleep more. It's almost like so obvious, but to this day, that's why we're in business a lot of people don't put that together but back then the research and then you have to go back there were no wearables back then Fitbit had just launched maybe Jawbone was about to launch soon there were like people trying to like you couldn't record your sleep that wasn't like a thing that you could do you could barely record your workouts so so now everyone like takes this for

granted that your Apple Watch or whatever 10 wearables you can do to kind of track your sleep and your steps and everything back then that wasn't a thing at all so a lot of the research was very focused on elite athletics which had very measurable outcomes and that was a world that at least could invest in tracking kind of very specific almost like academic grade lab grade devices uh so we started doing research with the sports teams at Northwestern and very quickly the coaches were like oh wow yes if we get players sleeping better things just like literally the outcome metrics that they care about do better right like where it's like literally speed on sprints or basketball three-point percent or like how fast you can like level up your bench press over time when you're kind of like training on the football season like all that stuff was just like literally linked to sleep uh so we started working on that

and the word in that community spread out pretty quickly and then other teams started reaching out to us and we started doing that to to a lot of teams and that's how we first launched launched the company that then uh over the years winded up focusing on the consumer side cuz we we just really wanted to then how we take what we learn on the sports side and bring it to every person with a smartphone without even a device and that's what we spend now almost a decade working on so it's been an exciting time

>> Yeah so it starts as this school project and then it was initially focused on sports teams and B2B partnerships and then you pivoted to consumer around 2020 in the pandemic but throughout sleep debt was really the core of the product and so let's talk a little bit more about that cuz sleep debt is something that I think a lot of people still don't understand why did you guys decide to focus on sleep debt as the nucleus of Rise >> We always were trying to do everything we can to follow the latest science. And if you do the science that almost every sleep researcher agrees is there's there's two systems that control your sleep. One is called the, you know, homeostatic system, which is you can think about it as sleep pressure, which is that the more time you're awake uh the more multiple type of of hormones and and and and proteins build in your body. One of them is adenosine and over time you

build something called sleep pressure. Uh so you get tired. >> [clears throat] >> The longer you're awake, the more you get tired. And there's a lot of then research that linked that to also historical sleep. So it's not only your sleep last night, which was maybe one of the biggest misconceptions back then. And to this day many people don't have a clue that it's like, "Oh, if I sleep good tonight, I'm good to go tomorrow, even if I haven't slept good in a week, right?" Uh where there was a lot of really interesting research that you can get sleep deprived for weeks. Like like the sleep deprivation from weeks ago impacts you um the next day. And that plays both ways. That helps you for example, if you were sleeping really good for a while and you have one terrible night of sleep, you're going to be fine the next day cuz cuz you didn't have sleep debt. Whereas if you flip it and it's like, "Hey,

you've been sleeping poorly for 2 weeks and then you have a big day tomorrow." This happened a lot in sports, right? That people were like, "Oh, they can sleep bad for a week and then they want to sleep great the night before the game." Well, that doesn't work that well. It definitely is better than nothing, but it took us a while to figure out how do we then make that concept of sleep pressure into something that is a little more understandable without saying without calling it the homeostatic sleep drive, right? I don't think that would be a good, you know, copy for a product name.

Uh uh and you know, sleep debt kind of became something that through a lot of research and customer kind of interviews sort of started getting connected. People could make sense. It you know, it's sort of something that you could put together. Like, "Oh, you kind of know what that is." And it sort of makes sense. And then the other big science angle is just circadian rhythm, which is a field that has really evolved and and become really interesting in the last 20 years even. Like it's pretty new from like a science and biology standpoint.

And that's where, you know, a lot of a lot of our app and a lot of the value is. How do you understand what your circadian rhythm is doing? We call that your energy schedule cuz again >> Mhm. >> even [clears throat] back then, but now to some extent, too, calling it circadian rhythm might not, you know, be as catchy as as as it is now. So, we basically base all of what we do in those two main sleep systems that research has kind of studied deeply.

>> So, I think that's a great segue into some of the specific growth strategies and tactics that have been so effective for you guys. And one of those is you just have an exceptional onboarding flow. I mean, I work with consumer and prosumer subscription apps all the time. I've seen thousands of onboarding flows. And yours is one of the best I've seen at very quickly getting the user to what's often called the aha moment where they're like, "Oh, wow, this product can do something for me that very few other products, if any, can can do." And that is help me track and manage down my sleep debt and improve my circadian rhythms and my energy schedule. So, let's talk through how your onboarding flow is structured and specifically what sort of inputs, whether those are qualitative responses to quiz questions or integrations with Apple Health or wearable devices, are you using to actually present a user's initial sleep

debt to them in that onboarding experience? >> Yeah, so I mean, onboarding is one of those fascinating topics cuz it is the moment that you get to sort of really win someone over and convince them in some persuasive way that you can help them. There's multiple almost philosophies with with onboarding, but for us it's always been and very early on we realized and we learned that the old, more like e-commerce based approach of reducing friction is not actually a good one for the type of product that we are building. It might be a good one for other type of products and services that it's like, "Oh, make onboarding shorter, reduce friction, and numbers go up." For us, it was, "Hey, once someone is in the door, they downloaded our app for a reason, right? They have a problem in their life that for some reason right now is is not great.

Uh in our case it's most people come in and they're like tired all the time. Like a lot of people are just exhausted all the time and they can't stand it anymore and they're just desperate. They could have tried a bunch of things, they could have tried supplements and tea and working out and just like more coffee and more energy drinks and tracking their sleep with another wearable and they're just like keep being tired. And then we get them, they download the app and you have their attention. You have a you know, people are being very graceful with their time and you can give something back in return. So spending those uh it's not about being efficient, it's about gathering the right information that you can then help someone click the concepts, right? What I described the sleep debt and the circadian rhythm. How do you explain that in a very succinct way with whatever data they already

have. So you can imagine for us uh that works very differently. If you might have some people that are coming in with, you know, an Aura or an Apple Watch and they have data for a year and they still are feeling tired by the way, but they they want more and you might have people that had no clue that they could track their sleep with their phone and they're also want to figure out. So how do you still connect the dots for those people and communicate this breakdown, their tiredness, the problem they're coming in for, tiredness, into oh, this is why you're tired and once you that clicks for people, then you almost like earn the right to be like all right, do you want to start a free trial? There's a lot more here, but at least you gave something back, which in our case it's like your sleep need, your sleep debt, uh something that now the person learned.

Uh we talk a lot of, you know, after user research and interviews from our onboarding, people are like I'm very happy that they learned a concept during onboarding. Um that said, while efficiency of onboarding is not a goal, but friction is never good. Uh you do want to be careful and and we've learned that it's important that it's very smooth in the sense of making it easy for people to answer a question is very important. Making it hard to answer a question. If you ask a question that could be hard to answer, like literally like the person will have to think hard about what to answer, that can be problematic. Open-ended text often is not super desirable, you know.

So so there's a lot of things that it's not about just removing friction. You can have more screens, you can have more questions, but it needs to be something that the person can keep the pace around and and and we've learned that that kind of works really well and less about kind of the length. So that's a little bit of where what we learned over the years uh

doing this. >> So a couple thoughts there. One is people often ask me like how long is the ideal onboarding flow? And my response to them, which I'm sure is frustrating, is it depends. And it depends on the problem you're trying to solve. And this is an oversimplification, but at the highest level, I think products that are primarily geared around entertainment or a sugar rush tend to have shorter onboarding flows. Whereas products like Rise or Noom, for example, Noom is an extreme case, but but products that are trying to help you improve your health, uh improve your finances, uh you know, learn a complex new topic, they tend to have longer and more involved onboarding flows for a couple reasons. One, it's just a more complicated product and so it requires a little bit more time to acclimate the user. And then two, it's a more nuanced use case and so oftentimes longer onboarding flows provide what you

referred to as good good friction in the sense that you're getting responses from the user that can help personalize the experience. And so for Rise, I think it very much falls into that category of you're helping people solve sleep, which is a huge part of health and life satisfaction. Uh and so you need to know a certain amount about the user before you can do that. That sort of brings me to another question, which is with AI, I find that more and more consumer companies are personalizing their onboarding experience to try to deliver a better first-time user experience.

To what extent are you guys doing that? Cuz I imagine you have people coming in who are everything from like elite athletes who are trying to perform their very best on the field to people who can't get out of the bed and they they have some sort of underlying medical condition or maybe they have sleep apnea and those are very different sets of problems, and I imagine very different levels of sleep debt. So, how do you bake that into the onboarding experience and then translate it into the core product experience?

>> The I think the thing people often forget or don't consider is like what are your users doing when they download the app or go to your service or whatever, right? In our case, we know vast majority of people in the world of consumer, there's nothing like 100% but a large number of people are literally sitting on the couch or literally in bed, kind of like scrolling through the internet, you know, winding down, wasting time.

They're not trying to like solve a problem very quickly in like a utility type way. So, in that way, we earned the right to almost like be a little we can kind of like be entertaining, right? That the length is less of a priority as long as like people are captivated cuz they're not, you know, they're not trying to like they're not sitting at their desk uh trying to get a problem solved in 3 minutes, which might be true for some other type of services that are maybe like, "Oh, I need to solve this task right now." Uh it's more like what are they doing when they're using your app?

So, that we know gives us quite a bit of headroom from a length standpoint. If something is good and it's entertaining, it's worth 5 10 minutes, like for sure. So, they're willing [clears throat] to. Personalization uh is one of the most fascinating things right now, and I think we're coming kind of full circle to where we started where if I go back to the athletics world, back then, we were kind of B2E service offering a very high-end white-glove sleep coaching for sports teams, and we were charged thousands of dollars per person per year cuz we had, you know, coaches on staff working one-on-one with these, you know, very important athletes that could really care about performance.

We could never figure out the right way of bringing that level of fidelity to the consumer experience. It was just not tractable. So, we had to distill it down to the basics, right? Like sleep debt affects everyone. Circadian rhythm affects everyone. But hey, there were like 20 other scenarios that we couldn't like perfectly consider, and obviously over time, there's segments of that that we have done specific improvements to the product to make it good and valuable for those users. But now with AI, it's it's, you know, it's obvious that that is the way things go, right? Like it's going to be an expectation that for all services like this, wellness, health, finance, any type of coaching, right? People will expect hyper-personalized experiences, and now we're getting uh to the point that we can do that, right? So, in the past, let's say it didn't really make sense for us as a small company to be like, "Hey,

let's optimize let's personalize experience for this like 10% of users that are 25 to 35-year-old people that are, you know, night owls and want to become a morning person." Uh or hey, let's let's personalize the experience for the subset of people that are also working night shift. Right? These are all small buckets of 10, 20% of people that before you couldn't really spend the calories on, but now you can actually personalize that level.

And I think more and more we're we're seeing users expect that level of personalization to the point that they start churning if the product feels too generic. So, it is an exciting time that you can now kind of personalize. And on the on the other side, kind of related to our growth machine that we can talk a little more about it, the bigger you get, you can now start actually AB testing a lot of things for smaller segments of people that when you're smaller, you don't have the luxury of the numbers aren't in your favor, right? That 10% bucket is too small to do any testing on. Uh so, you are trying to work product that you don't know if it's good or it's harder to know if it's good.

Whereas now you can actually test things when you're bigger with smaller percent of people. So, you actually create a really good feedback loop that you can work on problems that subset of people have, and at the same time you can measure them a lot more effectively. So, it is quite exciting.

>> So, that's a great segue into the last question I wanted to ask you in this section. That's around AB testing. We'll get to your growth machine in a moment cuz you guys have built a really impressive experimentation culture and infrastructure. But before we do that, let's stick to AB testing specifically within the onboarding flow and You guys have run dozens if not hundreds of tests on onboarding and paywalls. What are some of the tests that have been the biggest winners or or that have most surprised you over the years?

>> Yeah, so I mean, we've run a lot here and I would say almost like I would flip it a little bit. So I think people hyper fixate on paywall itself testing, which we do a fair amount. Uh and almost like undervalue onboarding testing and what leads into the paywall. So we have seen a lot of success actually sometimes not necessarily working on the paywall, but moving the paywall around or moving hey, what are the few screens before the paywall? How do you kind of create that feeling that you are building towards something? Uh so that has become really important. Like what are the few screens before the paywall, right? Are those setting the right expectations that maybe you're getting to somewhere, maybe there's something on the other side of the paywall that you really want that we're working towards. Uh so so that sense of anticipation is really important for a type of product like ours, right? That in some way it's a

it's a product that people don't need to buy, but you want to create a sense of they want to buy it because it's going to improve their lives. So you have to kind of tease a little bit into why they want that. So we call that pre-paywall uh onboarding phase, which is, you know, crucial. As we created a growth kind of engine, it become more and more important to start understanding what metrics really matter because we were surprised that often you look and and especially kind of early products, you look at a metric, you know, many people might look at like your free trial start rate or your paywall shown rate or onboarding drop-off rate or day one retention, uh but you might not be looking at your day 31 monetization rate or day 60 kind of net revenue rate, which ultimately is what matter. So you can actually make really bad decisions when you look at hey, trial start rate goes up, but you're actually getting the wrong users

start trials and you're kind of almost like not emphasizing on the right users that maybe aren't going to be the more engaged. Uh so we have over time gotten much better and invested a lot in how do you get predictive metrics on your day 31, day 60 to understand that the top of funnel metrics that you can see and measure quickly actually end up closing the loop once time evolves cuz you can it backfires. Sometimes you have something that actually lowers your, let's say, initial purchase rate but it's still beneficial in the long term cuz maybe those are more valuable users that retain uh for longer.

So getting the right measurement both early, top of funnel, and how do you going to get to the bottom funnel has been some of the most kind of fascinating changes that now with now almost default on our on our growth, you know, growth stack. >> Yeah, I'm so glad you said that because I think there can be an obsession, especially among solopreneurs, indie developers, smaller teams that maybe don't have the sophistication or the resources to build full-on AB testing infrastructure and stick with an AB test long enough, whether with a holdout group or otherwise, to understand the long-term implications of some of the tests they're running. And so as a result, they can like really juice their trial start rates and maybe even trial conversion, but then at the expense of long-term subscriber retention rates.

And if you're not keeping close attention, that can really come back to

bite you. Let's talk about your experimentation culture and how you cultivated this rigorous experimentation culture in the first place. When did that start and how have you sustained it over many years now where you're running, you know, multiple AB tests every single week even with a relatively small team? >> Yeah, I mean, I think it when I think about it, I think it often almost comes down to how we started as a science-based research project. Like we were always That was our tendency. How do we get the correct answer using the scientific method, right?

Uh you create a hypothesis, you test it, you measure it, and then you can possibly have an answer. Products maybe are not as scientific, you know, there's no central law of physics that applies to to product development, but you can apply the same concept. So I think that always served us a good North Star on how do we work. Uh cuz there is, of course, an art to building product and and it's not only a science and there's an art and there's a subjective side of aesthetics and visual design that matter immensely.

But at some point you have to know does something matter? Does something make the life of the user better? Does something help the company succeed cuz ultimately this is a company, right? Uh and ideally those two things need to be aligned. It was kind of a very obvious fit that you do want to measure a lot of things. That doesn't mean that every decision has to be a metric-driven decision. We have many things that we call a do no harm test that is like, "Hey, we like this direction uh as long as it doesn't make metrics worse.

We're going to roll it out. It doesn't have to win sort of thing cuz we we think it's the right thing that's going to help people and and we like the feature and we like the offering or or often we remove things also and and see if it does do no harm cuz we feel the product is getting kind of crafty. Uh so outside of that kind of upfront we decide is something a do no harm project or is something actually expected to move a metric. And that actually makes it very easy to decide down the line if something works or not cuz you don't have to rehash it like, "Well, this thing was supposed to move the needle but it didn't. Do we still keep it?"

It's like, "No, by default it doesn't earn its place uh unless it was kind of a do no harm test from the from the get-go." Um And the other thing is, especially as you grow, because you have more users everywhere in in what we call surfaces, you can now paralyze testing across many surfaces. So, for example, we can split an onboarding surface from a pre-paywall surface from a paywall surface from a win-back paywall surface from a day one app open surface. So, you can now have uh multiple layers of testing running at all the time. Uh and as long as you're stacked and you know how the analytics are working, you should be able to run them in parallel without getting too complex like that with with the the signal. So, that let us I think scale some of our experimentation where yeah, maybe you cannot be running 10 parallel tests at the same time, but you can certainly be running 10 tests across the stack at the

same time. Um yeah, and maybe 10 could be pushing it, but it's totally normal. So, I think that's kind of ingrained in how we we operate. >> This episode is brought to you by Paddle. Today's fastest-growing apps are multi-channel. More teams are using web checkout to improve margins, control pricing, and build stronger customer relationships. But external payments bring new complexity, from tax and compliance to fraud disputes, billing support, and entitlement logic. That's why Paddle launched the 2026 web monetization playbook. As the merchant of record for apps including Runna, GoodNotes, Codeway, and Hubex, Paddle handles the complexity, so you can focus on growth. Download the playbook at paddle.com/subversive.

>> Yeah, you said a couple of things there that I really want to emphasize cuz I think they're really important for folks trying to set up their own AB testing culture and infrastructure. One was there are different types of AB tests, and we had a taxonomy of these at Quizlet as well when I was leading the product growth team there. So, we had an AA test, which is where you're literally running the same user experience against itself, but just making sure that you aren't seeing statistically significant deviations, which could signal tracking or infrastructure issues underlying the product.

>> the way, I encourage everyone to try AA test. We have always running AA tests, and sometimes we get surprised that there's a lot more noise than you would expect in the real world. >> Uh >> Yeah. >> And even because, you know, our products are not fundamental, you know, random streams of users from like a physics standpoint, which is what most AB tests are determined to to do, right? Like, what's the probability distribution of of an effect here? Um but instead, our user base changes during the week, right? Like, we have very different type of users that come during the week or time of day. So, you actually do care about an AA test as like understanding that your machinery is working well. So, highly emphasize, and we took us too long to do that, but now we always have AA test running.

>> Yeah, I'm surprised by how many companies never run AA tests, but I think they can be very important for just ensuring data integrity. Then you've got a standard AB test, where you've got one North Star metric or a small set of target metrics that you're aiming to move by a specified minimum detectable effect over a specified time period to to hit statistical significance. That's sort of your down the fairway AB test. And then we actually called it a do no harm test as well. I don't know where that term originated, but a lot of companies seem to call it that now where it's similar to a standard AB test, but you're relaxing the requirement that it actually move a target metric. You sort of flip it. It's like innocent until proven guilty and all it needs to do is be neutral or better and as long as it doesn't statistically significantly negatively impact a key metric, then you're going to move forward with it

often because strategically it's a better direction to be moving in. So So

one I really like that you broke down those different types of AB test cuz that's something I think a lot of founders and growth leaders don't think about. But then two, you talked about how you've built infrastructure and tooling over the years that allows you to accelerate your AB testing velocity and run more concurrent AB tests across different surface areas or in some cases even on the same surface areas as long as you can make sure they're deconflicted. So you're the CTO of Rise.

I imagine you've been very involved in building a lot of the infrastructure tools you need. What sort of decisions have you made in terms of build versus buy and what sort of specific tools are you using in your AB testing stack? >> Yeah, I mean of course, you know, we measure a lot of things in analytics and we use pretty kind of standard stacks. We use, you know, Segment for tracking and Amplitude for product analytics, but and I think there's a lot of replacements that are, you know, work pretty well.

For us I think the biggest change was a few years ago now we implemented GrowthBook. And when I say implemented it's not as big of a lift that it sounds especially now with agents that can help you do it and and GrowthBook there's other replacements too, but one is sits on top of your warehouse so it's not a complicated data thing. So it's actually pretty cheap if you already have your data organized. Um but what it does best is standardizing the metrics and running the right statistics for you. I think there's a saying in in the computer science world like never do your own cryptography.

And I think there's a saying in in AB test that should be never run your own like never write your own statistics engine, right? Like don't want your product manager trying to do spreadsheets to like determine if a test is a winner, especially when it's like a complicated test and the setup is tricky and they have to check for all the things like SRM, uh sample ratio mismatch, and a lot of pitfalls of AB testing, which will happen to everyone. So, GrowthBook was like the biggest move in standardizing things. Well, we had a lot of internal tools to kind of do it and Amplitude and especially when you're trying to make a decision like hey, does this thing move revenue? That is a fairly complicated statistics engine that is doable. It's not crazy, but you probably shouldn't build it yourself. You shouldn't write code it. You should just let someone else that spent a lot of time thinking about, you know, how you determine the

success from a test. So, that's been the biggest move. That plus standardizing metrics. So, everyone knows hey, what's your trial start rate? You have like a clean definition of that because it's never as the user journeys are never as linear as people think. People think like oh, user downloads, user goes through onboarding, user completes onboarding, user makes a purchase. In the real world, it's like well, user downloads, user stops, user comes back 3 weeks later, user starts again, then sees a paywall, then clicks an email. Like it's so it's a very complicated journey that users go through that is non-linear. And the bigger you get, the less linear the flow gets cuz you have more existing users that are reinstalling the app and and how do you actually get visibility all of that I think has been key. So, standardizing metrics and standardizing on the statistics engine that you use, which we like GrowthBook. There's a

bunch of other products, uh highly recommend kind of doing that. And uh I think culturally, the other thing is this also took us a while, but when it comes to landing is there's a point that I think we and certainly me, uh almost a humility of saying like hey, we I don't know up front whether something's going to work or not at this point. Like my my predictive power of hey, I look at a test that we're about to launch and it's like is this going to be a 5% lift, a 10% lift, a negative?

Now I sort of don't even try predicting to some extent. We try for fun, but it's more like let's see what happens, right? Because sometimes you get very surprised about oh, this thing should have been a clear winner and it's not. And the other way around, too. Or hey, this is like tiny, there's no way this is going to make a difference. And it makes a way bigger difference than you think and you're like running for longer to like double-check and you spend all the time checking analytics to make sure that it's not a fake readout. So, having almost like a humility of being like we don't know up front if it's going to work or not and let just let the numbers determine. Just have an opinion, hey, I like this direction, but you almost are not trying to get hung up on like, oh, I like it, so it has to win or we're going to find a way that it wins. It's just, you know, we're going to find out. We're

going to discover whether it works or not. Uh, so I think that has been a cultural thing where it's in especially now that it's easier to build, you can just be like, hey, you don't need to hash it out in 20 conversations, you can just test it and see uh if it works or not.

>> Yeah, totally. So, what I heard there is you have three core components to your stack. You have a data and metrics orchestration layer in Segment that just makes it easy to sort of pass canonical metrics to other tools. You've got Amplitude, which is your product analytics tool. You could also use Mixpanel or PostHog. There are a number of alternatives, but they're all great products. And then you've got GrowthBook, which it sounds like has been a really big game-changer for you in terms of specifically handling AB tests and AB test results. Um, and then the other thing you said is just making sure that in addition to the tools and infrastructure, you're establishing the right culture. And in in your case, it sounds like what that means is standardized metrics where every metric has a clear definition and you keep your data warehouse really clean, so there aren't conflicting definitions bouncing

into each other. And then also just a culture of curiosity where like people aren't jumping to conclusions about what's going to be a good test or not. You're keeping an open mind because like you, I mean, I'm constantly surprised by which tests move the needle and which don't. >> I'll uh I'll uh one point there because the culture I think is somewhat the most important thing, of course, but uh one of the dangers of something like GrowthBook or any not GrowthBook itself, but the the art of like, hey, we look at these four metrics and if it moves, we win, if not, we kill it. It's almost like you can get lazy. I think laziness is a disease right now, especially with LLMs, where it's like wait, we added 10 screens, there's all these questions and parts of the flow, and people can put different things and try different things, and it's did someone actually look at what people were doing, or talk to users, or you

just looked at hey, they didn't move revenue day 30 by 3%, right? So, sometimes even if it is a loser, you actually get really interesting like, oh, I didn't know that 85% of users actually like this thing better than the other. So, what do you do next? So, I think one of the dangers of AB testing is that you get hyper-fixated on like some three metrics that you're going to be judged on. Uh and you kind of forget about what are people actually doing, what are they seeing, what are they doing, uh do they like it, do they not like it, are they confused by it? So, that hard product work is still there, and obviously you can you know, AI has changed the game on how deep you can go, but AB testing tools do not replace going deeper into what you actually ship. And did it help users do what they wanted to do, and does it teach you something else about what users want as a for a future test. So,

uh and I think that is, you know, the culture of curiosity that that you're talking about there. >> Yeah, I think that's a great call out. I mean, there's obviously the quantitative element of experimentation, which is really important, but there's also the qualitative element of understanding what users did in the product that led to those quantitative results. And there are plenty of tools that you can use to actually track user behavior in your in your mobile app or in your web product.

Uh and so, those can be really valuable as well. One One other thing I see happening more and more as as there's just so much data available with LLMs is companies sort of overload AB tests, and so, they have 20 metrics they're looking at. And just based on statistical randomness, you're going to have one or two metrics out of 20 hit statistical significance, even even if it's just random chance. And so, one thing we always try to work on with our clients is like, okay, if you're running an AB test, have like one or two metrics that are the North Star metrics you really care about, and make sure that you're making AB test decisions primarily based on those metrics versus having a laundry list of metrics and you happen to get lucky on one or two of them. That that's that's another trap that I see some companies fall into. Any any other things that you see other companies do wrong frequently on AB testing that that

you've tried to correct at Rise? >> I would say the bottom funnel like they can do bottom funnel and often reviewing data way after the fact is helpful. So often just like checking in on an experiment that you already called checking in 3 months later or 6 months later automatically to see how that cohort that supposedly won or lost how did that bake cuz we've had again fascinating results there where an experiment was like a loser or flat so you didn't release it and then you look 3 months later and that cohort actually performed a lot better from a retention standpoint. So it is hard to do well but I think that's one of the most interesting things right now that because you have the bandwidth to review things I think we're not have like a much longer horizon that often we run the test for whatever time it needs to run uh and then you also check in later regardless of what decision you made

cuz you can change your decision and and I think the you do want to run it as a science but you're also not trying to publish a paper and you don't need like the rigor sometimes like hey I'm not sure if it's a winner but it's more likely than not to be a winner so it might still be and we like it by the way we like the actual thing that we built so like you shouldn't just not release it because it wasn't like a stat sig winner. So I think there there's that part so bottom funnel metrics and getting confidence in your results. So >> Yeah, I think that's a great call out. I mean ultimately and Demetri Gurski at Flow has talked a lot about this.

Long-term subscriber retention is the foundation for almost any subscription app business and I think companies get especially early stage companies tend to get a little bit too obsessed with the top of the funnel but ultimately those bottom of funnel metrics are really really important and with LLMs it's much easier to go back and do a full analysis on tests weeks or months after they launched and so I think that's that's good advice.

Let's get to our last topic for today before we get to our lightning round, and that is the competitive ecosystem around sleep tracking. And I want to start with wearables because Oura and Whoop are two very popular wearable devices that have gotten a lot of momentum over the last few years. Um Oura launched sleep debt last fall. Whoop now also tracks sleep debt and circadian rhythms. So, how do you all at Rise view companies like Oura and Whoop?

Obviously, on one hand, they could be viewed as partners and and very complimentary to your product. On the other hand, they're potentially competitors as well. So, how how do you see that evolving? >> Yeah, I mean the way I've always thought about this competitive dynamic is you'd rather be in a market that has space for multiple people. Uh and and you know, I think wellness, sleep, uh fitness, all these markets have you want to kind of go where the the tides are going, and the market has shown to be so big. There's such a momentum around people just care more about how they're feeling, how they're doing, trying all these like wearables, whether long-term or short-term to get as a gift, to get obsessed for 3 months, and then they try something else. Uh so, I think the market is so big cuz people really care about this stuff now.

Uh that it creates you can carve your own little niche to serve certain user base that are in different parts of their journey. Uh so, for example, there might be someone that uh is feeling tired all the time, and they want to try something, and then, you know, like we are an easy thing you can just get started now. Uh maybe they get an Oura ring as a gift at Christmas, and they start using it, and they connect to the device, too. And now, they're, you know, they love using Rise with their Oura together. So, uh on a big market, when people have various different ways of of attacking a solution, and the journey is very long, um I think there's a lot of room for people. For example, Whoop historically, even our users are more the the fitness-oriented users than just the sleep users. Uh and there's many people that don't like wearables, and we kind of learn that time and time again.

Uh and for us, that works really well. We we work fantastically well with wearables or without wearables, and most of our users actually don't have wearables. Remember that I think it's easy to get to forget a little bit that most of us are not in San Francisco or New York, and just, you know, trying to just hyper-optimize everything and track all these things, and they're just going to feel un-tired, and they want a solution.

And And we're a great starting point for that. So, um I think the flip side was, "Hey, do you try being in a market that there's no one else?" And then it's like, "Sure, you can have it all for yourself, but there's a reason why there's no one else, cuz maybe users don't want it, or it's not actually a solution they want to pay for or use." So, competitive pressure still keeps you going, and you can riff on whatever people do, but it's I'd rather be in a place that there's enough people that have similar problems to other type of competitors that you can carve your own your own space.

>> Yeah, I think that makes a lot of sense. And especially with AI now, every category worth competing in has gotten so crowded. And And so, I think sophisticated entrepreneurs tend to view that as a signal that they're focusing on the right problem space as opposed to worrying too much about competition. But, you know, as as more and more

companies focus on sleep, it is a dynamic environment. And so, you all are continuing to adapt and evolve your core value promise and product offering. And to that end, you've launched a lot of new products and features over recent years. So, sleep debt and circadian rhythms continue to be at the center of Rise, but you've launched a number of new features. You've got this AI expert now, you've got Rise Sleep Clinic. So, can you talk a little bit more about some of the products that you're adding to your platform and how they fit into your longer-term strategy?

>> Yeah, so I mean, it goes back we're thinking like, "How do we keep making the product better?" And obviously AI drives this huge unlock of you can now personalize and help people in a much longer journey with a much deeper context than you were able to do before. And uh for example, we're trying to do things that before we thought needed to be done, but it was way harder to pull off, right? So, for example, women go through menopause and a period of years before that where sleep looks very different. Uh they have a very different problem than other people. You still have the fundamental sleep that to deal with and circadian rhythm, but there's a lot of other things going in your life. Uh, so how do you still create a product that is very specific?

Cuz what you tell someone in that phase is very different than what you tell someone in, I don't know, my phase, you know, as a 30, 40 year old with like two two toddlers at home, uh, right? Or two kids at home. So, uh, that personalization, I think, is where some of the AI movement goes and we are now trying to broaden our our scope there. On the other hand, something we discovered over time was we have a minority of of people that use our regularly that actually have a deeper illness, a chronic illness. Often, the two big ones in sleep are sleep apnea and insomnia. And historically, one most people don't actually know that has like a label. They're just like tired all the time, right? Uh, they might complain that they sleep poorly, but some people might not even complain about that.

They're just like tired. The reality is an app won't actually solve the problem, right? Like as a kind of general wellness app, you won't solve the problem, but turns out these two chronic situations have known clinical pathways to solve, right? On the insomnia side, there is something called cognitive behavioral therapy for insomnia. It's a type of therapy that is very specific on how you help basically reset your sleep schedule to solve insomnia.

Uh, that clinical psychologists can have been doing for decades. Uh, but turns out it's really hard to find them. It's really hard to find them covered by insurance. It's really hard to actually do it because, you know, you have to actually sleep less for a period of time, which is counterintuitive, but that's the clinical particle. So, we didn't invent the particle, but we're how do we help people get there? So, that's one and the other one is sleep apnea, where, you know, it's a very known, uh, respiratory condition that you basically have small periods of time, you can think about it, not exactly, but that you kind of suffocate in the middle of the night, uh, very brief awakenings. And so, you're you're actually sleeping way less than you think and you're tired all the time. There's known solutions, uh, both machines called CPAP. It's a type of kind of like a mask that you put on to the that you know helps you

whatever solves your apnea during the night. And there's now things like GLP-1s that are fascinating as well in that market. So, we believe that hey, if you can help that 5 to 10% of people that come to us already, but they have a much deeper clinical problem that there's no way we can solve with an app. But you we can now with AI, with a small team, actually close the loop, help navigate the complicated health care insurance. How do you get the test? How do you find a provider?

Right? Like good luck finding a CPAP provider that your insurance covers that takes an appointment like tomorrow, right? There's basically no chance, right? So, we are very interested and and we found that navigating health care now is something that we can do. Um >> Mhm. >> And one of the hardest things for health care new health care companies in the past was how do you get early users uh to get through? It turns out that we we know where the people that are tired are and they are in the Rise app. So, we can help them sort of uh navigate this complicated path. They thought they were tired, but really what they need is to solve their sleep apnea.

So, we are kind of going deeper and deeper on how do you fundamentally solve the tiredness problem and just help people feel better during the day. So, turns out that these two directions are where right now we're most excited of of of solving. >> Yeah, it makes a lot of sense. And I don't know if you would agree with this, but it seems somewhat analogous to what both Headspace and Calm did over time where they aggregated a bunch of people who were interested in some combination of meditation, mindfulness, and and over time sleep, which turned out to be a much bigger use case for a lot of people than meditation even. And then because they had aggregated all that demand as a consumer mobile app, eventually they layered on additional products and services on top of that, whether that was, you know, B2B partnerships to provide benefits to employees to help them with meditation, mindfulness, and

sleep, or it was layering on human clinicians to help people with mental health disorders or with sleep. And and that seemed to work pretty well for them. And in your case, you've not only aggregated demand among that group of people who are interested in better sleep, but you also have a lot more data than Calm or Headspace ever did around their sleep data, their circadian rhythms, and what's been working or not working for them.

Um so, it seems quite compelling. Last

question before we get to our lightning round, it's just around how you think AI will continue to impact this category over the next 3 to 5 years. Obviously, there's a lot of disruption happening with AI right now across the board, and that applies to how you build products internally, but I'm I'm more interested in it externally. How do you think our understanding of sleep and our ability to help people sleep better changes with AI?

>> I mean, I think we're going to keep seeing the end users have the expectations keep going up and up to levels of you can call it personalization. Users will never say that word of like, "The product should really understand my situation, and I don't have to explain it 27 times again that this is my current life situation." So, I think that becomes a new expectation that people will expect without like the word intelligence is maybe overused, but like people they expect their products to be just intelligent now and like actually smart and understand them well. So, I think that creating that context and in like closing the loop, I think will become table stakes in the future, right?

Whereas before you can kind of get along with like, "Hey, you just track my stuff and let me figure out." Now, I think people will expect their products to be as smart as their what whatever you know, ChatGPT or other LLM they're using all the time. They will expect products to be smart. That doesn't mean products need to be chat-based. We have found that most people don't actually want to chat when they just want to like solve a problem quickly.

Uh UIs are still fantastic, but people do want that understanding along the lines. So, it's rapidly changing, so I'm very curious and the whole the broad and health space I think is fascinating where the ability to provide care and get to the bottom of of health care situations and empowering users to do so on their own with light assistance and their data, I think is is the most exciting thing that has happened in a long time.

So, uh I think you're seeing a democ- health care where people can now take an action, go a lot deeper before they even get started with the formal hey talk to a doctor phase. They come in prepared. So there's a lot happening in that phase and and all this wellness industry in a way you can argue it's a preventative health care industry, right? Even though people don't don't look at it that way, but you are preventing long-term problems, right?

Sleeping better over the long run does prevent tons of long-term diseases and cardiovascular risk and metabolic risk and and pulmonary it's just you know, on and on. So we think about the wellness industry is basically the the the pathway into health care in a way to avoid health care when possible. >> Yeah, it makes a lot of sense. It goes back to the old adage that an ounce of prevention is worth a pound of cure and I think that's particularly true of sleep because it's so foundational to everything else in your life.

So with that let's get to our lightning round and I always ask the same question first, which is what's the best part about working at Rise and what's one thing you would change or given that you're one of the co-founders you're actively working to change? >> I mean as a small team and I think we're 16 people now and we've been around that size for a long time. The autonomy that you can go end-to-end in any part of the product, any part of the business and everyone has the same autonomy in the company. So you can go from idea to execution very quickly. You don't need to jump through hoops. That is kind of I think the best part of you know, that's why we kind of instill that value in us. It's almost like no gatekeeping and I think that is also the reason why many of our employees have been with us for so long that you can just take full responsibility which sometimes is scary because you're now

responsible for your actions, but it also means that you have full autonomy and you don't have to kind of play theater into your internal politics to get stuff done. So I think that's definitely some of the best best part here of the internal work. Right now I think we're going through a period of change that is unlike anything I've definitely certainly anything I've lived, but I've hearing from other people that are more tenured than me that anything they've lived, which is maybe not specific to us, but hey, we kind of got into a rhythm over the last 5-10 years of building product a certain way both internally as a company, but like there's kind of the general best practices as a quote-unquote industry of how you do things.

And it seems that everything is going out of the window in some ways, uh except very few foundational principles. The execution of everything is just totally upended. So, it's how do you fundamentally rethink product delivery, product execution, product strategy, even roles, right? Like what does it mean to be an engineer, a product designer, a product manager, a data scientist in in the new times where a lot of people can kind of do the same thing. So, roles are becoming a lot more self-serve, uh where you can have a team of one or team of two actually do a lot of damage when they they want and be extremely productive. So, I think we're going through this rapid phase of how do you let people be even more more effective in small tiny teams of one. So, um I think that's uh maybe less about us, but more about like we're definitely thinking about how do we change the way we work.

>> Yeah, I mean you're certainly not not alone. I I see AI impacting how teams are working across the entire tech industry and even for Elemental Growth, my my advising and consulting firm, like certainly impacted how we do our data work with clients as well. So, it's an exciting time to be building, but it also means a lot of change which can be stressful. So, to totally change pace, next question is what's the weirdest or most memorable dream you've ever had and how did you react to it?

>> Dream. So, I don't I don't dream a lot, but there was a period of time that I got into lucid dreaming and it's not about specific dream, but it's I think something everyone should sort of get into. Maybe we should build it in the Rise app, like how to coach you to get into lucid dreaming, which is where you can actually act in your dreams and there's techniques that you can go through in in a few days or maybe a couple weeks learn how to lucid dream, which means that you can control you're active in your conscious in your dreams, so to speak. So, um I had some like fascinating kind of where you you're dreaming about something and then you can just play the game while you're there. So, uh not a specific example, but more about like the act of lucid dreaming is incredible and I think everyone should kind of get into it at some point to live that experience and then uh yeah, and then you know, I haven't

done it in a few years, but it it's it's it's awesome. >> That's cool. I didn't even know that you could learn to control that. I've certainly had a couple lucid dreams in my life and usually it doesn't last very long before I wake up, but that's interesting that you can actually learn techniques to have more control over it. >> Yeah, there's a few and there I mean there's like a whole subculture of lucid dreaming, but but there's some techniques. One is called like the mnemonic induction of lucid dreams that you can kind of do certain things to prepare to lucid dreaming. You can kind of, you know, work through your senses so that you're more likely to to appear to dream about a specific environment. So you can kind of like control not only in your dream, but try to steer so that when you get into a dreaming phase, you are in a specific environment and you can explore that. So it's kind of this like

fascinating fascinating world. I I don't recall or I don't know. I should do some research on on the science behind it, how does that affect cognition or memory preservation, but uh it's just kind of fun to to kind of tap into that part of the brain. >> Sounds fun. Almost like a video game. Uh so you and your wife recently had your second child, which is notoriously difficult for sleep. So what's one surprising thing that you have found to be really effective in managing your own sleep debt with an infant at home?

>> Oh man, newborn sleep is hard. There's no easy solution, so I wouldn't say right to have a magic pill here, but uh ultimately one of the biggest things is getting a newborn to sleep. You cannot consolidate sleep as fast as possible. You know, by month two you can kind of get there. Month three certainly possible. Most people, you know, you can get there by month three, but from a research standpoint and what kind of seems to be working for us is there's actually a growing body of literature that suggest and shows how various hormones, including melatonin, transfer through breast milk. Uh and well right now there's no mechanistic clinical trials on it. Uh It is clear that for example, the breast milk the mother's melatonin actually and a bunch of other hormones also transfer through the breast milk. So, for example, when you are using the breast milk at night uh through a bottle, you actually want

to time it so that the milk that the baby is taking was pumped at the right time of day that it is for the baby so that it has the right amount of melatonin and the other hormonal kind of cycle. So, there's a fascinating kind of body of literature there to to time the breast milk if you if you know, for people that do kind of breastfeeding and and and pump, uh you can actually control which bottle you give when to help get what's called in the circadian biologist called entrainment to entrain the the baby actually faster to consolidate sleep and uh it seems to working for us uh and it's kind of a fascinating body of science cuz the body is just so fascinating that it has all these micro optimizations to kind of work well. So, >> Yeah, that is a great tip. I mean, my kids are 9 and 7 at this point, so I'm too far along to be able to use that, but if I were the parent of an infant or if I were going to have another child,

that would be gold. Uh related to that, so there are a lot of tips out there for how to sleep better, but a lot of it is been debunked. And so, if you could really boil it down, like what three tips do you think matter most for getting better sleep? >> It is true that there's a few things that matter for everyone and everyone wants like, "What's the magic pill? What's the one thing to do?" So, and surprise, surprise, it does change, uh but I would say that the things that seem to be for sure almost helping every single person regardless of the situation is one, get enough sleep, which is obvious, so I'm not going to repeat it, but goes without saying, get enough sleep and that often means like you need to make enough time for sleep, right? Like, you need to sometimes change your life so that you can get enough sleep. So, that's boring, but works. Time is sleeping at the right time is also crucially important. So,

there's a body of research right? So, depending on your chronotype, which is what time your body kind of runs on different schedules, uh some people might prefer, biologically, going to bed from 10:00 to 6:00 a.m. versus, let's say, midnight to 8:00 a.m. And there's a There's no like correct answer, but depending on your chronotype, if you are a later person, and there's ways of finding that out, you can kind of check online or use the Rise app to tell your chronotype, you can change when you sleep. And that is one of the biggest sort of misconception, that it's like, "Oh, as long as you get the time, you're good."

Often, the reason why you could be tired or not having the right sleep because you're just going to bed at the wrong time. So, often, it's hard to change. How do you change? Cuz you have, let's say, responsibilities during the day, and you have to wake up at a certain time, but there are ways to shift things. Uh so, sleeping at the right time is crucial. And then, the third, again, everyone can do it, extremely simple, you want to get sunlight in the morning.

You want to be outside, if possible. If not, you can just buy us a cheap uh you know, high-intensity light uh that you can use indoors, especially for people who live, you know, once it's wintertime, and you can't really be outside, but even 5-10 minutes of outdoor time, even if you're in the shade, uh in an overcast day, the intensity could be 100 times more, 1,000 times more than it's indoors, and you might not perceive it because your eyes are fascinating, and they let you see well even indoors, but the outside, your body has been developed over hundreds of thousands of years, if not longer, millions of years, biology in general, so that the sunlight is actually what entrains your body, and what synchronizes your body so that everything ticks at the right time, and the hormonal cycle, and cortisol, and melatonin, and a variety of hormones are kind of doing what they need to do at the right time. And there is no way to

actually override that without the right sunlight. So, sunlight in the morning is huge, uh especially in a time that we all use indoor lights and and phones in the evening, um it is even more important and there's fascinating research that sunlight during the day, especially in the morning, helps kind of avoid the pitfalls or of screen time at night. So, obviously, less screen time at night is good, but that could be hard. So, at least make sure that you have sunlight in the morning to help entrain your body into into the right rhythm. So, that is probably the most uh underestimated impact right now. What Some of the research is pretty new, which is why it's you know, in the last 10 years there's been a lot of movement on the science explaining why this this actually works the way it does. So, get sun in the morning. Go for a walk after coffee outside. It's great.

>> Yeah, so for those taking notes at home, three great tips. Get enough sleep. Simple, but easier said than done. Two is sleep at the right times based on your chronotype. And three is get sunlight in the morning. Uh I think I'm doing pretty well on all three of those, but those are those are important. Um last one before we wrap up here. What is one sleep myth that you wish everyone would stop believing and why? >> Uh sleep myth. Um I think people are obsessed about certain supplements. Everyone, and maybe this is not only about sleep, but certainly we see it in sleep. Everyone just wants like, "Hey, what's the one supplement that I can just get and it's going to fix all my problems?" And for most people, that's not the way to to kind of get started. And you can get try optimizing the right magnesium and magnesium could be impactful and maybe vitamin D. Low vitamin D levels can affect your sleep.

And oh, there's like different teas. Hey, if it works for you, do it, but I think people hyper fixate on "Hey, what supplements should I take to fix my sleep?" And often that is >> Right. >> very far away. And related to supplements is melatonin. So, taking melatonin, people think that it's a drug to help you fall asleep. And melatonin is not a drug, it is a hormone, and your body already has melatonin. Very few people need melatonin. You can use it to change your rhythm, but most people don't need melatonin.

And there's a lot of reasons why we can go deep into that. So, I think that is probably the most uh overused myth that people just want to take 5 mg of melatonin, which is borderline absurd for most people. There's few exceptions, but >> And in American culture in particular, everybody just wants a quick fix, and so often times that's medication or supplements, but going back to your three tips, there there are more natural ways to get the results you need.

Uh well, this has been great. Thanks so much, Leon, for taking the time. I love having you on the podcast. Any parting words to leave the audience with before we wrap here? >> No, thanks, Phil. If everyone is interested in in just adding sleep or or apps uh or AI and clinical services, I'm always uh I love chatting with with similar people, so it just, you know, you have my contact info, and it's going to be in the in the notes, so love meeting people in the community.

>> Awesome. Thanks again, Leon. >> Thanks, Phil. >> Thanks again to RevenueCat for sponsoring this episode. If you're building a subscription app, you can use RevenueCat to deploy native paywalls, run continuous AB tests, automate winback campaigns, and unlock more revenue. I work with dozens of subscription apps, and most of them are already using RevenueCat as their single source of truth to power their businesses. Learn more at revenuecat.com/subversive.