Do This If Your App Retention Sucks

What is appnation. It is Steve P. Young founder appmasters.com. The place you go when you want to actually pack content related to helping you grow. Yeah. Downloads, and more importantly, your revenue and part of that is going to be retention today. I've got phenomenal guests for you. We're going to talk about all the experiments so that you actually don't have to run all these experiments that he's running. We're going to learn about all the experiences, experiments that he's been running, and what's really worked on his end, but without further ado, let me introduce the guest.

His name is . Galot he is the CEO and co-founder of Listerine go check it out. It is the mystery like lists and story combined into one. It's gonna give you the best content, curated by the experts available on Google Blake. And of course, iOS, Denise. Welcome to the show. Thank you, Stephen. Thank you for having me. I'm a listener and we actually learn a lot and get ideas for experiments from your show. Um, so yeah, as a, as you've mentioned, Lee's story is, uh, an opinionated and mission driven content app.

Uh, we help users get. Happier and smarter through the content they consume. And as you said, we do it by two things. One, we refine the content and the web to find its finite list of recommendations, um, curated by experts and mentioned in newsletters and by curators. And to help you actually consume at least some of those pieces of content by giving you tools to create better habits for content consumption. So. That's kind of like, um, what we do.

Why do you say finite? I mean, is it the world? All INFINIT like an endless loop, like Instagram, you just keep scrolling, scrolling, scrolling. Yes. Um, we see that the more content you have available, the less you actually consume. So we took an approach that actually tried to do the opposite by saying we have less content, not all the content in the world, just the content that matters. The content that people who are experts in various fields pointed to.

We don't have infinite feed. We don't have all the topics in the world. All we care about is you reading something and feel very, very good after reading it, feeling that you learned something. Well, you think I want to talk about the strategy that helped you go from less than 10%. We've learned retention to more than 40%. I think retention, especially for an app like yours, it's going to be huge, right? Like in any app, like retention is so huge.

What have you been doing? Yeah. So retention is definitely important for us, especially as, you know, as an app that by the title or just it, what it does, it's just another content app within a broader, um, ecosystem of content. So. Our main strategy and it took us a little bit to get to, but the one that works very well for us is experimentations. So we realize that the only way for us to truly achieve our mission, our goals is one, put those, put that mission in front of us.

So we created a manifesto and we published that manifesto. And. To by being, data-driven not opinion driven where highly opinionated team. And we figured that the best way to kind of help us grow and improve is by experiment in experimentation by driving traffic, put options. Or different experiences to users and see what works and, um, yeah, to your point, we saw within the last couple of months, we saw a huge increase in, uh, retention in second, first week and second day retention.

Um, so experimentation that's the, that's the strategy. And, um, you know, the way we run experimentation is. Figuring out one before we even started, um, is to figure out what is an experiment? How do we measure how many users we will need in order to learn something? So. The first tactic we kind of took was to run AA tests. So basically using the same experience, just randomizing group of users and see at what number of users and what length of experiment we get to the same kind of results.

So this helps you plan later on how long an experiment you can run, how many users you need to drive to every experiment. Second, what I mean, what did you guys find from the user perspective? Cause I know when we talked about this during our mastermind meetings where it's like, all right, you are running this experiments, but how many people do you need? How do you figure that out? So we learned that depending on where you are in the funnel or where you are in the app, um, that number might shift.

But for up users for, for example, we, uh, we ran an experiment of the level of freshness of content that you'll see. So whether it do you, as an app user, will you see stories that were trending by newsletters in the last week? They. Month. So we ran an experiment showing each group of users, different level of freshness. And we started to see results only when we had around, um, 600 users in the experiment before that the data was just too noisy.

And, you know, if you're. Th th the more users, obviously you have the faster you learn, the more conclusive the results will be the better they effect, but the minimum of users we see that we need to see an experiment is around 600. The other thing I want to get back into, because I definitely want to learn more about some of the experiments that you ran, like getting into the details, but you said you created a manifesto.

So one, was there a template you followed because I want to create a manifesto too. And you said to you that you published it somewhere, where did you publish it and what was in the manifesto? So, yes, um, one. We didn't follow a template. Obviously when we roll it, we looked at examples. One example that I liked a lot because they are known for their opinion, native approach is, uh, you know, uh, base camp and with their new hay product.

So that was an inspiration for kind of like the cystic liveness of the manifesto. Um, the manifesto is. Basically state clearly what we believe in and what we do not believe in. We created it for ourselves because we wanted something to align us so we can better prioritize, better, decide what fits into the product and what not. And then we felt it will actually be a very good, um, marketing tool to allow us to establish.

We'll trust with our users. And, um, it actually also driven drove actions on our behalf. But the right now you can go to Lee's story.com and at the header, you'll see our manifesto and you can open it and, and kind of read what we believe, which is, think of the, if I said, what we want to do is to help you find the best content and actually consume it. This is kind of like, what are the philosophies that we're employing.

To do that. So for example, you've asked about the infinite feed. We say no infinite field. Got it. Um, more human curation, less technology, those types of beliefs. And they're mainly set publicly to help us. Kind of burn the bridges. That's what we believe it. That's what we're striving to and you need, I want to get back into the experimentation. You see a lot of experimentations. Are there ones that you want to pinpoint to be like, well, this really we saw, like we made this specific change we saw retention just was 40% day.

One day seven retention is phenomenal. Um, so it's hard to pinpoint one experiment that moved the needle so much. It's I always gave also for the team, the example of, you know, we want to lead a fire. It's not one match that do it all. You need to collect a lot of twigs and a lot of matches and put some oil here and there. And eventually you see huge result, but it's a lot of experiments. So the thing that you do realize is that the.

The the higher on the funnel. You do an experiment that works the bigger, the down the road results that you'll see, meaning let's say you have the app and you have an onboarding. And at the end of, at the end boarding user can take an action. Then you can run an experiment on that action. But to get users, let's say the 600 users that you want to get users to. That experiment. You need to actually get those users to buy two onto your product and download it and actually open it.

Imagine that you do that AB testing on let's say the Android store. So every user who see your messaging, and if that messaging, um, Relate to what you want to test in the onboarding. You can actually see a lot of users already in the store. They are cheaper. They, you, you will find more of them and you can get faster results. So the higher in the funnel, you do the experiment, the faster, the more impactful, the cheaper they experiment, the results that you'll get.

The, the experiments that we see that do work is when we. Use very concrete value. You know, I gave kind of like a remission. And when we put our mission in a lending page and explain what we aim for, you know, we want to make you happier and smarter and you have a call to action to download. This might work, but if we say read later, not never set your reading goals, something that is actionable. We see much higher response so that an experiment, we, we always run experiments around messaging.

And what we learn in general is that the more concrete you are in what you're going to get, the better, the expectation will be the better the retention will be as well. Well, let me try to understand what you said with the week later. You said, instead of being like, Hey, download now, here's what our mission is. You said. Something, something about read later. What did you do specifically with that? Was it on the landing page?

So, so, you know, our mission is to help you get smarter. And there are means we take to do that. There are features we have, one of the feature is read later. Our take on relator is. You know, when you see an article today and you say, Oh, that's an interesting article. I need to read it. I'll save it to my pocket. I'll save it to Instapaper you do it. And guess what? You never actually go there. So, you know, As our belief is that one story can be much more inspiring than seeing a lot of titles and scrolling through them.

So we said, what if read later will be a real read later? Like you will read it only later. So, um, instead of we, we think of it as the real retaliate later, so, you know, our messaging will be daunt. Procrastinate. Don't read, save for never the real read later. So we'll say at least three helps you get to the stories you never thought you will actually read or make use of your pocket stories that you never get back to.

Wow. So the value it's very good. Clear what you're solving for. It's less aspirational. It's more utilitarian. So we see that in general, that approach works better, at least for, you know, our type of app where just speaking about content, the quality of content discovery is something that we are all already blind to. Everyone tried to sell better content content that matters. But if you show the actual means by which you will help a user.

Consume we see that users are more responsive to that. Every experiment, every feature that you build, um, sometime it's, it's easy to focus on, you know, okay, here's the app. We want to change retention. Let's just do it. Test what you don't realize is that user coming from, for the app for a reason, as I've mentioned before, the more specific that reason, the more likely their expectation will be set on that reason.

So if you. Test a feature and that feature is not aligned with where you brought the users from. Then it doesn't matter what they experiment is. Users will just have different expectations and they might leave because you kind of, um, did something that different than what they thought. So one kind of insights that we have is that an experiment is. Full stack experiment. If you want to test something substantial, you need to make sure that when you bring users, they know what they're coming into.

Otherwise they're just targeting the wrong audience. And the second is what are the bottlenecks on the user journey are the battle button. Next next are. Conversion, are they secondary retention? Are they one week retention? Are they frequency? And every change you do kind of reset the whole button, the whole funnel. So if you change something in the app, it might mean you need to change the messaging. And it's, it's kind of everything connected.

And here it was. Yes. The second day retention is the goal, but we learn that actually we measure it in the wrong place.