Try These A/B Tests to Increase Your Subscriptions (APS SF 2023)

hey we're going to try to make these as interactive as possible all right so if you want to interrupt me anytime feel free I was the guy in class that would just like just talk about it right the teacher is like raise your hand so if you got any questions James is in the back he's going to come around just so we have the right audio for it so interrupt anytime all right cool let's get to this all right I've done some AB tests I've done a lot of ab tests and we're going to break down some of the most interesting AB tests that I've done all right just some social proof been in the app space since 2011 started a podcast in 2013 on the side right to try to learn from my heroes in the space so I interviewed the co-founder Shazam created the crossy road Mafia Awards color switch I try to talk to as many people as I possibly can right through that people started coming to me for marketing help and so I end up

leaving my corporate job to do at Masters full-time so theoretically we're an at marketing agency but I don't like to say I'm an agency I'm a media guy I like creating content all right again more social proof all right I love this slide and this is Rosie who was at APS London she had this slide I was like Rosie I'm going to copy it and it's like improving the monetization has Forex to impact rather than focusing on acquisition right and so really focus on the monetization side because it has a greater impact than just trying to lower your cpis so I'm going to talk more of the top of the funnel I know a lot of people are talking about churn and stuff like that but I'm going to talk more of the funnel and again we're going to make it as interactive as possible so loosen up your arms because I'm going to ask you to raise them I know so hard all right test number one here's what we did to engage or not to engage now I'm going to

trying to like pick some of the people and I these aren't the most accurate slides meaning calm does engage but like you know for the beauty of the story we're not going to have them engage so a which is Headway they're engaging they're asking for a lot of interaction during the onboarding process B Comm is just like hey look at us we're really cool 90% of our members blah social proof so who thinks that a is the winner in this AB test thank you Nico you're the only one I appreciate you who thinks it's me all right more hands cool it's actually a the more the more engaged you can get your users the better it is why because Lana talked about it customization people think that you're customizing the entire experience for them and I actually thought that's what they were doing I was using splice and I was like I think they're customizing this for me but as I talk to more people it actually increases conversions too

and the cool thing is the more you engage like you can see Headway super long all right I'll get out of the way I don't want to be in your guys' little photos the more you engage the more data that you have to then use in your acquisition so if you're finding like people I'm 3544 by the way 3544 male who's interested in non-fiction books they lead to higher conversions well guess what go to meta find 35 44 males who might be Founders and get them right you can get them copies so you can see super long pay wall I mean onboarding experience with a l of Engagement oh and when you're engaging guess what you can do you can have these screens that say I'm customizing your experience for you even if it does nothing else we've seen a 10% conversion uplift by just having this screen thank you I love the laughs too keep those coming all right stoping no interaction so far okay let's get to pay wall on on board oh

okay that was after account creation let's go back so you can see the account creation was right here now you can skip it way up there we can't skip it so but I created an account thank you all right pay wall and onboarding here we go you hear what I'm trading all right all right here we go so I picked on Flow period tracker onard right super long by the way too so and then the pay wall whereas B has no pay wall so here we go who you think who thinks it's a who thinks you shouldn't have the pay wall on the onboarding it's too soon who thinks it's B all right cool thank you still not no shows but it is a so if you see 2x more sales granted look I know I get it before they even use my app they're going to pay me yes let the data speak for itself right it's data I don't like opinions a lot of experts they base a lot of their stuff on opinions I like to base it off of data when I first heard this in 2018 by the way I was like

what really and then it works right and so if you do nothing else after this event say thank you to James and then lastly go put that pay wall during the on onboarding process it does work all right you can see Comm does it headspace no more Robo now where do people usually buy so of the people who end up buying your product not those non- buyers I don't care about them that much other the people who end up buying how many people think it's over 50% how many people think the opposite it's under 50% Thank you James it is around 60 to 80% of the people end up buying buy during that onboarding experience yes before they even use your app all right no questions okay discounting on onboarding all right let's go to this test now a has a discount you say no to that first pay well like Flo's like no you want to save 33% off or B you're like all right he said no here go on your Merry way so who do you who thinks

today who thinks it's B because it's too sou a Nico it's a winter winter chicken dinner you want to have why I'll tell you exactly why back to my early discount early look I believe I'm going to talk a lot about funnels in the top of the funnel like subscription because for me I like to think of phases one I got to make sure this product Market fit and how do I figure that out I got paying users so I'll worry about churn subscriber churn sorry v a little bit later once I've try to convert them right and then I'll start embedding it in because I need to get product Market fit look I like to keep things simple 60% to 80% of your people who buy buy during that onboarding process we rewards work I've seen other ways other strategies might share with you privately if you come and ask me but they work discounts work think with Google right 70 this is amplitude 77% of your an average app loses in just three

days three and then six days this is according to adjust six days users tend to keep an app before losing interest and deleting it all together so that's why I say discount early and I'll use Headway as well I just had them on my podcast all right we did a great live stream I talked to yaa who's and sheal we talked about this I'm tell me tell me the secret you know what's going on because you guys do a lot of discounting so here's their pay wall right you say no you're on the homepage free gift for you this opens up it opens up every single time second open and every app open it opens up look at their discounting here now all their things are locked Lana said this I think it was like show your content don't hide it behind a pay wall so that all their content is there you pick a book you see their pay wall you say no they give you a 70% offer now here's my opinion okay the best app that I've worked with has a

25% activation rate now I see that as 75% said no to me and so this discounting is to try to get the 75% to convert not the initial 25% and when I asked Headway I was like tell me what makes the most money they said it's the main pay wall it's the $89 offer not this 30 obviously because you need a lot more right it's a third of the the pay but it's that first initial pay wall that makes them the most money anyways so that's why I feel like you got a discount early on you're my favorite person right now is this like a trend that you see across all apps AC all categories I think so but I want more data to prove that out but yes the users that pay the $90 up front learn about this $30 option I don't have that data but I don't think so because you would never see this if you were already on the 90-day offer right because that initial pay wall during the onboarding is where most people buy and so they'll never see

it you can hide it right and I do know that it's working for a lot of my friends too this showing this pay wall showing a a discount right on onboarding right all right test number four I had to put just one screen to remind myself what the heck am I going to talk about later all right this is a from taplytics but I wanted to grab this just look the thing that I feel is the two places that you need to really optimize is your onboarding and your pay wall all right then you figure out the churn all that stuff subscriber churn I feel like those are the two most important thing look variation a so I'll just tell you the backstory so today takes with them they're like hey I want to get more people they they didn't know about this Rush ticket stuff right have a little pointer oh cool it's not working all right anyways they wanted to talk about that right so I'm gonna I'm not going to make you guys play a game

it was B all right just by that one little screen that at the bottom that talked about the hidden feature that they had that not a lot of people knew about they saw a 60% increase in registration and a 10% increase in subscriptions so that's why I'm like this whole thing was just to say look how one screen can make an impact now you're going have to AB test one of my clients he had seven screens on the onboarding process he's on his first date he's like what do you do I'm like I'm an app developer well let me take a look at your app girl friend they're married now decides to just swipe up after so many screens so he went from seven to four onboarding screens and he saw an impact in subscriptions so you're going to have to AB test there's no right number into the number of onboarding screens but I think the best practice is to try to get them to engage as much as possible in some apps long

onboarding sequences work better whereas certain apps it's just like very short three to five screens okay test your onboarding there it is so this is one of our clients too we have a bunch of onboarding screens then we ask you to sign up and then we show the pay wall so here's what we did we just flipped the two and I was like let's see what happens now again it is a hard pay wall meaning there's no X and there was no X to skip the sign up too so just keep that in mind when I'm when you're looking at this data but we saw 52 we saw twice as many pay wall views cuz the number one metric that you're going to pay attention to is your first open or install to pay wall view you're going to miss 100% of the shots you don't take right so his pay wall views went up from 40 94% and his trial activations went up 190% so that's 3x and a 52% increase in sales just by flipping the two right showing the pay

wall first and then forcing people to register now you might be thinking steeve yo yo yo yo I need them to register like otherwise I can't do it well here's what C does they're like hey if you want to you can skip the sign up but if you activate a trial guess what happens you can't do nothing else but sign in and that's it so don't give me excuses all right hard or soft payroll now he's one of our apps a b the only difference b as an next that's it see that little X it just means close all right who thinks let me reverse the engineer this who thinks it's B that one the little X who thinks it's a that one all right good job good job everybody it's 2x more sales don't believe me here's the data we had a hard pay well we launched with that now again theal where's my man he talked about this during our podcast interview he's like look from adapt he said I was like how would you launch a brand new app he said I would test a

hard pay wall why I was like what the hell is a hard pay wall he's like well it just means there's no X and I was like funny enough we just did that and my co-founder is like hey this is how we built it I was like yo that's scammy I want people to use our app love our app and then pay for it but the data says I was wrong right now again you could be thinking till Hest Point short-term gains or short-term wins for long-term gains or maybe it's a long-term loss now we're fixing it right in my opinion we're at a point where it's a six figure business now now I can start thinking about okay subscriber turn right if I'm not at the 10,000 I'm still at the 5 ,000 maybe that's enough but now I'm like okay maybe test a soft pay Woll we've already proved product Market fit now let's test a hard pay or soft paywall right put that X okay now if you don't want to do that and be more aggressive look you can just

lock up the content and that works pretty similarly if you just lock up a lot of your content now our app is very simple so we can't do that but if you have a lot of content you can just start locking it up too and same with Headway so here's some examples of a stiff hard pay wall no X right here's this jigsaw free doesn't work no X they got some social proof here here's semi hard pay walls you got I think it's lingo kids but you hit view all plans you then see this then you see maybe ler so there's really no X on that first initial pay wall until you hit view all plans and maybe later then you get into the homepage screen okay test it out right here's another one I love I love zero been using it for years it's an in casting app I have no data on this but I do want to test this they're 7 days free that's their annual start trial they're monthly it's just the first month it's $3 right no trial and then renews at 60

I want to test this I don't have any data yet but this is very interesting guess what there's no X right 5 minutes later literally if you guys pay attention up top it's 8:47 p.m. 852 I get start using zero for free no subscription required I tap on that get the normal payall suddenly the X appears and then I can use the app for free right look they didn't have this when I started using the app do I pay them no but would I have maybe all right test number six how many plans should we show should we just show one should we just show two so here's an interesting one this is for a very very big client so you know the data is statistically relevant okay all right who thinks it's a $10 product by the way I had to just find a figma on this pay wall so this is not the actual pay wall this is I just need to recreate it because of this case study now the $10 a year has a 7-Day trial that's the only

difference between the two B has no monthly all right who thinks A1 right here who thinks B1 cool a it's a you would think for a $10 product here's what happened all right it's $10 nobody's is buying the monthly they're like let's just remove it nobody's buying it so they removed it and they the sales just tanked why because you need to see some comparison so I believe in the rule of Threes okay and adapt shout out to them most probable combination is three plans now you have to test your LTV and all that jazz okay because as you have more products your LTV can actually go down products mean need more plans right so you're going to have to test it yourself but here's how I like to do it going into my my pay wall the cheap plan right monthly weekly for some categories of apps highest LTV plan sometimes it's weekly sometimes it's going to be yearly and then yo if you Steal This I know I said it first

it's the decoy plan right it's how do you get something so similar so for our app right the monthly is 10 bucks the annual is 60 about let's say 80% by the annual and then the six month which is semester I know we worded it poorly but is a complete decoy to get you you know if you think about like psychology there's price anchoring I like to do decoy just a little bit more gets you to the ideal outcome so I'm trying to yeah Lana tried your % off I like that no that three that one three all right don't forget our QR code all right next number seven is test your call the action super important okay all right so a you have a star the star just goes to the pay wall you see up top B has a crown all right here we go one last time ladies and gentlemen all right need your hands I need Almost 100% participation if we can get there all right here we go let's try it hey fa who thinks it's a oh who thinks it's B that's everybody oh

look at that good for you guys okay it is B we saw 57% increase by just changing the icon around now I didn't know at the time I was looking at the client's app and I was like Yo dude I don't know about the star thing but I've only seen a crown he's like funny enough Steve we're AB testing this right now I got permission to share it okay so thank you but he saw you can see you can see it the crown had a 20% clickthrough rate now the key is ATT trct to subscription into right because we found that unlock everything High click through rate not as impressive conversion rate so unlock Pro features lower click-through rate higher conversion rate so think don't just look at the click-through think about the actual trial activation but that simple icon change made a huge difference and so the the two places that most people buy are going to be on your onboarding and your homepage those are the call the actions that i' be

testing a lot of times I see like what GoPro right like go get premium like dude that nobody wants to get premium nobody wants to go pro right they want to unlock something you can chat GPT chat gbt tell me right a sales copy they're going to start with unlock unlock anything so unlock generally works better so blue Throne these are I'm an adviser for the company but they AB tested they're called to action from get plus all the way to all these different ones which one what do you think one Olivia what do you think one 1 2 3 4 which one one three three okay it's actually four I thought it was three too okay I get it wrong all the time so that's what you want to do is just test that call to action as much as possible that one little text change can make a pretty big difference and this is a big app so 133% increase is a lot better look get Plus versus all these other things one right like get plus was the

worst out of all of them all right there's some homepages here we saw 12x or Revenue just by messing around with the homepage copy too and the pay wall and then here's Headway so here's my big slide again two things you should always be testing onboarding pay wall here's what I think is the best practices test on your own not always going to be 100% right but this is what I feel is the best practices and these are the two areas that you should always be AB testing okay that's it I got a QR go too I thought I was being cool you want to check out our YouTube channel we got close to got over 30 th000 Subs but just hit that wait I I heard an Applause you can do an Applause now thank you Miss thank you all right I do have a bonus suckers all right one last thing I got to do the apple one more thing right trials now look I know there's some debate out there should you have a trial or should you not especially for chat gbt apps

right AI ABS you incur costs now one of our clients it's an invoice maker he ended up launching ing without trial then we switched it to trial not going to BL you play the game again but we saw 420 increase in sales when we added trials so I feel like during the onboarding process here's a slide just to prove it you should have free trials That's My Philosophy but what if they said no to that initial pay wall and they start using the app should you remove the trial now it's a lot more work you're going to need like four subscription plans one without a trial one with a trial right here's what one of my friends in the community saw we did a YouTube video about this so he removed the trial after you started using his app he's like uhuh no more trial because that trial the sub rate is never going to be 100% the highest I've heard is like 70% right and so he removed the trial he saw four 3x that's close to 4X he saw 4X more sales

he submitt $122,000 without the trial and he made $4,000 with the trial once they started using the app right right so now it's a different pay wall there's no more trials you're locking them out but if you have premium features consider removing the trial once they've started engaging within your app one more time I like that Applause one more one more one more thank you thank you