CaddieOpen in CaddieDownload for Mac

How to A/B Test Screenshots on iOS

Shared by Caddie Library · Updated
App Masters10:18Open on YouTube
Chapters show their main points

The bottom line

Apple Search Ads creative sets let you A/B test iOS screenshots by pitting alternative sets against the default on one exact-match keyword. The best-converting set shows which screenshots to ship, and that can lift downloads without touching keywords.

Key takeaways

  • Screenshots decide whether a search result becomes a download. 1:00
    • Only the first three screenshots show in search results, so their messaging matters most.
    • One client's impressions stayed flat while downloads doubled after a screenshot change alone.
  • iOS has no built-in screenshot test, so a cheap Search Ads campaign stands in for one. 0:36
    • Google Play makes this easy, but iOS needs this workaround.
    • The test needs little budget, though it matters most for apps running a lot of paid acquisition.
  • The test must be tightly controlled so only the screenshots change. 2:56
    • Bid on one generic keyword that describes the app, such as meditation, at a high bid of about $2.
    • Turn off search match and use exact match, so Apple can't show the ad for other terms.
  • Creative sets let you swap screenshots, and each variant is tested against the default. 4:50
    • Variants include adding breathing exercises, dropping the vague life-changing line, and adding expert social proof.
    • Video versus no video can be tested too.
  • The set with the best conversion wins, and the winner can differ by keyword. 8:48
    • The campaign shows average CPI and taps for each creative set.
    • Give each ad group its own creative sets, such as sleep stories for a sleep keyword.

We're flat with impressions but our downloads went up by double just because we change the screenshots.

Steve P. Young

Chapter summaries

0:00 Why test screenshots with Search Ads

Search Ads can show which three screenshots should lead, without waiting on a store update.

  • Ad results can show which screenshots deserve the first three slots. 2:01
    • A friend's live three underperformed, and three he wasn't using beat them.
    • Submitting an app update is the slower way to test the same thing.
    • On the client test, keywords stayed untouched, so only the screenshots could explain the change.
    • The test assumes the screenshots are already well designed.
    • Gut feel can pick the lead three, but a test gives evidence.

2:26 Setting up the test campaign

The test campaign is built like any normal Search Ads campaign, and the creative sets come after launch.

  • The campaign is set up for wide coverage on one market and one device. 2:29
    • It is named "screenshot test", for the United States and iPhone only.
    • The budget is about 200 to 300, with no daily cap, because more coverage is better.
    • Calm is the example app, but the bid goes on "meditation", not the brand name.
    • Exact match is shown with square brackets around the keyword.
  • Creative sets are added after the campaign is running. 4:27
    • Start the campaign, reopen it, then open Creative Sets.
    • The page lists the default text and images, and "add creative set" starts a variant.
    • One creative set can be added during setup, but this walkthrough adds them afterward.
    • Variants are built by choosing which screenshots show first, starting with the first three.

5:09 Building the variants against Calm's defaults

Each variant changes only one or two of the default's first three screenshots, so a win can be traced to that swap.

  • The default's first three screenshots are app of the year, the life-changing skill and sleep stories. 5:09
    • App of the year is a no-brainer, so the first variant keeps it.
    • Only the other slots change, so the result points to the swap.
    • The life-changing line is judged "decent", but it makes you ask "really are you really gonna change my life".
    • Sleep stories is read as a sign that Calm knows it ranks first for sleep stories.
  • Video can't be picked as a creative, so it is tested by leaving it out. 6:26
    • The default already carries the video, so a video versus no-video set is the way to test it.
    • There is no second video for the larger iPhone size.
    • The tester added a second set just before recording to make sure the flow worked.
  • The right screenshots depend on the keyword being bid on. 7:32
    • The last variant keeps life-changing and drops sleep stories, because the bid is on meditation.
    • Calm is number one for sleep stories and number four for meditation, so sleep probably drives its downloads.
    • Anyone bidding on sleep sounds would keep sleep screenshots in front.

8:48 Reading results and matching sets to keywords

Results guide what to ship, and each keyword group needs its own creative sets.

  • A keyword that converts better can change which screenshot leads. 9:44
    • Sleep stories may convert better overall, which may explain why Calm shows it third instead of meditation.
    • A sleep stories ad group would test relaxing-sounds and sleep screenshots against the default.
    • The test costs a little money, but it is the best way on iOS.

In the growth library

Steve P. Young, Founder of App Masters, app marketing agency. Shows how to A/B test the first three App Store screenshots on iOS by using Apple Search Ads creative sets on a single exact-match keyword. The payoff is a client who doubled downloads with flat impressions just by changing screenshots, and a repeatable way to find which three screenshots convert best.

More from App Masters · The whole library, and ask it anything

Transcript268 lines
0:00

coming up I'm gonna show you how you can a be test your screenshots on iOS to increase your downloads stay tuned what is up app nation it is Steve P young founder of app Masters dot-com alright in this video I want to show you how you can a be test your screenshots on iOS on Google Play it's really easy on iOS it's a little bit more complicated but I'm going to show you why you should be doing it now it's gonna actually help you with your search ads campaign because that is the tool we'll be using to a/b test our screenshots now if you don't know why these are important in my previous video called three ways to increase your downloads without touching your keywords I really talked about screenshots and how we helped our client double their downloads just by changing their screenshots and optimizing it alright so it is very very important that you get the right messaging within those screenshots because the first three are

1:07

the ones that show up in the search results and they're critical to making sure to converting your users into downloads and with that client we saw a flat and this is what I wanted to see we're flat with impressions but our downloads went up by double just because we change the screenshots we didn't change the keywords because I really wanted to make sure we a be test so you can do it that way by just submitting you update or you can do it the way I'm about to show you now stay tuned so here's what we want to do we want to actually use search ads now you don't have to spend a whole lot of money running this test okay but for critical especially when you're running a lot of paid acquisition campaigns this is critical critical to testing your a be testing your screenshots now one I'm going to assume that your screenshots are beautifully designed if you're in they're not make sure you definitely do

1:59

that but if they are then we're gonna use search ads to a/b test which three should show up first and a friend of mine told me that look by doing this he increased his downloads because he was a be testing the screenshots on iOS with search ads he found that three that he was actually using was underperforming and the other three that he wasn't using were over-performing so it's probably a combination of both I mean you can hide and use your gut to do that but I'm gonna show you how to do that right now alright so first going to search ads and we're gonna create a campaign and I'm gonna pick on comb because I have been picking on them a while anyways but it's a great app and that's why I love them that's why I keep plugging them away so I'm gonna call this screenshot test ok you want to name your campaign I'm gonna keep it in the United States shop tests you can pick other countries if you want but I'll

2:48

pick the United States I want to put like 200 maybe 3 whatever you want on there I'm not gonna put a daily cap because the more coverage I can get the better now for this I'm only gonna focus on one keyword so because I'm calm right I'm not gonna put calm I'm not gonna bid on that keyword I'm put meditation because that's what this app is about I try to pick a generic term that will probably work out the best so meditation you're gonna have to figure out which one which keyword it's gonna be the best for you but for me I would just pick the the keyword that specifically talks about your app and so here I'm putting meditation here I want to only put iPhone maximum bid I might put cuz I know meditations probably pretty competitive I'm gonna probably put like $2 per bid just keep it high for now and then turn this off so this is very key and this is the same way I would set up a search ads campaign all right we're

3:40

gonna I'm just showing you the screenshot test but this is literally how it's a search ads campaign okay now I've got a Chrome extension installed that's why I see the scores you might see a blue bar so definitely check out my other video on the Chrome extension I'm gonna click that now here's the key okay you don't now we want to turn this off right because we don't want Apple automatically saying this one seems relevant to meditation no no we just want to focus on meditation when a user searches for meditation we want to show up okay so this is very popular great and we want to put a big exact match that's key as well so make sure it's exact match these little brackets okay all users demographics fine fine fine now here it is the create creative asset so I'm gonna actually not do this yet I'm gonna hit start right now and I'll show you how to do it you can do one different ad set creative set right now

4:38

I'm gonna go in hit run and then go back into this campaign so meditation my patient now i'ma go up here creative sets and because it's meditation makes it a lot easier as you can see you see the default text and then the image so click on add creative set and here's where we can start testing around so as you can see here the first three and I'll show you the first two real quick see if I can see it up so you got this beautiful great first screenshot app of the year like duh that's a great that's a no-brainer secondly we've got learn the life-changing skill of meditation okay decent decent and the last third one is sleep stories that help you fall asleep okay they must know that hey maybe they're their number one for sleep stories if you check out our other videos but that's key as well right and then breathing exercises to help you relax master classes taught by world renowned experts these are screenshots

5:39

five four and five six relax with soothing nature sounds and this is the last one music to help you focus relax and sleep I don't know about the music part okay now because we're meditation focus right so I might call this I want to test it breathing exercises will help too so I'm gonna call this breathing exercises and I'm gonna pick I really like that first app of the year so I want to keep that in there but I want the breathing exercises in there and instead of showing the this one where it says life learn the life changing it's like really are you really gonna change my life you know I'm gonna put this one master classes taught by world renowned experts so I want to test against these these three defaults right so same thing here this is the video so I don't I don't maybe I don't want to show the video maybe you can't right so you can't select the video verses right now this is the default on this where it's video did so

6:36

maybe here I'm gonna here's the video right they don't have the other video on the larger app iPhone so I'm going to test one I want to keep this all the same two three great oh I went through this already Justin for you guys so I'm called to awesome that's breathing exercises I had to add to because I did this right before I hit red car just to make sure I went through it one more before you guys okay the next one I'm gonna do is maybe video vs. no video okay so now here this is the default so now I'm gonna go to do and then here I'm gonna keep this as default great now you can test right again I'm gonna add one more so this one I want to test like I think this is done really well but maybe I was wrong maybe this is a good screen shot learning the life changing I like this one the best bring the exercise to help you relax so maybe it's like life-changing all right let's keep life changing in there but

7:47

I'm wrong but I want to know master classes so I think we don't need sleep stories because we're focused more on meditation anyways so I want to knock sleep stories out so I want to put master experts is that what they said yeah experts so I want to put experts now this could be a different result so if you were if I were bidding on sleep sounds and I was part of calm then I would definitely have sleep sounds in there but if I'm getting if I know that there's higher traffic for meditation and getting more downloads through meditation then I might put meditation up front they are probably getting more downloads from sleep stories and sleep sounds because I know they're number one for sleep stories and number four for meditation but because I'm bidding on meditation I just want to see what would happen right so I'm gonna pick the first two that they currently have and I'm gonna pick the world experts because I

8:37

think that's pretty I like social proof anytime I can add social proof I'm gonna do that so I'll click those three no I hit save boom and then now that the campaign is running you can start to see okay here's the average CPI here's what's happening here's the taps and then which one whichever one is best converting then that's the one that you should use in the future all right so very simple very easy that you can use and then especially the last thing on a mention is if you are running search ad you should definitely be you should definitely have different creative sets for the different ad groups and different keywords that you're targeting right that's why I started that's why I put this ad group as meditation because I know I can select the right creative ads creative ads to then focus more on meditation okay and then different I might create another ad group that says sleep stories that way I can really

9:33

focus on sleep stories and that where that's where the relaxing sounds and the sleep those screenshots I start putting those in there as actual creatives to show versus the default okay so it's a fine balance and then maybe you find out that hey sleep stories is a way better converting keyword overall right and then that's why maybe calm has sleep story as the sleep stories as a third screenshot rather than meditation so a lot of things you can test here and they can test a bunch of different variables but that's an easy way for you to test so you have to spend a little bit money on Google Play there's another video if you want to check that out on Google Play but although you have to spend a little money on this this is the best way to do it all right guys till next time make sure you hit subscribe and I'll see you on the next video bye