Coasters, Consumers and A/B Testing

So Space Miner is going on a coaster this weekend. So what's a coaster?

Basically, a coaster is a term I'm coining to describe bargain pricing your app to take a ride up the charts. If things go well, then a day by day plot should show your app climbing up the charts very quickly (like the ascent on a roller coaster). Eventually it will peak, and hopefully maintain it's position for a few days before starting to fall again. Once the fall starts, the usual response is to return to normal pricing, which accelerates the decline, at least on the top download charts.

A number of companies do this to stimulate sales. You'll see Gameloft do this for brief stretches, and a recent example was the first week of April sale where EA discounted all their games (not all to $.99, but it is EA, not Joe Schmoe after all). However one game, namely Simpson's Arcade, went all the way to the top of the chart. At the end of the week, guess what? They left it there.

Why do this? Well there are a number of reasons, but the basic one is obvious - if it goes right, you'll make more money, even though your margin is reduced. Sales on the chart are in no way linear, and the top games sell A LOT more than games not in the top chart. I believe Lima Sky announced something like 4 million sales recently, so that tells you something.

However it may not go right, and there are consequences.

The obvious one is that your sales don't increase enough to make up for the difference in price. In Space Miner's case, we are going to 1/5th our normal price, and will need an increase of at least 5x to break even. And really something more like 8x-10x to make it seem worthwhile. Short of that means we're probably wasting our time.

Another consequence is eroding consumer confidence in the price of your games, and pissing people off. For whatever reason, $1 on the App Store is like $10 in any other marketplace, and you'll get more grief for that $1 than you can imagine. I do believe it's the vocal minority though - so time will tell how much stock to place in those complaints.

And finally, once you place your app at the hands of the $1 masses, expect to get a lot of customers who are not necessarily interested in your product for the right reasons. They do less research, spend less time on the App, and are more likely to hit you with the 1-star Rate on Delete. In general, the lower your price point, the lower your average ratings. There are certainly exceptions, but it follows normal consumer patterns. A great book that covers this type of behavior is Predictably Irrational by Dan Arielly, and it's a good read if you're into that stuff.

So why are we doing this with Space Miner? The official reason right now is marketing - we want more customers because we're launching a new game soon. With visibility being such a huge factor in App Store success, you could argue very easily that for a given daily profit, it's better to pick the price that gives you more customers, since if they like your App, they'll be likely to look at your other products favorably. We've also added a More Games type section to allow easier discovery from within our games. So hopefully we can get some sales synergies happening.

The second reason is probably, in reality, the real reason. And that is that I'm just damn curious as to what's going to happen. It is a compelling sale, since the game is normally $4.99, so if nothing else people should be interested in that. But do we have enough visibility that there will be 5x the number of purchases? I really have no idea. And what will be our peak? Can we actually break the top 25 with a game like this? I've certainly seen much worse games grace the top of the charts. We did a press release to lay the groundwork, but until we see what happens this weekend it's all speculation. It's pretty exciting to watch though, so it should be a fun weekend.

Our plan is to run it for three days, but like EA, I'll reserve the right to change my mind if we make a lot more money this way :)

Following up on last week, we did run an experiment with AdMob doing some A/B icon testing. We tested about six different icons, and the results were, unfortunately, not beneficial. Let me explain.

We had some technical difficulties just getting the test running. It was pretty difficult to get an even number of ads served in a single campaign, so I had to do a lot of manual turning on and off of different ads to get a high enough volume that each one had statistically significant results. But because of the different times of day ads were running, and since some ads just got served a lot more than others, it was hard to tell what was causing the deviations in ad performance outside of just the icons.

Be that as it may, at the end of the experiment the best performing icon was - the icon we're using. Go us! So when I say not beneficial, I mostly mean that there is no clear "better icon", at least superficially, for us to be using and hence no panacea to App Store success.

The sale is on, so it's time to sit back on the couch with the iPad and keep hitting refresh on the hourly rankings. I don't know if this will be profitable or not, but we'll have a bunch of new customers at least, and hopefully fans. Next week I'll update with some statistics from the sale, and any lessons learned.