30 April 2010

Coasters, Consumers and A/B Testing

So Space Miner is going on a coaster this weekend. So what's a coaster?

Basically, a coaster is a term I'm coining to describe bargain pricing your app to take a ride up the charts. If things go well, then a day by day plot should show your app climbing up the charts very quickly (like the ascent on a roller coaster). Eventually it will peak, and hopefully maintain it's position for a few days before starting to fall again. Once the fall starts, the usual response is to return to normal pricing, which accelerates the decline, at least on the top download charts.

A number of companies do this to stimulate sales. You'll see Gameloft do this for brief stretches, and a recent example was the first week of April sale where EA discounted all their games (not all to $.99, but it is EA, not Joe Schmoe after all). However one game, namely Simpson's Arcade, went all the way to the top of the chart. At the end of the week, guess what? They left it there.

Why do this? Well there are a number of reasons, but the basic one is obvious - if it goes right, you'll make more money, even though your margin is reduced. Sales on the chart are in no way linear, and the top games sell A LOT more than games not in the top chart. I believe Lima Sky announced something like 4 million sales recently, so that tells you something.

However it may not go right, and there are consequences.

The obvious one is that your sales don't increase enough to make up for the difference in price. In Space Miner's case, we are going to 1/5th our normal price, and will need an increase of at least 5x to break even. And really something more like 8x-10x to make it seem worthwhile. Short of that means we're probably wasting our time.

Another consequence is eroding consumer confidence in the price of your games, and pissing people off. For whatever reason, $1 on the App Store is like $10 in any other marketplace, and you'll get more grief for that $1 than you can imagine. I do believe it's the vocal minority though - so time will tell how much stock to place in those complaints.

And finally, once you place your app at the hands of the $1 masses, expect to get a lot of customers who are not necessarily interested in your product for the right reasons. They do less research, spend less time on the App, and are more likely to hit you with the 1-star Rate on Delete. In general, the lower your price point, the lower your average ratings. There are certainly exceptions, but it follows normal consumer patterns. A great book that covers this type of behavior is Predictably Irrational by Dan Arielly, and it's a good read if you're into that stuff.

So why are we doing this with Space Miner? The official reason right now is marketing - we want more customers because we're launching a new game soon. With visibility being such a huge factor in App Store success, you could argue very easily that for a given daily profit, it's better to pick the price that gives you more customers, since if they like your App, they'll be likely to look at your other products favorably. We've also added a More Games type section to allow easier discovery from within our games. So hopefully we can get some sales synergies happening.

The second reason is probably, in reality, the real reason. And that is that I'm just damn curious as to what's going to happen. It is a compelling sale, since the game is normally $4.99, so if nothing else people should be interested in that. But do we have enough visibility that there will be 5x the number of purchases? I really have no idea. And what will be our peak? Can we actually break the top 25 with a game like this? I've certainly seen much worse games grace the top of the charts. We did a press release to lay the groundwork, but until we see what happens this weekend it's all speculation. It's pretty exciting to watch though, so it should be a fun weekend.

Our plan is to run it for three days, but like EA, I'll reserve the right to change my mind if we make a lot more money this way :)

Following up on last week, we did run an experiment with AdMob doing some A/B icon testing. We tested about six different icons, and the results were, unfortunately, not beneficial. Let me explain.

We had some technical difficulties just getting the test running. It was pretty difficult to get an even number of ads served in a single campaign, so I had to do a lot of manual turning on and off of different ads to get a high enough volume that each one had statistically significant results. But because of the different times of day ads were running, and since some ads just got served a lot more than others, it was hard to tell what was causing the deviations in ad performance outside of just the icons.

Be that as it may, at the end of the experiment the best performing icon was - the icon we're using. Go us! So when I say not beneficial, I mostly mean that there is no clear "better icon", at least superficially, for us to be using and hence no panacea to App Store success.

The sale is on, so it's time to sit back on the couch with the iPad and keep hitting refresh on the hourly rankings. I don't know if this will be profitable or not, but we'll have a bunch of new customers at least, and hopefully fans. Next week I'll update with some statistics from the sale, and any lessons learned.

12 comments:

  1. I need to say... I've become a huge fan! Im pretty impressed with Space Miner, i knew it before, but i dont buy anything more than 2.99 because in Brazil 4.99 becomes easily more than 15 bucks, which means a full month of bus to university! But if you guys made 2.99 i didnt had problem at all! Was .99 the best Idea?! Please, when the coaster ends, tell us! Just one more thing, i think you guys are doing right by telling us, fans, how this stuff works! Is amazing to see the confidence with us by sharing these thoughts! For me and for a lot of people, the game is amazing, and becomes better because of the developer trust in us, players!
    THANKS!

    ReplyDelete
  2. This will be interesting to watch to see if it is successful. I love the game and just bought it on Saturday, but I didn't get it because it was $1. I bought it because of the Lite version. I played through it and immediately went to purchase the full game, expecting it to be $5. I was surprised to find it on sale.

    Anyway, you folks did a fantastic job and I hope that you are very successful. I also hope that you will extend the title with additional missions and options, either for free or as DLC. Reading through the comments in the app store, there is a lot of interest in seeing the game continue.

    ReplyDelete
  3. I love the game and found about the .99 sale from dealnews.com. I downloaded the lite version and played it all the way through and then purchased the .99 version. The graphics are beautiful and the games play is wonderful. I like the sense of accomplishment I get from mining and upgrading the ship. I'm looking forward to your next game. Space Miner looks great on the iPad.

    ReplyDelete
  4. This comment has been removed by the author.

    ReplyDelete
  5. Have you guys tried an icon with a character's face on it? Character seems to attract the eye quite a bit.

    A trick we like to do with our own icons is to take a potential icon, photoshop it into the top 25, and see if it jumps out at you at first glance. If you don't immediately focus on it from a millisecond look, the customers won't see it either.

    And yeah, A/B testing on AdMob is a pain. You have to babysit the process quite a bit. It's better than just taking a shot in the dark, though.

    Thanks for the great game!

    ReplyDelete
  6. Whoa, check back a week later and there are comments!

    @everyone
    Thanks for the compliments on the game! We are not planning any more major content for the original game, and we're concentrating our efforts now on a free to play take on the game, and a full on sequel.

    @natalia
    We did try character faces on the AdMob test. The big problem we had with our quick and dirty test though was that our portrait artwork for the game didn't shrink too well (and the artist who did it was out of house.) Our new game doesn't have a face on it per se, but does have a cute ninja, so we'll see if that has any appeal.

    ReplyDelete
  7. So yeah, this game totally rocks. I picked it up at 99c and would have paid full price for it had I tried out the free demo first. Anyway, did the coaster go as you had planned? I imagine it probably did, but it would be great if you could share a little detail with us.

    ReplyDelete
  8. The wonder of creating woodworks is that it can be as detailed or as simple as the crafter chooses. This means that lots of wood items and woodworking equipment may be used or, conversely, very simple wood items and woodworking equipment may be. Coasters can be made for their traditional use, such as placing glasses on them to prevent rings on beautiful and often costly furniture. Or consider non-traditional uses for coasters. Using fitting, appropriate dimensions, faux bamboo coasters can be used as kitchen pot holders, placed under plants in plant holders, or under pets' food and water dishes.

    Wedding Favors

    ReplyDelete
  9. Space Miner sparkles with originality, and its top-down action is pretty much as fun as its banjo tunes and hillbilly dialog.It's similar to a space-age western,with arcade combat and the final frontier aroused with a touch of silliness.It's right away amiable and altogether entertaining,regardless of the fact that it isn't totally accessible.A vein of quality runs through the game,in spite of not as much as ideal controls and trouble irregular characteristics that keep it from being pure gold.
    ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,@@
    hidden movie game for android


    ReplyDelete
  10. Even if you rent test equipment on a long-term basis you are able to upgrade and change your equipment to ensure you are using state of the art technology. Textile Testing instruments

    ReplyDelete