These are my notes from a presentation I gave Saturday at MobiCamp Boston 2.

The pitch

  • You are building a mobile app
  • You want it to be successful
  • How do you do that?
  • Is there something in product development that you can control? Of course, but we all know how to build a great app, so that doesn’t distinguish you.
  • So it’s all about the marketing, right?
  • I surveyed six mobile apps to find out why some of them are successes and other are not.
  • What are the key mobile success factors that will help you win?
  • Sit in on my talk to hear about the survey and find out why some apps succeed and some don’t.

Intro

  • We can all build excellent mobile apps
  • What it is that makes successful mobile apps?
  • If you were to build a new mobile app, what kind of app would be your best bet? What kind of marketing would be your best bet?
  • Be patient–the answers come later in the talk.
  • Disclaimer
    • This is early research.
    • Think of it as the seed of some useful information.
    • Needs to be tuned and improved

Motivation

  • Given two apps that are almost exactly the same, why is one successful but the other is not?
  • How can all the apps we build be successful?
  • Background reading
    • Theory of Constraints
      • A friend gave me the book The Goal: A Process of Ongoing Improvement by Eli Goldratt
      • A business novel. Instead of a dry how to guide, it is a dry novel, telling the story of a business team running a division of a failing company and how they fix it, make it succeed.
      • Got excited about it, read the two follow up books
    • Scrum
      • ToC, Toyota production system tuned for software engineering (really for any business)
    • Toyota production system
      • Eliminate waste
      • For software, see Scrum: don’t do things that have low business value
  • ToC process
    • Identify your goal
    • Identify the biggest constraint on achieving your goal
      • Globally, through the whole system, not just in your area of responsibility (for me, this means not just in software engineering)
    • Fix that constraint
    • Repeat
  • ToC in practice, for mobile apps
    • Identify your goal
      • Make money by building successful mobile apps
    • Identify the biggest constraints on achieving your goal
      • Product development
        • How can I build more apps faster?
      • Marketing
        • How can I sell more apps after they are built?
    • Prioritize
    • Fix that constraint
    • Repeat

Product development

  • Product life span
    • Product development is most expensive, most time consuming, riskiest part
    • Large revenue opportunity at end of product development
  • Definition: everything that goes into building an app suitable for sale to consumers
  • UX design
  • Software engineering
    • Most expensive, most time consuming, riskiest part
    • Address via Scrum
      • Scrum is like Theory of Constraints tuned for Software Engineering. When applied well here, Scrum mitigates this constraint, but doesn’t eliminate it.

Marketing

  • Product life span
    • Many smaller revenue opportunities beginning at product launch
  • Definition: everything that goes into selling the app to consumers and making money on it
  • Given two apps that are almost exactly the same (Music1 and Music2), why is one successful but the other is not?
    • Same product development work, so eliminate product development as biggest constraint
    • It must be the marketing–our biggest constraint on success

Methodology

  • Identify mobile apps to evaluate
    • Identified six mobile apps
    • Due to nondisclosure agreements, cannot name apps, who built them, or for whom they were built
    • Things these apps have in common
      • All apps are currently available to consumers
      • All apps have a monthly subscription revenue model, not one time purchase
      • All are data connected. Getting and storing data OTA on a remote server
    • None are iPhone App Store apps
    • Some are downloadable app’s, some are preloaded on phones, some are web apps
  • Gauge each app’s perceived level of success
    • Ask producer, sponsor: is this app successful?
      • Keep it simple: binary answer, Yes or No
    • Qualitative, not numeric. “I know it when I see it.”
    • Adequate for this study; people have internalized goals for the apps and have no hesitation communicating whether an app is successful or not
    • 3 successful, 2 not successful, 1 not sure
  • Enumerate obvious characteristics/dimensions
    • 22 characteristics/dimensions to help understand why some apps succeed and other don’t
    • Mostly marketing oriented, but some product development oriented
    • State each characteristic as a predicate, e.g. Is it a web app? Is it inexpensive? Does it include streaming video?
      • Keep it simple: binary answer, Yes or No
    • Try to construct the predicate such that the answer should be positive for a successful app
  • For each characteristic/predicate/dimension
    • Answer the predicate for each app: Yes or No.
      • Predicate nature keeps it simple.
    • Derive a success correlation score
      • The probability that a successful app has this characteristic
      • If you have this characteristic, are you likely to succeed?
      • The number of Yes answers divided the number of successful apps
    • Derive a failure correlation score
      • The probability that an unsuccessful app lacks this characteristic
      • If you lack this characteristic, are you likely to fail?
      • The number of No answers divided by the number of unsuccessful apps
    • Combine success score and failure score into a weighted score
      • weighted-success-correlation = success-correlation * num-successful-apps
      • weighted-failure-correlation = failure-correlation * num-unsuccessful-apps
      • weighted-score = (weighted-success-correlation + weighted-failure-correlation) / num-apps
  • Sort dimensions by combined weighted success score
    • High score => correlates to success
    • Low score => does not correlate to success
  • If combined weighted success score is >0.50, then the dimension is a success factor
  • In general, ignore the “not sure” app. It is not obviously either a success or a failure, so its results don’t help.

Results: top success factors

  • High: combined weighted correlation score == 1.00
    • Acceptable number of subscribers
      • A proxy for “acceptable monthly revenue”
      • “Acceptable” different from “high”. Depends on app’s target number of subscribers or target revenue.
      • A new app with 1,000 subscription events per month might be acceptable. An app that had high product development cost and has 100,000 subscription events per month might be unacceptable.
      • Based on your criteria, if the app’s number of subscribers meets its target, whatever that target is, then the app is successful.
    • Inexpensive (Retail price less than $2.75 per month)
      • Consumers will buy it if it’s inexpensive.
      • $2.50 per month is an attractive price point.
      • Two of the successful apps are quasi-free: included for free
        • In monthly membership of a related service, or
        • In the price of the carrier’s data plan.
      • Unsuccessful apps are as high as $5 per month
    • Web app
      • Two of the successful apps are web only
      • One of the successful apps is both web and download/preload
      • Web apps have a low barrier to use compared to download apps
        • There is nothing to download–just launch it in your mobile web browser
      • Successful apps are either
        • Easy to find on carrier deck, or
        • Have easy way to send URL to your phone via SMS
      • Easy to adapt web app to large number of devices
  • Next best success factors: combined weighted correlation score == 0.80
    • Small carrier size
      • Small carrier == has less then 50million subscribers
      • Two of the successful apps are on a small carrier deck
      • Both of the unsuccessful apps are on a large carrier deck
    • Few competing apps on deck / in app store
      • Two of the successful apps are one-of-a-kind on the carrier deck
      • The two unsuccessful apps have many competitors
      • May be a corollary to “Small carrier size”
      • Do app vendors perceive “large carrier” as “large potential market of consumers”? Are too many vendors attracted to large carriers, leading to too much app competition?
    • Long term deck / app store promotion
      • Two of the successful apps have been long term featured items on carrier’s deck
      • The two unsuccessful apps have not been featured items for more than one or two weeks at a time.
    • Carrier branding
      • Two of the successful apps are the “CarrierX Special Cool App”, extending the carrier’s brand name to the product category
      • Neither unsuccessful app is carrier branded.
      • Another corollary to “small carrier size” and “long term deck / app store promotion”? Does carrier prefer to promote it because of the branding?

Results: bottom success factors (non-success factors)

  • Product category
    • Not a numeric success score
    • The factor that got me started on this: if two apps are virtually identical, but one is successful and the other isn’t, why?
    • Survey uses category names from iPhone App Store. 2 Music apps, 2 Entertainment apps, 1 Lifestyle, 1 Sports
    • One Music app is successful, the other isn’t
    • One Entertainment app is successful, the other isn’t
    • Product category not a predictor of success.
  • Low: combined weighted correlation score == 0.20
    • Product >12 months on market
      • Product age does not correlate to success
      • One successful app is one month old; another is 24 months old
    • Short term deck / app store promotions
      • Similar to, but not the exact complement of “long term deck / app store promotion”
      • Occasional short term promotions do not yield success
    • Short term mobile web advertising
      • Targeted to supported devices
      • Occasional short term advertising does not yield success
      • No apps had long term mobile web advertising, so don’t know whether that would be a success factor
    • Famous brand
      • Only one of the successful apps has a famous brand name
      • Both unsuccessful apps have a famous brand name
    • Downloadable app
      • Similar to “web app”
      • Downloading is a barrier to installation?
      • More difficult to adapt app to a large number of devices
        • Device fragmentation: Java, BREW, iPhone, Android, screen sizes, etc.
    • Streaming video
      • Only one successful app is video oriented
      • Both unsuccessful apps contain stream video

Conclusion

  • If you were to bet on the success of a new mobile app, what should you look for?
  • Bet on
    • Inexpensive (Retail price less than $2.75 per month)
    • Web app
      • Or at least a web alternative to a download app
      • E.g. Remember the Milk
    • Few competing apps on deck / in app store
    • Long term deck / app store promotion
  • Don’t bet on
    • Downloadable app
    • Streaming video
    • Short term promotions or advertising
    • Famous brand
  • It’s only six apps with a specific revenue model (monthly subscription)–may not apply to you

Next steps

  • Identify additional apps to evaluate
    • Including single-purchase apps
    • Global survey?
  • Devise a quantitative measure of app success
    • Better than “I know it when I see it.”
  • Identify additional dimensions (including more non-marketing factors), evaluate apps in those dimensions
    • Free trial
  • Strengthen statistical model
    • Although these initial results pass the stink test
      • Stink Test: If it smells rotten, it probably is.
      • These results don’t smell rotten.
  • Grow this seed into a useful tree.