Futuristic Play by @Andrew_Chen

The multiplicative nature of retention
Retention metrics, like viral marketing, have a powerful multiplicative approach that makes them important to optimize. This is an often-overlooked metric that is easily sacrificed to the Great God of Viral Traffic I wanted to dig into this issue a bit more, and give my thoughts on how your retention numbers can be an incredibly powerful method of driving up total traffic numbers to your site, not just the user acquisition piece.

Matt Humphrey from Bumba Labs on user retention curves
A frequent collaborator of mine, Matt Humphrey, has often referenced some of these ideas in a public posting below, which I’ll excerpt. I’ve worked with Matt closely for over a year and a half now, and know a lot about his skillsets, background, and interests. His background is influenced by his time as a Carnegie Mellon alum, YCombinator entrepreneur, and working on quantitative marketing projects with me.

In my conversations with Matt, we often discussed the metric of “user retention rate” which tells you what % of users are revisiting after some period of time. So you might define 50% user retention as, 50% of the users on your site come back after a month, or what % come back after an initial signup, or whatever suits your particular business.

What’s amazing is the multiplicative aspect of this number:

[...] Having month-to-month user retention of 92%, 96%, and 97.3% will get you on average 1, 2, and 3 user-years respectively per user that ever signs up on the site.

Okay, in English? If each month you lose 8% of your existing users (92% retention) from the previous month, the average use will stay for 12 months. If you can hold just 4% more of your users (96% retention), then they will stick around for 2 years. If you can hold only 1.3% more than that (97.3% retention), they will be in for 3 years.

It’s easy to think of retention percentages in the 90’s as good. It just feels good. But over the course of time, products in the low-to-mid 90’s will fade super-fast, and ones only slightly more sticky will do much, much better. Single percentage points here are mission critical, that’s why attention to detail and rigorous analytics become so important on the web. [...]

To restate this, as you approach high levels of user retention, you begin to see powerful multiplicative effects.

Let me state, for reality-checking purposes, that retention rates over 90% are unrealistic, but are useful for discussion purposes because they bring out the extreme cases. More realistically, the numbers I’ve seen are generally much closer to 30-60% revisit rates after the initial registration. Similarly, the typically retention rates are not linear – you see the most churn initially, but then the cohort usually settles and becomes much more loyal.

But the core issues still hold, and the reason, of course, is simple – it’s simple arithmetic, which we’ll examine below.

An example of two subscriber cohorts
Let’s say you’re running a subscription site, and you compare two sets of subscribers, both starting in the same month, both numbering 1000. Let’s say that one cohort has 80% monthly retention, and the other is 90%.

In the short run, the numbers are close to the same:

• 80% monthly retention, after 1 month = 800
• 90% monthly retention, after 1 month = 900

Although obviously you’d want to have the 90% retention, the differences are not huge – you end up with 12.5% more users, after one period.

But let’s look at these numbers once you get to 12 months:

• 80% monthly retention, after 12 months = 1000*(0.8)^12 = 68
• 90% monthly retention, after 12 months = 1000*(0.9)^12 = 282

Of course, this ends up being a staggering 3X difference. Wow! And when you compare aggregate lifetime value, the numbers are even bigger.

Retention-focused features are very powerful
The point of all of the above is that retention-focused features are very powerful because they let you create dramatic improvements in all the important metrics, across the board – be it pageviews, total time usage, revenue, etc.

This means that you can, just as people do with addressbook importers and the like, put a tremendous amount of time into a whole host of retention-driven features like:

• A great product and value proposition
• Targeted notifications
• Fresh news and content on every return
• Desktop app-integration (which has a much lower rate of uninstall)
• The number of friends on the site (the more that are there, the more notifications can be generated)

All of the above contribute meaningfully to this user retention number.

As weird as it is to imagine that something as pedestrian as how you deliver your notifications can cause success or failure to your web business, in fact it can. The reason is that how you deliver notifications can have a huge impact on whether or not your users come to your site, and every percentage point of improvement may lead to many times the revenue, pageviews, and content. This is certainly an area worth focusing on, as much as Bay Area companies have focused on just the viral metrics.

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

UPDATE: Thanks for Bhanu Sharma for correcting some dumb typos on the post – I changed the numbers halfway through and forgot to update all of it.

I’ve been auto-posting all my tweets on my blog. I’m curious what people think about it – useful? Or not?

Take the poll below, and please leave comments if you have extended thoughts:

Thanks for the feedback!

Here are some links I posted to my twitter account over the last week or two. You can follow me on Twitter if you like these! Many are work unrelated.
• RT @sgblank: Lies Entrepreneurs Tell Themselves http://www.steveblank.com don't let this happen to you. #
• GReader share: The "Hidden Cost" of Privacy http://bit.ly/O9qdi #
• GReader share: Why Continuous Deployment? http://bit.ly/18QaPD #
• GReader share: Chris Anderson’s Counterintuitive Rules For Charging For Media Online http://bit.ly/Ojs5c #
• GReader share: Michael Lewis http://bit.ly/Xlhwc #
• GReader share: Dina Goldstein's Fallen Princesses photo series http://bit.ly/mRnsB #
• ew, gross. Comfort wipe infomercial on youtube: http://bit.ly/rV7Qp (via Slate http://bit.ly/yU2u0) #
• GReader share: Facebook Finally Catches Up To MySpace In The U.S. http://bit.ly/12fKwS #
• GReader share: BREADBOX64 http://bit.ly/uG7Qf #
• GReader share: Alex Payne's Open Ideas http://ideas.al3x.net/ #
• boy tries to give yellow rose to Megan Fox. Rejected: http://www.dlisted.com/node/32554 #
• GReader share: Times Square Tourists Don't Know What A Browser Is (CLIP) http://bit.ly/WVPRK #
• GReader share: Alice and Kev: homelessness in Sims 3 http://bit.ly/egxom #
• GReader share: The quirky genius of the Dos Equis ad campaign. – By Seth Stevenson – Slate Magazine http://bit.ly/oINCT #
• GReader share: Ross Racine's incredible artwork of aerial views of fictional city maps http://bit.ly/R9ZVS #
• GReader share: The Perfect Storm – Part 1 http://bit.ly/sXdvX #
• GReader share: The Young Entrepreneur Myth http://bit.ly/az7Jk #
• GReader share: Epitaph for an Entrepreneur http://bit.ly/Fye0Z #
• does being more chatty on Twitter lead to more followers or less? I use my twitter as a link dump, but is a different model better? #
• GReader share: Tagged CEO Greg Tseng On Accountability, TechCrunch and Leading the Third Biggest S.. http://bit.ly/LzBqZ #
• GReader share: Versions 1.0 Of Today's Most Popular Apps http://bit.ly/iMN5J #
• GReader share: Why is Golf So Fragile? http://bit.ly/7ZPrf #
• a fantastic, well-written explanation of lock-picking: http://bit.ly/CXmpI #
• Charles River Ventures Hires That FuckedCompany Guy http://bit.ly/PY0F6 #
• congrats to @pud on his new EIR gig at Charles River- don't spend too much time on the dark side! #
• Shared Experiences vs. Algorithmic Content http://bit.ly/660iO #
• Breakfast at cafe epi in palo alto! #
• Heading to see new scifi indie movie "moon." Thinking about what to write for tomorrow's blog post also… #

The perils of ignoring brand value
The nature of internet marketing makes it easy to have a highly accountable, metrics-driven view – but companies that are highly metrics driven easily overlook hard-to-measure issues like brand and user experience. The reason is that when all product decision-making is run through metrics-driven reports, soft things like “Brand” show up as costs, but never as benefits.

This leads to systematic erosion in many “soft” but important factors, like customer experience, brand value, and “love.” And ultimately you need all of these things to create a massive, enduring consumer brand – it’s not enough to optimize funnels.

Let’s discuss why:

Two worlds: Direct marketing and brand marketing
In the advertising industry, there’s been a long, historic distinction between brands and direct response – and this distinction echoes its way into the online startup building world as well.

In the brand world, you have companies like Coca Cola, Apple, and others who pour millions of dollars into high-reach vehicles like TV which lack any real accountability. Thus the saying:

Half the money I spend on advertising is wasted; the trouble is I don’t know which half
– John Wanamaker, US department store merchant (1838 – 1922)

To many people, the brand advertising world is irrational and fashion-driven, because of the complex interactions between agencies, their partners, and the publishers that rely on them. Just watch Mad Men.

On the other hand, you have direct marketers who thrive on accountability. They buy into marketing channels like direct mail, coupons, infomercials, and most recently online remnant ads, because they can purchase cheaply and use sophisticated statistical techniques to optimize their media buys.

Startup engineers tend towards metrics-driven
So what side do startups tend to side on? It obviously depends, but because of the highly accountable and measurable nature of online, it’s much easier to become metrics focused. Similarly, startups are mostly poor Thus, expensive brand efforts are mostly out of reach. (Probably for the better!)

Also, with the possible exception of GoDaddy, I don’t know a single startup that made it or not based on their brand advertising strategy. The typical path is focused on products and technology, and large organic growth which builds large consumer audiences.

And obviously, readers of this blog will tend to be much more metrics driven compared to the average entrepreneur!

You optimize what you measure
The first issue that causes metrics-driven startups to ignore brand value has to do with the fact that it’s very hard to measure brand, and you tend to optimize what you can measure. As soon as you throw some numbers on a big report, there’s an inherent human desire to make the numbers go up!

This is why one of the fundamental tenants of metrics-driven startups is to build lots of highly accessible reports that everyone in the organization can look at. Even if it’s easy enough to pull something out via a SQL query, it’s another thing for everyone to be able to hit a URL and load it instantly, no matter who they are on the team.

Measuring brand value is hard!
But measuring brand value, or user experience, or community “feel” or other soft things like that is very hard. I think they’re hard because while it’s clearly important, at the same time:

• The quantitative effects accumulate over large periods of time
• These might be “source” variables that drive lots of behavior, but it’s hard to measure past surveys and explicit information collection
• Some of the most important datapoints may be qualitative, not quantitative
• Changing these soft things may require big efforts above and beyond small A/B-testable changes

The companies out in the marketplace that try to measure brand value mostly just use surveys to detect changes. Or, many companies simply resort to a pretty ineffectual number like “reach,” which refers to the number of people who saw the campaign. This can sort of work, but self-reporting also sucks, and the quantitative data you get out may not be as useful as the qualitative data.

In my previous online ad career, I was shocked to hear that the standard way to measure a brand advertising campaign online was to fork $50k over to Dynamic Logic, whose job was to run a dinky little survey and tell you if your campaign worked. $50k to run a survey!

Reports show the cost of branding, but not the benefits
As a result of brand advertising being hard to measure, you get two systematic, interrelated issues:

1. Product changes that result in brand value are overlooked, whereas the costs of delivering that value is not
2. Features that negatively impact brand value but show short-term quantitative value are accepted

Here are two examples – let’s say that you think your site’s interface looks like crap, and you want to improve it to make it higher class and more trustworthy. But your metrics czar says, let’s make a really small improvement and see if it affects anything before we revamp the whole site. That sounds reasonable, but then you find out that in fact, making a visually compelling site just doesn’t drive better metrics, and in fact, it’s expensive and maybe lowers certain metrics. What do you do? (This is case #1)

Another example is that you make it really hard to unsubscribe from your mailing list. Maybe you don’t have a link, or you have to login first, or whatever. Making this change clearly affects your ability to retain users, but you get a small percentage of complaints, but the overall quantitative metrics look good. Should you keep this hard-to-unsubscribe mailing list issue? (This is case #2)

Ultimately, it should be clear that both cases are not clear cut issues at all. I could find reasons to go either way, but when you’re trading off a qualitative metric versus a quantitative thing, the numbers-driven approach tends to win. But this may not be the right thing. Similarly, sometimes the numbers may justify the decision, and the brand costs are actually quite low.

How do you make these decisions then? I’ll just wave my hands and say, “Entrepreneurial judgement”

Who’s the brand advocate?
One of the big, important roles that you need on every team as a result is someone who can advocate for the soft things. Who’s your brand advocate? Or customer experience advocate? Having someone on your team who can make logical arguments to balance out the quantitative stuff is hugely key, otherwise you’ll inevitably go down a path of brand-eroding quantitatively driven decisions.

Similarly, if you find that you’re never making decisions that go against the numbers, then frankly, you’re probably doing something wrong. If the data drives all the decision-making, then a lot of “soft” data is getting ignored.

Want more?
If you liked this post, please subscribe or follow me on Twitter. You can also find more essays here.

Here are some links I posted to my twitter account over the last week or two. You can follow me on Twitter if you like these! Many are work unrelated.
• GReader share: Flixel, a free Actionscript library for building complex games without Flash http://flixel.org/ #
• GReader share: Datablindness http://bit.ly/uYdFE #
• GReader share: What do American girls like? (crowdsourcing stereotypes) http://girls.yury.name/ #
• GReader share: Entrepreneurs on Twitter http://bit.ly/3TgJJ #
• GReader share: New Planetarium Projector Promises 16x 1080p Res, Might Pressure IMAX Format http://bit.ly/dMgIP #
• GReader share: Today, We Think Twitter Is Dead (for Now) http://bit.ly/E59ue #
• RT @garrytan: All models are wrong. Some models are useful. -George Box #
• RT @adachen @sbergel Three Rings continues our mad tradition of telling everyone all our MMO & virtual currency sekrits http://bit.ly/k6g7T #
• RT @dataspora: Also, "Never fall in love with a model." – G.P. Box (and several rock stars) #
• RT @cherrymochi @wesabe @palm: Awesome Billshrink post comparing costs of iPhone, Palm Pre, Android: http://bit.ly/1aaYgo #
• GReader share: Shocker: Study shows VCs plan to invest in fewer companies http://bit.ly/jiGoP #
• GReader share: Don’t Blame Venture Woes on the Economy; It’s the VCs’ Fault http://bit.ly/13glfn #
• GReader share: Revenue at Craigslist Is Said to Top $100 Million http://bit.ly/iFslh #
• RT @bryce @peHUB: @pkedrosky Don’t Blame Greedy VCs for Industry Bloat, Blame LPs: http://tinyurl.com/nklo55 #
• GReader share: Hey, Graduates, Check Out These CEOs' First Jobs (SLIDESHOW) http://bit.ly/VI2hm #
• GReader share: Twitter Hits The Ceiling http://bit.ly/12AlNW #
• GReader share: Kedrosky: Cut the VC Industry in Half http://bit.ly/b9M35 #
• GReader share: Trent Reznor Quits Web 2.0, Offers Cutting Advice To Haters http://bit.ly/1aDnp1 #
• RT @tworetzky: New Pew report on US use of online health content. Majority read UGC content but not on social networks! http://bit.ly/dhIrC #
• RT @angusdav: gotta love this one: http://billmyparents.com/ — the name says it all. cool payments system idea, cheesy web UI. #
• GReader share: I want… who works at EDGE Magazine? http://bit.ly/RUZUi #
• GReader share: Jimmy Fallon on Microsoft Natal: 'Oh my God — this is awesome!' http://bit.ly/EIZSu #
• GReader share: Inside the Startup Office from Hell http://bit.ly/19dm4S #
• RT @cherrymochi: weird but very cool shadow art made from junk: http://bit.ly/H2qyr #
• rt @m2jr Regardless of one's politics, Letterman's behavior to Sarah Palin has been out of line. Salon nails it. http://bit.ly/17B8IM #
• WSJ on benefits and perils of data-driven education: http://online.wsj.com/article/SB124475338699707579.html #
• GReader share: Friday Fun: Tennis Serves and New Balls http://bit.ly/IYcbz #
• GReader share: MySpace Is In Far Worse Shape Than Its New Executives Thought http://bit.ly/jHdoF #
• GReader share: Electronic Arts No. 2 exec foresees shift coming toward digital game distribution http://bit.ly/Bmczc #
• GReader share: Facebook Nabs The Man Who Engineered Google AdSense For Many Years http://bit.ly/3bNdW #
• GReader share: Is John Donahoe Finally Turning eBay Around? http://bit.ly/LePS0 #
• On Will Wright’s Team, Would You Be a Solvent, or the Glue? http://post.ly/qEZ #
• Why is America so fascinated with multiple simultaneous births? They are followed by paps, become celebs. Weird. #
• Overheard in Palo alto: new Stanford grad explaining full history of Apple in excruciating detail to his parents. Rite of passage? #
• Writing tomorrow morning's blog post about why you should make it easy for users to quit your product! Fun. #
• RT @ericnakagawa: Celebrity – We love to love them, love to hate them, love to create them, love to destroy them. #
• i'm tempted to change my safari search to bing.com, just to try it out… http://bit.ly/Je6Db #
• dear lazyweb: Is there a way to "print to kindle"? It'd take whatever page I'm on, PDF it, and then email to my @kindle.com address. Ideas? #
• GReader share: Hunch.com, decision-making engine, opens to the public http://www.hunch.com/ #
• GReader share: Shock Waves in Human Systems http://bit.ly/9HyJe #