The one email that decreased Moz's churn rate by 40% [Case Study!]

Every once in a while you’ll do something that has an impact beyond your wildest dreams. One of those moments for me was when I helped Moz lower their 30-day churn rate by a staggering 40%. We did this by sending just one simple, yet actionable onboarding email, which makes it even more impressive.

This is the first time I’m publicly talking about what we did. Thanks, Rand, for allowing me to!

In this article I am going to show you, how you can do the same for your SaaS - lets get going!

Step 1: Knowing you have a problem

When you’re trying to fix something, the first step always is knowing and accepting that you have a problem. It was no different for me.

When I joined Moz to work on my master’s thesis they were already a thriving company employing 120+ people. Their TAGFEE culture was excellent and I’ve never experienced a culture permeate every aspect of a company like it did at Moz. It was a great place to work.

Anyways, Moz ended 2013 with $29,3M in revenue - an amazing 33% growth over 2012. Sounds fantastic - right? There was just one problem lurking in the shadows: Their 90-day churn rate was around 40% - waaaay high and hindering future growth.

I think I learned this first from Sarah Bird - then the COO, now the CEO of Moz. One day she gave a talk for employees interested in learning more about the economic dealings of the company. She explained concepts such as annual run rate, monthly recurring revenue, churn, and customer lifetime value.

When Sarah explained churn she talked about Moz’s current numbers: 4 out of 10 customers leaving in the first three months. Their onboarding churn (or 90-day churn) was so high that Moz considered those three months just another step in the “become a loyal user” funnel.

Frankly, I initially thought that this approach was nuts. Clearly this is cheating yourself by obfuscating your real churn rate - right? It required me to launch my own SaaS and running into the same problems to realize that doing this is perfectly reasonable.

Sarah, thanks for teaching me an important lesson.

This problem of a high 90-day churn rate hits a lot of SaaS products, including my own. I like to call it onboarding churn, because it’s often a problem with onboarding. So looking at this phase as another step in the funnel towards becoming a loyal customer is reasonable.

What is your 90-day churn rate?

So the question now is: What’s your 90-day churn rate? And do you have a problem onboarding your customers?

Step 2: Finding a way to solve your problem

The next logical step is that you need to find a way to solve your problem. There is often more than one way to get to where you want to be, but this article is about actionable onboarding emails - so we’ll focus on that, aye?

With the knowledge about Moz’s high churn rate planted in my brain I took off to MicroConf in Las Vegas - a conference for self-funded, single-founder software startups. There I listened to a conference talk by Patrick McKenzie titled “Building Things To Help Sell The Things You Build”.

In the talk Patrick McKenzie mentioned what he called “get them promoted” emails: Weekly check-up emails that tell your users how much ROI they get from using your application. His idea was straightforward: If you show users their ROI exceeds what they’re paying each month by a sizable amount, their boss will be hard-pushed denying them a promotion.

He had a great example from his own SaaS product - AppointmentReminder.org - where he sends weekly emails saying “You made $576 this week by preventing no-shows with AppointmentReminder.org”.

I vividly remember how intrigued I was by that idea. I’ve had a founder crush on Patrick before but this email was exceptionally smart in my opinion.

Next thing that happened was that I talked to Jason Cohen (Founder of WPEngine), who had delivered an out-of-this-world keynote titled “Designing the perfect bootstrapped startup”. We talked about SaaS metrics, churn, and how Moz’s relatively high churn rate was hindering their growth.

I came back from Las Vegas with one question circling my mind: “How can I apply any of the things I learned at MicroConf to help Moz?”

I don’t remember when I finally found an answer, but it was rather simple: Since users of Moz’s software have already set up campaigns - i.e. websites they want to improve - in the software AND have usually entered at least 2-3 competing websites in each campaign, we should be easily able to do the following: We’ll take the backlink profiles (all links pointing to a website) of their competitors and intersect those profiles. What remains is a highly targeted list of websites that like to link to competitors - and thus will likely link to our users as well.

Then we’ll take that list, wrap it up in an email, and advice users to pursue getting a link from the websites on that list. This is done on the premise that those websites are interested in the niche and love to link out to valuable/interesting content.

Lastly, we will track both previous suggestions as well as newly acquired backlinks to our user’s websites. This way we can say “We suggested this website to you, you got a link, hence our approach is working and making you $$$”.

Fake it till you make it - Crafting a mockup to get buy-in from management

With this rough idea I went to work and coded up an extremely rough demo of the algorithm in 48 hours. And by “extremely rough” I mean “it works 1 out of 20 times and when it does it gives you the ugliest JSON file you’ve ever laid your eyes on”.

I showed that idea to my (super smart product manager) friend Jon White and told him about the idea I had. If I remember correctly I did so on the way back from a skiing trip at Crystal Mountain. He encouraged me to stop coding and instead present the idea to Rand and see if there is any interest at all.

Jon and I decided to mock up one email and send it to Rand. We spent quite a while on polishing the copy and here’s the final result we came up with:

In the email we showcased both parts - outreach suggestions as well as ROI demonstration - of the concept. The copy is written in an extremely playful tone, not something recommendable for most live emails. But one is allowed to have some fun while working - right?

We decided to “hide” the explanation in the PS section for two reasons:

1. We didn’t want to break the flow of the email
2. The PS gets read a lot in emails

Anyways, after we had mocked up our minimum viable email/product we sent it on its way towards Rand’s inbox.

It took Rand only a few hours to respond with a resounding “Yes”. Sadly I don’t have access to this email anymore, but he was definitely interested and handed my email (& me) to the marketing team.

The next step for me was to convince the marketing team of the value of this. I created a slide deck and presented the idea to a bunch of folks out of marketing - among them Roxana and Cyrus.

That was an interesting lesson, because I got a lot of great feedback. We decided on a few changes to the initial idea:

• we’d run the experiment only on users who started paying less than 30 days ago
• we’d set up the experiment as a proper A/B test to get credible results instead of a hunch
• we’d only send outreach suggestions and cut away the ROI demonstration part of the email

Candidly speaking, I didn’t like removing the ROI demonstration one bit. For me this was the core of the email and I thought that it would be utterly useless without.

But honestly, they did have good arguments against including it. I didn’t know how to properly tie a new link to an actual dollar value, so everything I’d come up with on short notice would be a bogus metric. And they were validly concerned about protecting the brand, not wanting to jeopardize it by looking like they were advocating selling/buying links.

Anyways, the race was on to build this thing!

Building the minimum viable product

The next few months I spent most of my free time (I was still writing my master’s thesis at the time) to work on this experiment.

There was still a lot of work to do. For one I had to make the script work reliably. That wasn’t as easy as it might sound because I downloaded all the backlinks for each competitor via the Moz API. This took some time and sometimes you’d get an error back from the API, which you’d need to handle gracefully (i.e. not killing the process and losing all the data acquired up to that point). It took me longer than I’d like to admit and I learned that I’m not a fan of node.js ¯_(ツ)_/¯

After maybe two weeks all API requests started to fail - I had exceeded the resources of their highest tier $10,000/month API plan - Oops…

Lessons learned: Use only a small data set for testing and for the love of all that is holy implement caching.

About 70% through we also took a look at the data and realized that some of the suggested websites were of poor quality. Thanks, Roxana, for catching this! It was back to the drawing boards and I had to implement a filter to exclude the poor quality websites based on Domain Authority and URL.

I’m sure there were a few more wrong turns and mishaps that cost me time. Luckily I don’t remember and thus will joyfully encounter them again in the future :-/

But eventually I had the app where it needed to be: We were good to go!

Actually running the test

The process for running the test were moderately complicated: The marketing team compiled a list of accounts that recently started paying for the software. They randomly selected every account on the list into two buckets - the “Test” bucket and the “Control” bucket. Then they handed me a list of all accounts in the “Test” bucket and I fed it into my application. The results were then handed back to the marketing team and they started sending emails.

The accounts in the “Test” bucket received a single email highlighting outreach opportunities, whereas the “Control” bucket did not receive that email. In other words this was a classical split test (A/B test) setup.

In total there were 695 accounts, 347 in the “Control” and 348 in the “Test” bucket.

The marketing team decided to collect two KPIs: 30-day retention rate and OpenSiteExplorer usage.

After the emails were sent it was a waiting game…

The results are in!!

After way too many days spent waiting on the results to roll in, I finally received this email (I had left Moz at that point and was back at university):

Note: Don’t worry, I’ve asked Rand before sharing this data.

But look at those results!

The secondary KPI (OpenSiteExplorer usage) improved by 55%. That’s great and all, but the real killer is the primary KPI. The 30-day retention rate has increased from 85% to 91%.

That doesn’t look like much until you convert it into the more commonly used churn rates. 30-day churn rates dropped from 15% to 9% - a 40% decrease!

This is a tremendous improvement. Getting them through the first month and improving their usage of the software makes them more engaged. This engagement makes churn rates drop off a cliff as users begin to experience the value of the product.

ROI calculations

Here’s a back of the napkin calculation on the ROI of this experiment. We’ll first establish a lower bound for the ROI.

Establishing a lower bound for ROI

Assuming 700 new customers (likely more by now, given that Moz has doubled annual revenue since then) each month, sending this one email results in 42 customers paying for at least one additional month. 42 customers at a minimum of $99/month (their lowest tier) results in an additional $4,158 in revenue each month.

This comes down to an additional $5.94 per email sent - nice ROI! And this is assuming a 100% churn rate after 30 days - which is completely unrealistic.

A more realistic ROI calculation

Let’s look at a slightly more (but not overly) optimistic calculation.

Take that same 700 new customers and again assume that 42 customers stay longer because of that single email. But instead of assuming a 100% churn rate after one month, we’ll assume that they churn at the same rate as other customers after that.
Remember: Since they are already more engaged with the product (55% improvement in secondary KPI!) they are less likely to churn than an average customer. I’m being cautious in this calculation.

So 25% of those 42 customers (=11) will cancel in the next two months - which leaves us with 31 customers. Those 31 customers will experience an average churn rate of ~2.5% [source] after the first three months - or stay with the company on average for 40 more months.

I’m discarding the 11 customers that cancel completely - although in reality they do contribute some ROI. I’m also discarding revenue from the first three months and assume all customers are on the lowest plan. Again, I’m being cautious in my calculations.

So… 31 customers at $99/month for 40 months comes out to an amazing $122,760!

Since I love numbers: That means an ROI of $175 per email sent.

So we’re looking at 12 * $122,760 = $1,473,120 of additional annual revenue - roughly a 5% overall improvement!

Or to use my favorite unit of measurement: 9.3 Tesla Model S P100DLs with all the bells and whistles. :-)

How you can apply this to your own SaaS

Okay, at this point you’re saying “But this is Moz. They are a special snowflake and there’s no way I can do this with my product” - WRONG! You won’t suddenly make $1.4M, but you can definitely do something about the high onboarding churn if it plagues you.

Find the right hook

First of all, you’ll need something valuable that you can deliver to your users. In our email we delivered actionable insights from the user’s data. Think about which insights you might give your customers.

Alternatively, think about which features really deliver value to your users. This is often not the feature that you think is the core of your product - e.g. for my own SaaS LinksSpy.com the core feature is link intersect analysis, which gives you better search results when looking for outreach opportunities. BUT this does not deliver nearly as much value as the automated email outreach feature, which actually gets you guest posting opportunities & more links.

In my experience the features delivering most value are also the features that make your users stick around. It makes a lot of sense to heavily promote usage of those features in your onboarding emails.

Make the email SUPER-actionable

When you’ve found your hook, make sure that the email you send is super-actionable. Don’t settle for “You should do more of this - Click here to do more of this” emails:

source: Really Good Emails

“Getting started” is not specific enough. Instead take a look at an email from Wistia, where they encourage users to upload the first video:

source: Really Good Emails

You immediately get what needs to be done. In an ideal world users could upload a video right from that email, but as far as I know there’s no technology for that (yet).

Still, this is what you’re aiming for - a 1-click-to-success action item prominently placed in your email. Remove friction where possible.

Conclusion

This experiment proved to be an overwhelming success. Sending new users an email with actionable next steps to help them during onboarding dramatically decreased the 30-day churn rate.

If you’d love to receive more content like this PLUS a free chapter on lead generation from my book PLUS a discount when you purchase the book - sign up below: