There’s something beautiful about building products from the ground up. You theorize about them first, sketching problems they are supposed to solve and trying to plan what is seemingly unplannable.
Then you begin prototyping, adding screens, and trying to find someone crazy enough to give it a try. Once you do this and it seems you actually picked the right path, problems start sprouting like mushrooms after rain.
Fast-forward a few years into the future, and you’re probably thinking about that great idea you once had that never really took off. But if you found the right niche, worked on it diligently, and listened to your users – whether there were 50 or 500,000 of them – you could be talking about a really successful product.
That’s briefly the story of Mailtrap, which we founded in 2009 to solve our own problems. Today, 11 years later, it’s a rapidly-growing product with over half a million users, and arguably the most popular safe email testing tool on the market.
Today, we’re going to shed some light on how we managed to take Mailtrap from a tiny prototype to the profitable, independent business it is today.
Product Studio concept
Mailtrap is just one of the many successful products we’ve developed since 2007. We owe such a result to the concept of Product Studio that we’ve polished over the years.
We’ve transformed Railsware into a platform that excels at repeatedly shipping products that solve real problems – products that are loved by their users. We established processes that allow us to quickly validate an idea, pick the right approach, and find the smoothest way of delivering a concept to the market.
This approach proved to be really successful for dozens of client projects we have worked with over the years. We’ve also shipped a number of our solutions using the very same processes. Today, tools like Mailtrap, Smart Checklist for Jira, and Coupler are used by some of the world’s greatest companies – Atlassian, GitHub, SendGrid, Adobe, and Pivotal being just a few out of hundreds of thousands of others.
If you’re curious to know how the Product Studio concept works and why it’s so successful, definitely check our article on Product Studio. And now on to the story of one of the most successful results of the Product Studio engine – Mailtrap.
How it all started
The Mailtrap story begins in 2009. As Railsware, we were involved in the development of a platform for sending marketing campaigns for one of our clients. Everything was moving smoothly and we were in the process of testing core functionalities. So we thought, let’s send a large batch of test emails and see whether it doesn’t break the system.
It didn’t, but that was the only good news. The bad news? On this day, tens of thousands of users of this platform were flooded with a wave of test emails that they were never meant to receive. As you can imagine, the customer-facing folks weren’t exactly delighted about the responses they would receive in the following hours.
To say we were unhappy about this situation would be a major understatement. But instead of whining and looking for excuses, we quickly came up with a concept of a script for safe email testing, which would soon become known as Mailtrap.
In a few days, we created a simple script that would serve as a fake testing environment. It would emulate sending emails, but it would capture them immediately after so they don’t even pass through any SMTP servers. No emails sent -> No emails received on the users’ ends.
The alpha version was developed by several devs in a matter of days, and it was giving promising results. So, some time later, another group of Railswarians built a more sophisticated beta that could work as a full-fledged testing environment for our projects. And we started promoting it in the dev communities on Stack Overflow.
The reception was overwhelmingly positive. We received tons of comments and messages about how useful the tool was and how the problem it was solving was real. By 2011, we already had 2-3 thousand users with absolutely zero marketing efforts. All the traffic was coming through recommendations and upvotes, and it seemed as though we’d really hit the target.
One of the first versions of Mailtrap’s website, as seen between 2012 and 2014.
First paid plans
2014 brought the v2 of Mailtrap to the world. Since 2011, the code had barely changed and the infrastructure was heavily outdated. Despite that, we added several thousands more users and the needs of the current user base were growing.
Instead of improving the old prototypes, we opted for a much more efficient solution. We built a v2 of Mailtrap from scratch. It had a much better performance and improved the quality of SMTP standards we were using. More servers and features were added. It simply became a much better and more reliable tool for everyone.
And it wasn’t just our opinion. The reception was fantastic. Twitter, Stack Overflow, as well as our support channels were overflowing with positive comments. We saw a spike in new signups too and we started to seriously think that Mailtrap could become a significant piece of our diverse portfolio.
Around that time, we also started getting another question that would recur on a regular basis: “When are you guys going to start charging money for Mailtrap?”
Many devs wanted to commit to Mailtrap and didn’t want us to suddenly pack up our bags like many free tools do. And we were solving an important problem for them – a problem that probably no one else on the market could address at the time.
With the improved infrastructure and more regular involvement from our devs, it was a very sensible thing to start charging for Mailtrap. Especially since, by 2015, we already had over 10,000 users, many of them actively using the platform.
We had little idea about how much we could charge for such a product as, once again, there was hardly any competition in this particular field. We did a fairly detailed analysis of other tools for developers and how they shape their pricings. We analyzed the market, but the results were far from conclusive. So we’ve chosen to reach out to the best advisors we could think of – our users.
Customer Development
In a short time, we conducted a few dozen interviews with them. These allowed us to gain a much better understanding of our customers and the problems that Mailtrap was solving for them. Many users would say straight away that they would pay for Mailtrap if we kept developing it.
Simultaneously, we also reached out to all our active users and decided to be very transparent with them. We shared with them the drafts of the plans we had in mind and asked, “What do you think?”
We received over 300 replies and were able to stop theorizing and actually use some data to our benefit. Eventually, we decided to play it safe and set the price tag a bit lower than we probably could have done.
Shortly after, we released a paid plan on top of the free one. We also committed to keeping Mailtrap free forever in a limited capacity. In the first four months after adding a paid plan, we registered 400 new, paying users. Looking at this from a time perspective, we could have easily done it a year or two earlier with similar success.
2014 also brought us a redesign. Here is the Mailtrap website since mid 2014.
The growth that followed
The improvements to the platform and the marketing efforts we started around that time led to significant growth. In the following four years, we’ve grown to around 280,000 users. A lot of signups came via paid traffic – Google Adwords, Facebook, and Twitter ads. We started crafting content and distributed several videos explaining how Mailtrap works in the dev communities. And it brought us some really good results.
But we also owed a lot to the community. As the numbers grew, word of mouth was a major growth contributor for us. Mailtrap was getting better, with better infrastructure and more features, and the users appreciated it.
They wrote numerous tutorials on using and integrating Mailtrap. Mailtrap was regularly mentioned as a solution to questions asked on Stack Overflow or Quora. We also ended up as a recommended email testing solution for Laravel. It drove hundreds of new signups and is still routing dozens to our site every month years later.
Aside from some investment in ads, everything else happened with nearly zero effort from our side. And it wasn’t all just a coincidence or dumb luck. We were delivering a product that users loved and they would pay us back big time.
A data-driven approach to the new pricing
In late 2018, we started preparing to launch new pricing. We hadn’t touched it since 2015. In the meantime, a lot of features have been launched and there was already a dedicated team working on Mailtrap. And the platform was a lot bigger than a few years earlier.
Some paid users would barely use the premium features, while others would send thousands of test emails, heavily overloading our servers. And both would pay the same price. We had to differentiate between them and also do one of the hardest things for any business – increase the prices.
We knew how risky it was, especially since we would literally ask most users to pay more without giving anything new in return. So we decided to make a really data-driven decision and, as always, get our favorite advisors involved in the process.
Our marketing manager at the time would spend the next several months gathering the data we desperately needed. She would run another series of customer development surveys to understand the value users see in Mailtrap and the features they find essential. She would carefully create buyer personas and analyze keywords people find us with when searching.
Competitors research
We made very detailed research on our competitors. A few similar products popped up over the years and there were many ways to go about email testing. We had to know all these tools and understand their features and pricing.
To have a wider understanding of where we were and how much we could ask for, we also analyzed hundreds of other tools for developers. Many of them had nothing to do with email and were not competing with us in any way. But, by seeing how they shape their pricings, what they offer on free tiers, and how some transitioned from free to paid products gave us a tremendous amount of knowledge of what we could afford to do.
Analyzing users’ behavior
We also looked at the actual data generated by our users and started collecting new data points. We had a deeper look at:
- how many emails users send/receive
- how frequently they use the platform
- which features they use
- what API requests they send
Google BigQuery and Google Analytics were of tremendous help in putting all of this together.
Our goal in all of this was to keep the platform free for users, the majority of whom are casual, irregular testers. Years earlier, we promised that Mailtrap would always remain free in some capacity, and we intend to keep that promise.
To determine what types of Mailtrap users there are, we analyzed one of the most important metrics – the number of emails users capture with Mailtrap. We then split users into cohorts, each representing different sending needs.
The first cohort was of more occasional senders. The other cohorts used Mailtrap more regularly, but the group wasn’t homogenous. Some would send 100-200 emails every month. Others’ emails would easily go into 5-digit numbers. The simplified version of the chart we used looks like this:
Blueline – users on a free tier, Red – paying users. X-axis – the number of emails sent, Y-axis – accounts in a given cohort. Green and yellow lines (almost flat) refer to older reporting methods and can be ignored.
After the first cohort with the lowest needs, there was a significant drop in the number of accounts. Yet, the number of accounts in the second and following cohorts was significant enough to break down those cohorts further and assign paid tiers among them.
To do that, we’ve used a more detailed chart. Our idea was to place the limitations for paid plans at the point when the number of accounts in a cohort started to drop. This way, users with similar needs would likely fall under the same plan. Those sending slightly more emails would mostly land in the next plan, and heavy senders would follow in the next groups.
The significant drops below are marked with orange circles, while black lines indicate where we placed the limits on Individual (5k emails), Team (15k), and Premium (100k) plans. To make plans more balanced, we opted to use a 3x multiplier and made some adjustments to round some numbers evenly (no one likes being limited by the 93,528 emails they can send each month).
Red and green lines – the number of companies in each cohort. Blue and yellow lines represent individual accounts. There are 4 lines instead of 2 because of 2 different reporting methods we’ve used over the years. X-axis – the number of emails sent, Y-axis – accounts in a given cohort
We also thought that the gap between the latter two plans was too significant, so after talking to some users, we added a Business plan (50k) in between.
Pricing as of June 2020. This can also be seen on the Mailtrap Pricing page.
Gathering feedback
We designed the paid plans along with a free tier, just as if we were about to implement them. Then we selected 500 paying users that were actively using the platform, and showed them the screenshots of what we were about to launch.
The feedback was mixed. Some were hesitant about upgrading to new plans, as in nearly all cases it would mean a higher bill. At the same time, plenty of those we talked to showed their understanding, and committed to keeping their subscription active when new plans are launched.
We also decided to create legacy plans that would allow current users to stay on their current plan for as long as they continue their subscription. Due to the inconsistent nature of email testing, many of these legacy plans ceased at some point and eventually upgraded to new plans.
Finally, we began implementing the plans on staging. Just as we do for each major update to our products (internal and external), we ran a typical Railsware TestFest with the entire Mailtrap team and several Railsware engineers involved in other projects. We quickly caught a number of bugs and inconsistencies and fixed them before the launch. We also gathered a lot of feedback and revised the features set for each plan once again.
Mailtrap today.
Converting free accounts into paid ones
First, we launched the plans only for the new users. At this point, we also estimated that about 10% of the existing free users would eventually be affected by these changes. In other words, they would need to start paying to keep using the platform the way they used to.
We sent an email to all the users explaining the changes and our reasoning behind them. We talked about how we wanted the pricing to be fair to everyone and really relevant to the way they use the platform. If you use Mailtrap more, you pay more. If you use very little of it, it could even be free for you. All in all, we announced that, in two months’ time, all free users would be migrated to a new (lower) free plan.
And it worked out really well. When the time came to make a decision, around 20% of active free users decided to upgrade their subscription and start paying for Mailtrap. At the same time, very few paying users downgraded their plans, so our revenues immediately got a major boost.
Ahead of the launch for the existing users, we prepared three possible scenarios – optimistic, neutral and pessimistic. We listed precisely what we were going to do if for two weeks in a row our revenues:
- increased
- decreased
- stayed about the same
Product teams usually don’t want to plan for a pessimistic scenario. And they’re caught by surprise when things don’t work out the way they hoped they would. We were prepared for that. But, luckily, we never had to implement that plan.
How we come up with ideas for product improvements
Before we move on to Mailtrap’s current state, here’s a quick note on how our teams come up with new ideas for product improvements.
BRIDGeS is an ideation and decision-making framework designed by Railsware. It allows you to dive deeper into a problem and define the benefits, risks, and issues surrounding that problem. By the end of a BRIDGeS session, your team will have reached a solution — one that can easily be turned into a roadmap for implementation.
Like Mailtrap, we originally created BRIDGeS for internal use only. However, when we saw how it could help people save time and design more effective solutions, we released a free Figma template to the public. For context on how to use it, you can check out the Uber Case Study.
Mailtrap today
The first reports we built quickly proved that the optimistic scenario was the most accurate one. The upward trend continued throughout the rest of 2019 and into 2020. Both user count and revenues keep soaring. We were affected a bit by the Covid-19 crisis but quickly recovered, as you could read in the interview with our Managing Director.
We’ve hit half a million users logging into Mailtrap from every country and territory around the world. More and more people are aware of what we do and see the clear benefits of using Mailtrap in their projects.
What were the key factors in the transition from a free to a paid tool? There were a few for sure. Of course, we found the right niche. We’ve built a product that really solves a significant problem that many developers and QAs face. We’ve also combined our gut feeling with a lot of numbers and made really informed decisions on a number of crucial occasions.
But what we all agree on is that the unique connection we’ve managed to build with our community is what made a difference. It’s the countless hours we’ve spent interviewing developers about their problems, analyzing their responses, and contemplating how we can best address their needs while keeping our business goals in mind too.
It’s also the community’s devotion to spreading the word about Mailtrap, suggesting improvements, giving constructive feedback, and showing understanding when we had to make difficult decisions. If it wasn’t for our #BestClientsEver, we wouldn’t be anywhere close to where we are today.