There’s a well-trodden path for developing a viable startup.
Come up with an idea, build an MVP, and do beta testing to catch bugs while making the startup better.
It’s simple right? One thing I’ve learned is that simple is NOT the same as easy. After four months and tons of beta testing lessons, we’ve officially ended our testing phase and have rolled out a product we’re proud of.
It wasn’t easy and we had a lot of sleepless nights. It was worth it because acquired a lot of insights into what our customers wanted from a solution like ours.
In this post, I’ll share some of the insights we learned from beta testing. I’ll also make the case for why everyone building a software enable startup (or any startup for that matter) should perform a round of beta testing.
1. Some people are at different stages of familiarity with the internet
Going into our beta test, I was confident that people were familiar with the underlying use cases for the technology.
You make an opt-in form or quiz to capture leads, connect with your email marketing service, and promote. I knew we’d have to explain how to use our specific solution but we were prepared for that.
That’s not exactly what happened. I found myself fielding live chat messages and emails asking what someone would need an opt-in form for.
At first, I chalked this up to poor messaging or targeting. I rewrote the copy on key pages but the messages kept coming in.
I ignored the temptation to link a few stats and instead did a bit of research.
It turns out that 59% of American adults find the internet overwhelming. This blew me away. We took it for granted that people would just get our solution.
It made me reassess how we’d educate both our audience and our new users.
Right now, we’re in the process of refining our onboarding process. I’ll share more about that when we have more conclusive results.
2. Documentation is super important even while beta testing
There’s never enough time to do all the things you’d like to do. That’s why efficiency and effectiveness are important.
You’re juggling one million processes, customer support, and acquisition. Anything that can cut down the amount of time spent on repetitive tasks is worth its weight in gold.
When the beta launched, we didn’t have a single knowledgebase article. We were worried about over optimizing for features that may not be used.
We decided to wait and see what our early users would ask for.
My expectation was that most of the process would be intuitive and people wouldn’t need to any additional help.
I was wrong.
Even though there were clear indications of success like this one:
People would still reach out just to confirm they were doing it the right way. That happened because there was no documentation so they were taking stabs in the dark.
Stabbing – and darkness – aren’t good.
Once we made the documentation, we used Intercom to set up notifications on pages that caused friction.
This cut down on support requests but also opened up a new line of inquiry from our users.
3. Even when you have the documentation, people still want to talk.
We eventually made the most requested documentation which revolved around integrating email marketing services and setting up quizzes.
Even though we made the documentation, people would still reach out.
Their messages would run along the lines of “this is my first time making quizzes, would you check it out and let us know if it’s any good?”
I was ecstatic about this. I mean, we do this all day every day so I was more than happy to help our users make quizzes that’d convert.
It was interesting that people didn’t ask about opt-in forms much. My guess is that people are more interested quizzes because opt-in forms are a staple of the internet.
When we were able to talk to people about their quizzes, we naturally started asking them about their niche, their goals, and why they signed up.
This gave us deep insights into why people were signing up for KyLeads which helped us refine our copy.
4. An intuitive user interface isn’t optional
This is related to documentation. Our initial interface was downright embarrassing. I was ashamed to share it with the world.
I know it’s been said if you’re not embarrassed by your first version then you’ve launched too late.
That doesn’t make it any easier to put something downright ugly into the world. Here’s what our initial app looked like.
Here’s what our current app looks like.
People were utterly confused about how to use KyLeads because it wasn’t an intuitive process. Some of the elements were hidden behind buttons that you wouldn’t click without a prompt.
I blame myself.
When we were building the initial MVP, we experienced a lot of challenges. We went through multiple iterations of the product and the user interface took a backseat.
After launching the MVP and getting initial feedback, we also invested in a UI designer to revamp the app.
It was worth it.
When you’re deciding on whether to use your resources to develop another feature and polishing your UI, choose the UI.
5. You have to fight for engagement and feedback
I was surprised at this beta testing lesson.
We thought the major challenge would be getting our initial users. It turns out, that wasn’t so hard.
The hard part was keeping engagement up and getting unsolicited feedback.
Here’s how it would go.
Someone would sign up for KyLeads, click around a bit, maybe set up their email marketing service, and then disappear.
At the initial stages, we hadn’t set up Intercom to capture the emails of people who signed up for the app. That means there were no automatic onboarding messages.
We did everything manually.
When we looked at our logs, we noticed people were leaving and not coming back. We initiated a manual outreach campaign to re-engage our users.
We sent simple emails that focused on highlighting some of the things they could do and direct emails asking them to jump on a call.
Here’s an example of one of the emails:
Subject: Did you know KyLeads could do this?
Hey firstname,
KyLeads was built to help you increase your conversions, at the same time, it’s possible to segment your leads and send better messages.
When you create a quiz, each outcome can be mapped to a different list or tag in your email marketing service.
It’s as simple as creating the outcome and mapping it to the list you want to segment your subscribers into. As long as you’ve added your email marketing service, this information will be populated for you.
Sign in to your account and try it out for yourself.
Regards,
Daniel Ndukwu
This email generated a 15% click-through rate and helped us keep our engagement up. We’re in the process of rolling out an onboarding sequence for new users. We’ll let you know about that when we have more data.
6. Use cases and features I would’ve never thought of
People use your product in whichever way best suits them. Kleenex was originally marketed as a disposable face towel to remove makeup. When the manufacturer, Kimberly-Clark Corporation, conducted market research, they realized people used it as a disposable handkerchief instead.
They changed their marketing to reflect that and their sales rose as a result.
We realized that people wanted to use KyLeads in more ways than we thought and got a lot of feature requests as a result.
- Copying quizzes
We thought split testing would be all people needed to test different versions of their quizzes. Though our users liked that, they also wanted the ability to copy quizzes, change a few things, and publish it differently.
- One answer, multiple outcomes
We also discovered that our users would like to have one answer that would be mapped to multiple outcomes. For example, if there was a question like “what’s your favorite food?” and one of the answers was sushi, they wanted to map it to multiple end results.
That way, people wouldn’t be pigeonholed into a specific outcome.
- Survey forms
Though you can hack quizzes into a certain kind of survey form (we’ve done it multiple times), people wanted more. They wanted dedicated surveys that they could use to capture lead information.
We’ve taken note and we’re on it.
These are just a few of the features people requested that we took note of. There’s a lot more added to our long term roadmap.
7. The faster you respond to people, the more they’re willing to forgive you
Early on, I decided to take customer support seriously. Coming out the gate, our competitors had more mature apps than us.
We couldn’t differentiate on features and we’d spent a lot of time on the app as is. The only thing we could do was be human.
I made everyone on the team download the Intercom app on their phone and keep it open at all times. Though we have office hours, they don’t really matter.
We have a global user base and they contact us whenever they run into a problem. If someone didn’t reply within a few minutes, I’d jump into the conversation.
Our app, naturally, had a lot of glitches. There was an ugly error screen that displayed the code to tell you there was an error, the app wasn’t so fast, and Infusionsoft was being a bitch.
Our users would contact us and we’d reply as fast as humanly possible. Though we don’t apologize much (we don’t use sorry lightly) we do empathize and tell it like it is.
If it’s something we can fix now then we’re on it. If it’ll take some time then we let them know. I’ve even turned away users after having a conversation and figuring out KyLeads wasn’t right for them.
We don’t have enough users for word of mouth to kick in yet, but the few we do have appreciate the effort we put in. They also let us slide when our app decided to malfunction.
How do I know?
They told me so.
We’ve also gotten our support times down to below five minutes.
People aren’t grateful for features – they’re grateful for outcomes.
8. Not everything that can be automated in the beginning should be.
We have an automated email that goes out asking people to give us feedback about why they signed up for KyLeads. It gets opened most of the time and even gets a few responses. I’m happy with it.
The problem was that we didn’t have enough users to make it matter.
If you have a 5% percent response rate from one hundred users it’s still a negligible amount of feedback.
If you personally send those emails, you’ll get a much better response rate.
That’s what we did.
We sent out personal emails to users the day after they signed up and asked them why.
For the people that replied, we’d follow up and ask if they’d be willing to jump on a call for a free consultation.
Obviously, we’d need to know about their business before we could give them useful insights. A few people took us up on the offer which proved to be invaluable for us.
If we’d automated all of that then those insights would’ve gone right over our heads.
Even if you can automate something, that doesn’t mean you should – at least not in the beginning. Do it manually for a while until you understand what’s going on or have achieved the maximum amount of learning.
9. Be there and be helpful
This is one of the most important beta testing lessons we learned.
In the end, our product – your product – is a solution to a problem. That problem could be life-threatening or a commodity.
It doesn’t matter.
The faster you can help someone solve that problem, the better. There’s only so much a good onboarding flow, a well-designed product, and documentation can achieve.
Sometimes, you’ve just got to show up, be helpful, and put in the work.
When you’re beta testing, you don’t a thousand customers so it’s an efficient use of your time.
Conclusion
I was reluctant to beta test because I knew our product would solve a need and was anxious to put it out there and get our first few customers.
Wiser heads prevailed and it has been a valuable learning experience for everyone on the team.
We stayed in beta testing mode for only four months. Some startups do it longer and some don’t do it at all. Whatever timeframe you choose, go in with a learning mindset and you’ll realize just how valuable it is.
Let me know what insights you’ve gained from beta testing in the comments and don’t forget to share.