On May 20th, we launched a Beta version of our platform to the engineering community. Over the last two months, more than 375 engineers from 30+ countries have signed up for our Beta program. Our initial goal was to have just 50 active Beta users, so we’re thrilled with how many people are tinkering with our tools.
Like many other companies preparing for launch, we’re lean and agile and wanted to get something out to the world quickly. Our goal was to connect with engineers as soon as possible. We needed feedback. We needed our vision to be validated, and we needed to better understand how people use our log visualization tools.
Were there common use cases we can better understand?
Little did we know how much we would learn. From user experience testing and product usage data to specific suggestions from excited users, our Beta program has been a great success, both for our users and for the Initial State team!
Below are five of our biggest insights from the Beta program.
1. Think Logistics & Provide Appropriate Support
The easier it is to become a Beta user, the more people will sign up. But the easier it is to be a Beta user, the more you’ll learn and the stronger your community will become.
After signing up for Beta, our users receive four emails over the course of a couple of weeks. Each email serves a purpose, and throughout the campaign, Beta-specific resources are explained and promoted. We track click-through rates to better understand engagement.
In our final step in the nurturing sequence, we offer a clear and specific call-to-action, asking recipients to reply directly to the email with feedback. Some of our best comments, feature requests, questions and bugs have been communicated to us as a result of that email.
2. Capture and Sort Feedback
We needed one centralized location for all comments. We opted for Trello, which makes it easy to sort feedback into different lists. We created four lists: General Feedback / Questions, Suggestions, Compliments, and Complaints.
With these four lists, our whole team is aligned with how the Beta community is responding to our product.
This board has been very helpful. Repeat comments in both our Suggestions and Complaints lists have helped us to better prioritize our product roadmap and to more effectively triage bugs.
For example, we learned that an unplugged version was desperately needed. Turns out defense contractors have issues with uploading data into the cloud, and so does one very large shipping company. A number of users requested the ability to run an offline, or native/local version of our visualization tool. We spoke more with these users and prioritized this in our product roadmap.
We learned which file formats to build support for. We also learned that our users wanted to more easily share their data with team members.
3. Be Transparent and Genuine with Responses
Our Beta users are kind enough to play with our tools and to give us their time. The least we can do is to be as thoughtful and helpful in our responses as possible.
We try to be transparent. We don’t yet have a formal support process, so we try to respond as quickly (and with as much gusto and aide) as humanly possible.
4. Conduct One-on-One Interviews
Feedback is great, but intimately understanding how an engineer uses our product is priceless. Who are these people? Systems engineers? Firmware engineers? Software or electrical or product development engineers?
We conducted persona interviews. We circled back with folks to better learn about their job, their company, and their feelings toward our platform. These conversations are invaluable.
5. Publish Use Cases / Case Studies
We thought it might be useful for prospective engineers to better understand how other engineers use our platform. We also knew that it would be a great internal team exercise to take a step back and focus on the story — who are we trying to reach? What problems are we helping them to solve?