Case Study: Segment President Ilya Volodarsky On How To Effectively Use Data Analytics In A Startup

Learn from Segment's President how to properly leverage data analytics in your startup.

Jul 11, 2020

Apply for the Segment Startup Program, which gives startups free access to tools and unique training to help master data analytics from the earliest stages of their business.

Executive Summary:

The Successes - Solving Data Integration Problems And Giving Back To Other Startups
Segment achieved success by first solving their customer’s data integration problems, then turned around and use their knowledge to help other startups earlier in their journey.
Action Item: Once you figure out how to solve your core customer’s problems, see how you could build a community around your solution.
The Failures - Struggling With PMF & Nearly Losing His Friendships With His Cofounders

Volodarsky captured the mistakes of founders succinctly: “Too many founders waste their time building irrelevant products because they focus on what's possible, not desirable.”
Action Item: Stay focused not only on the problems that need to be solved but also prioritize your relationships with your cofounders - success won’t taste as sweet without them.
The Lessons - Be An Egoless Learner And View Your Business As An Experiment

Volodarsky learned to not take success or failure personally, and look at Segment dispassionately when trying to find the best way forward.
Action Item: As much as your startup becomes a part of you emotionally, you still have to view its success and failure in an objective lens to move forward.

Founder File: The Analytics Stack for Startups

Segment is an analytics API and customer data infrastructure used by over 19,000+ customers worldwide. We work with the world's leading marketing, analytics, and engineering teams to collect and centralize their customer journey, including the teams at IBM, Intuit, Angie's List, DigitalOcean, and New Relic. Our iOS and Android libraries have been open-sourced since March 2013, and are now powering the analytics stack for over 5,000 mobile apps, including HomeAway, HotelTonight, Instacart, VSCO, and DraftKings.

Under the hood, Segment collects 10 core customer data streams: mobile, web, server, offline, support, sales, email, push notification, advertisement, and payments, and integrates these streams to 180+ third-party tools (Google Analytics, Amplitude, Facebook, etc..) and data warehouses (Redshift, Postgres, BigQuery).

Volodarsky, cofounder and President of Segment.

Note: For further information on data analytics, I recommend reading the following - Case Study: Viewing Data From A Leadership Perspective With PlanGrid's Ralph Gootee (acquired by Autodesk for $900M)

The Successes: Solving Data Integration Problems And Giving Back To Other Startups

What has been the most substantial professional success you've had in resolving the challenges associated with data analytics while building Segment?

There are two successes I would mention. One is related to Segment as a solution, and the other on how we are giving back to the startup community to solve their analytics challenges.

The first is that, with Segment, we've solved the problem of data integration. There are so many fantastic analytics and marketing tools on the market right now, but companies struggle with connecting those tools to their data. They end up with a messy data stack that causes more problems than it solves. But with Segment, you can connect all of the tools you want, using it as the single source of truth - and integrations can be performed in minutes, taking the pain and hassle out of switching tools in and out as you finesse your stack.

The second is the Segment Startup Program, which we launched in 2018 to provide early-stage companies with the financial, educational, and business support they need to reach key product milestones and effectively grow their business. The program includes free access to Segment's CDP for two years. It has been an enormous success, helping hundreds of startups with some of the biggest challenges in analytics. We go round to early-stage startups, and we tell them not to make that same mistake that we did and waste half a million dollars over two years! Thousands of startups have signed up already, and we're continuing to see companies go from strength to strength by following the advice we've given them.

As Segment has evolved throughout its journey, did success in data analytics takes on a different form as it grew? If so, why? How does success in data analytics for Segment change as the company scales?

Segment's evolution has shown us the need to adapt your analytics stack throughout your journey. Having the flexibility to change things up as you focus on different metrics is essential, and learning how to do this has been crucial to our development as a company.

As our company has grown, we have layered different tools into the analytics stack. Our focus shifted from building a minimum viable product (MVP), to searching for product-market fit and eventually scaling the business. The tools have changed as the metrics changed, starting with monitoring the number of user signups and retention in the beginning and qualitative feedback on our product. Then later, more marketing-focused metrics as we scaled and began to roll out advertising, and so on.

The confidence and ability to do this wasn't always there but is something we've learned and developed. That has helped to shape Segment as a product. For example, in the beginning, we were incredibly confused by which tools to use, which metrics to look at, and how to integrate them. Then we stumbled on an idea. What if we implement our analytics with a simple single API? The API would be as simple as possible: who is the user? And what are they doing? Having a simple foundation like that meant we could build our data stack without unnecessary complexity, which was essential to keeping the whole company laser-focused on the metrics that mattered.

These two API calls serve as the basis of what Segment is today and are essential to how we've been able to switch out different tools to meet our business's changing needs.

What general advice do you have for founders who struggle with success in using data analytics for their startup? What should they take away from attempts and subsequent success at Segment?

Of the companies I have spoken to, about a quarter of founders know how to use analytics and use methodologies, and 75 percent don't... Too many founders waste their time building irrelevant products because they focus on what's possible, not desirable. It's crucial to spend your time searching for unmet needs before you dive into building solutions. Otherwise, you'll be tackling non-existent problems and creating products nobody will want to buy. Once you have that clear focus, it'll be easier to use analytics in a meaningful way.

My biggest tips:

Focus on the metrics that matter.
Don't build tools in-house.
Make sure you make the metrics accessible to the whole company so every team can act on them.

Success doesn't happen when teams work in silos.

With the current wealth of tools available on the market, even the youngest company can use first-party data to make better decisions earlier. Google Analytics is a powerful tool for primary, top-of-the-funnel analytics for young startups when you're just getting started. Then, when things get a bit more advanced, Mixpanel and Amplitude are great choices.

Finally, don't discount the value of qualitative feedback. If you're still in the test-and-learn phase, live chat software such as Drift and Intercom provides an excellent way to source direct, qualitative customer feedback to work into your developing products and services. And make sure to open Slack channels with your customers, and never stop iterating based on their feedback!

Having the flexibility to change things up as you focus on different metrics is essential, and learning how to do this has been crucial to our development as a company.

The Failures: Struggling With PMF & Nearly Losing His Friendships With His Cofounders

What's been a notable or defining failure that you personally or professionally had while building Segment?

Eight years ago, my co-founders and I had no easy way to quantify product-market fit (PMF), and as a result, we burnt through $500,000 trying to launch six different products.

Our first idea was called ClassMetrics - a classroom lecture tool that allowed students to say whether they were confused, and gave the lecturer a live cardiogram of the class confusion. We took this idea to the prestigious startup accelerator, Y Combinator, in 2011. We went out to different classrooms to start beta testing it. We found out that when the professor would say, "everyone takes out your computer and start using ClassMetrics," the kids would last like two, three minutes before they would go back to Facebook or start reading their email.

So very quickly, we realized that it's not a good idea to disrupt classes where people pay attention and take notes on paper by asking them to take out their computer. Our mentor at Y Combinator, Paul Graham, eventually turned to us to say "so you've spent half a million dollars and you have nothing to show for it…" We ran out of money, and it felt like we were just a few months away from disbanding as a team. Luckily, it was that we came upon the idea that became what Segment is today.

How did this failure impact you on a personal level? Did it change the way you view yourself as a person?

Those two years were incredibly painful while we wandered the desert with no PMF. I wish I had learned earlier how to find out what customers need, so I didn't fall into the trap of what I thought they needed. We had a massive fall out and nearly lost each other as friends. As a 21-year old co-founder, I found the whole experience pretty shocking at the time!

How did this failure impact you at a professional level? Did it change the way you view yourself as a founder, and Segment as a whole?

It taught me the importance of listening to our customers and finding out what they want and need. Only by solving real customer pain-points is your company going to be viable. And it was clear that data is crucial to this. Without having those early failures, we wouldn't have been forced to learn how to use analytics properly - and on a professional level that's been key to our current success, and my focus as leader of Segment's Startup Program.

Too many founders waste their time building irrelevant products because they focus on what's possible, not desirable.

The Lessons: Be An Egoless Learner And View Your Business As An Experiment

How should founders view and handle success?

Know what success looks like. And ask yourself, is my success now really sustainable?

We often see teams mistake tracking feel-good metrics like Total Active Users or Total API Requests. But measuring totals gives you only half of the picture - they're just vanity metrics.

Too many times, I've seen startups focus on the wrong thing. For example, I remember a startup that was profitable and working hard to increase its revenue. But it realized that the numbers are stagnant at the end of the year despite hundreds of new customers signing up.

Analytics is crucial in moments like this, and looking at the data reveals the fuller story. The company is only retaining users for about three and a half months, causing them to lose as much revenue as they gain during the year.

How should founders view and handle failure?

Learn from it! It's all part of the process.

I find that the most motivating times for me are when it becomes clear that I have failed at something over the previous quarter or two. It means that I now understand what I need to get better over the next quarter.

We reflect on our company goals every quarter. We reflect on the annual plan once a year. I also set personal goals once a year and reflect on them quarterly. That's probably most of the reflection process.

How should founders manage their emotions in the event of success or failure?

The happy path is treating your business as an experiment for as long as possible; failure means you got time back from investing in the wrong way; a success means you should invest further in the experiment. This allows for quick egoless learning and makes the perpetual ups & downs bearable.

Analytics is crucial in moments like this, and looking at the data reveals the fuller story.

Apply for the Segment Startup Program, which gives startups free access to tools and unique training to help master data analytics from the earliest stages of their business.

Previous F2F Q&A Articles:

Latest Forbes Articles:

If you enjoyed this article, feel free to check out my other work on LinkedIn and my personal website, frederickdaso.com. Follow me on Twitter @fredsoda, on Medium @fredsoda, and on Instagram @fred_soda.

Founder to Founder

Discussion about this post