Shipping to Learn: Increment Design and Go-to-Market

One of my favorite topics in Product Development is increment design. As a Product Manager, I have always been enamored with the strategic process of breaking down large initiatives into small increments that drive learning.

I especially loved the challenge of doing this during the phase in Product Development when products were being introduced to customers. The dual challenge of needing to continuously learn, while simultaneously creating a positive, safe experience for customers and the teams that supported them was such a fun challenge.

At Ellevation, the customers of our SaaS products are K-12 school districts across the country. Change management in EdTech is a notoriously hard nut to crack; technology adoption is inconsistent and the ebbs and flows of the school year make timing particularly unforgiving. Despite those challenges, Ellevation has been successful over the years at bringing big complex changes to market thanks in large part to smart increment design.

One of the more successful initiatives was the introduction of our Student List which has driven the way customers use Ellevation to understand their student population, take actions to inform instruction, and ease the burden of compliance requirements.

These are some of the lessons from that journey.

Background: Discovery and Delivery

At the time, Ellevation had around 700 customers and was growing most rapidly in Texas, California, and Florida. Customers were using Ellevation to solve increasingly complex data analysis and workflow-related challenges.

Our products, which were originally designed to satisfy core compliance needs, were not designed to handle this type of complexity. This resulted in workarounds where customers would patch together workflows across multiple features. We tried to adjust the product to meet those needs, but our systems were brittle and hard to adapt.

We knew we needed a new approach.

To kickstart that effort, we conducted a cross-functional Design Sprint focused on ensuring we deeply understood the problems our customers were trying to use Ellevation to solve and had a fresh approach to addressing them. Coming out of the Sprint, we had an interactive prototype to drive customer conversations and test key concepts.

A cross-functional Design Sprint is a great way to build momentum for an idea.

After just a few weeks of talking to customers, it was clear that the problems we were trying to solve were real and important. Our solutions were still a ways off, but we knew we were on the right track.

As the Engineering team made progress, there was a moment where we were able to swap out the prototype for real, production code. Customers didn’t notice a major shift. For them it was a continuation of the journey. Internally, that was not only an important milestone, but also a major learning in how we could bring the product to market by minimizing change.

As our development continued, the sense of excitement around the new solution gave way to urgency. It was at this point that the primary challenge flipped from building the right product to developing the right strategy to ensure customers could benefit from the new feature as soon as possible.

Go to Market Strategy

The Go to Market (GTM) challenge facing us was immense. Getting customers excited about something new was the easy part. Creating a pathway to move 700+ customers who were using different permutations of legacy systems that they had configured, trained teachers on, and grown accustomed to was the hard part. Doing so quickly and safely was downright daunting.

Speed mattered, but we didn’t want to use a big bang launch. We had a small engineering team and introducing the new feature meant that we would have to support old and new at the same time. This wasn’t ideal, but it was clearly better than trying to introduce something new to replace something old in one action.

Pro Tip: try to avoid adding something new and replacing something at the same time.

We knew we would have to support both systems at the same time, but wanted a fixed window to complete the transition and sunset our legacy systems.

To navigate this complex challenge, we would need to consider many difficult tradeoffs. Early on, we knew that there was no such thing as a perfect plan. We wanted to build in optionality to adjust our plan based on new information that we would inevitably learn. In other words, we focussed on strategy over plan.

To do this we developed a GTM Strategy based on a few key principles:

  • Internal readiness over external communication
  • New value over minimizing change
  • Opt-in moments over rigid segmentation
  • Continuous delivery over polished launch

Having a defined GTM Strategy was important for two main reasons. First, it helped others on our team understand the activities and tactics we were using. Second and more importantly, it allowed us to say no to some things that were genuinely good ideas, but didn’t align with our GTM Strategy.

Internal Readiness over External Communication

Before we undertook any external training or adoption campaigns, we focused efforts on preparing our internal teams. This included the early development of talking points, numerous dogfooding opportunities, and an interactive all-hands offsite session focused on developing confidence with the new tool.

The idea was that if we prepared our internal teams to fully understand the new feature, they would become our greatest evangelists. We didn’t do much external customer messaging. There were a few opt-in webinars that were well attended, but most of the transition effort was conducted through normal check ins led by the Account Management and Partner Success teams.

This internal readiness created a much faster (and safer) pathway to engaging with customers for GTM. We ignored scale, and focused on a limited set of high-touch engagements with customers. Because these engagements were highly controlled, we felt a high degree of safety.

New Value over Minimizing Change

Marrying multiple disparate features together into a single, coherent product was immensely satisfying. It allowed us to standardize our approach to things like filtering and forced us to think in terms of systems and scale from the outset. But this product approach also came with a risk of assuming that everything you could do in the various legacy features would be in the new system – these assumptions applied to data availability, certain interactions, and configuration.

To counter this, we focused on establishing a clear set of talking points form the earliest moments of the initiative. Our first effort focused on parity - how internal teams and customers could understand how the new system was similar to the systems it was replacing.

That served us well early on, but when we started sharing those talking points with customers, the conversations became narrow and specific and we found that those conversations prevented us form having the opportunity to talk about the benefits of the new system.

So we pivoted and changed the focus to talk about benefits first. We untethered ourselves from the legacy systems and talked about what was new, exciting, and better about the new systems. We wrote the new talking points as if the old system didn’t exist. It was both liberating and effective.

Creating visibility into what a new system does that an old can't can be more strategically valuable than creating a crosswalk to a legacy system.

The most important lesson that we learned was that the single most effective tool to manage change was to build an awesome new product.

Opt-in Moments over Rigid Segmentation

We knew from the outset that we wanted to avoid any “big bang” cutover events.

Early on we identified that our own ability to determine preparedness levels for customers was imperfect at best, and more likely than not wrong. In lieu of trying to develop complex segmentation plans, we prioritized self-selection through the creation of opt-in moments.

This was one of the highest stakes decisions, but one that paid off in the end. We created 4 key opt-in moments through the year – the last of which was a firm deadline. The natural distribution was unsurprisingly heavily weighted towards the later opt-in dates, but Partner Success and Revenue identified a number of customers to pull-forward their transitions balancing out the distribution. If despite our proactive messaging and personal outreach, a customer wasn’t ready to or didn’t communicate, they were put in the last cohort.

The power of having these options was that it enabled our customers to feel in control; this was important because it allowed us to avoid conversations where we were forcing customers to do a hard thing. Rather, we were able to flip the script and work to help them avoid waiting to do the hard thing.

It’s important to note that one of the keys to enabling this change was that internally, we were all aligned on a key immovable sunset date for the legacy systems; that clarity created the right incentives for the Revenue and Success teams to leverage the latitude on timing to help customers find the right timing for them.

Another one of the important learnings was that after we got through the first few transitions, we realized the technical cost of a transition was relatively low. We told our internal teams that we could cutover any customer at any time; the unambiguous goal was to get all customers on the new tool. Internally, we were clear that the opt in periods were marketing / positioning and not a product or engineering limitation.

Continuous Delivery over a Polished Launch

We had key milestones identified, we had 700+ customers depending on us to meet deadlines, we had internal teams nervous about timelines and our ability to deliver. Those things usually spell doom for a team’s ability to work fast. But we stayed focused on delivering incremental value.

We continued to ship code as often as possible (usually 2-3 times a week) and introduced new functionality and closed gaps that customers identified along the way. Being able to move quickly while bringing a new feature to market was instrumental. The perceptions internal teams and customers may have had that a big launch only happens once in a blue moon and that if there was a problem, it would take forever to fix it, were myths that we had to bust.

Overall, the ability to respond quickly, proved that we were continually building and improving the new feature worked to our advantage, demonstrating to customers that we were focused on seeing this through to be successful and weren’t just satisfied with having the new thing out there.

Lessons Learned: Perfect IS the Enemy of the Good

In retrospect, we came up with a really good strategy to launch the Student List. Not everything went exactly to plan, but we had a framework for what we thought we make us successful and stuck to it. Along the way, there were a few unexpected lessons that we learned.

Aim for the Stars, Enjoy the View from the Moon

Early on, we aimed our product discovery and development efforts on solving the Reclassification use case in California. We learned the ins and outs of the most complex use case with the confidence that if we could crack this nut, we would hit on a number of more linear use cases along the way. This served us well and created a lot of options for early increments outside of California. The key was being able to update our strategy as we learned from the market.

Creating Value Means Embracing the New

For all of the reasons outlined above, the effort to transition customers was an urgent priority. Customers wanted the new feature, our teams didn’t want to bear the cost of supporting two systems, and we needed to sunset the original Student List and Data Dashboard. As we evaluated our ability to drive change, we always came up short of a sure thing. There were concerns across the board – small vs. large, new vs. old, California vs. North Carolina and everything in between.

If we had waited to resolve all issues and create consensus before moving, we would have failed. To generate value and drive adoption, we needed to jettison the old, and focus fully on the new. Making this choice when we were still far from 100% confidence included risk, but it’s a decision we would make 10 times out of 10 again today.