Design discovery process at Smile

This is a document that I created to help improve the discovery process at Smile.

What?

  • Dual-track agile - cross-functional teams (squads) performing “discovery work” that feeds directly into the backlog of “delivery work”
  • A template for research and discovery that aims to tie all feature work to solving a business problem

Why cross-functional?

  • Cross functional teams raise the “product IQ”
  • Engineers being a part of discovery means they can catch backend tasks early and start making their own tickets without waiting for prototypes
  • Backlog items that have been planned by the people that will work on them
  • Having members of Success/Sales/Marketing assigned to a product squad creates an open channel for ad-hoc communication - without the other party feeling that they have to do this on top of their ‘real work’

How?

The PM provides a problem statement and the squad collectively brainstorm, following a specific template, with the goal of creating a set of testable hypotheses. These then get broken down into deliverables.

Template

Problem statement

Usually comes from PM, should cover 3 areas:

  1. Current goals
  2. A problem that needs to be solved ie. where the goals are not being met
  3. A request for a measurable improvement (important: there is no proposed solution)

Example problem statement:

(1) It’s our intention that merchants on the Free plan are able to identify a paid plan that meets their needs and upgrade on their own. In practice, however, we have observed that (2) only xx% of signups upgrade and of those merchants only xx% self-serve. (3) What can we do to improve those metrics?

Any data points that can be surfaced to back up this claim are very useful, not only to help validate the problem in the eyes of the team but also to provide a sense of what success would look like. This could include analytics reports, usability tests, revenue data, etc.

Assumptions

The first task of the squad is to come up with assumptions. Assumptions are statements that we believe to be true but need to be verified.

Some example assumptions:

  • Users don’t upgrade because the Free plan has enough features already
  • Users don’t want to pay for something without trying it first
  • The paid feature that most free merchants are interested in is app integration

Ideally, assumptions should cover these four areas:

  • Business outcomes - a measurable change we want to see in the world or in customer behaviour.
  • Users - the people for whom we believe we are solving a problem.
  • User outcomes - the goals of the people for whom we are building products.
  • Features - the product changes, additions, or improvements we believe will help our customers achieve their goals.

Hypotheses

Based on the assumptions that were created we should be able to create a set of testable hypotheses. Some of these can be tested by research/surveys, others by creating prototypes and sometimes only by releasing an MVP. Examples:

  • If we reduce the number of branding features in the Free plan more users will upgrade to Starter. We will know this is true if we successfully increase upgrade requests originating from the Branding area after reducing the feature set.
  • If we offer free trials of the paid plans we will improve conversion rates. We will know this is true if merchants offered a trial version upgrade and maintain their subscription for at least xx months.
  • If we offered a basic plan for $9 that only included app integration we could convert more Free merchants to a paid plan that are not interested in the branding features. We can see if this is true by conducting a survey of Free merchants to find out if they would be willing to purchase stand-alone app integration.

Prioritisation and delivery work

We should start with the PM performing prioritisation at their discretion to begin with. A hypothesis is what gets turned into deliverable tasks and added to the backlog.

More articles…