Background

This research is informed by the existing analytics data and previous studies aiming to improve Day 1 user retention. Previous research insights suggested that users don’t perceive our core product value after initial exploration of the product and user flow is random and messy. Stakeholders involved are Design, Product, and Growth teams.

Process

Previous study / Secondary study

Our analytics data indicates a significant drop in users on Day 1. Uninstall survey results show that many users disengage during their initial experience. These insights drove us to evaluate our current onboarding experience. Research insights suggested that users don’t perceive our core product value after initial exploration of the product and user flow is random and messy.

Research Objective

• Evaluate user flow of the new onboarding design
• Measure and compare user attitude in the early user experience among two versions of design

<aside> 🏁

The new design aims to improve four key attributes, including perceived value and identity association. This is improved by previous study insights.

1. How might we effectively emphasize on the core values and do it early?
2. How might we use product to reinforce the values promised in marketing? </aside>

Research Method

• Unmoderated testing
• Hypothesis testing
  • H1: There is a difference in user attitude after exploring the 2 versions of design
  • Control group: shipped product onboarding
  • Treatment group: new design onboarding

Recruiting

• Criterion sampling: sample participants based on the power user personas, in which case includes usage of product, past behavior, value recognition, etc.
• Statistical analysis: Mann-Whitney U Test (for non parametrical data)
• Sample size n=120 for statistical significance, between participants