Validating FTUE and In-Game Support
Project Overview
This study aimed to validate the design improvements in My Reading Academy’s First-Time User Experience (FTUE) and game mechanics, ensuring that children could easily navigate and engage with the app. The primary focus was on identifying any pain points and confirming the ease of use to support long-term engagement and educational success.
My Role and Responsibilities
As the Associate Director of Design Research, I led the usability testing sessions and conducted the analysis. I collaborated closely with the Director of Design, Senior UX Designer, and Game Designer, overseeing the study design and delivery of insights to guide product release decisions. I reported directly to the Product Owner and VP of Marketing.
Research Goals and Questions
Primary Objectives
Test the ease of use of the First-Time User Experience (FTUE).
Validate the effectiveness of the updated game mechanics and identify any blockers or challenges faced by children.
Key Research Questions
Can children easily start and engage with the game?
Are there any blockers or difficulties with the game mechanics?
Do children utilize in-game assistance features like the Helping Hand Icon?
Methodology
Research Methods
I conducted In-person usability testing to observe how children interact with the updated app directly, focusing on the FTUE and game mechanics.
Participant Recruitment Strategy
Screened 12 children aged 6 who regularly play online games, ensuring familiarity with digital interactions but without prior exposure to the app’s new features.
Timeline and Constraints
The study was conducted over several sessions as part of a bi-weekly sprint cycle, aligning with My Reading Academy's iterative development process.
Research Process
Step-by-Step Approach
1. Moderated Usability Tests: Conducted 20-minute moderated usability tests via Zoom, observing children as they interacted with the game.
2. FTUE Navigation: Focused on understanding how children navigated the FTUE and engaged with the game mechanics, including selecting characters and starting the game.
3. Device Interaction: Collected feedback on gameplay across various devices (laptops, tablets, mobile phones) to assess consistency in experience and usability across platforms.
Tools and Techniques
Used Zoom for remote testing, Miro for real-time note-taking, and collaboration with the team. The research team and product stakeholders observed the testing sessions live to gather immediate insights.
Key Findings and Insights
Main Discoveries
1. Smooth Navigation: All participants (100%) successfully selected a character and started the game, demonstrating an understanding of the mechanics and confirming the FTUE’s effectiveness.
2. Assistance Utilization: 40% of children used the Helping Hand Icon, indicating a need for in-game support features and validating its importance for younger players.
3. Engagement Across Devices: Children were able to engage effectively with the app across all tested devices, showcasing robust cross-platform usability.
Visual Representation
The bar chart above titled, "Engagement Success Rates for My Reading Academy FTUE" illustrates the engagement success rates of 12 children with various tasks during the First-Time User Experience (FTUE) in My Reading Academy. The chart highlights key areas of success and opportunities for improvement:
- Character Selection and Game Start (100%): All participants successfully completed the tasks of selecting a character and starting the game, indicating that these initial steps are intuitive and effectively designed.
- First Level Completion (85%): A slightly lower success rate was observed for completing the first level, suggesting a minor drop in engagement or understanding as gameplay progressed.
- Using Helping Hand (40%): Only 40% of participants utilized the Helping Hand Icon, highlighting a significant area for improvement. This suggests that in-game support features may need to be more accessible, visible, or engaging to encourage their use.
This visual representation underscores the strengths in the initial navigation of My Reading Academy and identifies opportunities to enhance in-game assistance features, ensuring a smoother and more supportive user experience for young learners.
Recommendations and Impact
Actionable Suggestions
1. Maintain the Helping Hand Icon Mechanic: Keep this support feature as a critical element to assist young users and ensure a seamless gameplay experience.
2. Continue Iterating on FTUE: Regularly refine the FTUE to keep gameplay smooth and intuitive, enhancing the learning journey for new users.
3. Monitor Cross-Platform Consistency: Continue testing across different devices to ensure a consistent and engaging user experience.
Impact on Product Decisions
The study validated the design improvements, confirming that the FTUE and in-game mechanics are effective and intuitive for young users. This validation ensured that the release schedule remained on track with only minor adjustments needed, supporting a smooth launch.
Measurable Outcomes:
Expected reduction in user drop-off rates due to improved navigation and effective in-game assistance.
Higher engagement and satisfaction levels as children navigate the FTUE and game mechanics seamlessly.
Reflections and Learnings
Challenges Faced
A primary challenge was observing how different devices impacted the usability and engagement of the game, highlighting the importance of consistent design across platforms.
What I’d Do Differently
Incorporate more diverse gameplay scenarios in future studies to further test the robustness of the FTUE and in-game mechanics under varying conditions.
Skills Developed
Enhanced my ability to conduct usability tests with young children, focusing on their interaction with digital products and refining observational techniques to capture nuanced feedback.
Next Steps
Follow-up Research
Monitor user feedback post-launch to identify any unforeseen issues and areas for further improvement, ensuring the app continues to meet user needs effectively.
Long-Term Implications
Continuous user testing and feedback will be essential for maintaining the effectiveness and appeal of My Reading Academy. Ongoing iterations will help keep the app engaging and supportive of young learners' needs.