Overview
Campaign Builder allows users from Oscar’s Marketing and Population Health teams to easily create HIPAA compliant campaigns that range from simple, single-step actions to more complex, multi-step workflows with conditional logic. Prior to joining this team, the old Campaign Builder was built and launched in March, 2019 without research, usability testing or design consideration. The app itself was capable of handling users’ needs, but suffered low adoption due to a frustrating user experience as reported by the intended users.
Role and team
Senior Product Designer
Responsible for the entire end-to-end design and development process which includes; product strategy, research, usability testing, design sprint facilitation, information architecture, wireframing, visual design, prototyping, QA.
Additional team members
1 Product Manager, 1 Product Specialist, 7 Engineers.
Fundamental problems
By January 2020, there were only 49 campaigns that were ever built since the original launch, which is low because our users had a cumulative total of more than 60 additional campaigns they wanted to launch by this time. The low number of campaigns also meant low engagement with our members considering that members could be enrolled in several different campaigns at the same time, but since campaign creation was low, so were the total number of members ever enrolled in campaigns, which was around 536k. Finally, of the 24 total potential active users across the different user groups, only 5 were using it in their daily workflows.
49
campaigns created
536k
members enrolled
5
active users
Research and strategy
As the Product Design Lead for Campaign Builder, my initial efforts included:
Familiarizing myself with the existing app’s user experience, functionalities and capabilities.
Conduct research by establishing feedback and usability testing sessions with our users to fully understand their frustrations with the app, their workflows and motivations
Make product and engineering part of the research process by inviting them to participate in feedback and usability testing sessions with the ultimate goal of spreading team-wide learning from the users.
Reinforce the value of user input to product and engineering teams by regularly sharing synthesis and insight from the research sessions.
Strategize with product on how to address our users’ pain points and how to prioritize those efforts on the product’s roadmap.
Our users
Campaign Builder has 2 primary target users groups.
Marketing
Users who work in Marketing are responsible for member engagement by running campaigns that encourage members to participate in the broader Oscar offerings, i.e., Sending members an email encouraging them to download the Oscar mobile app and booking a virtual wellness visit with an Oscar doctor.
Population Health
Users who work in Population Health are responsible for health outcomes for a group of individuals by running clinical-focused campaigns that impact members’ health and care, i.e., Sending members with pre-existing health conditions a survey to assess if they are at-risk for COVID-19 transmission.
User pain points
Initial research uncovered 3 shared paint points between our users as the primary reasons for why they felt frustrated using the app and why they did not want to continue using it.
Configuration felt overwhelmingly long and used plenty of engineering jargon.
Too technical due to the reliance of writing YAML code to build population queries.
Limiting in flexibility and complexity by only allowing 1 action to occur per campaign.
An additional point of frustration was a mandate to use it from a HIPAA compliance perspective, and they could only set up campaigns with the help of a dedicated product specialist, which ultimately slowed down their workflows and limited the types of campaigns they needed to build.
Old Campaign Builder which caused frustration for our users.
Collaborating on the problem
To understand our users’ problems at a more granular level and to ensure we would build solutions that truly served them, I worked with the Product Manager and Product Specialist to host a 4-day design sprint in July 2019 and invited the leads of Marketing and Population Health to collaborate as stakeholders.
During the design sprint, we collectively discussed our users’ pain points, learned about their respective workflows outside of Campaign Builder, brainstormed various solutions, storyboarded a happy-path user experience, built low-fidelity prototypes and finally tested those prototypes with other Marketing and Population Health users.
Takeaways
The design sprint led to 3 solutions that would directly address our users’ 3 main pain points.
Paginate campaign configuration, allow for progressive saving and use natural language to make the flow less overwhelming and complicated.
Allow users to build population queries by using an input-based query builder instead of requiring them to write YAML code.
Build a flexible workflow editor to allow for a multi-step flow with multiple actions and conditional logic in a single campaign.
The impact of these solutions would warrant a fundamental redesign of Campaign Builder. I strategized with the Product Manager and engineers to scope product requirements and prioritize design work on our roadmap in order to deliver an MVP by February 2020.
Product design lifecycle
Every feature of the redesign incorporated multiple rounds of usability testing in order to learn and iterate on our design work and to keep our users involved in the process before handing off high-fidelity designs to engineering. QA and spike testing sessions with engineering occurred regularly prior to each feature launch.
Campaign Builder’s previous single-page flow next to the proposed multi-page flow
Paginated flow
Breaking out the user experience into several pages allowed for more focus on the individual parts of a campaign configuration and alleviated the overwhelming feeling our users previously had, and using natural language gave our users confidence in understanding what they were configuring.
Query builder
Building a population query through a UI was easier to use and understand for our users by removing the need to write complicated YAML code and relying on a Product Specialist to check their work.
Workflow engine
Giving users the ability to create multi-step workflows with conditionality opened up new possibilities in the types of campaigns and experiments they could run. This feature also streamlined much of their process by eliminating the inefficient need to sketch out workflows on pen and paper and then relying on the Product Specialist to work with engineering to custom code the campaign.
Outcomes
The Campaign Builder redesign launched in February 2020 and was met with an overall increase in positive sentiment from our users, with many of them reporting that Campaign Builder was no longer a blocker in their daily workflows. As of November 2020, the following metrics have been impacted:
147 (+200%)
campaigns created
Increase of 98 total campaigns created due to the flexibility of the redesigned Campaign Builder.
1.36m (+154%)
enrolled members
Increase of roughly 824,000 total enrolled members in new campaigns created.
62 (+1,140%)
active users
Increase of 57 total active users who have adopted Campaign Builder as part of their daily workflows.
Sentiment from a user on the Population Health team as reported by our team’s Product Specialist.
What have we learned since launch?
Our entire product and engineering teams learned the value of keeping our users involved in every step of the design and development process. This built trust with our users and created excitement as we continuously shared progress with them. Since launch, users were able to create campaigns entirely on their own, including but not limited to, COVID-19 risk assessments, prescription refill reminders and virtual primary care announcements.
COVID-19 symptomatic workflow
COVID-19 high-risk assessment workflow
What’s next?
In a larger effort to make Campaign Builder a full service application, we will be working on bringing analytics and insights into the app with the goal of removing users’ reliance on third-party analytics tools.