The grocery store weekly-ad is a cornerstone for helping families plan their meals for the week and find the best deals in their area. But print is dying out, becoming increasingly expensive, and offers a brittle experience that is vastly outclassed by digital. Our goal was to develop the weekly ad from the ground up based heavily on research with our customers in an effort to begin nudging folks from using the paper printed circular to seeing the benefits of the digital experience. Here is the current weekly ad for reference (at the time of publishing this case study, these designs are not yet live).

I led this project alongside 2 UX researchers, 2 UX designers, a Sr. PM, and numerous partners across marketing, merchandising, and data science. This was a massive project, so the first step was creating a UX playbook of what we wanted to complete in order to define success for this project.

UX Playbook

1. Blue Sky Designs
2. Research: Diary Study & Interviews
3. Research: Competitive Landscape and Current State Audit
4. Proof of Concept for Personalization
5. Design Iterations
6. Usability Testing
7. Measurement Plan

Blue Sky Designs

Conversation starters for the 2 week diary study

The true impetus for this project stemmed from two distinct moments: 1) one of our brands lost print distribution for a week costing them an estimated $300k in sales 2) a vendor demoed some vaporware demonstrating 'personalization' using 'real data' and the brands got really excited.

As a result, our first step in this project was to unpack the value of a personalized weekly ad. We had already committed to the idea that this wide-reaching of a project required an ambitious research study. In order to support the right questions for a 2 week, 50 participant diary study, we wanted to design some real conversation-starters.

Enter our 'blue sky designs' which strove to simplify the ad while maintaining the graphic-designesque look and feel of the printed circular. We went for two approaches to share with customers which we named Personalized & Organized and Value & Inspiration.

The design on the left offered a wider range of filters and sorting, immediately leading with personalized offers (Top Deals for You). It then went category by category with interrupting blocks highlighting other collections of deals the customer might be interested in. This design strove to put the customer's personalized picks front and center.

The design on the right simplified the tools and led with seasonal inspiration. The biggest difference between the two was that for this design, we chose to group offers by type rather than by category (coupons, meal plan, 10 for 10, BOGO offers, etc.).

One thing to callout: the weekly ad is the same for every customer based on the grocery store's sales for that week. As such, each person is seeing the same 300 (let's say) offers each week... the question is: in what order and emphasized how? I am over simplifying the complexity of these designs, but these served as the backdrop for a one hour conversation with 11 customers chosen from the diary study, which I'll discuss now.

Research: Diary Study & Interviews

We gathered over 700 in-context moments from 50 participants using weekly ads over a 2-week period. We then used 2 designs to facilitate a discussion with 11 ADUSA customers about the digital ad of the future.

The diary study used the "Jobs to be Done" framework to understand three key questions:

1. Why do customers turn to print and digital weekly circulars and what do they seek to accomplish?
2. How do customers interact with print and digital circulars across different brands and how does it fit into their shopping journey?
3. How might we encourage customers to transition from a print ad to a digital weekly circular?

After following 50 customers for two weeks, observing over 700 contextual moments, we invited 11 representatives for a one hour deep dive each to discuss their observations and our blue sky designs. Across the discussions several "jobs" emerged: help me save money, help me plan what to buy and where, help me find inspiration.

When asked if the printed paper ad were to go the way of the dodo, we heard a wide gamut of answers ranging from, "I will be devastated if paper circulars go away" to "I understand this is where all retailers are going. I don't love it, but I'll adjust" to "I'm already fully digital and others should catch up". For print fans, it meant they could collect all their local brand's circulars and splay them out on their kitchen table to plan their week.

We realized print loyalists would be too hard to sway, but felt we could influence those that were intrigued by having the ad be more customized to their past purchases and likes. However, we found many folks were skeptical of an overtly personalized experience, saying things like, "That's a little creepy"; "But I don't want to miss out on other great deals"; and my favorite:

You don't know me, I want to see all deals even if they don't apply to me.

From our key research questions and the main "jobs" customers hired to be done, we developed four key principles that defined the rest of this project.

1. Optimize the weekly ad to help customers save money, plan what to buy, and find inspiration.

Customers use weekly circulars to complete three main "jobs to be done". If what we build doesn't help them accomplish these jobs, they will find another place to shop. The most important task being to help customers save money.

2. Start with UX Improvements that strengthen the pillars of a good digital ad.

There are three pillars of a good weekly ad experience:

1. Clear and captivating pricing
2. Intuitive navigation and organization
3. Interactivity that supports processes

Any digital ad redesign should include clear and captivating pricing such as highlighting the top deals at the store, show the amount of the savings, and explain how to get more savings like additional coupons. Organization by categories can make the ad easier to use as it matches how people shop in-store. Options to navigate with filters and search can give customers control and reinforce relevancy of the ad. Customers don’t want to have to create new routines to use a digital ad. Enhancing interactivity such as 'circling' items, clipping coupons directly from the ad, and integrating with external shopping lists will help digital adoption.

3. Personalize the weekly ad quietly and with intent.

Personalization ultimately did not help customers complete their jobs to be done. There were too many skeptics that felt it might help them, but that they would still need to review the entire ad even if offers didn't pertain to them. Beyond saving time, we must develop personalization elements that clearly show customers how this will help them save money, plan shopping, and find inspiration.

4. We must nudge customers to adopt digital circulars while the paper circular still exists.

People naturally resist change. We can’t fundamentally alter their routines. We can’t expect customers to adopt digital overnight. We need to start with nudges now, while both paper and digital formats exists so paper readers can come to appreciate what the digital ad offers.

Small nudges that chip away at the reliance on paper look like:

• Integrating QR codes in the ad and store to link directly to digital experience and offers​
• Instituting SMS, email reminders, or notifications informing paper readers the digital circular is available​
• Focusing on features that honor the paper readers process such as print view defaults, interactive meal planning, and flexible digital navigation

These four key principles formed our path forward for the remainder of this journey and served as a touchstone for each choice we made along the way.

Research: Competitive Landscape and Current State Audit

Define the lay of the land and note our existing pain points.

Concurrently with the diary study, myself and my two UX designers began scouring the competition and auditing our own experience. In the tech space of weekly ads, there aren't very many vendors - with regional grocers picking between one of two camps. Smaller grocers even employed fully static assets, making their digital experience a glorified PDF.

Competitive Landscape Analysis

For the competitive landscape we looked at five other grocers, chosen due to their variety of weekly ad styles and technical vendors. Some common themes that we observed were that many brands were already transitioning from paper to digital by heavily promoting their website/app on the front page; most brands defaulted to the 'print view' experience that feels more familiar than a digital-first layout; most vendors included shoppable offers, the ability to clip coupons, and monetized zones.

Our high level takeaways from the competitive landscape were:

1. All brands have similar digital counterparts to the paper circular
2. Mobile web is poorly optimized across the board
3. Clarity of pricing has a lot of room for improvement
4. Balancing pricing clarity with clean and consistent design is a challenge

Current State Audit

We conducted an end-to-end audit of the paper circulars as well as the mobile web, desktop, and app experiences across all of our brands. We sought to uncover pain points and review all of our voice of the customer data on this topic. We also wanted to fully understand our API implementation and any technical restrictions. Lastly, we wanted to identify any areas of opportunity.

Our summary from all of this broke it down as such:

1. Our current vendor comes with a lot of baggage and does not get access to 1st party data
2. Performance issues and bugs prevent the reason customers come here: for meal planning and inspiration
3. The mobile experience poorly restricts pinch and zoom and has unoptimized mobile interactions
4. Customers shared the sentiment that they felt there were more deals available on the paper circular than the digital counterpart

With knowledge under our belts about what others in the weekly ad space were doing as well as a firm technical understanding of our current approach, we then combined this with the diary study to begin initial concepts.

Proof of Concept

A quick aside to some technical stuff

The goal of this POC was dead simple: demonstrate that we can display our current weekly ad grid view using real, personalized data.

Part of my role in leading this project includes working cross-functionally to help define our algorithm and understand business needs for the weekly circulars. Luckily for me, the data scientist we paired with on this project was a former PM that I used to work with which made collaboration super fun. We sat down to identify how we would structure our approach.

The first step meant we had to bypass our current vendor and send real 1st party data through our data science algorithm and then pass it to the back-end. We had to learn how to get this feed from our internal app and clean the data for web. We also needed to vet the feasibility of personalization using one of several different models the DS team had developed. We wanted to be able to demonstrate to stakeholders the value prop of native personalized ads. Lastly, we needed a way to test the effectiveness of the algorithm, identify user personas, and observe behavior across our many customer segmentations.

After a few sprints, we developed a staging environment that included a toggle on the weekly ad page allowing us to switch between any of 11 different personas that represented the main customer segments our DS algorithm defined. Overall, the goal of the POC was to assess the viability and potential impact of using personalized ads in a real-world scenario and make informed decisions about the features and experience for a full-scale implementation. We used this artifact later in our demo to the brands alongside our final design approach.

Design Iterations

This is what you're here for, right?

In the early moments of iteration, we began quite broadly - taking great liberties with information architecture, image quality, and things we probably knew were impossibly tagged. While ultimately anything was on the table, I didn't want to stifle my team in the early weeks of designing as some of these ideas paved the way for others and some even lived on to the final designs. However, as the POC developed, I was able to share insights with my design team to bring us back down to earth. We took a lot of early inspiration from the more magazine-like approaches of some of the better players. We hoped that our CMS could eventually accommodate complex components with fluid layouts.

Wireframes / Low-fi Designs

I tasked my two designers to each focus on a different aspect: offer tile and layout. Our product tile is something we have been perpetually designing since my time with ADUSA, so tackling a potential redesign of the offer tile was no small feat. However, with the key principle of clear and intuitive pricing, we at least had a north star to head towards. Similarly, for layout, we knew that intuitive navigation and organization was paramount. We also knew that enhanced interactivity would be a key for customer adoption.

After spending a few sprints iterating (roughly aligning with the finishing of the POC), we pivoted from designs to a discussion around defining the MVP. Meeting with our PM, lead data scientist, and research manager we mapped out what an MVP might look like. We white-boarded in the office in Chicago and walked away with a new concept of running a pilot with a single brand. We documented which features would make the cut and which were deemed out of scope. We identified risks and mapped out a timeline. My team now had a clear goal of when to begin usability testing in order to meet the development timeline.

With the MVP scope and requirements fully defined, transitioning from wireframes to mid-fidelity designs felt quite natural. Part of what we left behind was the idea that we were going to drastically redesign the tile itself and instead focused on updating the existing IA with some new elements that we wanted to highlight. During these few weeks, myself and my lead designer meet constantly to critique and hone in on our approach with full team critiques weekly as well. By the end of our self-imposed deadline, we had something we were truly proud of and felt would confidently pass usability tests with flying colors.

MVP Pilot Approach

In order to better understand where we landed, let's first define the pilot and what we hoped to accomplish. We wrote a vision statement so that everyone (including stakeholders) could be aligned to what we were building.

More specifically, we decided to focus on what we called an "enhanced grid view" and target mobile web only. Most of our traffic goes to mobile - either to native apps or to the mobile browser, so it makes sense to focus our efforts on solving for that platform first. Larger layouts on desktop makes everything easier, so this forced us to ruthlessly edit and pare down.

Additionally, we decided to focus on one brand as part of the pilot and committed to running a second diary study with 50 participants for 4 weeks. We knew that a simple a/b test wouldn't cut it as we would not gather enough qualitative feedback. A usability test would help prove simple usability, but without using real customer data to test the validity of the data science algorithms we wouldn't know if our approach to personalization was working as well as we hoped. We narrowed down a few more criteria and then set forth to finessing the final designs.

Final Designs

To summarize our final design choices, we can group the changes into four buckets: Navigation, Filters, and Sorting; Offer Tile; Overall Structure; and Other UX Improvements.

Navigation, Filters, and Sorting

• Clear and dedicated search specific to the weekly ad
• Filter drawer exposes most common filters with docked access to "All Filters"
• My List allows customers to create a weekly ad short list (that is also added to their full shopping list)
• Simple CTA for switching to the "Print View"
• Sort by defaults to personalized categories with the option to sort by best % and % off (as well as most recently added 'page order')

Offer Tile

• Large, personalized image for the offer (using product images instead of 'offer' images). The image shown will reflect the top item in the offer for that customer (think flavor, variety, type, size, etc.)
• Clear and intuitive pricing showing offer price, percentage off, and dollar off.
• Coupons available callout will be accurately shown wherever additional coupons are available
• CTA changed from "Shop Now" to "View Eligible Items" to support both ecommerce and brick-and-mortar customers.

Structure

• By far the biggest change: default to personalized category view of offer carousels
• "Front page deals" carousel allows for customizable CMS featured offers - a key business strategy
• Large tiles allow for non-offer content to be highlighted (a shortcoming of the current weekly ad grid-view experience)
• Monetized zones allow for further CPG partnership opportunities
• Sort by % or $ changes offers from carousels to list view, sorted by best discount first
• Selecting multiple categories in the "All Filters" drawer also puts offers into list view

To expand on the topic of personalization, this will be done three-fold in the pilot. Firstly, the order of carousels will be personalized such that if I buy mostly produce and diary, I will see those carousels at the top of my list. In other words, they will not be in a static order, but unique to each customer segment. Secondly, the order of the offers within each carousel will be sorted by past purchases and adjacent items so that the offers that a customer might find most enticing will be the first they see.

Lastly, as offers can house as many as 50+ items, the order of those items will be personalized so that the customer sees any past purchases or DS algo identified items at the top of the 'shoppable items' modal. Let's take the example in the prototype embedded below: a customer sees "buy 1 get 1 of equal or lesser value" for "Blueberries, Raspberries, Strawberries, Blackberries". If a customer is more prone to buying raspberries, they will see the product image for raspberries as the offer image and they will see raspberries first in the list on the shoppable modal. By showing the customer the variant they tend to buy rather than a collage of products or a poor best guess for all, we feel that quietly personalizing this experience will greatly improve engagement.

Other UX Improvements

• My List - a weekly ad specific tool for list building
• Add to shopping list CTA spelled out on offer modal to connect back to the icon on the offer tile
• Back to Top CTA upon reaching the bottom of the page to easily jump up to navigation
• Full filter drawer with a wider range of options
• More robust search

With these concepts highlighted and the caveat that this case study may soon be out of date when comparing the below prototype to this live page in production... here is the prototype we created for usability testing.

Advanced Prototype

Usability Testing

Two usability tests with different goals in mind

Our final stop in the UX playbook was to put these designs in front of customers and gather some qualitative feedback. We split the tests in two:

1. The first being very task focused: can the user do this, can they access that?
2. The second test attempted to validate the jobs to be done.

At ADUSA, we have a pretty standard methodology for usability testing - beginning with a research brief written by the lead designer and then reviewed by their UX Research partner. We typically recruit 8-12 participants and limit each test to a handful of specific tasks with follow-up questions digging into more qualitative aspects of their experience.

For the first study, we asked:

• Can a user filter using the exposed filters? Can a user filter using the unexposed filters?
• Can a user add a weekly ad deal to their list and know where to access it?
• Can a user do a local search on the weekly ad and understand the difference between local and global search?
• Can a user navigate to the print view of the weekly ad?

Each task is the scored on a 1-5 scale and averaged across the users with a 4 and above being considered a passing score. A 4 means the user completed the task easily but with hesitation or minor errors while a 5 mean they completed it immediately, easily, and with no errors.

Some takeaways we observed from this first test were that we needed to reinforce the label "Filters" next to the icon for 'All Filters'. We also observed a common theme that customers had trouble connecting the add-to-list icon to that piece of functionality.

For the second study, we asked bigger and more nuanced questions:

• What are your first impressions? What do you feel you can do on this page? What are your thoughts on the organization of this page?
• Sorting by discounts - how would you organize the page to display deals sorted by the best value?
• Pricing clarity - what do you notice about the item prices and the amount of savings? What about the pricing do you think you would find helpful when grocery shopping?

This test performed very well and reinforced a lot of the design decisions we made. One improvement led to us better calling out the "Sort by" button and adding color to indicate interactivity.

Overall, everything performed quite well and aside from a few minor tweaks, we felt confident in what we were building!

Measurement Plan

...and next steps

3400 long-winded words later and we arrive at where we're at now: prepping to run the pilot and enacting the measurement plan. As stated previously, the pilot will run with one of our grocery store brands, include 50 customers across a range of data science segments, and gather contextual moments over a 4 week period.

One of the biggest changes I haven't discussed yet has to deal with how our marketing teams input this '1st party data' and what ramifications the success of the pilot may have on those teams. This is part of the reason we chose to do a limited pilot - we will need to manually clean a lot of data based on the current way these teams publish the weekly circulars (imagine a mix of InDesign, XLS, and data entry). Our designs require a clearer product hierarchy and offers need to match the taxonomy of our product assortment. Therefore, a large portion of post-pilot analysis will be documenting all of the data complexities and potential changes to workflow needed to support this effort going full-time.

We are currently in the initial stages of defining the next diary study: participant criteria, research questions, and analysis deep dives. As we get closer to the pilot launch we will collaborate with our brand partners to understand what questions they seek to answer as well. We are also working now to draft our main KPIs and measures for success for this pilot.

Assuming all goes well, our plan is to digest the pilot diary study and share out to the brands. We will then schedule 1:1 workshops with each of the 5 brands of ADUSA to understand their end-to-end workflow, unique challenges, and business goals. We will then layer-in these goals with the sentiments observed from our diary study and make any pivots needed to the next-gen weekly ad designs. One final stage of usability testing to ensure we didn't disrupt any crucial flows and we will then be ready to a/b test. The a/b test will likely roll out brand by brand with a subset of customers seeing the new experience and the control group receiving the current production experience.

Our hopes are that by structuring the weekly ad in a clear and organized manner, showing simple pricing and value, and quietly personalizing the order of offers, customers will find better upfront value and use this as the key place to kickstart their weekly meal planning. Over time, we hope that print loyalists will take a look at the benefits of digital and learn to rely on the new functionality to help save time and money.

As part of standing up our DesignOps practice at ADUSA, I collaborated with other Sr. Managers to develop a "UX Playbook" of activities to help our designers think through which UX methodologies would define success for their project. Our UX designers at ADUSA are given a large amount of autonomy to intake a problem statement or customer pain point and ideate freely on what the best customer and business solution may be. By embedding this playbook in our process, we ensured consistency in process across our growing team.

The funnel of Evaluate, Execute, and Measure corresponds to our product development stages and maintains consistency across disciplines. When beginning a feature or a redesign, we must begin by asking the question: What information is needed to begin solutioning. This stage in the evaluation process relies heavily upon research, but creates a framework to reference throughout the design phase.

After spending time to fully understand the problem, ecosystem, and the customer intent, we move on to the execution phase and ask, “What steps are needed to bring this solution to life?”. Without a strong customer-centric foundation, everything here would fall flat, however, by building our design approach upon research and analytics, we can confidently point back to how our solution solves a customer need. By the end of this process, we have produced the typically assumed UX artifacts of high-fidelity designs, advanced prototypes, and stakeholder presentations.

Lastly, and most often overlooked by designers earlier in their careers, we develop a measurement plan defining, “How will success of this solution be measured?”. Product KPIs and UX KPIs are a venn diagram, but some things are mutually exclusive. Here, we consider another playbook of honing in on UX metrics for success, indicators such as time on task, feature utilization, back button usage, task success rate, etc. This phase often leads to a recurring loop back to a/b testing and further iteration.

By defining a UX playbook at ADUSA, we took some of the guesswork out of creating a successful feature launch and applied consistency to the design process across our organization.

We gave customers the ability to redeem their earned grocery dollar loyalty points directly in cart rather than having to interact with the rewards section.

Who are we doing this for?

Meet the customer where they are and reduce the friction of having to go out of their way to visit our Rewards section. Many customers were missing out on using these earned points and were simply letting them expire. By offering this feature in the cart, our goal was to increase the perceived value of the loyalty program and increase curiosity and engagement with our rewards section.

My Role

I led this initiative alongside a UX designer, content designer, product manager, and product analyst.

Introduction

Any feature like this begins with identifying a UX playbook of activities we feel we need to conduct in order for it to be successful. In this case, we began with a Lean UX canvas and empathy map based on a wide customer pain point: how can we stop customers from losing out on expiring loyalty points.

Our grocery store brands all have slightly different loyalty programs, but to simplify, customers earn points on every purchase that can be used for taking dollars off their basket (grocery dollars), used for limited time offers such as a free item, or donated to charity. These points are continually expiring on a rolling 30 day basis. As such, we found that many customers were letting their points expire (point breakage) without using them in any way whatsoever.

From a business perspective, grocery dollars are the best value to the grocery store brands as it typically leads to more items in the basket and a higher dollar spend. With this in mind, we conducted our Lean UX Canvas and Empathy Map to better understand what might be the best option for our customers.

Lean UX Canvas

The Lean UX Canvas is a handy tool that centers our focus on the “why” and shifts our thinking from outputs to outcomes. Despite often having a preconceived notion of what we might be building, it asks us to check our assumptions at the door and look at the problem from multiple angles.

Some takeaways that emerged from this activity were:

• A broader understanding of customer pain points
• That this is a small step forward for what needs to be a much larger initiative from the brands
• That we could have developed multiple hypotheses to serve the different user types
• Clear next steps to pursue via a quick usability study

Empathy Map

To complement some of the outcomes of the lean canvas, we looked through our voice of the customer program for feedback from our customers around the loyalty program. Through this we were able to:

• Better understand what drives customer behavior
• Uncover customer needs that they may not be aware of themselves
• Guide us in that we are delivering what’s best for the customer

By looking at customer feedback, some themes emerged that we need to teach our customers how to interact with the program (learn by doing); introduce less friction, but not make it frictionless; put the value front and center. Many customers expressed not taking the time to fully understand the program - but everyone can easily understand taking dollars off your basket total!

Business Case

Working directly with our Sr. Product Manager, I helped define our business case with supporting metrics to share to our external and brand stakeholders. Digging into the numbers showed us that only about 25% of customers are actively redeeming their grocery dollar points - a huge opportunity. Additionally, point breakage was approximately 40% across our three loyalty point-based brands (Giant Food, Stop & Shop, and The Giant Company). A secondary goal from this project was to reduce the number of customers redeeming their points for fuel, which is very costly to the brands. We felt that by allowing easier access to grocery dollars, it would sway some fuel customers.

Competitive Landscape

Conducting a broad landscape analysis across grocery and grocery adjacent loyalty programs revealed that redeeming your points within cart and checkout is a common retail pattern. We also noticed that it helps connect to the branding and value of each of these programs.

With cart being an area of high impact, we consider all the moving parts of what is currently in the cart as well as planned features that would be released soon. We worked closely with our cross-team partners that manage the cart and checkout experience to ensure that our feature would fit well into the customer experience.

Initial concepts

Thinking from a test-and-learn mindset with an MVP approach, our initial concepts asked us to consider two approaches: a single action vs. multiple. Did we want to overly simplify the interaction and just allow customers to redeem their maximum number of points for dollars off their basket? Or would we allow them a similar toggle to what they have within the rewards section where they can choose which denomination to redeem? As I stated earlier, we wanted to reduce friction, but not make it entirely frictionless as we want the customers to be aware of what they’re doing and also reflect upon the value of the program. We did not, however, want to detract in any way from this crucial ecommerce flow.

Usability Testing

We took both of these approaches into UserZoom for a quick usability study with some questions around that balance of flexibility vs. ease of action. Both designs scored very well, with the single button winning out slightly for ease while the multi-select had the edge for which one was preferred overall.

Further Exploration

We continued iterating on a few different rounds of ideas, from adding more prominent branding to the component, adding the main interaction and more description into a modal experience, to toying with several different interaction patterns for the component. Ultimately, we kept going back to wanting to simplify the approach from an MVP perspective. We felt that despite customers preferring the hypothetical multi-select, the data showed that most customers redeem their full point value. Since the points are always expiring on a rolling 30 day basis, most customers typically had between $2-$3 to use. With this in mind, we felt that our first MVP a/b test should focus on the single interaction.

A/B Testing

With everything in mind, we narrowed in on this final MVP solution where our control was what was in production(A), and we tested between two variants where the promo code is shown(B) or only in cart(C). Our thinking behind this was to further simplify the features shown to customers at the cart, further reducing friction and moments where the customer may bounce to go exploring for promo codes.

Things we measured during this a/b test were:

• Primary metric - Adoption Rate. Determine the percent of customers who redeem their points in cart when this feature is available.
• Secondary metric - Redemption Lift. To determine the impact that the in-cart redemption feature has on customer behavior
• Business metrics - average order value and promo code redemption. Would there be downstream impacts on KPIs?

Results

We saw a 38% increase in point redemption with this first round of testing for variant B as well as a 2% increase in average order value! Variant C noticed at 30% decline in promo redemptions though had no negative impact on conversion or average order value. Despite this, we felt the best experience for customers was to keep promo code available in the cart. This test was conducted with a single brand, so our next steps were to test with the remaining brands, extend tagging in GA for better event tracking, and roll this out to our customers full time.

Next Steps

Once this feature has baked for some time, we plan to continue to iterate on the interaction, add some more delight, and expand so that customers may also choose to donate to a charity at this step in the flow.

Our objective was to improve the ease of basket building within our browse aisles (categories) section for our grocery sites and apps.

Why focus on this?

The vast majority of our customers build their baskets through search (~65%) and past purchases (~25%). This is true for most returning customers that are habitually building their baskets week over week. However, for new customers or those returning customers looking to be inspired, a big remainder comes from within the browse aisles section. For new customers, BA represents around 15% of their basket. Additionally, this section is meant to encompass the entire taxonomy while also serving as landing pages and jumping off points for inspiration.

When we first launched PRISM, our white-label app to our brands, there was a big disconnect from the mobile to the desktop experience. Desktop exposed a left rail navigation where customers could see all of the subcategories (L2’s-L4’s) while also browsing products and L1 landing pages. The mobile experience, however, forced customers to dig down through the taxonomy to the deepest subcategory before ever seeing products. Imagine you want to browse for a new type of jam. As a mobile customer, you would have to tap Condiments and Sauces > Jams, Jelly and Fruit Spreads > Fruit Spreads > Raspberry Fruit Spread before ever seeing any products. This limited product discoverability by forcing customers to dig so deeply into the product hierarchy. What’s worse was that each of these taps loaded a new page.

Our goal was to show products sooner, reduce taps, and allow for easily navigating within a browse aisles category.

My Role

I led the team working alongside two design strategists, a UX researcher, and a product manager. I also put my dev hat on and helped prototype our high fidelity HTML / Vue.js prototype. In terms of overall process for this project, we went through various project stages:

• Research (landscape analysis, current state audit, gathering usage data, defining measures of success)
• Ideation (Information architecture, wireframes, low-fi designs)
• Feedback gathering (Internal, dev, and stakeholder reviews)
• Design Stage (High fidelity designs and an advanced HTML/CSS/JS prototype)
• Testing (Usability testing and designing the plan for a/b testing)

Competitive Landscape

This project began by taking a close look at our competitors and running a usability study between our site, instacart, and walmart grocery. We asked users to go through a series of tasks across all three sites gauging what they liked and disliked along the way.

Some things we observed across competitors were that many had quick access to other categories through exposed navigation or breadcrumbs. This allowed for inspiration through seeing related areas to explore. No competitors required their customers to dig as deep in the taxonomy as we did - with most showing products either immediately or on the second level. We often observed competitors using product carousels on top level landing pages to represent a peek into some of the more specific lower level categories. And lastly, we observed many competitors using imagery to reinforce the categories rather than long, descriptive titles.

In the unmoderated usability test, we tasked 15 participants with finding 3 specific products while not using search. We immediately noticed that users quickly and easily adopted Instacart’s side-scrolling pattern. Customers commented positively on having images paired with the category names so they could more quickly identify them. On our site, we noticed a usability issue where customers would simply go home each time they wanted to add a product rather than browse through the aisles. And lastly, we noticed that names truly matter when trying to find a specific product: something our merchandising team knows all too well.

Ideation

With some research under our belts, we set out to define some of the goals for the design process based on our current pain points and moments of delight we observed in our competitors.

• Improve the ability to navigate more freely
• Allow for more discoverability of products
• Show products earlier within the shopping experience
• Organize content in a digestible and logical manner
• Delight customers with beautiful design
• Don’t change the taxonomy or current structure
  

With these goals in mind, we developed our key metrics for success for this project. We knew that we wanted to decrease the amount of time it took for customers to find and add products to their cart while using browse aisles. We wanted to define a new pattern that followed more up-to-date trends that would allow for a more seamless navigation across top level and sub-level categories. A simple metric we wanted to gauge was the number of add-to-carts as customers would go to browse aisles, but often get discouraged and revert back to search. Lastly, we wanted to improve customer satisfaction in this area - hoping that improved product discoverability and better design patterns would lead to this.

After many rounds of rapid design iteration and critique, we came away with a design that drastically reduced clicks before a user saw products and greatly improved visibility into the product taxonomy. We also separated our search and sort/filter capabilities into its own section making it even easier for a customer to narrow down their selection.

Advanced Prototype

After ideating through various approaches, we landed on a solution that we felt addressed many of the customer pain points and UX metrics for success we had identified. However, since this redesign involved a new type of interaction, we decided that an advanced HTML prototype would be needed in order to move forward with usability testing. We focused on building a similar test to what we ran in the competitive landscape analysis and asked users to navigate Browse Aisles to explore the product hierarchy and find three different products. This prototype was hand coded by myself and one of my design strategists. We worked with our merchandising partners to gather new category background images across all the different top level aisles.

This prototype was optimized for mobile usability testing, so my iframe embedded here will work best on a mobile device or touchpad. For desktop viewing, visit the prototype and select “inspect” and then emulate your favorite viewport.

Usability Testing

With the advanced prototype built using real data and product images, we put this before another group of customers to gather qualitative feedback. We wanted to test:

• Do users understand the side scrolling ‘pills’? If not initially, how long does it take them to learn and adopt the pattern?
• Do users drill down to the third level of categories & filters? If not, do they still find the right products easily?
• How do users choose to move between categories? Do they use the browse aisle nav or the hamburger menu?

Where did we go from there?

We were happy to see that this test resonated quite well with users and gave us confidence going into designing the a/b test. For the a/b test, we chose to feature flag this and test with a single brand. We used Optimizely to roll the test out to 10% of the customer base and slowly increased over a two week period. For a large change like this, we knew there would be some negative comments in our voice of the customer platform, but we expected those to level off over time.

I would like to say it was all roses and unicorns, but we noticed some of our key metrics not meeting our expectations. Specifically, we saw issues with how customers were interacting with the “search within” feature as well as sort & filter alongside a dip in our overall metrics.

In addition to a/b testing follow-up refinement to this feature redesign, we implemented an SEQ (Single Ease Question) research intercept on the browse aisles section to gather real time qualitative feedback from customers. We wanted to understand how customers of different shopping intent (in-store, delivery, pickup) might differ in how they were using this feature.

We continued to iterate before rolling this out to all of our grocery brands. Two major changes we adapted were in how our ‘sticky header’ behaved and also the naming and iconography of our Categories search. Our theory was that too much of the viewport was being taken up by top level categories, sub level, and search/sort/filter. We iterated on a few different approaches and ran further a/b tests to hone in the right balance ultimately landing on a design pattern that works well for our customers and our business.

Alongside some more in depth case studies, I wanted to share a few a/b tests that were simple to implement, but had a huge impact on revenue. One such test was in tweaking the mobile product image size on our product tiles.

We had observed that many of our competitors were using drastically larger product images (Amazon being the most overt). We ran a simple test between control at 80px by 80px, Variant B at 90x90, and Variant C at 100x100.

What we saw from this incredibly minor adjustment was simply astonishing. This test was run across all of our brands for 30 days.

• Revenue per session went up 8.56%
• Basket size went up 3.65%
• Order conversion rates went up 7.23%
• Add to carts per session went up 6.46%

We were ecstatic with these results and all joked that we should just make the product images take over the entire viewport! Sadly, we ran a v2 of this test several months later increasing image size all the way up to 130px by 130px and noticed diminishing returns. This data was used to inform our new fully native app where product tile images sit at 112x112. Our team will continually look to optimize the image size of product tiles as we perpetually iterate on this crucial component.

I photographed, designed, and built my wife Meaghan's jewelry website - Artulia Jewelry. This site was built using Shopify and I developed a photography workflow that allowed us to photograph each piece and seamlessly publish between Shopify and Etsy. We redesigned her logo, redefined her typography style, and additionally created all new print assets including her business card, thank you cards, and polishing cloths.

Defining a new photography aesthetic

The photograph truly set the tone for this project. Meaghan knew she wanted to reflect the delicate nature of her work by complementing the pieces using natural materials. From a color perspective, we wanted the photos to be luminescent and feel larger than life. For on-person photography, we shot from the rooftop of our Chicago loft to give a metropolitan vibe with the city in the background.

The Finished Product

While developing the new photography, I dove into the world of Shopify. Having spent the better part of my career focused on the ecommerce experience across Vivid Seats and ADUSA, working with Shopify felt intuitive and natural. Additionally, I've built many other sites using WooCommerce, so Shopify came quite easily.

This entire project took roughly two months. We launched the site just ahead of Meaghan's holiday season and saw a massive increase in revenue for that first year. Shopify continues to be a consistent channel for Meaghan in addition to her primary storefront on Etsy.

In the Spring of 2017, I led the efforts to redesign and responsively develop one of the most impactful templates on the Vivid Seats mobile and desktop websites - the performer page. This template, shared across all performers in sports, concerts and theater was often a direct landing page for many users coming in from organic and paid search and therefore their first impression of Vivid Seats.

View a current example of this page for reference.

Project Goals

We wanted the visual design to shine through and for the user to feel connected to the performer they were searching for. The challenge was to capture the attention of the users and set expectations for where they were in the flow while improving conversion. The goal was to increase click-throughs and decrease bounce rates while presenting a clear hierarchy of information and getting the user excited.

User Research

I began by conducting a user study in usertesting.com with 10 participants asking them to perform various actions throughout the page and provide general impressions and feedback. In addition to this qualitative feedback, I worked with the web development team to add GA event triggers to many neglected aspects of the template so that we could create a thorough map of all interactions. Lastly, I interviewed several of our senior customer service representatives to ascertain any pain points discussed through phone sales that this step in the flow may have offered.

Pain Points

With this information, I was able to map out customer pain points with my team and begin affinity mapping and high-level brainstorming. At this point in the process, I would always invite the project manager and one or 2 devs from the product team to participate in this session as it would often shed light on new aspects we had not considered. It would also help ground our choices in technological considerations.

Design Solutions

This project involved several major stakeholders across engineering, product, and executives. After collaborating and receiving sign off on the wireframes, I worked with one of my UX designers guiding and critiquing dozens upon dozens of hi-res comps iterating through different approaches. Across several stakeholder presentations over multiple sprints, our final top three distinctive comps were then agreed upon to create a multivariate test.

Split Testing

Our test ran for two weeks and had three variants plus control. We began by slowing rolling out the template to a few sports teams at a time before eventually releasing it to all sports teams for MLB, NBA, and NHL (the current in-season sports). In addition to the multivariate test, I ran a second user study with 20 participants in a script created to avoid order bias. At the conclusion of the testing, I had quantitative and qualitative feedback to present to our stakeholders and make a firm case for releasing the winning variant live to production.

Conclusion

Our final results were significant. Conversion rate increased by over 3%, bounce rates for this page went down nearly 60%, and we reduced the number of front-end templates from 91 to 8. Visually, I feel that we significantly improved the impact of the page, helped to excite the customer about their potential purchase, and brought it inline with larger design efforts we were looking to make throughout the site. In the end, we iterated a few more minor tweaks, then quickly released to all performer pages. This project was the touchstone for an upcoming UI overhaul and was a constant point of reference for design components and a champion for how the UX process works.

The production page otherwise known as the ‘map’ page or where customers choose their tickets is the most crucial page of the Vivid Seats flow. I had worked on several refactors to this page over my years at Vivid Seats, but in May and June of 2018, myself and a lead engineer on the web development team set out to lay a new foundation for this page.

View a current example of this page for reference.

Project Goals

This new foundation would allow us to rapidly test new features and products across mobile and desktop. Previously, this was an adaptive template which served up a much smaller payload for mobile. The page was refactored to be fully component based and mobile first. The challenge was in the fact that the mobile and desktop versions of this page were quite dissimilar in information architecture. The goal was to unify both experiences while also improving several customer touchpoints and doing no harm to conversion.

Competitive Research

Researching our competitors was something we did quite regularly, but for this project, we wanted to fully understand the advantages and disadvantages of several different approaches. I developed a user test that had 10 participants go through several general activities across TicketMaster, StubHub, SeatGeek, and our website. This feedback helped to ground some of the larger information architecture changes we ultimately made to the page.

User Research

User research was of paramount importance as I had to separately test both mobile and desktop platforms to determine the customer delights and pain points. Additionally, with such a complicated page, keeping the customers focused on the areas we were changing proved an additional challenge. I began by creating a broad usability test of the page to gather data on likes and dislikes of the interface. In usertesting.com, I created a 20 question script that reached 10 qualified customers. With this information, we mapped out a matrix of mobile vs. desktop customer pain points. This gave important insights into what we knew had to stay as compared to what we had the opportunity to improve.

Wireframing

As I've mentioned, the information architecture was slightly different for the mobile and desktop views of this page. This presented a unique challenge in marrying the two together as a mobile-first responsive page. I focused on the modular components of the page, iterating through low-fidelity sketches, inviting my team to brainstorming sessions, and bouncing ideas quickly through Slack and InVision. These IA-heavy wireframes went through several rounds of revisions, presenting to our key stakeholders: the CEO and VP of Product. By comparing and marrying the mobile and desktop IA, we reached a solution that we felt confidently conveyed the critical information to users.

Design Solutions

The purpose of this project was not a complete UI overhaul, but it still held opportunities for making several QoL improvements to the page and bring components inline with our design guidelines. The largest design change came for desktop in which we added a key feature from the mobile version: the ‘ticket details’ view. This introduced an intermediary step that highlighted three key pieces of information: where their seats were located in the venue, all of the disclosures about the tickets, and a clear indication of quantity and price. It also gave us the opportunity to create language through motion. The motion of our animations helped to connect users to the page in a delightful manner.

Split Testing

This project required a layered approach to testing so that we could be certain that the technology had no impact on conversion. The first test simply pitted control vs. the refactored code styled to look identical to control. After that baked for a couple of weeks and we gathered enough stats to prove it was winning, we released the main test. This multivariate test introduced the fully responsive template that added ticket details to the desktop view. It also opened the door for dozens more impactful tests that would come later down the road.

User Testing Again

As we ran the multivariate test, I interviewed a group of 20 users comparing control and the ‘ticket details’ variant. The results were overwhelmingly positive with a clear indication that users greatly preferred our new variant. I like to end many of my users tests with several questions that ask users to rank on a scale of 5 how much they liked or disliked certain features. With this data, I created a graph that makes it very easy for stakeholders to quickly digest the trends.

Conclusion

The result was a page that unified the mobile and desktop experiences, allowed for rapidly creating new multivariate tests (which we later did to much success in the form of ticket scarcity, urgency, and promo offers), and improved customer confidence in their ticket selections. We reduced and simplified the front-end code substantially which made it significantly easier for future changes.