This guide explains how to interpret the Personalization Report and understand where personalization creates value in your site experience.
Unlike traditional performance reports, personalization metrics require a slightly different way of thinking. This article helps you read the data correctly and set the right expectations.
What personalization is designed to do
In-session personalization is designed to improve relevance early in the customer journey.
It works by adapting product lists based on what a shopper has recently shown interest in during their current session.
The goal is not to always increase conversion directly, but to:
- Help users discover relevant products faster
- Reduce the need for repeated searches or filtering
- Support decision-making when intent is still forming
Because of this, personalization has different levels of impact depending on the type of journey.
Two types of customer journeys
To understand the report, it is important to distinguish between two common types of behavior:
Exploratory journeys
These occur when users:
- Browse categories or landing pages
- Compare multiple products
- Refine their preferences over time
In these journeys:
- Intent is not fully defined
- Users evaluate different options
- Personalization can significantly influence what they see and choose
This is where personalization typically creates the most value.
High-intent journeys
These occur when users:
- Search for a specific product
- Already know what they want
- Move quickly toward purchase
In these journeys:
- Intent is already clear
- Product discovery is limited
- Personalization has less opportunity to influence outcomes
This is expected behavior, not a limitation of the system.
Why the report uses “Contribution”
The Personalization Report uses Contribution metrics instead of comparing personalized vs non-personalized traffic.
For example:
- Clicks Contribution
- Add to Cart Contribution
- Purchases Contribution
These metrics answer the question:
“How much of the total outcome is driven by personalized ranking?”
This approach is important because:
- Personalization is part of the ranking system, not a separate variant
- It is continuously applied where relevant
- It should be evaluated as part of the overall experience
This avoids treating personalization like an A/B test and instead reflects how it operates in production.
Why Add to Cart is a key metric
Add to Cart is one of the most important signals for evaluating personalization.
This is because:
- It reflects active product consideration
- It captures behavior in exploratory journeys
- It happens earlier and more frequently than purchases
In many cases, users:
- Add multiple products
- Compare options
- Continue browsing before purchasing
This makes Add to Cart a strong indicator of whether personalization is helping users move forward in their journey.
By contrast, purchases:
- Often happen quickly in high-intent scenarios
- May not reflect the influence of personalization
Understanding Scale vs Impact
A key part of interpreting the report is understanding the relationship between:
- Scale – how often personalization is applied
- Contribution – how much it influences outcomes
These two dimensions should always be considered together.
Common patterns
High scale + high contribution
→ Personalization is widely used and performing well
→ This is the ideal state
High scale + low contribution
→ Personalization is applied often but has limited impact
→ May require tuning or better signals
Low scale + high contribution
→ Personalization is effective but underutilized
→ Opportunity to expand usage
Low scale + low contribution
→ Personalization is not yet creating meaningful value
→ Investigate configuration or data quality
Navigation vs Search: what to expect
Personalization behaves differently across surfaces.
Navigation (categories and landing pages)
- Typically more exploratory
- Users browse and compare products
- Personalization often has a stronger impact
Search
- Often more intent-driven
- Users may look for specific products
- Personalization impact is usually lower but still meaningful
This difference is expected and reflects natural user behavior.
When personalization is working well
You can recognize strong performance when:
- Add to Cart Contribution is high
- Navigation shows strong engagement
- Scale and contribution are balanced
- Users move smoothly through the funnel
This indicates that personalization is helping users:
- Discover relevant products
- Evaluate options
- Progress toward purchase
When to investigate further
You may want to investigate if you see:
- Low personalization usage (low scale)
- High usage but low contribution
- Large differences between Navigation and Search
- Weak Add to Cart contribution
These patterns can indicate:
- Limited or noisy session signals
- Suboptimal configuration
- Misalignment between product data and user behavior
What personalization does not do
It is equally important to understand the boundaries of personalization.
Personalization does not:
- Optimize exact-match or highly specific searches
- Override strong user intent
- Replace merchandising strategies
Instead, it complements your existing setup by improving relevance when:
- User intent is unclear
- Product discovery is needed
Best practice
Personalization is most effective when it supports exploration and decision-making, not just final conversion.
When interpreting the report:
- Focus on Add to Cart and engagement signals
- Evaluate both scale and contribution
- Compare Navigation and Search thoughtfully
By understanding these patterns, you can better evaluate performance and identify where personalization creates the most value in your site.
Article last updated