Good design is as much about evidence as it is about creativity. Every decision, from layout choices to content placement, has the potential to influence how users engage, understand and act. But when those decisions rely on opinion rather than evidence, even small changes can create uncertainty.
In practice, that uncertainty often surfaces in design debates, moments where creativity meets constraint. One person may see opportunity, whilst another sees risk, creating friction and slowing the team’s search for clarity. That tension is familiar across projects, especially when user research cycles are limited or feedback is fragmented. The question then becomes how can design teams keep moving forward whilst making informed choices when the data is not yet clear?
The challenge: Making informed design decisions
In UX design, even small content changes can spark debate. A new element might promise value but also raise questions about accessibility, usability or user impact. When continuous user research isn’t always possible, these discussions can stall progress, especially when strong opinions differ.
That was the challenge facing one of our design consultancy teams. Their goal was to improve engagement with articles published by a large content platform. The proposed idea, a new informational element showing estimated reading time, showed promise, but had a mixed reaction internally, and from the audience. Rather than relying on assumptions, the team decided to find out what really worked.
Our approach: Experimentation and iteration
Experimentation on projects is exciting. It gives space to validate various design decisions with real data. In this case, the team has been able to test and explore outcomes with real users to make confident, user-centred decisions. An in-built, rigorous measurement approach has bolstered any debate with data and allows the team to act quickly on evidence.
Working closely with their editorial colleagues, our designers adopted an iterative, exploratory approach. Instead of committing to one design, they created multiple variations to understand which would genuinely improve engagement.
Design and build: The team developed nine visual and positional variations of the new element. These ranged from small styling and layout tweaks to differences in copy. For example, showing a numerical read time versus a qualitative label such as ‘Quick Read’ or ‘Long Read’.
User research: Competitor analysis uncovered useful insights into common practices around showing (or not showing) estimated reading time. Also learning from previous user research conducted in similar domains helped drive design ideas and decisions.
Testing infrastructure: A two-week A/B test was launched across several services on all article pages. This setup allowed the team to collect enough data to reach statistically reliable results quickly.
Data analysis: Our data specialists focused on one clear metric – whether the presence and placement of the reading time element encouraged readers to scroll further through each article.
This approach was deliberately designed to test hypotheses and uncover insights early.
What the data showed
By testing directly with live users, the team could measure how real-world readers interacted with the changes – not just what they said they might prefer. The experiment delivered a mix of expected and surprising outcomes, each one valuable in guiding the next stage of design.
Positive impact for new users: For first-time or less-engaged visitors, one variation, where the element sat directly beneath the article’s timestamp, produced a measurable increase in engagement. This showed that the new information acted as a helpful signal, encouraging people to continue reading.
Neutral effect for regular readers: For loyal visitors, the change made little difference to engagement. This confirmed that the addition was valuable for new users without disrupting the experience for returning ones.
Accessibility insights: Some designs performed less effectively, highlighting the importance of inclusive design. For instance, purely numerical read times may create challenges for users with dyslexia or cognitive differences, reinforcing the need for accessible communication choices.
These insights provided immediate clarity. What might have taken weeks of discussion was resolved in days, with clear data showing which designs worked best, and which didn’t.
From experimentation to continuous improvement
The experiment defined the copy, styling and positioning that most positively influenced user behaviour. But its real value lay in how it shaped the team’s mindset.
By embracing iteration, the designers could make evidence-based decisions, scale successful versions to more services and identify the next areas to test, such as how read-time indicators affect onward journeys from a homepage.
Designing for experimentation also encouraged creativity and collaboration. Knowing that changes could be tested and reversed quickly gave the team confidence to take measured risks, learn from the results and improve continuously.
Key takeaways
Let data guide your direction: When access to qualitative feedback is limited, A/B testing provides the clarity needed to move forward.
Embrace an experimental mindset: Testing in live environments allows teams to validate ideas quickly and roll back underperforming versions as soon as insights are gathered to prevent degradation of the overall experience long term.
Iterate with purpose: Each experiment should inform the next. Incremental learning builds lasting improvement.
Design for testing, not perfection: Treating design as an experiment encourages creativity, alignment and accountability across teams.
It’s important to acknowledge too, that this is one approach with a focus on the experimental, and this approach can’t be used in all cases. While an experimental, data-driven approach offers speed, clarity and continuous learning, it isn’t universally applicable. Some contexts require greater control, predictability, or ethical consideration than live experimentation allows. For instance, testing in production may pose risks in cases where privacy or safety are at the forefront.
Turning insight into action
This project shows how data-driven experimentation can transform uncertainty into confidence. By testing ideas with real users and acting on the evidence, Hippo’s designers were able to deliver measurable improvements in engagement while building a culture of creative and informed decision-making.
A/B testing isn’t just about optimising interfaces, it’s a mindset of curiosity and learning that empowers teams to learn faster, design smarter and create digital experiences that truly work for everyone.
The effectiveness is, like with anything, based on its application. Having an understanding of when other methods are better suited is crucial, but by combining data-driven testing with sound judgment and a clear sense of purpose, teams can balance innovation with responsibility and ensure that every decision contributes meaningfully to the user experience.
