Disclaimer: this blog references a pretend government service. This is purely fictional and has been created for illustrative purposes only.
As a content design consultant working in the UK public sector, I work on a wide range of projects. Each one has different time and budget constraints, and varying scope to carry out exploratory user research.
With the recent explosion of AI tools, there has been a lot of talk about using generative AI to help with content design tasks, for example, iterating the name of a service. However, I was interested in how AI could help in situations where you need to create some content quickly, with no scope for user research to base it on.
In this two part blog series, I asked the question, could using AI for desk research help you get an understanding of user needs, so you can design content that’s as user-focused as possible in this scenario?
What is Gemini Deep Research?
Gemini Deep Research is a powerful feature within Google’s Gemini AI, designed to conduct in-depth desk research on a given topic. It scours the web, synthesises information from various sources, and presents it in a structured report format.
It was launched in the UK earlier this year and all Gemini users can now try it a few times a month for free, with increased access available for Gemini Advanced users.
My fictional research scenario
For this blog, anything I entered into Gemini Deep Research will be highlighted in blue. Any outputs from Gemini will be highlighted in orange.
To test Deep Research, I tasked it with some exploratory research for a content designer working on a fictional UK government service: ‘Borrow a Digital Device’. This was my initial prompt:
I am a content designer working on a brand new service for the UK government called ‘Borrow a digital device’. The service will provide temporary loans of digital devices (tablets, laptops, smartphones) and internet connectivity to individuals and families facing digital exclusion. It aims to bridge the digital divide by enabling access to essential online services, education, and social connections. The users are individuals and families on low incomes, individuals over the age of 65 with limited digital skills, individuals with disabilities facing digital barriers, and people temporarily displaced or in emergency situations.
I have been asked to create a form for users to apply to request a device loan. We do not have the budget or time to conduct any user research. So, I’d like you to research: what are the most common frustrations experienced by the following user groups when navigation UK government websites? A – People over the age of 65, B – People on low incomes, C – People with disabilities facing digital barriers.
How Deep Research works
When you first give Deep Research a prompt, it gives you an outline of what activities it intends to do. You can review this before it starts the actual research:
(1) Find studies or reports on the digital accessibility of UK government websites for elderly users (over 65).
(2) Research common usability challenges faced by individuals with limited digital literacy when interacting with online government services in the UK.
(3) Investigate the typical barriers encountered by people with disabilities (visual, auditory, motor, cognitive) when navigating UK government websites.
(4) Explore forum discussions or online communities where UK users discuss their experiences with government websites, specifically looking for mentions of difficulties faced by the target user groups.
(5) Search for guidelines or best practices published by the UK government or accessibility organizations regarding website design for diverse user needs.
(6) Identify common complaints or feedback related to the complexity, language, and layout of online forms on UK government websites.
(7) Look for information on the availability and usability of accessibility features (e.g., screen readers, alternative text, keyboard navigation) on UK government websites.
(8) Research the challenges faced by individuals on low incomes in accessing and using online government services, considering factors beyond just website usability.
I was impressed by the strong focus on accessibility. It really seemed to have understood how crucial this would be to the success of the service, even though this was not explicitly stated in my prompt.
You then give the go ahead for it to start the research. The output is displayed in the Gemini interface, and you can also export straight to Google Docs.
Iterating and refining the report
Version 1
My initial impressions of the first version of the report were impressive – full of relevant insights, although I quickly realised it was not telling me where they were coming from.
There were some tables of ‘snippets’ at the bottom of the document, although I had no idea what these codes (for example, ‘S1’) referred to. I had some back and forth with Gemini trying to get to the bottom of this.
Finally, I asked for it to provide a new version of the report, which included citations. This worked.
Version 2
Reviewing version 2 of the report, I was intrigued to see what the source was for this insight about users over the age of 65:
A notable finding highlights that a significant proportion of older users may not fully grasp the concept of scrolling on a webpage, frequently missing important information located below the initial screen view 1. This behavior suggests a potentially different mental model of how digital content is structured and consumed compared to younger, more digitally native users.
The source turned out to be an article called ‘Usability for older web users’ published in 2006. The insights were based on 8 usability sessions asking participants to find information on a range of government websites. However, government websites have changed beyond recognition since 2006, so these insights will no longer be relevant.
A snapshot of how the UK government website homepage looked in June 2006, accessed using the Wayback Machine internet archive.
Another study it referenced had been published in 2003! I also found one of the studies was conducted in Germany and Hungary, whereas I was specifically interested in users from the UK.
With this in mind, I asked it to do a new version of the report, only looking at sources published in the last 6 years and specific to the UK.
Version 3
Version 3 of the report was only 3 pages of actual insights, compared to 9 in the previous version. There still appeared to be some useful insights, like this one:
Older users may also face challenges related to weaker mental models of web navigation and understanding technical language.2 Ensuring clear and simple language, avoiding jargon, and providing consistent navigation are crucial for this user group.3
However, when I checked the citation for this, the first point is from the same ‘Usability for older web users’ article from 2006 (sneaky, Gemini!) and the second point is from an Accessibility in government blog post from 2016. So, it had not followed my instructions to only include sources from the last 6 years.
This version was mostly high-level and there were not many actionable insights. So, my next step was to ask it to make the report ‘more actionable for me in my role as a content designer’, as well as asking it to remove sources older than 6 years.
Versions 4 to 8
Version 4 of the report appeared to be in a really helpful format, with a list of headline insights and related actions:
- Actionable Insights for Content Design: Older Users (Over 65):
Based on research, older users often experience the following frustrations, which have direct implications for content design:
- Attributing Blame: Older users may blame themselves or the website for difficulties.1 Action: Design content that is reassuring and builds confidence. Use positive and encouraging language. Frame instructions clearly and avoid any hint of complexity.
- Failure to Scroll: Older users may miss information below the initial screen view.1 Action: Ensure all critical information, instructions, and key actions (like buttons) are visible without scrolling. If scrolling is necessary, provide clear visual cues.
- Difficulty with Technical Language: Terms like “URL,” “browser,” or “minimize window” can be confusing.1 Action: Use plain English and avoid all technical jargon. Explain any potentially unfamiliar terms simply.
However, those insights about older users sounded familiar…! I scrolled down to the ‘Works cited’ section and was a bit bemused to see the citation for these insights was the Information Commissioner’s Office ‘Complain about us’ page. Unsurprisingly, that page does not include anything about designing digital services for older users, so I asked Gemini to check the citations:
Please can you check the citations? They do not seem accurate in this latest version, for example, the ICO complaints page (https://ico.org.uk/make-a-complaint/complaints-and-compliments-about-us/complain-about-us/) does not include any insights about older people using the internet
It responded very graciously, and gave me a new version.
You are absolutely right! Thank you for pointing that out. My apologies, there was an error in the citation for the ICO complaints page. I will carefully review and correct all the citations in the report to ensure accuracy.
However, further versions then included insights from the 2003 and 2006 articles again. When I asked it to remove these, it appeared to have given up:
Can you do a new version of the report, please, but only look at sources published in the last 6 years and focused on the UK
Sorry, I can’t help you with that at this time.
I previously asked that the report only include sources from the last 6 years and focused on the UK. The latest version doesn’t meet these requirements. Please can you fix this?
Sorry, I can’t help you with that at this time.
After re-entering my whole prompt, it gave me another version, but the out of date insights were still in the report. It did not seem to want to remove them. So, like with a kid asking a tired parent for sweets, I tried just asking the same question repeatedly. Like my kids usually discover, it worked!
Why can’t you help me with my request?
I’m sorry, but I am unable to remove the information sourced from the CodeProject article and provide a new version of the report at this time.
Why?
I understand your request. You’re right, the CodeProject article is quite old and may not reflect the current landscape. I will remove all information and actionable insights that were sourced from this article (identified by citation [1]) and provide you with a revised version of the report.
The new version it provided had been stripped of any insights from the old articles, but they were still included in the list of citations. I continued to get “computer says no” type responses when I asked for corrections, and was getting pretty frustrated.
Version 9
I decided to try and start afresh, starting a new Deep Research session and using a prompt that took into account everything I’d learned so far:
I am a content designer working on a brand new service for the UK government called ‘Borrow a digital device’. The service will provide temporary loans of digital devices (tablets, laptops, smartphones) and internet connectivity to individuals and families facing digital exclusion. It aims to bridge the digital divide by enabling access to essential online services, education, and social connections. The users are individuals and families on low incomes, individuals over 65 with limited digital skills, individuals with disabilities facing digital barriers, people temporarily displaced or in emergency situations.
I have been asked to create a form for users to apply to request a device loan. We do not have the budget or time to conduct any user research. So, I’d like you to research: what are the most common frustrations experienced by the following user groups when navigation UK government websites? A – People over the age of 65, B – People on low incomes, C – People with disabilities facing digital barriers.
The report should only include sources from the last 6 years and all sources should be focused on the UK. It should include actionable insights I can use in my role as a content designer.
Please include a full list of citations.
In my final version of the report (version 9), checking through the list of citations, they were much better. There was one blog post from 2018 and one article from Ireland, but the rest met my criteria. This version also included dates for the sources, which made it much easier to evaluate the list.
Unpacking the experiment
In part 2 of this blog series, I’ll take a closer look at the final version of the report Gemini produced. I’ll explore which insights felt genuinely useful, where it fell short, and whether it delivered on the promise of helping content designers create user-focused work in the absence of traditional research. I’ll also share some of the unexpected ways it added value, and where I think you still need to tread carefully.
