Evaluating content on a content-heavy site

I’ve got a particularly content-heavy site for doctors and I need to work out what to keep and what to lose.

I have personas and experience maps to help but, not being a subject matter expert myself, I’m looking to propose some research with doctors to better understand the content priorities section-by-section. The only problem is that the content takes a while to read so I’m not sure if face-to-face research sessions are suitable.

How would you structure the research? I’d love to hear any recommendations on methodologies or exercises that will enable good feedback on the content. Thanks.

I’ve worked on similar

You can’t beat a bit of qualitative research with doctors, face to face. Ignore the existing website and find out what they’re doing day to day when they need to access the website. This will give you a starting point and help you at least prioritise what should be on the website.

I’m guessing that doctors need to know lots and lots but what they need to know depends on what they’re doing at any particular time.

What type of content is it?

How did you create the personas and experience maps?

We think we’ve got their goals and content needs sorted and that’s represented in the personas and experience maps I’ve been given.

As we’re a bit further along the process as we just need to validate content that’s included in a site prototype we’ve put together. As this is the case, I’m not quite sure how to make best use of a face-to-face session. I have visions of saying “please spend 15 minutes reading this page and give me your thoughts”, which isn’t something I’ve done before in research. I’ve generally done it with shorter content.

It’s clinical content so, from my perspective, it’s difficult to understand if the language is at the right level of complexity. There are technical terms, but doctors know them so I can’t simplify it as I would with a site for the public.

I didn’t create them, but the experience maps are basically structured around the patient’s journey. Each stage in the journey has got the doctors’ tasks and information needs at each stage, which we’ve included in the site prototype. It’s essentially an information resource.

So you’ve got a large database of information that doctors may or may not need.

Has the existing site got any analytics set up? this can really help

That’s right, yeah.
Thanks. We’ve got analytics on the current site. Suspect some of the page view numbers are slightly skewed by lack of findability. Fun fun fun

Wait, doctors’ needs in the patients’ journey? Not doctors’ needs in the doctors’s journey? That’s an interesting approach. How do you know they match up?

And how were the personas developed? Are they patient personas or doctor personas?

It’s unlikely site analytics will help much at all. Unless you can match them to the journeys and personas you have.

As for “validating content”, have you’ve spent any time studying how they use content today? Do you know what these doctors look for in the content?

We’ve done a lot of testing of doctors and medical information. Watching them read the content is common protocol. (Though, it’s unlikely you’ll see a doctor spend 15 minutes on a single document, except certain types of training docs for CME credits. They are very much skimmers and cruise through medical journal articles extremely fast.)

One technique is to print the documents out and hand them a pen and a highlighter. Ask them to highlight anything that’s important to the scenario. Have them write in the margin any questions that occur as they are reading. Tell them to strike any content that is unnecessary and circle any content they think is questionable or unclear.

After they’ve marked up the piece, as them to walk through it with you and explain their notations.

I agree with jmspool. Testing it out with print forms is a good and fast way to determine what is working and what is important. All information needs to be prioritised. Rating it against a point of scale of 10 is a good way to start sorting out the most relevant. Then the next would be to match those with the high priority users and further with priority scenarios. Trying to answer - what are the worst pain points, may be.

Thanks Jared and Render. Very helpful.

Yeah, the personas and experience maps have captured information needs so I’ve used those to inform the prototype. I get your point about the doctors’ needs being mapped to the patient journey but it’s been enough to give me a (hopefully respectable) prototype.

No CME content just yet so the printed approach sounds like it could work! Great suggestion. I’d love to hear about anything else along those lines that’s worked for people in the past.

(We have patient personas and doctor personas, both focused around patient+doctor touchpoints.) There’s other information needs for doctors outside of these touchpoints that I’ve noted down separately.

Is your expectation that doctors will read this content during their time with the patient? (I’m not sure I understand how patient+doctor touchpoints help here.)

Also, that you didn’t collect the data that went into the personas or journey maps makes it all a bit suspect. At best, you’re a victim of the game of telephone. At worst, you’re going completely down the wrong path.

Not really expecting them to read this content in consultations.

Apologies, I haven’t given enough context about my approach really:
The documents I’ve inherited (personas, maps) are focused around how the doctors can support patients. The opportunity for doctors to support patients is at the touchpoints so there’s an opportunity to add content to help with this. The personas and maps are pretty useful for this.

However, there is existing content from a previous version of the site, which is the content I need to evaluate and edit where necessary. It’s not directly relevant to the patient interactions, so I’ve only got limited knowledge of where and when it’s relevant to doctors. With a deadline looming, I’m organising this round of research with doctors to minimise the risk of giving the wrong information or pitching it at the wrong level.

It’s clear that the scenarios in which doctors will consume this content are different so I need to be careful and think outside of the context of the personas and maps to get the complete picture, but do so in (probably) only one more round of research.

Hope that gives a bit of background :smile:

And does your site have a search on it? Can you look at that data too? See what folks are looking for?

Maybe do a usability interview with the users of the previous version of the site. Ask them some pretty deliberate questions as they’re moving through the site. Maybe - What content do you find yourself looking for the most? Why? Something which gets them moving to content and talking about it. This will no doubt provide deeper qualitative insights.

Is there a way for users to “bookmark” content from the previous version? You could analyze which content was most bookmarked, which would give you a sense of which was most useful.

Also, you could survey as many users from the previous site as possible. Maybe something like: “What 3 pieces of content do you find most useful from the previous version of the website?” “Why?”

Finally, if the Information Architecture of the previous version of the site lends itself to some sort of arrangement of content into categories, maybe you could synthesize the reams of content into main content categories. Then you could do something of a card sort with the doctors to put the main categories into groups. Then those groupings could be ranked by priority from “Most helpful” to “Least helpful” in the context of when the doctor is engaging with a patient. Then you’d know at least that certain categories of content would be best brought into the new website.

At the end of the day, you will always be able to go back and further analyze the content from the previous version of the website. Just make sure you don’t lose that content. Then, over time, as you migrate users over to the new version, keep your ear to the ground. If you build a mechanism into the design of the new site which allows your users to provide you with immediate feedback when the content they’re looking for isn’t available, then you can tack more quickly and add the content that they want.

It’s the difference between migrating everything to the new site in a “flip the switch” method verses something more gradual.

Of course, most all of this depends on how technologically robust the back-end architecture of the sites are, in particular the search function.

The new solution is going to have problems. It won’t be perfect. But, as long as it’s better than the old solution, you’ve hit designer gold, my friend.

Kudos to you for taking on this project from (what appears to be) someone else’s initial work. A mid-project handoff is never fun, but you seem to be on the right track.

I hope this helps! Keep us posted!

Thanks very much. Already had a good look at analytics and also launched a survey so I’m glad that was suggested. The next step is learning whether the tone of voice, level of language and depth of content is right.

Thanks for the suggestions ^^