Messaging Research

Jun 29, 20213 min read

This is a process I developed for testing clarity and relevance on research-based messaging copy.

About the product

My Perfect Cover Letter helps users create a cover letter without having to write one from scratch.

Content is personalized based on their target job and experience
We give step-by-step prompts so they can customize and make edits
They can browse templates and set a tone for their letter

The process in four phases

Discover—review existing value propositions and gather insights from existing research.

Define—synthesize and brainstorm proposals for new messaging copy.

Develop—create a testing plan and validate clarity and relevance.

Deliver —present findings and recommendations.

Discover Phase

Reviewing the existing content

The existing messaging was not tested
There's ambiguity on product feature comprehension
Lack of supporting research and data

Gathering insights from existing research

According to a recent research, here's what I discovered about job seekers and cover letters:

Define Phase

Synthesizing and brainstorming proposals for new value propositions with stakeholders.

Through brainstorming, we created a new research-based variant

Develop Phase

Creating a test plan to validate the value propositions (VP) derived from the research.

Methodology:

Create 2 variants of the SEM landing page (current and new messaging)
10 participants, 5 participants for each variant
Unmoderated sessions via usertesting.com
Users are presented with a scenario of needing a cover letter
Part 1: Present the full landing page for a 15-second test
Part 2: Ask questions about the previous page
Part 3: Present the copy and ask them what they think about it

In detail:

Part 1: Show the new value propositions in context of the SEM landing page

Scenario: A friend recommended this website to you. This website can help you create a cover letter. Imagine that you’ll need one to start applying for a job.

[Go to landing page with VPs]

Script: Without clicking anything, take 15 seconds to explore the contents of this page of the website.

Script: Go to [exit page link] when you're done.

Part 2: Questions about the full landing page

Script: Here are two questions about the previous page. Read each question out loud and share your answers.

What did you recall from the previous page?
What stood out or spoke to you?

Script: Now, let’s go to the next page [VP page only] when you're ready.

Part 3: Questions about the offerings

Script: We’ve shown this to you before but take some time to look at them and answer these questions when you’re ready. Read each question out loud then share your answers.

How interested are you in these 5 offerings? Rate them as a whole and share your reason. Rate your interest from 1-7: 1 not interested at all to 7 very interested.
Out of the 5 offerings being shown here, pick 3 that are most appealing to you, and tell us why. Start with the most appealing, followed by 2nd, and 3rd.
Is there something here that seems unclear, confusing or irrelevant to you? Share your thoughts.
In your own words, explain what each of the 5 offerings means to you. You can start from the left most offer.
Would you want to add something that you feel is missing here? Tell us about it.
Last question, is there anything else you’d like to tell us that’s related to this website, product, or your experience just now?

Participants for variant 1

Participants for variant 2

Deliver Phase

Synthesize findings and create recommendations.

Summary of Findings:

Users are most interested in an offer that they don't fully understand
The new variant scored 4.0/5.0 in comprehension and readability
The baseline variant scored 1.8/5.0 in comprehension and readability
5/5 participants ranked saving time the highest in relevance
The new variant has a higher overall score for clarity and relevance

Recommendation

Further Validation

The new copy was tested on the SEM landing page to measure its impact. While the initial test was inconclusive, further tests are being conducted to find an touchpoint where the new messaging can create an impact in conversion. (This metric is beyond the scope of this research.)