We see it time and again. Critical policy messages that miss the mark.  The reason is simple: they were vetted with the wrong people.

If you are based in the Washington, DC area, whether you’re a corporation, association, think tank, or politician, here’s some free advice: Test Your Message Outside the Beltway.

We do policy message testing for a living.

➤ Please reach out- we’d love to help.

That’s what we’ve been telling our clients for nearly two decades. And even though they know that—of course they know that—clients are always surprised at what outside-the-Beltway testing turns up.

Here’s why it matters:

➤Average Americans Only Consume a “Trickle” of News

Most people outside the Washington metro area are not immersed in the details of your issues—sometimes they’ve never even heard of the issues. We’ve validated this again and again through our Swing Voter Project, which taken us to swing states every month for the last 18 month. Average Americans by and large get their news is small doses from LOCAL SOURCES. Yes, some watch cable news  but are almost as likely to get their news from Facebook.

➤Average Americans are Fixated on Themselves

This is not to imply that they don’t care about other people or the world as a whole. It simply means that while trying to pay their bills and care for their family, they don’t have the LUXURY of immersing themselves in DC news.

Some of them are working multiple jobs.
Some of them are a health crisis away from bankruptcy.
Some of them are working 18 hour days to keep the family farm afloat.
Some of them are caring for a chronically ill family member.

Their challenges are real and immediate.  

➤Validate That Your Policy Messages Are Connecting

And the best way to do that is to test your messages with them. Get in their heads. See their reactions. Hear their thoughts. Then refine based on what you’ve learned.

Again, our clients know all this. But they still—regularly—say things like, “I did not expect that one to tank!”; “I am really surprised none of the respondents have heard of that.”; “I didn’t realize our language was so full of jargon until this dial test.”

And we especially like: “Good thing we tested this!”

We do policy message testing for a living.

➤ Please reach out- we’d love to help.

——–

To demonstrate how different Washington, D.C. is from other parts of the country, here are some facts and comparisons:

—Bigger salaries: highest per capita income in the country ($75,569; the national average is $49,571)[1] with an annual growth rate of 4.5% (vs. 3.6% national average)[2]

—Educated: 55% of the population 25 years or older has at least a bachelor’s degree (29.3% nationally);[3] 31% have an advanced degree (11.2% nationally)[4]

—Expensive housing: median value of a house is $460,700 ($163,900 nationally)

—Lower rate of home ownership: 40.4% own their own homes (63.7% nationally)

—High crime: 6,408.6 crimes per 100,000 people (3,295.0 nationally)[5]

[1] http://www.courant.com/business/hc-per-capita-income-connecticut-20170328-story.html
[2] https://wtop.com/business-finance/2017/03/dc-per-capita-income-25-percent-higher-national-average/
[3] https://www.iowaeconomicdevelopment.com/50states
[4] https://factfinder.census.gov/faces/tableservices/jsf/pages/productview.xhtml?pid=ACS_15_5YR_S1501
[5] https://www.iowaeconomicdevelopment.com/50states

This group of swing voters think the country is in trouble but don’t lay the blame on President Trump. Read this month’s findings to see what they had to say.

Here’s a couple of interesting things this group told us:

President Trump still has their vote — at least 7 of the 9 of them. Only 2 in the group say they would vote for Biden over Trump if the election was held “tomorrow.”

They feel life between now and election day will be chaotic, divided and emotional.

They feel that the demand to “Defund the police” is going too far and could have a negative impact on Biden’s chances.

And they largely want schools to re-open in the fall

View the highlights for yourself: 

Couple of direct takes from this group:

On the pandemic and re-opening the country…

“I believe [scientific experts] should be part of the guidance, but… if we listen to nothing but scientific guidance, we’d be shutting down every business in the country.”

On social unrest and the ramping up of protests in some cities… 

“They (protestors) are still burning buildings. They’re rioting. They’re doing things that aren’t going to bring us together. They’re doing more things that are going to bring us apart.”

➜ Check out all the key findings in our July summary report

Should We Wait To Do Research “Until this is Over?”

We have been fielding these questions since March. We can now say definitively that yes, now is not only a good time, but a great time.

Back when this all started we did not know (nobody knew) how the coronavirus economy and lockdown would impact qualitative research. Will people fixate on the short-term? Will their mindset impede insights?

Back then, we were advising our clients to push the PAUSE button until things changed, stabilized or we got a better read. 

But here’s what we have found, and why we have so much confidence in the quality of online qual research being done right now:

5 Reasons Why Now is a Great Time to
Do Online Market Research

➤People are bored with the same-old quarantine-based conversations. They are very eager to participate in research right now, if for no other reason than to communicate with a different human being than their quarantine buddies. (It is not an over statement to say that many respondents are down right thrilled to have a chat!)

➤People are living thru a most unusual (arduous? painful? frightening?) time. Their emotions are raw, and their walls are down. This vulnerability means they are opening up much quicker than usual during online focus groups.

➤People have had a lot of downtime to dissect their angst. This new level of self-awareness is leading to super candid conversations in our online IDI’s.

➤People have had to forfeit so many of their simple pleasures: dining out, retail therapy, family events. This deprivation has given them a fresh take on what your products and services really mean to them. 

➤The silver lining to this slow-moving tsunami we are living through: people have never been more connected with the world around them. That leads to great insights.

Have a research project you’ve been thinking about doing? Now might be the right time!

Reach out and we can talk it through.

Concern is on the rise, especially as COVID-19 cases spike throughout the U.S. “Is it safe to go to (fill in name of business)? Are they following proper cleaning protocols?” (What ARE the proper cleaning protocols? A lack of federal mandates around reopening businesses has further complicated the issue.)

This means that, as a business owner, the onus is on you. 

➜Need help putting these messages into action?  

We can help.

Our study revealed that hearing from a company directly can greatly greatly impact a customer’s comfort level. In short- they need to hear from YOU.

But keep in mind you are talking to 2 distinctly different groups:

1. 1. 1. 1. 1. 1. 1. 1. Those who can’t wait to come back (ready-to-go)
                     2. Those who need some reassurances before they will spend at your establishment (assurance seekers)

                   

From our research with returning customers, we have created a messaging cheatsheet.

Here’s some guidance to help you home-in on the right Welcome Back message for your audience.

Assurance seekers want to see directive signs: what should you do after trying on a pair of shoes? Put them back on the rack? Give them to an associate to be properly cleaned first? They want to see that you are taking customer’s safety seriously and guiding them.

On the other hand, people deemed “ready-to-go” do not need as detailed messages. They want to hear that unlike many businesses, you are open again and further that you’ve got their back, all they have to do is enjoy.

At an unprecedented time like this, more communication is always better. Share your plans, procedures, and expectations upfront to address concerns. Surround your customers with “Welcome Back” messages that ease their mind. 

Every business is unique. We can help you fine-tune these messages for your audience. Reach out➜

*Photo by Artem Beliaikin on Unsplash – Image from Bali, Indonesia

Have you noticed consistent trends in each month’s SVP report? You’ll want to read this month’s findings—there’s been a shift. 

Trump’s support among these swing voters has fallen

• • • The country is in chaos, and he’s not bringing order, he’s bringing more conflict.
    • They’re still not crazy about Biden. But they’re more willing to give him a chance.
    • How are they feeling in general? No surprise here—anxious, depressed, frustrated, fearful.

• • • They’re horrified about George Floyd’s death, and they’re supportive of the initial protests; but they think the protests have made their point. And protest violence is causing more division. They want it to stop.
    • They oppose defunding the police, and if Biden supports those efforts, he’ll lose many of their votes.
    • They want a national conversation about race, but they don’t want politicians to lead it.

“Our entire year has been taken away from us,” they said, with the prohibitions on weddings, funerals, and other public gatherings. Meanwhile, thousands are allowed to crowd together for protests. They want consistent pandemic gathering rules.

➜ Like we said, you’re going to want to read the whole thing

Have you noticed consistent trends in each month’s SVP report? You’ll want to read this month’s findings—there’s been a shift. 

Trump’s support among these swing voters has fallen

• • • The country is in chaos, and he’s not bringing order, he’s bringing more conflict.
    • They’re still not crazy about Biden. But they’re more willing to give him a chance.
    • How are they feeling in general? No surprise here—anxious, depressed, frustrated, fearful.

• • • They’re horrified about George Floyd’s death, and they’re supportive of the initial protests; but they think the protests have made their point. And protest violence is causing more division. They want it to stop.
    • They oppose defunding the police, and if Biden supports those efforts, he’ll lose many of their votes.
    • They want a national conversation about race, but they don’t want politicians to lead it.

“Our entire year has been taken away from us,” they said, with the prohibitions on weddings, funerals, and other public gatherings. Meanwhile, thousands are allowed to crowd together for protests. They want consistent pandemic gathering rules.

➜ Like we said, you’re going to want to read the whole thing

Engagious/Focus Pointe Global Swing Voter Project turned to eastern Iowa for May’s research, meeting with swing voters in online focus groups.

︎ For the complete report, go here.   

︎ For video clips, go here.

What’s on their minds? Coronavirus, of course. They say it will be their number one voting issue in November. It’s on our minds, too; and we wanted to know how the pandemic is influencing their thoughts about the presidential race:

• Six of the eight are very likely to vote for Trump again.

• They don’t blame President Trump for the situation we’re in, and for the most part they think he’s doing a decent job in an unprecedented situation. At the same time they are less confident in his leadership, saying they think he’s delegating too much power to the governors.

• Unemployment is high, but they’re not going to hold it against the president. Unless the pandemic intensity recedes, and unemployment doesn’t.

• Trump brought us a great economy once, they said; he can do it again.

• Are they worse off than before COVID-19? Yes. Are they better off than four years ago? Yes. For now, at least, respondents seem to be mentally bracketing the COVID-19 economy.

• That said, they’re more worried about their financial health than their physical health.

• The two respondents who said they are likely to vote for Vice President Biden want him to bring a new outlook to COVID-19. Though the group as a whole is not at all familiar with what Biden plans to do about COVID-19,

• Not one of the respondents feels that Biden’s choice of running mate would affect their vote.

• The six likely to vote for Trump cited Biden negatives: behavior with women, outbursts on the campaign trail, dishonesty, his length of time in government, doubts about his mental acuity.

• Reopening restaurants, bars and theaters? Respondents support a local approach, with slight support for protestors who demand that their governors allow them to reopen their businesses (but please, wear a mask).

• They’re somewhat uncomfortable with Chinese ownership of Smithfield Foods, the world’s largest pork processor with packing plants in Iowa. (Iowa is America’s top pork producer).

*All Respondents voted for Obama in 2012 and Trump in 2016.

Last week, we conducted a virtual focus group made up of 10 swing voters all of whom live in the Canton, Ohio area, and voted for President Obama in 2012 and President Trump in 2016.

Key Takeaways from the Canton, Ohio swing voter group:

• Some advice for the president:  Take some cues from Governor DeWine. They’re happy with how the Ohio governor is handling the crisis, giving him high marks for taking the problem seriously early on, and being decisive, and apolitical. They’d like the president to follow suit.
• Some of the shine has faded but he’s still their pick: they think the president’s initial response to the coronavirus crisis has made it harder to contain the virus, but overall, they still approve of how he has handle the crisis, and plan to stick with him.
• Has their confidence in Trump taken a hit? The short answer is yes. Half of our group has lower confidence in the president as a leader now then they did three months ago.
• What do they worry about the most right now? It was even split between their financial health and their physical health.
• No rush to normalcy: At least half of the group said they’d be willing to wait until August or September to return to more normal activities, with some indicating that they’d need to have several conditions in place before they’d go back.
• WWBD (What would Biden do?) Most of the group were not familiar with the Democratic challenger’s comments on the coronavirus crisis or what he would do different than what Trump is doing.
• Pro or against protesting? Generally against those protesting the stay-at-home restrictions and business closures.
• And what about suspending immigration? Unanimously in support of the president suspending immigration into the US due to the coronavirus crisis.

Next month, we will NOT be heading to Davenport, Iowa, but we will virtually meet with a new set of swing voters who live there. Stay safe and well.

(This most recent group took place over a week ago- which feels like a lifetime ago with the new ‘social distancing guidelines. Be assured we did not hold this group after the current guidelines went into place. Nor would we.)

Engagious traveled to Edina, Minnesota, to meet with swing voters: ten respondents who voted for Obama in 2012 and Trump in 2016, and one respondent who voted for Romney in 2012 and Clinton in 2016.

Key Takeaways:

• Top worries: Cost of health care; immigration; coronavirus (and why doesn’t the president seem more concerned about it?).

• Trump or Biden? 8-3 Trump.

• Trump or Bernie? 8-3 Trump.

• Why Trump: Good economy; immigration; he’s doing a good job for the country—let’s stay the course.

• Why Biden: Time for a change. He would fix the lack of diplomacy in our country.

• Why Bernie: He’s been consistent with his positions, and he’s trustworthy. Medicare for all. He was robbed in 2016, and “deserves his chance.”

• This is Minnesota. What if Biden or Bernie put Senator Klobuchar on the ticket? It wouldn’t shift any of their votes.

• 2016 do-over: Trump over Hillary 10-1. (they would all vote the same as they did in 2016)

• Obama vs. Trump hypothetical: 7-4 Trump. (3 of the Trump voters would choose Obama over trump. The one Clinton voter would also pick Obama.)

• They’re happy about the economy, but a coronavirus-triggered recession could dampen their enthusiasm. Although they say they wouldn’t blame the president for it.

• Should President Trump pressure drug companies during the coronavirus crisis? It’s probably okay to lean on them to come up with a vaccine, respondents said, but they’d be wary of the government requiring that drug companies provide free or deeply discounted drugs to treat coronavirus—it would set a bad precedent.

• Should we get rid of Trump so we can get back to our normal political civility? Civil like when Vice President Aaron Burr killed former Treasury Secretary Alexander Hamilton? Respondents said this question sounds like a Democratic political strategy to beat Trump rather than an authentic goal.

Next month we will be talking to swing voters in Ohio.

Dial testing is a key part of Engagious’ research approach. And right now many people are asking “Can we do qual research online?”  Yes. We are in the midst of #coronavirus and yet we all still have research to execute and clients who need insights ASAP. Online research options are your friend. And we’ve got you covered. 

Specific questions? Get in touch and we’ll talk it through: hi@engagious.com

Our Dialsmith group has been in the dial testing game for a long time. As the developer of the Perception Analyzer dial testing tools and services for focus groups and surveys, we support global clients with in-person and online research every day. 

Specific questions? Get in touch and we’ll talk it through: hi@engagious.com

So, without further ado, you might be wondering…

What is Dial Testing?

Here’s the deal; if you’re trying to find out what people are really thinking, memory stinks. Asking someone what they thought, or how they felt, about something is bound to give you all kinds of less-than-reliable results. We’ve studied this and with the help of industry and academic experts have tons of resources for you.

So, how can dial testing help? Dial testing is how you overcome the negative impact of flawed memory and recall bias by capturing what people think in-the-moment. This is done two ways; with hand-held dials in focus groups and with an on-screen slider in surveys.

As participants watch your content (online or in person), such as ads, TV shows, presentations, etc., they use in-person dials or an online slider to give continuous, second-by-second feedback. This gives you real-time, gut reactions that you can then use to probe deeper and refine your content.

The second-by-second results are overlaid on top of the content being tested. Colored lines represent different audience segments. 

So, Dial Testing is Just for Media Testing, Right?

Oh, contraire, my friend. Dial testing is used for a heck of a lot more than that! Yes, what makes dial testing most unique is its ability to to capture moment-to-moment reactions through a dial or slider. But, those same devices can be used to ask all kinds of other questions.

For example, in a focus group, participants can use their dials to answer closed-ended questions. Some may be as simple as a gender question to help segment the moment-to-moment ratings, while others may be Likert scale or attitudinal scale questions to gauge feelings and opinions. Most any closed-ended question can be asked with the dials in a focus group.

 Dial testing results for Attitudinal Scale and Discrete Choice questions

This has the added bonus of eliminating groupthink by allowing each participant to answer privately and individually before opening things up to group discussion. You’re paying every respondent, so why not hear from each of them for every question?

How Do Respondents Answer Questions with the Dials?

Good question! The dials use a 0 – 100 numeric range, but can also use any range in-between. So, for instance, for a gender question, respondents would dial 1 for Female and 2 for Male. For a 5-point Likert scale, respondents would dial between 1-5 for the range of Strongly Disagree-Strongly Agree. For a 5-, or 7-, or 10-point attitudinal scale, respondents would dial to the appropriate number based on the scale labels.

For every question, the moderator, as well as colleagues and clients in the viewing room, can see the results charted in real-time to help drive group discussion. Any of those questions can also be used to segment the results of moment-to-moment ratings so you can see how different subgroups reacted to the content.

How Does Dial Testing Work Online?

Online dial testing is very similar to in-person dial testing except respondents use an on-screen slider instead of an in-person Perception Analyzer dial. Video or audio content gets embedded into a survey and respondents adjust the slider continuously as they watch. There is also an optional hotkey that respondents can click to indicate they would take a particular action.

For example, during a TV show they may click a “tune-out” button if at some point during the show they would stop watching if they could. That moment is recorded and the respondent continues rating the show to the end.

Overview of our online dial testing tools and moment-to-moment reporting. 

Want to Learn More About Dial Testing?

Here’s a short video from our in-house Dialsmith group:

And here’s some additional recommended reading:

Dial Testing Group vs Focus Group: How Are They Different?

10 Ways Dial Testing Will Improve Your Research

Moment-to-Moment Research Playbook

Hope that helps. If you have follow-up questions or want to discuss how dial testing can help with your research, please feel free to hit us up.