That’s what we’ve been telling our clients for nearly two decades. And even though they know that—of course they know that—they are almost always surprised at what outside-the-Beltway testing turns up.

Getting close to their issues is our clients’ job. They’re policy experts. They need to know every in, out, and upside down. That’s what they and their colleagues think, read, write, and talk about. It’s the air they breathe.

Most people outside the Washington metro area are not immersed in the details of these issues—sometimes they’ve never even heard of the issues.

It’s also important to recognize that life experiences outside the Beltway can be very different, so the concerns can be different: employment and unemployment rates, income levels, blue collar or white collar, religion (a lot or none), political attitudes, and attitudes toward politics. Communities vary from densely urban to intensely agricultural, with their own neighborhood values. What works in DC might not work in Chicago and Seattle, and probably won’t work in Worcester, Mingo Junction, and Tupelo.

But if they’re your audience, you need to verify that you’re speaking their language.

And the best way to do that is to test it with them. See their reactions. Hear their thoughts.

Again, our clients know all this. But they still—regularly—say things like, “I did not expect that to tank”; “I am really surprised none of the respondents have heard of that”; “I didn’t realize our language was so full of jargon until this dial test.”

And we especially like: “Good thing we tested this!”

Some Washington, D.C., facts and comparisons:

—Bigger salaries: highest per capita income in the country ($75,569; the national average is $49,571)^[1] with an annual growth rate of 4.5% (vs. 3.6% national average)^[2]

—Educated: 55% of the population 25 years or older has at least a bachelor’s degree (29.3% nationally);^[3] 31% have an advanced degree (11.2% nationally)^[4]

—Expensive housing: median value of a house is $460,700 ($163,900 nationally)

—Lower rate of home ownership: 40.4% own their own homes (63.7% nationally)

—High crime: 6,408.6 crimes per 100,000 people (3,295.0 nationally)^[5]

^[1] http://www.courant.com/business/hc-per-capita-income-connecticut-20170328-story.html
^[2] https://wtop.com/business-finance/2017/03/dc-per-capita-income-25-percent-higher-national-average/
^[3] https://www.iowaeconomicdevelopment.com/50states
^[4] https://factfinder.census.gov/faces/tableservices/jsf/pages/productview.xhtml?pid=ACS_15_5YR_S1501
^[5] https://www.iowaeconomicdevelopment.com/50states

Some Time to Reflect

Milestones are made to be celebrated and our 1^st anniversary as Engagious is no exception.

When the legacy companies of Dialsmith and Presentation Testing came together to bring new and enhanced value to the industry we knew it was the right thing to do, by the right people, at the right time. The proof would be in the pudding and oh what delicious pudding we made! Merging two companies is not for the faint of heart and we learned a ton along the way.

We also had a lot of fun and are proud of the new business we’re creating. After achieving our goal of making 1+1=3 in year-one, we are excited for what the future will hold for our company, team, and wonderful clients.

The Story audience is an eclectic bunch and I knew that in order to earn my time on that stage I’d have to deliver a lot of value and a few surprises. With that in mind I got to work crafting a talk and a fun, interactive experiment.

The talk was based on how principles from the field of behavioral science can be used to craft more effective and persuasive stories. I told two versions of a compelling story, the second of which was deliberately written to capitalize on known cognitive biases to elicit a certain reaction. I then pulled back the curtain to show the audience how it was done and gave them some tools to use for themselves. The experiment looked at how powerful “framing” is when communicating a message. Participants listened to two versions of a story, then answered questions about it. Again, one version was deliberately written to elicit a specific reaction and it most certainly did.

Feedback from the audience was quite positive, with many telling me that they appreciated the tangible and practical takeaways they could deploy in their work. My biggest takeaway from this experience is to know your audience. There are many directions I could have taken and many were more self-serving. But when on that stage, I was in service to the audience, not myself. Through that lens I was able to craft a great experience for them and for me.

By the way, if you want to hear more about the talk or the outcome of the experiment, please let me know. I’d love to discuss it or share it with your audience.

The focus groups are the result of careful planning

The scripts that we write, the video we show, and the discussions we lead are the meat of the session—the submission of your message for the respondents’ judgment. We work closely with you in preparing every element, to maximize the time in the sessions and get you the information you need.

The discussion guide is our road map

To loosen up the room, we try to make a personal connection. So we start with introductions. Once respondents introduce themselves, we have them answer a series of questions using their dials—demographic questions, and general questions concerning the issue at hand, to record what attitudes they’ve brought to the session, or the level of support for your issue that they already have.

Then it’s time for the dial test. We show the video, usually a point-counterpoint debate performed by two actresses, one presenting your message, the other the best arguments against it. Respondents watch, listen, and register their agreement or disagreement at each moment with their dials, using a zero to 100 scale.

This gives us far more actionable information than simply asking questions in a traditional focus group: we can quantify how each person in a group is reacting to every piece of your message.

How you participate

During the groups you’ll be in the control room, observing; as you watch the agree/disagree lines spike and dip in real time, you can make note of things you want us to ask during the discussion. We’ll be doing the same.

After the video debate, we ask more dial questions, including whose point of view they sided with more. We may ask some of the pre-video dial questions to see if your arguments (or your opponents’) changed their opinions at all.

Once we know what works, and what doesn’t, we can probe more deeply with respondents. We replay clips with significant spikes or dips, and discuss why they reacted the way they did, and explore ways that would work better.

Afterward we’ll analyze the results, edit the scripts as necessary, and then (usually) do another round of testing.

In the end, we will land on a set of best practices for you to follow. We pare it all down for you into the ‘What to say ; What not to say’ matrix. It is a favorite of our clients because it is so simple to use and weigh messages against. Check out the article on the ‘What to say ; What not to say’ Matrix.

Feel free to reach out if you want to brainstorm any upcoming projects.

We have two new faces in the NYC office Morgan and Madison who are interning for the Summer, so I wanted to get the skinny straight from them and give you a chance to get to know them:

MORGAN
I’m spending my summer at Engagious because I’m a huge psychology nerd and want to learn more about the business/market side of psychology. The team here is so awesome, and I’m hoping that they can help me learn even more than I already have in the short time I’ve been here.

Fun facts about me: I’m very into the arts. I love to draw and paint when I have free time, and one of my favorite things to do is go to art shows and museums. I also love to read and am OBSESSED with Dan Brown and his books. I love music as well, my favorite genre being hip-hop. Documentaries are also on my list of favorites – I’ve probably seen every single one on Netflix, but 13TH is my favorite.

One of my favorite accounts to follow on social media is a “Dogspotting” group on Facebook, where people basically just post pictures and videos of cute dogs they see…so that pretty much sums me up.

MADISON
I actually knew nothing about market research until last year when I got an interview with Presentation Testing and started to prepare for it. I was pretty unprepared for that interview and didn’t really have any background at that point, but that failure was what prompted me to take the right classes, read the right books, and get the experiences that have led me to Engagious now.

Like Rich, I regard Thinking, Fast and Slow almost like my bible.
By the end of my internship, I hope that I will be able to identify a real-life application of the biases I have learned about.

A fun fact about me: I am OBSESSED with making friendship bracelets and have made them in at least 100 different patterns.
That along with a love for Greek mythology led me to making a mini bracelet tapestry inspired by the story of Arachne for a final project my freshman year.

My favorite social media is probably a Facebook page called Haverbro Quotes where students submit funny quotes from “bros”

—-

Morgan hails from New Jersey and is a junior at University of Miami majoring in Psychology and minoring in Political Science and Criminology.
Madison is from DC and is a senior at Haverford College majoring in Economics with a minor in Statistics and Psychology.

Welcome Morgan and Madison!
We hope you have a good productive Summer at Engagious.
Please let us know how we can help.

Not to mention a news cycle with the briefest of attention spans.

But if you’re beginning your political career, you need to figure out how to communicate what you stand for in a way that resonates.

And if you’re launching reelection campaign, you need to make sure your messaging hasn’t become irrelevant.

It’s essential to test your message (in all its iterations) outside the echo chamber of your campaign team. They aren’t your audience. They already (probably) agree with you, and they (obviously) support you, and they (likely) could be inclined to tell you what they think you want to hear. So test your message in front of your audience.

The prologue to the testing process is crafting a great message—or at least an outline of it.

You’re running for a reason—what is it? Who are you? What do you believe—what are your non-negotiables? What’s the fire in your bones?

You’re willing to be a public figure, sacrifice privacy, open up your life and finances to public scrutiny—and meanness. What do you have to offer voters? What’s the mission that transcends all the negatives?

Your message should flow from that.

This seems obvious, but it’s incredibly important. No amount of message refinement can impart core values. Or sincerity.

Your philosophy will inform your attitudes toward the issues: the current ones, the campaign surprises, and the policy decisions of elected office. Have a solid, reasoned basis for those decisions.

Then take this thing you believe, this passion you have, and give it form. Have a line that sums up what you stand for that’s easy to say, easy to remember, easy to expand upon.

Play with words, bat around ideas with family and friends, and sure, bring in professional teams.

Here are five proven ways to test your message, both in the building process and after you have something closer to a finished product.

1. Talk to friendly audiences and ask for feedback. Your family, friends, and close associates form a natural, informal focus group. Explain to them what you want to say and ask what they think. This forces you to articulate your message and see reactions to it. You can begin to see what works, and get a sense of what kinds of questions come up.
2. In-depth interviews. A moderator can conduct a series of one-on-one interviews with people in your target audience to see and probe reactions to pieces of your message. This is incredibly useful, especially in the crafting stage; conversations with objective outsiders consistently provide insights that you and your team can’t.
3. Focus groups. A moderator tests your message in front of a larger group. This is classic political message refinement. “It’s been focus-grouped.” So classic it’s been verbed.
4. Another political classic, polling is the quantitative research side of message testing. This is you putting a number to favorable/unfavorable reactions to your message.
5. Dial tests. A dial-test focus group involves presenting your message (usually in video form) to a group that records their moment-to-moment reactions with handheld dials, enabling you and your team to see exactly which parts of your message are fantastic, meh, or cringe-inducing. If it’s set up as a point-counterpoint debate, you can see how you fare against attacks. You will be surprised at the what works and what tanks. Our clients always are. You can dial test things online as well—useful especially later in the campaign, when you need to respond swiftly to attack ads.

Don’t forget to include these things in your messaging:

• Your bio. The things people need to know to be persuaded that they can entrust this job to you.
• Stories. Fill your quiver with them, and practice telling them.
• Speeches for different groups.
• Answers to potential questions, especially the ones you hope no one asks.

Good luck on the campaign trail! If you need assistance, please reach out. We’d love to help you find the perfect tone.

This is a distillation of what you wanted to find out when you started the message refinement process; your own personalized cheat sheet. You and your team will want it handy when you are designing any and all of your communications.

When dial-test respondents record strong agreement with your message (scores above 70), we’ll note that in the “What to Say section.” Messaging that causes significant dips or scores below 60 goes in the “What Not to Say” section. For example, which phrase sounds more credible to this group: “clean energy” or “renewable energy”? During post-video discussions we’ll find out why respondents turned their dials the way they did.

Both the “What to Say” and “What Not to Say” sections are organized by topic and ranked according to what’s most valuable to you.

And if certain language works great with one group of respondents but tanks with respondents of a different type, we’ll note that, too.  The result? All this data and discussion enables us to craft your story in a way that hits all the right notes.

If you are interested in talking through your project, please reach out to us.

Engagious was out in force last night. With our technology in their hands, several focus groups watched and rated the President’s speech- and we captured the highs and lows.

BIPARTISAN HIGHLIGHTS

With all the talk about a divided nation, there were several quotes that received a unanimously favorable reaction:

“America stands with the people of Iran in their courageous struggle for freedom”

“They defend our flag abroad. And they are strong moms and brave kids. They are firefighters & police officers and border agents, medics, and marines. But above all else, they are Americans. And this capital, this city, this nation, belongs entirely to them.”

Other unanimously favorable topics: Rebuilding America’s infrastructure, reducing the cost of prescription drugs, and vocational job training.

LOWLIGHTS

One of the low points for Independents and Democrats was around the phrase: ending the war on “beautiful, clean coal.”

One of the lowest points for Republican respondents? During the immigration reform section of the speech, the phrase a “Path to citizenship” was not received well.  This was one of the few times during the night that the Republicans’ response dipped below 50% favorability–and dipped far below the Independents.(Note the drop off in the red line below).

WHERE DID WE NOTE THE BIGGEST DIFFERENCES BY POLITICAL AFFILIATION?

We noted some of the widest gaps were around social statements such as: …”why we put our hands on our hearts for the Pledge of Allegiance, and why we proudly stand for the national anthem.” This phrase in particular did not resonate well with Democrats (blue line) or Independents (green line).

One of our other focus groups last night was also documented by the USA Today. You can see that article here.

I’ve been moderating focus groups and performing dial testing many years. What I have learned is that dial testing puts your message under a microscope in a way that traditional focus groups cannot.

Dial Testing is Far More Valuable than Traditional Focus Groups. Here’s Why…

Imagine you were giving a presentation or wanted to understand an audience’s reaction to your messaging.  By messaging, we mean the words you use to describe your policy position, your sales pitch, a product rebranding or even your CEO’s upcoming keynote presentation. What options would you have?

Focus Groups (after-the-fact feedback)

One option is to conduct traditional focus groups.  In that setting, you would probably show respondents a video of your presentation or public policy messaging, and afterwards, you would then solicit feedback.  You would ask what they liked or disliked about it.  Maybe someone would say something is missing and needs to be added.  Your audience would mostly be reacting to the video’s content in broad-stroke, focusing on overall themes and concepts.  After all, you’re relying on their memory to produce feedback that you would find valuable.  Also, you would have to worry about one or two dominant voices in a focus group influencing others’ perspectives or maybe even silencing other points of view.  You have to ask yourself: Is this really an effective method to gather feedback that you and your organization would find valuable?

Dial Testing Groups (in-the-moment feedback)

Dial testing is a far more comprehensive and valuable tool used to understand an audience’s reaction to your presentation or public policy messaging.  During a dial test, each member of the audience uses a hand-held dial and rates what he or she is hearing on a scale from zero to 100, for how much they agree or disagree, or for how much they like or dislike what they are hearing at that moment.  With the dial technology, we take a reading from every dial every second, and we are able to monitor the group’s average score, second by second, as they provide continuous feedback.  They are scoring the video in real-time with their visceral reactions to what they are hearing in an anonymous and unbiased way.  During the dial test, the moderator is reviewing a printed text of the messaging content, and he or she is writing the group’s average score at the end of each sentence.  The moderator and clients can view the results on a monitor in real-time, and the moderator carefully looks for spikes or dips in the scores, and sometimes, the scores remain flat when he or she was expecting significant movement toward agreement/disagreement or like/dislike.

Dial testing helps identify the strongest and weakest parts of the message

Dial testing provides a competitive advantage for those who utilize this technology because they are able to easily identify the strongest and weakest parts of a presentation.  You don’t need to guess which messaging is working and which isn’t—it’s right in front of you as an easy-to-understand visual.  With dial testing, you can show video clips to respondents after the dial test with their dial results superimposed on the screen if you want to ask why they reacted the way they did during particular parts of the video. This helps them to remember the context of that part of the presentation and it could help them to remember why they reacted so positively or negatively to it.

Dial groups help a moderator adjust their questioning 

The value of dial testing is that with these results, the moderator is able to focus on a more specific line of questioning that would produce deeper, more valuable insights in follow-up discussions. The visual nature of dial testing is very persuasive to clients who may believe that one of their key arguments is a compelling message—when it really is weak messaging and a problem for their organization.  It is also possible they have a good argument that is not being presented to audiences in an effective way.  Dial testing provides actionable insights while saving clients time and money.

Dial tests provide tangible results

Here’s an analogy: How much more does a scientist know looking through a microscope than with his or her own naked eye?  Dial testing puts your messaging “under the microscope” and shows you how audiences are reacting to it sentence by sentence.  Dial testing provides tangible results for reporting and helps you to determine which messaging you should keep, scrap, or revise.  Clients gain confidence that their original messaging content is receiving the evaluation and analysis it needs and that they will be in a position to communicate their message effectively in the future.