Informed Polling and Getting it Right
Cross Posted from Fako & Associates' Political Polling Blog.
Every pollster will tell you that political polls are a snapshot in time and at best consecutive polls can elicit trends, but should not be used to predict turnout or the outcome of an election.
While the national attention is on how the pollsters got it wrong in several contests with surveys taken only days before an election, few are talking about how early surveying often-times gets things right. We've occasionally written about the importance of informed trial heats in some of our past posts.
To refresh, an informed trial heat is designed to simulate the effects of an engaged campaign, presenting balanced positive and negative messages about each candidate (or multiple candidates). The end result of the election scenario tells a campaign what is possible with their messages and themes they plan to implement. Most importantly, this section will determine if the core message works in direct contrast to the opponent(s)' message and can identify movement among the various demographic and attitudinal groups -- helping refine strategy.
F&A recently conducted a benchmark survey for a client running in a Democratic primary for open seat in a multi-candidate open-seat race. The benchmark poll was designed to evaluate the political environment, determine voter’s top issue concerns, examine opinions of the candidates and other significant figures, and test messages in support and in opposition of various candidates. As we always do in comprehensive benchmark polls, we included an informed trial heat question in the survey. The client was on a tight budget and didn't want to include "minor" candidates in the informed trial heat. After some debate, we were able to convince the client to include the "minor" candidates in the question.
We accounted for the ability of the "minor" candidates to get their message out given their budget constraints. Their messages were curtailed in the informed scenario to a simple bio-statement, while "major" candidates received bio, supporting, and opposing information, simulating an engaged campaign.
In our poll, the results of the informed trial heat were unexpected; a "minor" candidate took a 22 percentage point lead above the assumed frontrunner in the informed scenario, an increase of 25% above the candidate's level of support in the initial trial heat (the uninformed horse race question). A "major" candidate jumped up 7% and the assumed frontrunner stalled with a gain within the margin of error. Undecided voters in the initial trial heat heavily sided with one of the "minor" candidates. The percentage of undecided voters was reduced by over 40% in the informed trial heat. At this point we recognized the minor candidate's growth potential and advised the client to pay close attention to this so-called "minor" candidate. We noted that this individual clearly had the basic background and simple message that would break through the clutter of a highly engaged multi-candidate race, despite an initial perception of not being viable.
As the campaign progressed, the so called minor candidate ended up raising some serious money and gained significant earned media attention in addition to their own paid activities. It became apparent that this minor candidate was not minor, something our polling has observed only three months before the election
The "minor" candidate ended up winning this election, slightly ahead of the "major" candidate that we also observed gaining traction through the informed trial heat. This highlighted the usefulness and importance of utilizing informed trial heat questions in polls and why clients should never ignore perceived "minor" opponents. Polls that include an informed trial heat are one of the most useful strategic planning tools available to a campaign. It gives campaigns the information needed to determine if their message works (in the above example, out client’s message was not working); provides detailed strategic planning information, particularly at the demographic sub-group level, and gives campaigns information to prepare and adjust strategy for unanticipated situation (such as an unexpectedly strong opponent).
Every pollster will tell you that political polls are a snapshot in time and at best consecutive polls can elicit trends, but should not be used to predict turnout or the outcome of an election.
While the national attention is on how the pollsters got it wrong in several contests with surveys taken only days before an election, few are talking about how early surveying often-times gets things right. We've occasionally written about the importance of informed trial heats in some of our past posts.
To refresh, an informed trial heat is designed to simulate the effects of an engaged campaign, presenting balanced positive and negative messages about each candidate (or multiple candidates). The end result of the election scenario tells a campaign what is possible with their messages and themes they plan to implement. Most importantly, this section will determine if the core message works in direct contrast to the opponent(s)' message and can identify movement among the various demographic and attitudinal groups -- helping refine strategy.
F&A recently conducted a benchmark survey for a client running in a Democratic primary for open seat in a multi-candidate open-seat race. The benchmark poll was designed to evaluate the political environment, determine voter’s top issue concerns, examine opinions of the candidates and other significant figures, and test messages in support and in opposition of various candidates. As we always do in comprehensive benchmark polls, we included an informed trial heat question in the survey. The client was on a tight budget and didn't want to include "minor" candidates in the informed trial heat. After some debate, we were able to convince the client to include the "minor" candidates in the question.
We accounted for the ability of the "minor" candidates to get their message out given their budget constraints. Their messages were curtailed in the informed scenario to a simple bio-statement, while "major" candidates received bio, supporting, and opposing information, simulating an engaged campaign.
In our poll, the results of the informed trial heat were unexpected; a "minor" candidate took a 22 percentage point lead above the assumed frontrunner in the informed scenario, an increase of 25% above the candidate's level of support in the initial trial heat (the uninformed horse race question). A "major" candidate jumped up 7% and the assumed frontrunner stalled with a gain within the margin of error. Undecided voters in the initial trial heat heavily sided with one of the "minor" candidates. The percentage of undecided voters was reduced by over 40% in the informed trial heat. At this point we recognized the minor candidate's growth potential and advised the client to pay close attention to this so-called "minor" candidate. We noted that this individual clearly had the basic background and simple message that would break through the clutter of a highly engaged multi-candidate race, despite an initial perception of not being viable.
As the campaign progressed, the so called minor candidate ended up raising some serious money and gained significant earned media attention in addition to their own paid activities. It became apparent that this minor candidate was not minor, something our polling has observed only three months before the election
The "minor" candidate ended up winning this election, slightly ahead of the "major" candidate that we also observed gaining traction through the informed trial heat. This highlighted the usefulness and importance of utilizing informed trial heat questions in polls and why clients should never ignore perceived "minor" opponents. Polls that include an informed trial heat are one of the most useful strategic planning tools available to a campaign. It gives campaigns the information needed to determine if their message works (in the above example, out client’s message was not working); provides detailed strategic planning information, particularly at the demographic sub-group level, and gives campaigns information to prepare and adjust strategy for unanticipated situation (such as an unexpectedly strong opponent).
2 comments:
Fako -
You leave one great question answered: armed with a perfect poll that told your candidate everything they needed to know, how did they end up losing?
You can lead a horse to water, but you can't make them drink... (spend money wisely, fund raise, etc)
-- Steve
Post a Comment