Perception Is Reality: Why Subjective Measures Matter, and How to Maximize Their Impact
By Robert S. Gold, Former Vice President, Palladium Group, Inc.
From Balanced Scorecard Report
BSC strategic objectives often involve changing the attitudes and perceptions of customers, employees, and other key stakeholders. Yet creating reliable, objective measures of subjective factors is a particular challenge. Many leaders resist using surveys, believing them not worth the cost and trouble. In addition, designing and implementing surveys that yield rich, actionable answers can be tricky. Robert Gold makes a compelling case for using such tools, offering insights and practical advice on how to make them an effective part of your BSC reporting.
When helping organizations design measures for their Balanced Scorecards, the moment comes when we float the idea of surveying employees or customers. Invariably, there is an uncomfortable silence, followed by loud protests. Executives complain that surveys are expensive, that they don’t tell them anything new, and that a steady diet of them annoys people and can thus defeat their purpose. An unspoken source of resistance is leaders’ fear that survey results will challenge the comfortable fictions they may be sustaining to support their decisions.
True, surveys can be expensive, especially when professionally designed, researched, and administered – although technological advances over the past decade have gone a long way in offsetting their cost and complexity. Moreover, in the absence of clear-cut results, surveys may raise more questions than they answer. Yet investor, customer, and employee expectations and perceptions all matter to a company. Understanding them is vital to predicting the behaviors of these stakeholders: whether they’ll invest in your shares or buy your products or work hard. When done properly, surveys are essential for developing a balanced portfolio of leading and lagging indicators. And when survey results inform key management decisions that involve making large investments in pursuit of even larger revenues, the value can be substantial.
Understanding, shaping, and fulfilling the expectations of stakeholders is central to successful strategy execution. In for-profit organization strategy maps, the financial perspective is rooted in the need to satisfy shareholders’ expectation of a return on their investment. The decision to invest (and, by implication, the stock price) is driven by investors’ collective expectations of the firm’s future performance. The customer perspective captures the firm’s value proposition—that is, the reason a customer would do business with the firm. The customer’s perception of the value proposition predicts his or her behavior toward the firm, namely, whether he or she buys its products or services. Value, like beauty, is in the eye of the beholder. While executives may envision what the value proposition should be, it is customer perception that determines what the value proposition actually is. The concept of the customer value proposition applies to the firm’s internal customers as well. In our work with IT organizations, we’ve learned that expectations and perceptions shape behaviors that influence the quality of these internal service provider–customer partnerships—and ultimately, how efficiently the resources that drive enterprise performance and strategy execution are used.
Of course, human capital is a key intangible asset necessary for creating value. A company can influence, but not control, employees’ expectations and perceptions of the firm—two factors that largely drive their behavior: how hard they work, how well their actions support the firm’s interests, and ultimately whether they’ll continue working for the firm.
Perception drives behavior; that’s why the behavior of these stakeholders is indeed the firm’s reality. Woe unto the firm that doesn’t understand what its investors, customers, and employees are thinking.
Ensuring Survey Success: Skillful Research Design Is Vital
A survey program is the best way to regularly monitor stakeholder perceptions. E-mail and Web-based survey tools enable faster design, execution, and analysis, and have reduced the cost of surveys considerably. Many enterprises already have e-mail address lists from the Web sites and customer databases they maintain for direct communication and marketing purposes. Wireless telephony and text messaging enable nearly real-time data collection and analysis. Technology, however, is no substitute for good research design, and in amateur hands, such tools amplify the risk of getting unactionable results or even causing adverse consequences.
Regrettably, to save money, many firms opt for do-it-yourself research design, even when they lack the expertise to do so effectively. The principles of research design are beyond the scope of this article; suffice it to say that unless you have a professional in-house market research staff, do-it-yourself research design is risky, unlikely to yield much useful information, and can actually do harm. The “Hawthorne Effect”1 demonstrated that the very act of studying a group of subjects changes its members’ behavior. A poorly designed survey can antagonize its subjects, or, if they fear repercussions from any negative responses, discourage candor. Either way, it can taint results.
While an inexperienced research designer would simply begin by writing questions, a seasoned designer would start by studying the environment and its population groups, as well as the organization’s strategy, to decide the best method (e.g., surveys, focus groups, or data mining) and frequency, and how to segment the research population. Drawing on an academic and practical understanding of human behavior, statistical analysis, and survey design, the designer would then develop a comprehensive research plan and plan follow-on research.
The research designer must be both unbiased about the research topic and familiar with the study environment. Unwittingly or not, do-it-yourselfers often allow their firm’s biases to creep into survey questions. But subtle differences in the way a question is phrased—even the order of the questions—can dramatically affect how people respond. Allow leaders to contribute their opinions when establishing hypotheses, but keep them a safe distance from the actual design.
BSC-driven research begins with the strategy map and the cause-and-effect relationships among strategic objectives. Objectives in the customer and learning and growth perspectives require perception measures to give a full picture. While financial measures like sales volume or market share reflect the results of customer behavior, leading measures of customer perception can help executives anticipate changes in these ultimate outcome measures.
Changing people’s perceptions is often central to strategy. Well-designed research can provide leaders with insights about which perceptions need to change or help test hypotheses about how a change in perception will actually occur. A customer perception measure can validate the effect of an initiative in the internal process perspective.
For example, as part of its strategy to develop more profitable relationships with its customers, a bank undertakes an initiative to have customer service agents spend more time on the telephone with customers who call for support. The bank believes that if agents are equipped with better customer information, they will be able to probe beyond the original scope of each call and promote additional products appropriate to the caller. A sound customer feedback program can enable the bank to validate the assumptions underlying its strategy, or to uncover negative perceptions (e.g., calls are taking too long or agents’ questions are intrusive). Only by measuring agents’ new behavior (for example, through average call duration), along with agent and customer perceptions, can increased sales reasonably be attributed to the new strategy.
Conquer Survey Fatigue
Leaders seeking “silver bullets” for intractable problems don’t often have the patience to execute surveys repeatedly. But few surveys provide full value in one execution. Periodic sampling of a population reveals trends that are impossible to see in a single snapshot. Often the absolute value of a perception measure is meaningless; the insight comes from tracking the direction and magnitude of change in the measure over time.
“Survey fatigue” is a common reason why organizations resist using surveys. Employees in firms with Internet survey tools are often bombarded with narrow, one-off surveys from multiple sources whose timing is not coordinated. Over time, these do little more than discourage response. All research requests should be funneled through a single coordinator, ideally in the Office of Strategy Management.2 Rather than surveying the entire employee population annually, the organization should send the same basic survey to a controlled random sample every quarter (25% of the employee population), polling different subgroups each time—thereby reducing survey frequency and minimizing fatigue. The sample size is determined mathematically, according to population size and acceptable margin of error.
A basic set of questions could be asked each time, and carefully targeted new questions could be included when needed to understand new challenges. Under this model, no employee in the target group would receive more than one survey per year. By generating quarterly findings (rather than the usual annual ones), this model is especially appropriate for BSC reporting.
Consider Focus Groups or Interviews
While most perception measures come from surveys, focus groups and interviews are also valuable tools. Focus groups can be a component of a survey (answering the complex question, “Why are employees unhappy?”), or can simply serve as a way of capturing the perceptions of a small group when surveys would not be effective or practical. A focus group can reveal complex root causes for perceptions that may not be anticipated in a set of multiple-choice responses.
Focus groups typically involve eight to 12 subjects and one or two facilitators, and deal with generally no more than five questions. Experienced facilitators skillfully elicit and structure responses from participants, which may be captured through flip charts, note taking, or audio and video recording. Because facilitators affect results, it is vital that they be experienced, neutral, and knowledgeable about the topic. Specially designed focus-group facilities feature one-way mirrors that enable trained observers to monitor participants’ body language and group dynamics without influencing them.
One-on-one interviews are another valuable technique. By asking members of a group with similar characteristics a standard set of questions—for example, asking salespeople the same questions about follow-up calls to customers after a store visit—you get the benefits of a focus group (more detailed, qualified responses and more flexible dialogue) without the burdens of scheduling and travel. Internet and other technologies—from simple discussion boards and blogs to real-time sessions assembling people from different locations to interact with text, audio, and even video—now enable on-line focus groups.
Put Your Findings in Context
With survey data in hand, how do you best present it? It’s easy to present survey data in graph form, but graphs by themselves don’t tell a very useful story. Including all the detailed findings in the BSC report is usually not necessary, though they should be available to the leadership team so they can drill down if they want. What is necessary, however, is context. The “performance advocate”3 for the overlying strategic objective should work closely with the research designer to analyze the survey findings, considering them in the context of data from prior surveys and other measures in the BSC report—and presenting that context in the reporting. Focus group feedback, including carefully selected participant quotes (paraphrased when necessary to ensure anonymity), should be summarized through the facilitators’ written analysis. The research designer should attend the management discussion of the findings to answer technical questions and help shape subsequent research requests. Because of the very real danger of simply replacing the old “comfortable fictions” with new ones, leaders should balance their reasoned judgment with a healthy skepticism when making decisions resulting from their enriched understanding of stakeholder perception.
1. The Hawthorne Effect, well known in the research field, was first observed in the 1920s at
the Western Electric Hawthorne factory.
2. See R.S. Kaplan and D. P. Norton, “Strategic Management: An Emerging Profession,” BSR
May–June 2004 (Reprint #B0405A). Also, look
for the forthcoming BSR Reader (article collection) on the Office of Strategy Management at
3. A performance advocate, typically the leader
and his or her direct reports, acts as project manager for a strategy map objective or objectives.
For a more detailed explanation, see R. S. Gold and J. R. Weiser, “The Balanced Scorecard Strategy Review Meeting: What to Expect the First Year,” BSR March–April 2005 (Reprint #B0503E).
Bob Gold’s “Perception Is Reality: Why Subjective Measures Matter, and How to Maximize Their Impact” is an excellent discussion of subjective measures such as employee satisfaction. (July-Aug 2006, Vol. 8, No. 4, Reprint #B0607E)
© 2008 by Harvard Business Publishing Corporation and Palladium Group, Inc. This article is reproduced from July -August 2006 Balanced Scorecard Report with permission from Harvard Business Publishing Corp. To obtain a reprint (Reprint #B0607E), visit www.harvardbusiness.org.