What got me thinking about this is the fact that my car lease is ending and I'm shopping. The last three times I've leased a new car, the process of picking it up has been identical. I go in, pay some money, sign a bunch of forms I don't understand, get a tour of the car's features and then I'm told that I'll be getting a survey and that I should give the highest marks on everything. Sometimes the salesman says "if there is something you can't give the highest mark on, tell me what I have to do to earn it", but they always say, "If I don't get the highest mark it will hurt my commission."
I recognize that the car company might not view this or use this as they would pure market research. In many respects they are like response cards (like hotels or restaurants use) or invitations to do a survey found on receipts. Even without all the controls pure market research puts in place, the data generated by these efforts can have tremendous value. My firm, for example, has used them to help establish the bottom line impact of various attributes.
No question that knowledge of the survey program's impact on compensation will encourage many employees to work hard to satisfy customers, but it will encourage other to game the system by cheating. This could range from the "give me high marks" example or could take the form of trying to control who participates (encourage those you have a good interaction with by asking them to participate when called or pointing out how to participate/ discourage those who had a bad experience by entering their phone number wrong or not telling them how to participate).
In the end, if the system is easy to game, it calls into question the integrity of the data…something no one should be willing to compromise on. Such results are not a fair way to base compensation (once employees see that it will encourage even more cheating) and they are useless for understanding how to do a better job (try running a regression on data that has almost all top box scores). Ultimately, it could even cause customers to participate less often, not just in these surveys, but in all surveys.
So, before firms decide to make survey results a means for compensation, here are some questions that you should ask:
- Is there a better measure? To the extent you can track real world measures (like sales, new customers, profits) down to the employee or location level, they are far superior to using survey results.
- How much control can you exert on the process? Things like signs on the register or recorded messages that encourage participation make it harder for employees to deny access to upset customers. In situations where you have little control (such as the car dealer), simply asking respondents what was said and/or having a manager talk to customers before they leave can make a difference.
- How many customers do you have? The more you have the harder the system is to game. It can be hard to detect someone telling their few customers each month to give high ratings, but if they have hundreds of interactions (such as in a call center or retail setting) then this becomes more difficult.
- What do the results look like? Careful data analysis will help you see to what extent results have been gamed. Straight lined results or low response rates for an employee should be cause for concern and warrant a closer look…perhaps even the use of mystery shoppers.
- Is there a way to dissuade employees from gaming the system? Setting clear rules on what they can and can't say and taking action for violations of this policy might not worry the truly unethical, but it will cause most employees to have second thoughts about gaming results.
- Do results track over time with the real world? Before choosing to use the results as part of compensation you should find the best way to tie them to real world results that you can. If some attributes improve, but the predicted real world impact is not realized, it likely indicates that gaming is rampant. Worth noting that doing this will also help you communicate why this is important…messages like "if we cut our wait time we will see an increase of $$$$$ in profits" bring credibility to the compensation system.