You are planning to take a trip to the city of brotherly love to visit the world famous Philadelphia Flower Show, and would like to book a hotel near the Convention Center venue. If you’re like most people, you go online, perhaps to TripAdvisor or Expedia and look for a hotel. In a few clicks you find a list of hotels with star ratings, prices, amenities, distance to destination – everything you need to make a decision. Quickly you narrow your choice down to two hotels within walking distance of the Flower Show, and conveniently located near the historic Reading Terminal Market.
But how to choose between the two that seem so evenly matched? Perhaps you can take a look at some review comments that might provide more depth? There are hundreds of comments which is more than you have time for, but you quickly read a few on the first page. You are about to close the browser when you notice something. One of the hotels has responses to some of the negative comments. Hmmm…interesting. You decide to read the responses, and see some apologies, a few explanations and general earnestness. No such response for the other hotel, which now begins to seem colder and more distant. What do you do?
In effect, that’s the question Davide Proserpio and Georgios Zervas seek to answer in a recent article in the INFORMS journal Marketing Science. And it’s not hard to see why it’s an important question. Online reviews can have significant impact on a business, and unlike word of mouth they tend to stick around for years (just take a look at the dates on some reviews). Companies can’t do much to stop reviews (especially negative), and so they often try to coopt them by providing responses to selected reviews. It is a manual task, but the idea seems sound. By responding, perhaps they can take the sting out of negative reviews, appear contrite, promise to do better, or just thank the reviewer for the time they took to write the feedback – all with the objective of getting prospective customers to give them a fair chance. The question then is whether such efforts are useful or just more online clutter.
It turns out that’s not an easy question to answer, and as Proserpio and Zervas document in the article, there are several factors that first need to be controlled. But their basic approach is easy enough to understand – they examine whether TripAdvisor ratings for hotels tend to go up after management responds to online reviews. An immediate problem to overcome, ironically enough, is management response. That is, in reaction to bad reviews a hotel may actually make changes that then increases future ratings. That’s great for the hotel, but not so much for the researcher who is trying to study if the response to the online review had an impact, not whether the hotel is willing to make changes in response to the review. So, that’s an important factor that needs to be controlled. How to do that?
Enter Expedia. As it happens, hotels frequently respond to TripAdvisor reviews while they almost never do so on Expedia. So, they use Expedia as a control cell and compare the before-after difference in ratings on TripAdvisor and Expedia (the difference-in-difference approach). Hence they are able to tease out if the improvement in ratings was because of responding to reviews or real changes. Another check they use is to compare the ratings of guests who left a review shortly before a hotel began responding with those who did so shortly after the hotel began responding. Much of the article is actually devoted to several more clever and increasingly complex maneuvers they use to finally tease out just the impact of management responses. What do they find?
There is indeed a positive effect of management response to online hotel reviews! It is to the tune of 0.12 stars on a 5-star rating scale. Yes, I can see that you are underwhelmed, but consider this. In practice (as market researchers know) all five of the scale points are not used, mostly just a narrow sliver, hence the impact is magnified. Further, review platforms usually round average ratings to the nearest half star, so small changes can have a significant impact. For example, if a 4.24-star hotel can cross the 4.25 threshold its rounded average rating jumps by half a star. That’s a half star jump for simply responding to some online reviews! In the data used in this study 27% of responding hotels increased their ratings by at least half a star within six months of the first management response!
What happens when hotels respond to online reviews? Something quite interesting. The overall number of reviews, and number of positive reviews goes up. The number of negative reviews go down, though they seem to get longer and more detailed. “Our research points to an interesting trade-off: hotels can decide to respond to reviews and increase their ratings at the cost of longer negative reviews,” Proserpio said. “While longer negative reviews could be seen as a negative outcome, we think they can benefit hotels that use them to improve their business,” Zervas concluded.
Especially given the low cost of responding to reviews, this research generally supports the idea of hotels responding to online reviews because they increase both firms’ reputation and the quality of the (negative) feedback.
Oh, if you were indeed looking to book that hotel in Philadelphia, go right ahead. We hear that the Flower Show (themed Wonders of Water) is going to be a five star experience!