Analyzing Guests' Sentiment: A Scientific Look at Reviews

    The proliferation of online reviews and ratings has aroused interest from science and business sectors alike in the process of analyzing the sentiment of customers. By identifying relevant content in reviews, this data can be utilized not only as a marketing tool, but also to assist in controlling a business’ reputation and getting to know customers better.sentiment.JPG

     

    Reviews can be a contentious topic for vacation rental owners and managers. Each traveler’s experience is utterly subjective, and can be based on a myriad of ever-changing criteria. There are no common rating standards or service levels in the vacation rental industry; what constitutes a 5-star stay to one guest may be a 3-star experience to another.

    HomeAway has an extensive collection of reviews across our multiple websites. Brent Schneeman, one of HomeAway’s senior software engineers, decided to dig into this data to find out if 5-star reviews have certain characteristics which distinguish them from 4-star reviews, and so on.

     

    Ideally, the star rating in a review matches the sentiment expressed therein. However, there are often examples of failure in this process. For example, the text may indicate unhappiness with some attribute of the property, but the star rating is high. Alternatively, the text may describe an incredible experience, but the traveler, in error, gives a low star rating. In both cases the owner/manager suffers from the inaccuracy, and trust in the reviews system is compromised on both sides.

     

    A study of our small sample concluded that this kind of error occurs about 5% of the time. While in the future, we may be able to improve the accuracy of star ratings by using sentiment analysis, for now here is some interesting data we have collected.

     

    Schneeman also extracted the top three-term collocations, or sequences of words, that occurred in the sample set of reviews. Here are his findings in descending order of significance.

     

    1-Star

    2-Star

    3-Star

    4-Star

    5-Star

    pots and pans

    pots and pans

    washer and dryer

    vacation rental was

    washer and dryer

    accused us of

    needless to say

    thank you for

    washer and dryer

    comforts of home

    needless to say

    washer and dryer

    letting us stay

    helpful overall recommend

    look forward to

    thank you for

    walking distance to

    look forward to

    within walking distance

    glass of wine

    did not work

    taken care of

    across the street

    recommend this vacation

    thank you for

    during our stay

    within walking distance

    within walking distance

    satisfied with the

    across the street

    when we arrived

    did not work

    vacation rental was

    thank you for

    exceeded our expectations

    upon arrival we

    during our stay

    thank you so

    property manager was

    plenty of room

    there was no

    when we arrived

    recommend this vacation

    plenty of room

    forward to returning

    do not rent

    make sure you

    away from home

    look forward to

    within walking distance

    security deposit we

    vacation rental was

    helpful overall recommend

    happy with the

    work with and

    would not recommend

    prior to our

    property manager was

    more than satisfied

    thought of everything

     

    Taking an objective and methodical look at reviews can reveal a lot about your customer base. In this case, it is clear that providing quality pots and pans is a seemingly simple way to meet travelers’ expectations and avoid a negative review (as the phrase pots and pans occurred frequently in 1 and 2 star ratings.) Consider your reviews as data from which you can pull valuable information to continually improve your processes, and provide ultimate customer satisfaction.