General

A mystery shopper survey assesses if the standard of service delivery is in compliance with the client’s protocol.

• A mystery shopping survey is the tactical cousin of a customer satisfaction survey and addresses detailed service aspects that affect satisfaction, but which respondents from a satisfaction survey would not be able to recall in detail.
• An MS survey measures the extent of which service is implemented in accordance with (sometimes documented) service standards that the client adopts.
• Their purpose is to remedy (for example through additional training or upgraded communication to staff) gaps between the protocol of standards and the reality of experiences customers may have during visits, over the phone or online.

Objectives

• Assess services from the customers’ point of view objectively – untainted by mystery shoppers’ partiality – through measuring a series of service aspects.
• Monitor employees’ compliance with the service standards at the various customer touch-points in the order of the typical customer journey.
• Evaluate employees’ product knowledge and service efficiency.
• Appraisal of staff’s appearance, attitudes and sales skills (e.g. up or cross selling).
• Presentation of the facilities, cleanliness and maintenance levels.
• Standard of equipment and/or consumables, e.g. forms, menus, dispensing machines.
• Mystery shopper surveys can monitor any attributes visible at the point of sale; e.g. promotions, customer traffic, number of staff.
• Identify strengths and weaknesses, opportunities and threats.

The power of mystery shopping data resides in tracking performance over time. Typically, the report recommends a list of steps to be taken to improve service delivery in order of priority, starting with those that have the greatest potential impact on raising the overall service delivery index.

Information Needs

The measurement of service delivery may include any service channel, such as a retail branch, telephone service and even communication from and through websites.

• Presentation (e.g. cleanliness, maintenance standard) of the premises, preparation areas (where relevant), facilities, equipment and consumables, as well as accessibility.
• Hygiene level of washroom facilities where relevant.
• Presentation and clarity of signage as well as any promotional material visible in the premises.
• Staffing level in relation to the number of customers in the store. Specifically focusing on waiting and transaction times.
• Presentation of staff, which means adhering to the company’s dress and grooming code.
• Politeness/hospitality standard. Making the customers feel welcome through appropriate body language/behaviour. Alertness of staff to anticipate customers’ need for service.
• Communication skills. More than just language, aspects that reflect the ability of staff to pro-actively share information.
• Product knowledge. Do staff provide sufficient and accurate information in response to queries?
• Efficiency of service generally and in handling transactions in particular.
• Mystery shoppers to provide a short qualitative appraisal of their experience by noting down any occurrences – if any – that raised or spoilt their experience.
• The survey divides service attributes into categories in the order of the customer journey and the report is able to present results accordingly.

Methodology

• Researchers carry out the research under the guise of being genuine customers and check from a mystery shoppers evaluation form if certain services are passing the standard through a simple yes or no. In a mystery shopper survey there is no room for subjectivity and therefore, unlike a customer satisfaction survey, ratings from a scale do not belong in a mystery shopper survey.
• The survey applies weights to scores depending on the importance of them (to the client or extracted from a CS survey).
• Findings of a mystery shopper survey are so much more meaningful with competitive bench-marking. Clients may include this as an option. Without a measurement of rivals, the benchmark is whatever standard the management set through the service delivery protocol.
• Mystery shoppers follow the typical customer journey when evaluating the service in sequential order starting from first impressions to bidding goodbye.
• The survey divides these experience steps into categories with each consisting of a series of attributes (typically 6 – 10) that they rate with a yes or no.
• A mystery shopper survey usually includes between 60-80 attributes, grouped into service categories.
• The research team and client agree on the attributes to be measured before fieldwork proceeds.
• Scores are based on the number of readings. For example, if out of 6 visits an attribute complies with the standard on 3 occasions, the score is 50%. If this happens to be a critical service factor, the figure lifts on account of its weight within the service category.
• Attribute scores calculate the weighted performance of a category and, when consolidated in full across all attribute groups, into an overall weighted score of the branch, call centre or website.
• Mystery shoppers test product knowledge and complaints handling through “scenarios” that have been agreed with the client in advance.
• Other than measuring a series of attributes, mystery shoppers can collect anything else observable, such as:

• Number of staff visible.
• Visible promotions, membership initiatives etc.
• Presence and standard of equipment and/or consumables.handout menus.
• Customer footfall.

• Mystery shoppers also add qualitative feedback, such as commenting on events that spoilt or raised the customer experience, as well as their own view on customer service. These are reported as anecdotes to illustrate the mystery shopper indices and are not used in measurements.
• Visits to a branch may last up to 30 minutes and online/call contacts around 5-7 minutes.
• The client will only be able to gain the full benefit of a mystery shopper survey if it measures performance over time across different waves at regular intervals.
• Typically schedules consist of fieldwork waves – each spanning four consecutive weeks – at bi-monthly or quarterly intervals, which gives the client the time to digest the results and to take actions, ready for the next wave of fieldwork.

Mystery Shopper Tools

• These days mystery shoppers rely on online tools to accurately record findings instantly. Instead of an evaluation paper in paper format, the mystery shoppers record their experiences through an online version, which they can complete inside a venue surreptitiously.
• Mystery shoppers can take a limited number of photos, usually reserved for illustrating a poor or excellent rating of specific attributes.
• Mystery shoppers can video record the entire visit to the restaurant and the research team extract clips to illustrate particular findings.
• For telephone contacts to branches or to a call centre, we can audio record all the MS calls. The mystery callers enter ratings in the online evaluation form immediately after completing the call.
• For online contacts, mystery shoppers screen-record their interaction. This will illustrate opportunities for improvement if ratings indicate any development glitches that could irritate users.

Reporting

Every mystery shopper survey is customised and so is the reporting of results. Given the tactical nature, it is imperative that results are made available within the shortest time possible as soon as fieldwork is completed. The client receives:

• Online interactive data tables and charts available immediately after completion of the fieldwork. The default version consists of an online Excel based dashboard. The client may prefer upgraded visualisation from Shopmetrics, which has been specifically developed for the online reporting of mystery shopper data.
• The report is first made available to the management and accessible to a wider audience after this has been greenlighted by the management. The client may set different access levels to data by user groups. For example a branch manager can only see the results of the store he/she manages versus total.
• In addition to the online reporting dashboard, there is a PowerPoint report at the end of each wave of research complete with an interpretative summary and with recommendations. This too is available online (downloadable and/or viewable).
• Presentation by the research consultant as and when required. This is an optional service.