Methodology background
Here, we explain how we get customer satisfaction performance results.
Regulatory transactions
Research methodology
Our research programme into how satisfied our regulatory transactions customers are is currently conducted by GfK NOP - an independent market research company that complies with the Market Research Society Codes.
All decision letters we send to firms advising them of the outcome of their application now contain a paragraph explaining that they may be approached to take part in a customer satisfaction survey. Although we would encourage firms to take part, they can advise us that they do not wish to be contacted or, of course, may decline if the market research company contacts them.
Periodically we give GfK NOP details of firms who have completed one of the surveyed regulatory processes. We will supply details of the main compliance contact or, if different, the individual who made the application. In the case of applications for a change in control we will supply the details of the new controller applicant. In all cases we do not pass on any details until after the regulatory transaction has been completed and we have notified the applicant of the outcome.
GfK NOP then independently selects a sample of these firms and telephones to invite them to complete the customer satisfaction questionnaire. This covers the various stages of each process from finding and completing the application form through to receiving the final decision. As well as gathering views on their satisfaction with aspects of the application, firms are invited to offer their suggestions for improvements.
These results are processed and analysed by GfK NOP to produce a detailed report of aggregated views for operational improvement purposes and the customer satisfaction index (CSI) that is published here. The FSA has no way of knowing which firms have taken part in the research.
Statistical methodology
We use the CSI because it provides a better picture of overall satisfaction than relying on a single overall satisfaction question. The CSI process works by weighting the satisfaction scores respondents give on each of the aspects of service included in the survey and aggregating these to produce a single overall score for each process. The weightings for each aspect of service are derived using a ridge regression analysis.
The regression analysis allows us to compare the importance of each attribute to the others. We then rank the elements of service in terms of their derived importance to users. This does not mean that those elements at the bottom of the ranking are not important or worthy of consideration, but that any improvement in these is likely to have less of an impact on overall satisfaction than a similar level of improvement in the higher ranked elements.
Using the CSI approach effectively requires that the aspects of service that are surveyed are those that influence overall satisfaction. We have gone some way to ensure that the aspects of service on which we seek opinions are those that influence overall satisfaction. We are confident that this objective has been achieved given the following:
- As part of the pilot stage of the research we conducted a qualitative study to identify the key drivers of satisfaction among the applicants for the various regulatory processes. These drivers were tested again during the quantitative stage of the pilot.
- When using the ridge regression analysis, an R-squared value of 76.3 was obtained. The R-squared value indicates the proportion of the overall satisfaction that can be statistically explained by the other satisfaction elements in the survey. This figure is very high for this type of survey and strongly suggests that the main drivers of satisfaction are covered in the survey.
Contact Centres
Research methodology
Our Contact Centre customer satisfaction research is currently carried out by IFF Research, an independent company with significant experience in market research across the financial services industry.
To conduct the research, we send IFF a sample of recent correspondence, together with contact details. This sample is selected at random from all the correspondence received over the reporting period. All contacts are notified in advance that we would like to carry out market research, and are given the option to opt out if they wish. To ensure accurate results, quotas are required for the minimum number of interviews that must be performed to yield significant results.
Depending on the method of initial contact, feedback is collected either online, or using CATI (Computer Assisted Telephone Interviewing) interviews.
IFF Research analyse the results to provide overall satisfaction scores for the two Contact Centres and a breakdown of the satisfaction rating for each method of communication. They also provide a detailed report including information on areas which would benefit from further improvement, and issues raised through the additional comments sections of the questionnaire.
Statistical methodology
A CSI score is used to present our performance because it takes account of the different factors which drive customer satisfaction. It also reflects the relative importance of each individual question against the customer's overall perception of satisfaction.
Each composite score is derived by calculating an efficiency score for each element, based on the mean results from the questionnaire. The correlation of each element to the overall score is then used to aggregate and weight these efficiency scores, providing the overall score. This ensures that the key performance drivers IFF identified have the largest impact on the overall results. These results are then further broken down into component correspondence types by weighting them according to the likelihood of occurrence.