Eastern Mojave Vegetation | Measuring Help Desk Customer Satisfaction |
| Tom Schweich |
Topics in this Article: Introduction Customer Satisfaction Postcard Program Comments Appendix | (No Preface) |
|
| IntroductionIt is very easy for a Help Desk to become known as the "Helpless Desk" or for an end user support organization to become known as "End Useless Support." I know, I've been there. |
I felt the reputation was unfairly earned.
I knew I had good people who were well trained.
I thought I had technology, telephone and procedures in place.
My biggest critic was my own Vice President.
He just did not believe we did good work.
| |
At about the same time, my company began a Quality Improvement Program.
To keep the story short, I was told to measure customer satisfaction
and do something about it.
Our original thinking was a long and involved customer
survey with a sophisticated statistical analysis to
provide valid measurements along independent sociological
axes, i.e., we were going to do a factor analysis and
cluster on factor scores.
Fortunately we did not pursue that path.
| |
I learned of a successful and startlingly simple
customer satisfaction measurement system.
This system was seen in a Florida fast food restaurant by someone
who joined our Quality department from Florida Power and Light.
We adapted the program to our business and implemented it in a matter of days.
Over time we evolved an extremely effective program of continuously
measuring customer satisfaction and concerns, coupled with an effective
system of adapting our organization to respond to customers, and reach
increasing levels of customer satisfaction.
This document describes how we did it.
| |
And the best thing is ... faced with data showing customer satisfaction,
my Vice President came to appreciate the work of his End User Support
organization.
| |
This program has actually been used twice.
Most of the description that follows derives from the first time,
which was a period of about 3 1/2 years.
The second time the program was used only about four months.
It was stopped because of a merger and changes in services contracts
which had no relation to customer satisfaction.
| |
| Customer Satisfaction Postcard ProgramThis document describes the End User Support - Customer Satisfaction Postcard program. This program is extremely important to maintain and improve the satisfaction of our customers. It has a proven track record when used in other organizations. To make it work, everyone in the End User Support organization must participate. |
The heart of the program is a very simple instrument -- a postcard --
which is kept accessible to the customer all the time.
| | Front of paper postcard.
The postcards are available in two media: paper and electronic. The face of the paper postcard is shown at left (click on the thumbnail to see the full size image).
| |
The format was jointly developed by Tom Schweich and Brendan Collins
of the Southern Pacific Quality Improvement Department.
Brendan had previously seen a similar card in a small restaurant.
The three questions on that card were
|
|
These questions were adapted to reflect what we thought might be important for customers of End User Support. At the beginning, our managers thought there should be more questions, or they should be more probing, or ... all kinds of suggestions that, in retrospect, we were wise to resist. We had to stay focused on a very few essential questions which which capture the ideas of:
|
|
The original restaurant postcard had check boxes with the scale
of "Excellent," "Fair," "Poor," etc.
We changed these to graphic faces.
The idea for use of happy, neutral (so-so) and sad faces rather than
some sort of check-off scale came from the order form for Road Runner Sports,
a mail order sports store in San Diego, California.
| |
I prefer the faces to the descriptive scales. Descriptive scales require definitions, as in, "Just what do you mean by poor service?" On the other hand, everyone knows when they are happy or sad. Moreover, it's the customer's state of mind we're after, not someone's definition of good or bad service.
| |
The staff of End User Support recommended that a note be added to the paper postcard telling customers that an electronic form of the postcard is available. However, we never added the comment.
| |
The reverse side of the paper postcard was pre-addressed to me, the Manager of End User Support.
There were three blank lines in the upper left corner, where a customer could write in a return address.
We did not require a return address, however. The upper right corner contained a company internal mail frank, that could be covered by a stamp, or a postage meter frank, if necessary.
| | Electronic Postcard.
The electronic postcard was implemented as an AEFORM on PROFS. The customer could access the form by typing "postcard" at any PROFS or VM system prompt. The format of the card, and the screen seen by the end user when they type "postcard" is shown below:
| |
We had a few comments from customers requesting more room to write comments. However, I resisted lengthening the card. The eight lines for typing seems adequate to me. I think if the card was longer, some people would write a novel, when a few words will capture the essence of their message. Customers could always send a long PROFS note, or call on the telephone.
| |
| Postcard Volume GoalsOur goal was 100 cards received per month. That may seem like a lot of postcards. However, think of it this way. We probably had 4,000 customer contacts each month. The Help Desk itself receives over 3,000 calls per month. In addition, we provided a "walk-in" service for assistance, shared equipment, company-logo diskettes, and training material. One out of forty, or 2 1/2%, need to result in a postcard to meet the goal. |
The goal for number of postcards received probably could be set objectively, from an analysis of variance, while preparing the control charts. The number comes from the question of how many cards are required to claim the sample size yields a valid sample. From previous experience, in preparing p-charts for fraction defective, 100 cards per month yields acceptable-looking upper and lower control limits. The control limits broaden quickly if the number of cards falls below 100. This indicates to me that 100 is about the minimum number of cards needed.
| |
| Getting the Postcard in Front of the CustomerThe Help Desk had a dispenser for paper postcards. Every supervisory person also had a dispenser, or a ready supply of cards. The dispenser was for the use of customers and staff. It was replenished daily, and a paper postcard was counted as being "out" when it was removed from the dispenser. |
Support staff carried a supply of cards when they visited
customer locations.
The standard procedure was to leave a postcard behind after every
visit to a customer desk, or other location.
| |
The postcard was included in hard copy newsletters which were published
about three times per year.
The newsletter was supposed to be quarterly, but we could never
dedicate the resources to make such a production schedule.
| |
We kept the electronic postcards in front of the customer
with system logon messages, by imbedding a reminder in system
screens, by including a reminder in every piece of electronic
mail, and by reminding customers verbally.
| |
Every person who had contact with a customer
was expected to request a postcard when closing.
We found that we had to continually remind customers to send
in the postcards.
If we stopped reminding customers,
the number of postcards sent would decrease rapidly.
| |
| Instructions to Support Staff |
The instructions given to the support staff for
the Postcard program were:
|
How are we doing? ... Please let us know ... ... type "postcard" at the PROFS Main Menu
|
| The Manager's Role in the Customer Satisfaction Po |
The manager's role is customer satisfaction is crucial.
My staff took their cues from me,
in deciding how much emphasis to put on this program.
Telling them that customer satisfaction was important had no real lasting effect.
Only when I lived and breathed customer satisfaction every day would they come to believe I meant it.
| |
| Daily
|
These are the things that I did daily, as the Manager, End User Support.
| |
| |
| |
| I also had frequent customer contact. The comment above about BART, above, is not facetious. It was quite common for a disgruntled computer user to tell me their frustrations on the way home. Sometimes customers would call me just to complain. I would often help them frame their frustrations into an action-able postcard comment. We would accept a postcard about any aspect of computer services, whether it pertained to my department or not. If I received a comment about another department, I would send it to the manager of that department. Sometimes a manager would resent my appearance with a problem for them. However, I generally got pretty good cooperation from the other managers. |
Completed electronic postcard.
| |
| |
| The work sheet was accessible to any End User Support staff. However, I don't have the sense that many of them ever regularly looked at it on the LAN. As you will see below, they had other effective ways to find out what cards were being received. |
| Cards that were complimentary to a staff member were acknowledged immediately, with a comment like, "Thanks for the happy faces for Louis, I'll pass your comments on to him." Cards that were complaints were also acknowledged. The kind of acknowledgment depended on the type of complaint and what kind of organizational response we could make. Postcards are most commonly acknowledged through electronic mail. Sometimes the acknowledgment would be by telephone, but the telephone is usually reserved for times more information is needed. An example might be the case of a somewhat vague complaint, in which case clarification is sought by telephone. |
| I sometimes received complimentary postcards about people who did not work for me. Those I also forwarded to the person and their manager. (I also logged them and took credit for the happy faces.) |
|
|
| |
| |
| WeeklyThe End User Support staff meeting was every Tuesday at 11:00 am. The entire organization, including the Help Desk, was shut down for an hour to an hour and a half. (We were, however, immediately available for any user-described emergency.) |
The first order of business at the staff meeting was the current postcard summary.
I brought copies of the current worksheet,
like that shown in the Appendix.
As each person entered the room, they would pick up a copy, sit down,
and read each card.
The room was silent for a few minutes as people read the worksheet.
Then we would begin to discuss the postcards.
Someone would giggle; another would call out, "I like number 12," etc.
Someone else might say, "What happened on number 26?"
One-by-one, nearly every postcard would be discussed.
| |
Very often a postcard would be the starting point for a discussion about
a problem we were having.
I tried to let the table have a good discussion of the issues,
and then guide the discussion to synthesize a solution.
| |
Sometimes the postcards would consume only a few minutes of a staff meeting.
Other times nearly the whole meeting revolved around a postcard,
or a larger issue that was represented in a customer's comments.
I think the key point is:
we heard the voice of our customer as the first order of business
in every staff meeting.
| |
| MonthlyAt the end of every month, a final worksheet was prepared and distributed to every staff member. Summary statistics were transferred to a spreadsheet and several charts were prepared. |
Chart of Postcards Sent Out and Returned.
The first chart shows the number of
postcards sent out versus returned.
Some explanation is required.
The number of postcards sent out is the number of paper postcards.
The number of postcards returned is the sum of paper and electronic postcards.
The percentage of return is the number returned divided by the number sent out.
Since we sent out thousands of paper postcards in newsletters,
this number was quite low.
| |
As you can see, the number of postcards fell off the last few months.
This was because the group was being outsourced,
along with the rest of the Information Services department.
The outsourcer was expressing no interest in the postcard program,
and it was hard to maintain any enthusiasm for the customer satisfaction
program when people knew they would be losing their jobs.
However, the postcard customer satisfaction program was restarted
a year later when a critical customer satisfaction problem
developed with the outsourced Help Desk.
| |
Another thing that can be seen in this chart, though,
is that the volume of cards would often drop off.
I used this chart mostly to track the volume, and to have something to
point to when I was encouraging the staff to get more postcards flowing.
| | Postcard responses by month.
The next chart shows the number of happy faces, versus neutral faces and sad faces.
| |
The purpose of this chart is to show that the overwhelming proportion of
responses were happy faces.
I did this chart specifically for my Vice President,
and called it to his attention each month.
| |
As you can see in this chart, the percentage of Happy Faces stayed at about 90% or more for 18 months.
I think this is about the maximum that can be expected over a long
period of time, and I also think 90% customer satisfaction is about the
minimum level to be considered "good."
| |
Percent of Sad Faces by Reason.
The next chart begins to focus on the problem areas.
Specifically it displays the causes of sad faces
by the question to which to sad faces were marked.
Often a customer was either happy or unhappy,
in which cases they marked three happy faces or three sad faces,
respectively.
Sometimes, though, they would discriminate among the causes for their
satisfaction or dissatisfaction.
As a general rule, we got most sad faces for not being timely,
and least sad faces for not being friendly and helpful.
The last few months were anomalous because of the outsourcing.
| |
u-Chart for Sad Faces per Card.
The last chart is an attempt to be rigorous about the measurement of satisfaction.
We defined a postcard as a product which could have zero to three defects,
i.e., sad faces.
We then prepared a u-Chart for defects per product.
| |
I think this was the only statistical control chart ever prepared for
any process in the Information Services department.
The chart shows the process was out of control during the period
of April to November, 1993 because there are 8 points below the center line.
Actually, we had made some process improvements and should have recalculated
control limits from April 1993 on.
| |
Twice during this time period, we also categorized all comments received
by topic.
We then prepared pareto charts indicating causes of dissatisfaction
and requests for additional or improved services.
The primary conclusion was that our customers most wanted us to be
available when they needed us.
We addressed that need by combining our three help desks into one.
This permitted a sufficient depth of staff to keep the Help Desk fully staffed,
Monday through Friday, 8:00 am to 5:00 pm.
We also installed a multiple-level ACD telephone system.
This allowed the Help Desk to route telephone calls into other staff
areas when the volume of calls exceeded the capacity of those
working the Help Desk at that moment.
| |
| CommentsThere are a few general comments that I want to make about help desk customer satisfaction and this program. |
It is my belief that most Help Desks provide good service to
the majority of their customers.
I have two data points to support this belief.
Both times I started this program, I was led to expect extremely
poor results.
However, the opposite occurred.
The first time I started this program, we received a 79% happy face rating.
The second time it was 74%.
Based on this data I believe most help desks provide valuable assistance
to their customers, and this assistance is appreciated.
| |
Let me quickly add that
there often remains much room for improvement, however.
In both cases improvements were needed in the telephone systems,
staff skills and procedures.
I believe a Help Desk customer satisfaction rating should be above 90%.
| |
I believe the bad reputation Help Desks receive is from a small proportion
of customers who will be unhappy at any level of service.
These people tend to be vocal and often have access to senior executives.
A corporate folklore that the Help Desk is lousy can be quickly established
and is hard to counter until you have real satisfaction data.
| |
In my first experience with this program,
my own Vice President boldly predicted that we would be
lucky to get a single happy face.
When we received 77 happy faces and only 10 sad faces
in our first month, July 1991,
he had to admit his surprise publically.
He became an ally instead of an adversary.
| |
Regardless of our early success,
we still had a lot of work to do.
We combined three help desks into a single facility,
installed a new telephone system and problem management system,
cross-trained our staff, and built a new set of procedures.
Over time our support improvements resulted in 95% happy faces
being the norm.
| |
In the second time that I implemented this program,
customer dissatisfaction with the help desk had become
a critical issue at executive committee meetings.
There was, of course, no data about customer satisfaction;
only anecdotes from several (truly only 2 or 3) vocal people who,
I think, would never be satisfied with any level of service.
When some real data became available,
it became apparent to this executive committee that
many people were satisfied with the support received.
Help Desk customer satisfaction ceased to be an issue at the executive level.
To be honest, this support organization still had a lot of improvements
to make.
However, having some real data showing satisfied customers quickly
got the executive committee off their back, and let them make
improvements in an orderly program.
| |
There are a couple of issues that executives seem to raise about this system.
In my mind the common thread in these issues is executive fear
of the unknown or what they perceive to be uncontrollable.
| |
Every executive has resisted the implementation of this system.
My sense is that they fear asking the customers about satisfaction
will stir up a
maelstrom
of dissatisfaction.
This is unfortunate.
If the customers are truly so dissatisfied,
it can't be much worse by having an accurate count of the
dissatisfaction along with the name and phone number of
every dissatisfied customer.
(You can, at least, call them, or ... take them out to lunch!)
The facts, though, point to the opposite case.
Most customers like the services they are getting and
they are willing to say so.
| |
Having a lot of satisfied customers to balance out the vocal
dissatisfied customers puts some sanity back into the support business.
It puts you in the position of helping senior management achieve some
perspective about the complaints they hear.
It also allows you to manage the chronically dissatisfied.
They can put their dissatisfaction into words on the postcard,
or find themselves marginalized by the overwhelming proportion
of satisfied users.
| |
Occasionally an executive has worried about publishing the comments of
customers. Might the customers object to seeing themselves in print?
So far, I have not had a complaint about this.
The approach I take is that someone owns the words they type.
If it's not acceptable for public consumption,
then it's better left unsaid.
I'm not personally brilliant about this.
I take my inspiration from my Internet Service Provider,
the Well, which prominently displays this phrase,
"You own your own words"
on its logon screen.
| |
The last topic that executives have worried about is manipulation.
A support staff member might somehow manipulate the customers,
to skew the results.
Customers might also try to manipulate the results by submitting
multiple cards.
Of course they will.
Someone is going to try it, and succeed ... for a time.
There is an example of this in the
Appendix.
| |
Several years ago we bought a new washer and dryer from the large
department store that everyone uses for this purpose.
When they delivered and set up the equipment, I signed some papers.
One of papers indicated that I had received the equipment.
However, there was another paper the delivery men wanted me to sign.
They had placed it on the clipboard and covered all but the
signature line by some other papers.
When I asked them what it was, their answer was that it was just a paper
they needed to fill out later; sufficiently vague
to raise my suspicion.
The paper was in fact a customer satisfaction survey
which I was supposed to complete.
The line they wanted me to sign stated that I had filled out the survey.
| |
So, yes, people will try to manipulate the system.
Any manager who has any understanding of their staff will
instantly know, however, which of their staff is likely to try it,
and can apply the necessary correction.
Attempts by the customers to manipulate the system are also very easy
to spot.
| |
So, that's my system for measuring customer satisfaction.
I don't think it is the intellectual property of anyone
because the concepts were borrowed in the first place,
and the organizations that have used it either do not exist
or are no longer using it.
| |
If you would like to send feedback, correspond or consult, you may reach me at my personal address:
tom@schweich.com or my business address:
schweich@us.ibm.com. Other Help Desk information may be found at the Help Desk FAQ.
| |
| AppendixThe following document is real data from part of the first full month of the second time this program was implemented. All the names and telephone numbers have been changed, but the remainder of the comments have not. |
Between the date and the text of the customer comment are symbols
for which faces the customer marked.
A "+" is a happy face, a "0" is a neutral face, and a "-" is a sad face.
Thus the symbols "(+-0)" says, "I'm happy about getting what I needed,
sad about how long I had to wait, and neutral about your helpfulness and friendliness.
| "How Are We Doing?" PostCard Analysis Work SheetTime Period Covered: From: July 1, 1995 to July 18 1995, number of working days: 10 |
| Cumulative at End of Previous Period | Cumulative at End of This Period | This Period | Approximate Return Percentage | |
| Postcards Out | 0 | 400 | 400 | |
| Postcards In | 0 | 1 | 1 | 0.25% |
| Electronic Postcards In | 4 | 53 | 49 |
| Happy | So-So | Sad | Row Totals | |
| Did you get the assistance you needed? | 21 | 2 | 4 | 27 |
| Was it delivered in a timely fashion? | 16 | 4 | 6 | 26 |
| Was the service friendly and helpful? | 20 | 2 | 2 | 24 |
| Column Totals | 57 74% | 8 10% | 12 16% | 77 |
|
New Postcards This Month
| |
| If you have a question or a comment you may write to me at: tas4@schweich.com I sometimes post interesting questions in my FAQ, but I never disclose your full name or address. | Date and time this article was prepared: 5/8/2008 9:42:48 AM |