MEASURING TELEPHONE EFFECTIVENESS
By Jon Hastings
Phone work is important to every successful organization. Yet one of
the areas that has been underdeveloped in recent years has been the
measurement of telephone effectiveness.
Most modern phone centers, which are technologically sophisticated,
are still very primitive in their communication measurement techniques. Certainly, they have machine-derived data about a rep’s telephone time on and off calls, as well as average call length. But they have very unreliable information about a given rep’s telephone effectiveness, or communicative quality.
The typical approach to assessing call quality is the use of check-
lists by supervisors. These lists are usually concerned with whether a
rep said something. This kind of “presence-or-absence” determination
is one of the most superficial ways of measuring any phenomenon.
Imagine watching your television news and hearing the meteorologist
declare, “Yes, we had weather today!” Of course, we had weather, but
what kind of weather was it, how does it compare to yesterday’s, and
what is the prediction for tomorrow?
To get this kind of quantitative and qualitative report requires precise language and a meaningful use of mathematics. To say, “It rained
today,” doesn’t tell us whether the rain was normal, above average, or
below average. Tell us it rained four inches today, and we have knowledge that is much more meaningful.
This analogy illustrates two of the requirements for measuring phone
work. To make it meaningful we need:
1. Clear categories
2. Quantification of data. There are several other criteria that
phone measures need, in order to be meaningful and to improve
telephone performance and customer outcomes.
3. Telephone communication categories need to be operationally de-
fined. For instance, in defining “articulation,” we don’t say it
is being understandable to listeners. We define it as “the full-
formation of words, so they are immediately comprehensible to a
listener of reasonable sensibilities.” This tells phone reps to
form words fully–in other words, it prescribes a course of behavior
that they can follow.
4. The categories need to be exhaustive. All meaningful events in
conversations must be captured by our measures.
5. Each category must be isomorphic. Categories need to mean one
thing, and only one thing, and not usurp another category’s
territory.
6. They need to be flexible, to allow for unusual telephone events.
7. They need to seem fair to phone workers.
8. The measures need to create inter-judge reliability. They need to
be usable by different managers with different experiential and
educational backgrounds. These managers need to score the same
conversations within a margin of difference of no more than 2
percent.
9. They need to be tested and proven across companies and across
industries.
10. They need to relate to customer values and improve customer sales
and satisfaction with greater reliability than the systems they
replace.
Jon Hastings is a telemarketing consultant based in Napa, California.
