Wiley Inside Information Making Sense of Marketing Data.pdf

Inside Information

Making Sense of Marketing Data

D.V.L. SMITH & J.H. FLETCHER

JOHN WILEY & SONS, LTD

Chichester · New York · Weinheim · Brisbane · Singapore · Toronto

Copyright © 2001 by John Wiley & Sons, Ltd

Baffins Lane, Chichester,

West Sussex, PO19 1UD, England

National 01243 779777

International (+44) 1243 779777

e-mail (for orders and customer service enquiries): [email protected]

Visit our Home Page on http://www.wiley.co.uk or http://www.wiley.com

All Rights Reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, in

any form or by any means, electronic, mechanical, including uploading, downloading, printing, recording or

otherwise, except as permitted under the fair dealing provisions of the Copyright, Designs and Patents Act 1988,

or under the terms of a licence issued by the Copyright Licensing Agency, 90 Tottenham Court Road, London,

W1P 9HE, UK, without the permission in writing of the Publisher. Requests to the Publisher should be addressed

to the Permissions Department, John Wiley & Sons, Ltd, Baffins Lane, Chichester, West Sussex, PO19 1UD, UK

or e-mailed to [email protected] or faxed to (+44) 1243 770571.

Other Wiley Editorial Offices

John Wiley & Sons, Inc., 605 Third Avenue,

New York, NY 10158-0012, USA

WILEY-VCH Verlag GmbH

Pappelallee 3, D-69469 Weinheim, Germany

John Wiley & Sons Australia, Ltd, 33 Park Road, Milton,

Queensland 4064, Australia

John Wiley & Sons (Canada) Ltd, 22 Worcester Road

Rexdale, Ontario, M9W 1L1, Canada

John Wiley & Sons (Asia) Pte Ltd, 2 Clementi Loop #02-01,

Jin Xing Distripark, Singapore 129809

British Library Cataloguing in Publication Data

A catalogue record for this book is available from the British Library

This title is also available in print as ISBN 0 471 49543 3 (Cloth)

Typeset in 11/15 pt Garamond by Mayhew Typesetting, Rhayader, Powys

Contents

Foreword by Andrew McIntosh vii

Preface x

Acknowledgements xii

1 Mastering Twenty-First-Century Information 1

The information paradox 2

Twenty-®rst-century information craft skills 4

A new holistic way of evaluating information 7

About this book 8

2 Acquiring Effective Information Habits 11

The seven pillars of information wisdom 13

Understanding the evidence jigsaw 24

Developing a personal information strategy 28

Robustness checks 33

Getting to the storyline 42

Acting on information 48

3 A Primer in Qualitative Evidence 51

Softer evidence here to stay 53

Making `faith' decisions 54

The quality of qualitative research 63

Understanding the overall analysis approach adopted 74

Making judgements and decisions from qualitative evidence 79

The safety of qualitative evidence for decision-making: a

seven-point checklist 83

4 Understanding Survey Data 85

A recap on the key characteristics of survey-based research 87

Seven key checks 93

5 Designing Actionable Research 145

Step 1: is formal research the answer? 146

Step 2: de®ning and re®ning the problem 149

Step 3: start at the end: clarify the decisions to be made 153

Step 4: pinpointing the information gaps 158

Step 5: developing a ®tness-to-purpose design 158

Step 6: deciding on the research design 165

Step 7: choosing an agency 167

Appendix A: An overview of the market research `toolbag' 168

Appendix B: A ®ve-step guide to writing a market research

brief 171

6 Holistic Data Analysis 177

The key principles of holistic data analysis 178

The main techniques underpinning holistic data analysis 180

Putting it all together: holistic analysis summarised 183

Ten-step guide to holistic data analysis 185

7 Information-Based Decision-Making 219

Decision-making cultures 221

Organisational decision mine®elds 222

Why we ®nd decision-making dif®cult 226

Applying information to decision-making 229

Decision-making frameworks 232

Implementing marketing decisions 240

Good practice design and implementation guide 247

Bibliography 253

Index 255

vi Contents

Foreword

Everybody knows how to distrust statistical information ± `lies, damn

lies, and statistics'. And a few people even know how misleading popular

conceptions of probability are, to the extent that some can give the

counter-intuitive, but correct, answer to the question `what is the

probability that two children in a class of 30 will share a birthday?' ± a

much higher probability than most people think.

But how many of the hundreds of thousands of people who use

survey data in their work or lives, let alone how many who read survey

®ndings in the media, have had any serious training in their analysis or

interpretation? It is precisely because there is much more to the

understanding and use of survey research than statistical formulae, that

this book is necessary.

A very public example in recent years has been the debate on the use

of focus groups by political parties in the formulation and presentation of

policy. This raises two kinds of issue, each addressed by Smith and

Fletcher in this challenging book.

First, the issue addressed by Chapter three of how qualitative research

is carried out, when it is appropriate (and when not), and what pre￾cautions should be taken in the interpretation of qualitative evidence.

Historically, most qualitative research has been widely ± even mainly ±

used as part of the problem de®nition stage of a research project. Focus

groups, or as they used to be called, discussion groups, were used to test

how comprehensible ideas, language, or images, would be if used in a

quantitative survey. Even motivation research, originally conducted by

psychologists seeking to explore unexpressed motivation rather than

conscious attitudes or behaviour, would commonly be reported as part of

a study embracing both qualitative and quantitative data.

But the public image of focus groups, mainly triggered by political

parties and their spin-doctors, has been as a short-cut to understanding of

public opinion, not complementing but replacing the measurement of

opinion and behaviour on political issues, among signi®cant groups of

the population, which can only be achieved by quantitative surveys. It is

not just the media who over-simplify an issue of public concern: it is clear

from their own accounts that those advising political parties in Britain

have indeed misused focus groups, and neglected the proper use of

survey research.

Dick Morris, President Clinton's spin-doctor, did not rely on focus

groups to give his tactical advice to the presidential candidate in 1992, but

commissioned 800 telephone interviews every night during the campaign.

Not cheap, but effective. Spin-doctors to British political parties would do

well to follow that example. Smith and Fletcher help to explain why.

Second, the issue of how research ®ndings are to be used in making

business decisions, which has dominated business texts on marketing

research since Green and Tull. Again, the focus group controversy

illuminates the issue. Too often, public reporting of research for political

parties, often fed by leaks of internal documents, gives the impression

that parties wish to use research, not to guide them in the presentation of

policy, but as a replacement for political, social and economic analysis in

the formulation of policy itself.

Perhaps they do: perhaps popularism without principle is gaining

ground in our political life. But as a politician, I profoundly hope not;

and as a survey researcher, both in business and in public policy, I

deplore such distortion of our discipline. Survey research should assist,

but never seek to usurp, the role of decision-making based on proper

business or policy objectives, and in possession of all the relevant facts.

Again, this book provides practical illustrations of the dangers of

misinterpretation of research ®ndings ± what the authors call the `craft

skills necessary to scan, gut, and action information'. Textbooks of

market research already expound many of the rules of interpretation ±

caution when dealing with small sub-samples, re-percentaging when

bases change (or better, avoiding changing bases), and so on: the authors

rightly rehearse these rules. But in emphasising the importance of

inductive reasoning, in what they call `the seven pillars of information

wisdom' they address issues which are well known to those experienced

in the craft, but which have not before, to my knowledge, been suf®-

ciently expounded in print.

viii Foreword

It has always seemed to me that there are two dif®cult problems for

those who ®nd themselves required to commission research, or to make

business or policy decisions using research ®ndings.

The ®rst is to remember that commissioning original research is a last

resort. If effective ways can be found to use business or of®cial statistics,

or to re-examine or re-interpret existing research data, then that will be

preferable to commissioning original research, which runs the twin risks

of costing more than the bene®t to be derived from it, or of being carried

out on an inadequate budget, with the potential for untrustworthy results.

Second ± and there are constant reminders of this in the book ±

survey research essentially provides the customer viewpoint, to counter￾balance the producer bias which is inherent in business life. It does not

mean that the customer is always right.

To give merely one example: for many years, economic and business

researchers both in the UK and in the US devoted considerable resources

and great skill to analysing the validity and reliability of anticipations data

as a tool for forecasting consumer purchases. They took into account the

obvious psychological truth that buying intentions will become less ®rm

and actionable the further into the future they go; they allowed for the fact

that large purchases, such as home or cars, are more likely to be anticipated

than purchases of, for example, small electrical appliances; they even,

eventually, caught up with the fact that anticipation of replacement

purchases will follow a different pattern from ®rst-time buying.

But what they failed to do was to recognise that other factors, them￾selves capable of forecasting, but necessarily unknown to the consumer at

the time of interview, would in¯uence consumer buying intentions.

Without the best available forecast of trends in in¯ation, in consumer

disposable income, in product development and pricing, anticipations data

are almost certain to be misleading. Here too is a lesson from market

research for public policy, and indeed for political polling.

If this book can help users of survey research, whether they be infor￾mation professionals, research practitioners, or more generally people in

business or public life, with the insights necessary to understand and

bene®t from the skills of the researcher, it will have well justi®ed itself. It

is a worthy objective.

Andrew McIntosh

Foreword ix

Preface

In this book the authors argue that we need to develop a new infor￾mation paradigm that provides data users and suppliers with the fresh

insights and practical hands-on information skills and competencies

needed to cope with the `information explosion'. We are aware that the

term paradigm is a much overused word. But we believe that information

professionals ± most notably market researchers ± urgently need to put

into the public domain a clear set of guiding principles about how

they are currently tackling the world of marketing information in the

twenty-®rst century. The authors ± both of whom are practising market

researchers ± believe that this issue places the market research industry at

a crossroads. The industry could stumble on pretending that many of the

principles and concepts spelt out in existing market research textbooks

still apply to the way they now operate. Or, as we believe, they could

seize this golden opportunity to articulate the way that New Market

Research really `works'. This would explain how, increasingly, we are

relying on more holistic analysis techniques than has been the case in the

past. In this new Millennium market researchers must learn how to

assemble a jigsaw of imperfect evidence using the skills of the `bricoleur',

rather than falling back on some of the more methodologically pure, but

now rather stale, approaches of the past. In short, we outline what

market research practitioners have been doing behind closed doors ± but

not articulating to the world ± for a number of years. So we are not

inventing new analytic techniques for the ®rst time. But the ideas this

book contains are new in the sense that this is one of the ®rst books that

make explicit what may be termed the hidden market research practi￾tioners' paradigm. We believe that unless market research practitioners,

and other information specialists, now start to articulate and make

explicit many of their day-to-day data analysis practices, then we will not

have a platform upon which to realistically debate the techniques being

used to make sense of marketing data. It is a debate that is much needed

if we are to develop the appropriate training for prospective information

professionals.

Preface xi

Acknowledgements

The authors wish to thank Jo Smith and Andy Dexter for their helpful

comments on the structure of the book. In addition, we are indebted to

Phyllis Vangelder for her contribution to the editing process. But we are

most indebted to Chris Rooke and Sandra Mead for the professionalism

that they have demonstrated in typing various drafts of the book. Sandra

needs a special mention for all the dedication shown in painstakingly

working on the ®nal stages of the preparation of the book.

CHAPTER1

Mastering Twenty￾First-Century

Information

Overview

This chapter:

· introduces the view that new analysis skills are needed to cope with

modern twenty-®rst-century business information

· explains that these new skills require information to be analysed in an

holistic way

· reviews the way this holistic approach is characterised.

ONE

Mastering Twenty￾First-Century

Information

`Where is the wisdom we have lost in knowledge? Where is the knowledge

we have lost in information?' ± T.S. Eliot

This book is about how to make sense of the data and evidence that is

arriving at us from all directions in this the `information era'. Some might

think that the information era is already at its zenith. But the real infor￾mation explosion is still a little way off. True revolutions are the result of

changes in infrastructures, rather than just the arrival of a new invention.

Thus, it was not the invention of the car that revolutionised transport, but

the creation of our road network. Similarly, it was not the ability to build

washing machines and other electrical labour-saving devices that changed

household life, but the setting up of the National Electricity Grid. And so it

is with the information era. It is not the invention of the personal computer

that lies at the heart of the new information era, but the creation of the

Internet distribution channel that allows information to ¯ow from business

to business, home to home and so on. And because this infrastructure is

not yet quite in place ± not all businesses are `wired' with each other and

not all homes are interconnected ± the full information explosion has still

not hit us. Just how far away this will be is dif®cult to judge. In the United

Kingdom the Prime Minister has announced that the target is to ensure that

everybody has access to the Internet by the year 2005.

The information paradox

The arrival of the information era brings with it an information paradox.

One might have hoped that, given the busy time-pressured lives we lead

and the need to master increasing amounts of information, we could now

spend less time deciding on the robustness of each piece of evidence

with which we are presented. But this is not the case: this is the paradox.

At the very time when we have so much more information, we also have

to spend more, not less, time delving into exactly what this information is

trying to tell us. This is because a feature of the modern business

information world is the emergence of a wide range of less than `perfect'

information drawn from a myriad of comparatively unknown information

sources. In the past, decision-makers in the world of marketing have

been able to rely on a small number of reasonably methodologically

sound sources of marketing data. But today, increasingly, we are faced

with more information, much of which will have a question mark over its

robustness.

In some ways, the arrival of concepts such as Knowledge Manage￾ment is helping to keep us on top of this new array of marketing

information. But this ± and the hope that the computer technology will

come to our rescue and help us better sort, classify and even `interpret'

information ± only goes so far. At the heart of the challenge facing us is

recognition that we need a new set of twenty-®rst-century information

competencies in order to handle this new world of multi-source, `imper￾fect' data. There is talk of a high proportion of the workforce now being

`knowledge workers', but comparatively little new thinking on how to

help these knowledge workers make sense of the new sources of

business information. It seems that an assumption is made that indi￾viduals will, by osmosis, learn to dissect and absorb all the new

information swirling around and use this for effective decision-making.

But in this book we argue that these knowledge workers are going

to require a new set of twenty-®rst-century `information skills and

competencies'.

We should stress that when we talk about applying information to

decision-making, we are de®ning a decision as being a `choice made

between alternatives'. (The word `decision' is derived from a word

meaning `to cut'.) And given this de®nition of a `decision', in this book

we will not be looking just at the way information is applied to big

strategic decisions about the overall direction of an organisation, but also

at the way in which information is applied to more tactically focused,

day-to-day decisions.

Inside Information 3

Twenty-®rst-century information craft skills

It seems to be the case that if someone has successfully negotiated the

educational system, then it is assumed that they will have automatically

acquired the key craft skills necessary to `scan', `gut', and `action' infor￾mation. But the majority of people in business and commerce ± notwith￾standing the prowess they may have demonstrated in their chosen

academic discipline ± still need speci®c, practical guidance on how effec￾tively to process and action modern marketing and business information

to maximum competitive advantage. Speci®cally, we believe that there are

®ve key skill areas that new entrants into marketing must learn if they are

effectively to master the new world of marketing information.

· The ability to instantly classify and reduce incoming information. A

clear difference between the current marketing environment and that

of only 10 years ago is the need for practitioners to be able to make

decisions quickly about what information to accept, reject and store.

So, in this book, we will be providing a series of practical tips to help

the reader keep on top of the sheer volume of incoming marketing

information.

· Getting underneath the evidence. In today's marketing environment it

is important to understand the strengths and limitations of incoming

evidence from all angles. This means getting behind, and underneath,

the data to identify any `sources of error' that might have implications

for their subsequent interpretation. This is an approach that squares

with those who argue for data to be analysed in an holistic, rather than

a solely statistical way. Here, by `error' we do not mean a mistake, but

any feature of the research process that may have introduced some

form of `bias' ± something that takes us away from the `truth'. This

softer (more qualitative) assessment of data provides the platform for

the subsequent, more statistically-based, interrogations of the data. In

this book we will be providing the reader with a number of insights

into what questions to ask about the origins of different types of

evidence. In short, we will give the reader the skills needed to check

out the `full service history' of incoming data.

· Embracing intuition. Business history abounds with stories of

individuals whose success has been founded on sparks of dazzling

4 Mastering Twenty-First-Century Information

`intuition'. This has been de®ned by Jung as the `perception of the

possibilities inherent in a situation' and Spinoza claimed that intuition

was the `royal road to truth'. And there are numerous captains of

industry who will testify that the hard taskmasters of logic and

rigorous analysis were only part of how they made `big' decisions.

Richard Branson tells us that his decision to go into the airline

business in the mid-1980s was `a move which in pure economic terms

everybody thought was mad, including my closest friends, but it was

something to which I felt I could bring something that others were not

bringing'. Similarly, Sir David Simon, ex-boss of BP, is on record as

saying: `you don't have to discuss things. You can sense them. The

``tingle'' is as important as the intellect'. Thus, in this book we will be

arguing strongly that the market research and market intelligence

process needs large doses of intuition in order to realise their true

potential.

Psychologists tell us that we are conscious of only a small part of

what we know, pointing out that intuition allows us to draw on our

unconscious knowledge ± everything that one has experienced or

learned, either consciously or subliminally. But this does not make

intuition a `mystical' phenomenon. If we arrive at a solution by

intuition this simply means that we have got there without consciously

knowing exactly how we did it. It does not mean that we have not

been following a `process'. It means that things are happening

automatically, at high speed, without conscious thought, in a dif®cult￾to-de®ne process. A Grand Chess Master considers far fewer alterna￾tives when making a move than an amateur player. The Chess Grand

Master has incorporated into his/her implicit memory, knowledge

of the probability of the success or failure of different moves. This

provides a rich reservoir of knowledge which means the Grand Master

does not formally have to search through all the alternative moves.

The Grand Master can quickly eliminate the unworkable, and focus

only on the potentially winning moves. For this reason, intuition has

been called compressed expertise. Of course, the idea of attempting

formally to codify and make explicit `tacit intuitive knowledge' is a

paradox. But, in this book, the authors ± in pursuing their belief in

the value of the `holistic' analysis of data ± provide various frame￾works that help ensure that in any decision-making process intuitive

Inside Information 5