Tài liệu User Experience Re-Mastered Your Guide to Getting the Right Design- P3 pdf

86 User Experience Re-Mastered: Your Guide to Getting the Right Design

me what you are thinking as you are grouping the cards. If you go quiet, I will

prompt you for feedback.”

Whenever participants make a change to a card, we strongly encourage them to

tell us about it. It helps us to understand why they are making the change. In a

group session, it offers us the opportunity to discuss the change with the group.

We typically ask questions like

John just made a good point. He refers to a “travel reservation” as a “travel

booking.” Does anyone else call it that?

or

Jane noticed that “couples-only resorts” is missing. Does anyone else book

“couples-only resorts?”

If anyone nods in agreement, we ask him/her to discuss the issue. We then ask

all the participants who agree to make the same change to their card(s). Par￾ticipants may not think to make a change until it is brought to their attention,

otherwise they may believe they are the only ones who feel a certain way and

do not want to be “different.” Encouraging the discussion helps us to decide

whether an issue is pervasive or limited to only one individual.

Participants typically make terminology and defi nition changes while they are

reviewing the cards. They may also notice objects that do not belong and remove

them during the review process. Most often, adding missing cards

and deleting cards that do not belong are not done until

the sorting stage – as participants begin to organize the

information.

Labeling Groups

Once the sorting is complete, the participants

need to name each of the groups. Give the fol￾lowing instructions:

Now I would like for you to name each of your

groups. How would you describe the cards in

each of these piles? You can use a single word,

phrase, or sentence. Please write the name of

each group on one of the blank cards and place

it on top of the group. Once you have fi nished,

please staple each group together, or if it is too

large to staple, use a rubber band. Finally, place all of

your bound groups in the envelope provided.

DATA ANALYSIS AND INTERPRETATION

There are several ways to analyze the plethora of data you will collect in

a card sort exercise. We describe here how to analyze the data via pro￾grams designed specifically for card sort analysis as well as with statistical

TIP

We prefer to

staple the groups

together because we do not

want cards falling out. If your

cards get mixed with others, your

data will be ruined; so make sure

your groups are secured and that each

participant’s groups remain separate!

We mark each envelope with the

participant’s number and seal it until

it is time to analyze the data. This

prevents cards from being

confused between

participants.

Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.

Card Sorting CHAPTER 3 87

packages (e.g., SPSS, SAS, STATISTICA ™ ) and spreadsheets. We also show

how to analyze data that computer programs cannot handle. Finally, we

walk you through an example to demonstrate how to interpret the results of

your study.

When testing a small number of participants (four or less) and a limited num￾ber of cards, some evaluators simply “eyeball” the card groupings. This is not

precise and can quickly become unmanageable when the number of partici￾pants increases. Cluster analysis allows you to quantify the data by calculat￾ing the strength of the perceived relationships between pairs of cards, based

on the frequency with which members of each possible pair appear together.

In other words, how frequently did participants pair two cards together in the

same group? The results are usually presented in a tree diagram or dendrogram

(see Figs 3.4 and 3.5for two examples). This presents the distance between

pairs of objects, with 0.00 being closest and 1.00 being the maximum distance.

A distance of 1.00 means that none of the participants paired the two particu￾lar cards together; whereas 0.00 means that every participant paired those two

cards together.

FIGURE 3.4

Dendrogram for our

travel Web site using

EZCalc.

Books

Links to travel gear sites

Luggage

Travel games

Family friendly travel information

Currency

Languages

Tipping information

Featured destinations

Travel alerts

Travel deals

Weekly travel polls

Chat with travel agents

Chat with travelers

Post and read questions on bulletin boards

Rate destinations

Read reviews

(Average)

0.50 1.00

Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.

88 User Experience Re-Mastered: Your Guide to Getting the Right Design

Create a new message

Send current message

Attach file to a message

Spell-check current message

Reply to a message

Forward a message

Print a message

Get new messages

View next message

Delete a message

Save message to a file

Append message to a file

Create a new folder

Delete an existing folder

Rename an existing folder

View another folder

Overview of folders

Delete the trash folder

Move message between folders

Copy message between folders

Overview of messages in folder

0

2000

Single linkage Complete linkage 4000 6000 8000 10000 12000 14000 16000

18000

20000

22000

24000

26000

28000

FIGURE 3.5

Tree diagram of

WebCAT data analysis

for an e-mail system.

BRIEF DESCRIPTION OF HOW PROGRAMS

CLUSTER ITEMS

Cluster analysis can be complex, but we can describe it only briefl y here. To learn more

about it, refer to Aldenderfer and Blashfi eld (1984), Lewis (1991), or Romesburg (1984).

The actual math behind cluster analysis can vary a bit, but the technique used in most

computer programs is called the “amalgamation” method. Clustering begins with every

item being its own single-item cluster. Let’s continue with our travel example. Below are

eight items from a card sort:

Participants sort the items into groups. Then every item’s difference score with every

other item is computed (i.e., considered pair-by-pair). Those with the closest (smallest)

difference scores are then joined. The more participants who paired two items together,

Hotel reservation Airplane ticket Rental auto Rental drop-off

point

Frequent-guest

credit

Frequent-fl yer

miles

Rental pick-up

point

Featured

destinations

Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.

Card Sorting CHAPTER 3 89

the shorter the distance. However, not all the items are necessarily paired at this step. It

is entirely possible (and in fact most probable) that some or many items will not be joined

with anything until a later “round” or more than two items may be joined. So after Round 1,

you may have the following:

■ Hotel reservation and frequent-guest credit

■ Airplane ticket and frequent-fl yer miles

■ Rental auto, pick-up point, and drop-off point

■ Featured destinations

Now that you have several groups comprised of items, the question is “How do you con￾tinue to join clusters?” There are several different amalgamation (or linkage) rules available

to decide how groups should next be clustered, and some programs allow you to choose

the rule used. Below is a description of three common rules.

Single Linkage

If any members of the groups are very similar (i.e., small distance score because many

participants have sorted them together), the groups will be joined. So if “frequent-guest

credit” and “frequent-fl yer miles” are extremely similar, it does not matter how different

“hotel reservation” is from “airplane ticket” (see Round 1 groupings above); they will be

grouped in Round 2.

This method is commonly called the “nearest neighbor” method, because it takes only two

near neighbors to join both groups. Single linkage is useful for producing long strings of

loosely related clusters. It focuses on the similarities among groups.

Complete Linkage

This is effectively the opposite of single linkage. Complete linkage considers the most

dissimilar pair of items when determining whether to join groups. Therefore, it doesn’t mat￾ter how extremely similar “frequent-guest credit” and “frequent-fl yer miles” are; if “hotel

reservation” and “airplane ticket” are extremely dissimilar (because few participants sorted

them together), they will not be joined into the same cluster at this stage (see “Round 1”

groupings above).

Not surprisingly, this method is commonly called the “furthest neighbor” method, because

the joining rule considers the difference score of the most dissimilar (i.e., largest difference)

pairs. Complete linkage is useful for producing very tightly related groups.

Average Linkage

This method attempts to balance the two methods above by taking the average of the

difference scores for all the pairs when deciding whether groups should be joined. So

the difference in score between “frequent-guest credit” and “frequent-fl yer miles” may

be low (very similar), and the difference score of “hotel reservation” and “airplane ticket”

may be high but, when averaged, the overall difference score will be somewhere in the

middle (see Round 1 groupings above). Now the program will look at the averaged score

to decide whether “hotel reservation” and “frequent-guest credit” should be joined with

“airplane ticket” and “frequent-fl yer miles” or whether the fi rst group is closer to the third

group, “rental auto” and “rental pick-up point.”

Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.

90 User Experience Re-Mastered: Your Guide to Getting the Right Design

SUGGESTED RESOURCES FOR ADDITIONAL

READING

If you would like to learn more about cluster analysis, you can refer to:

■ Aldenderfer, M. S. & Blashfi eld, R. K. (1984). Cluster analysis. Sage

University paper series on quantitative applications in the social sciences,

No. 07-044. Beverly Hills, (CA): Sage Publications.

■ Lewis, S. (1991). Cluster analysis as a technique to guide interface design.

Journal of Man-Machine Studies, 10 , 267–280.

■ Romesburg, C. H. (1984). Cluster analysis for researchers. Belmont, (CA):

Lifetime Learning Publications (Wadsworth).

You can analyze the data from a card sort with a software program specifi cally

designed for card sorting or with any standard statistics package. We will describe

each of the programs available and why you would use it.

Analysis with a Card Sorting Program

■ At the time of publication, there are at least four programs available on

the Web that are designed specifi cally for analyzing card sort data: NIST’s

WebCAT® ( http://zing.ncsl.nist.gov/WebTools/WebCAT/overview.html )

■ WebSort ( http://www.websort.net/ )

■ CardZort/CardCluster ( http://condor.depaul.edu/~jtoro/cardzort/

cardzort.htm )

■ XSort ( http://www.xsortapp.com/ )

■ UserZoom ( http://www.userzoom.com/online-card-sorting-study )

■ OptimalSort ( http://www.optimalsort.com )

Data analysis using these tools has been found to be quicker and easier than

using manual methods (Zavod, Rickert & Brown, 2002).

Analysis with a Statistics Package

Statistical packages like SAS, SPSS, and STATISTICA are not as easy to use

as specialized card sort programs when analyzing card sort data; but when

you have over 100 cards in a sort, some packages cannot be used. A program

like SPSS is necessary, but any package that has cluster analysis capabilities

will do.

Analysis with a Spreadsheet Package

Most card sort programs have a maximum number of cards that they can

support. If you have a very large set of cards, a spreadsheet (e.g., Microsoft

Excel) can be used for analysis. The discussion of how to accomplish this

is complex and beyond the scope of this book. You can fi nd an excellent,

step-by-step description of analyzing the data with a spreadsheet tool at

http://www. boxesandarrows.com/view/analyzing_card_sort_results_with_a_

spreadsheet_ template .

Please purchase PDF Split-Merge on www.verypdf.com to remove this watermark.