Industrial applications of affective engineering

Junzo Watada · Hisao Shiizuka

Kun-Pyo Lee · Tsuyoshi Otani

Chee-Peng Lim Editors

Industrial

Applications

of Affective

Engineering

Industrial Applications of Affective Engineering

Junzo Watada · Hisao Shiizuka · Kun-Pyo Lee

Tsuyoshi Otani · Chee-Peng Lim

Editors

1 3

Industrial Applications

of Affective Engineering

Editors

Junzo Watada

Graduate School of Information

Production and Systems (IPS)

Waseda University

Kitakyushu

Fukuoka

Japan

Hisao Shiizuka

Kogakuin University

Tokyo

Japan

Kun-Pyo Lee

Department of Industrial Design

KAIST

Yusung-gu, Daejon-shi

Korea, Republic of South Korea

© Springer International Publishing Switzerland 2014

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part

of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,

recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or

information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar

methodology now known or hereafter developed. Exempted from this legal reservation are brief excerpts

in connection with reviews or scholarly analysis or material supplied specifically for the purpose of

being entered and executed on a computer system, for exclusive use by the purchaser of the work.

Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright

Law of the Publisher’s location, in its current version, and permission for use must always be obtained

from Springer. Permissions for use may be obtained through RightsLink at the Copyright Clearance

Center. Violations are liable to prosecution under the respective Copyright Law.

The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication

does not imply, even in the absence of a specific statement, that such names are exempt from the relevant

protective laws and regulations and therefore free for general use.

While the advice and information in this book are believed to be true and accurate at the date of

publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for

any errors or omissions that may be made. The publisher makes no warranty, express or implied, with

respect to the material contained herein.

Printed on acid-free paper

Springer is part of Springer Science+Business Media (www.springer.com)

ISBN 978-3-319-04797-3 ISBN 978-3-319-04798-0 (eBook)

DOI 10.1007/978-3-319-04798-0

Springer Cham Heidelberg New York Dordrecht London

Library of Congress Control Number: 2014935146

Tsuyoshi Otani

Department of Kansei Engineering

Faculty of Textile Science and

Technology

Shinshu University

Matsumoto

Nagano

Japan

Chee-Peng Lim

Centre for Intelligent Systems Research

Deakin University Geelong Waurn Ponds

Campus

Waurn Ponds, WA

Australia

v

The concept of emotion-based design, production, and marketing was considered

or dreamed in Japan in the 1950s. Since then, numerous methods have been stud￾ied to support humans in decision making, e.g., artificial neural networks and liv￾ing structures as well as artificial intelligence were investigated in the 1960s and

1970s. Various methodologies such as fuzzy sets were also proposed during the

same period. In parallel, in economics and marketing, people recognized that the

influence of human psychological behaviors and preferences plays a pivotal role

in decision making. Daniel Kahneman was awarded the Nobel Memorial Prize in

Economic Sciences for his contributions in this line of research. Indeed, emotional

experience is important in shaping our future behaviors.

Researchers started to investigate the practical use of human emotion and affec￾tive recognition in design in the 1980s. Such research was then extended from

design and questionnaire-based analyses to more physiological, biometrical, and

bio-measuremental experiments. Recently, studies on human emotions and affec￾tive senses have become prosperous. As highlighted in the book titled “Descartes’

Error: Emotion, Reason, and the Human Brain” by Antonio R. Ramasio, human

emotions play an important role in our thinking, and our rational behaviors are

greatly governed by emotions. Therefore, it is imperative to take human affective

feelings into consideration when we tackle problems in various domains.

In essence, affective (or Kansei) engineering is a scholarly field that focuses

on discovering and utilizing the value of human emotions for the development

or improvement of products or services, i.e., by incorporating human affective

feelings and impressions into the product or service design, development, and

delivery cycle. Indeed, the concept of affective engineering has become increas￾ingly important in the economic value chain. This is apparent when the Ministry

of Economy, Trade, and Industry of Japan launched the “Kansei Value Creation

Initiative,” and placed “Kansei” as the fourth value axis in the product or service

value chain. In other words, the affective value now joins the other three axes

(performance, reliability, and price) to help organizations maintain and improve

their competitiveness, i.e., producing products or services not only come with

high performance, high quality and reliability, and low price, but also with high

human affective values. As a result, affective values need to be embedded into

the whole economic value chain, ranging from upstream goods such as materials

Preface

vi Preface

and components to downstream goods such as finished products, services, and

contents.

This edited book stems from the First International Symposium on Affective

Engineering (ISAE2013) held at Kitakyushu, Japan, from 6 to 8 March, 2013.

ISAE2013 managed to attract numerous participants from different backgrounds,

which included academics, engineers, and practitioners to present and exchange

knowledge, experience, results, and information related to the broad aspects of

methodologies and applications of affective engineering. Following the success

of ISAE2013, participants have been invited to extend their research works and

contribute their findings as book chapters. Following a review process, a total of

22 chapters have been selected for inclusion in this edited book.

This book consists of two parts, i.e., methodology and application. Each part

has 11 chapters. In Part I (Methodology), attempts and efforts in the design and

development of a variety of methodologies related to affective engineering are pre￾sented. These include

• controlling the temperature and realizing a comfortable space based on human

brainwaves;

• proposing a new method that is useful for estimating human social emotions by

measuring micro body movements;

• developing a bi-level human migration model based on conjectural variations

equilibrium;

• evaluating signs in the artisanal sign-making area with the aim to improve the

level of customer satisfaction;

• defining the design subjects for creating attractive products and improving user

experience without relying on designers’ heuristics;

• devising an icon strategy to cultivate and attract consumers’ loyalty that helps a

company able to differentiate its products from others in the market;

• analyzing aesthetic experience as a Kansei element and a cognitive process in

product design and development;

• understanding Kawaii (an affective value) feelings pertaining to shapes, colors,

sizes, texture, and tactile sensation caused by product materials;

• evaluating the emotions for traditional Vietnamese clothes for women based on

computer vision and machine learning methods;

• assessing the sound effects in e-book reader software packages based on near￾infrared spectroscopy;

• investigating the relationship between cognitive style and webpage perception

from people with different cultures.

In Part II (Applications), the effectiveness and usefulness of a variety of affec￾tive engineering models and techniques in practical environments covering differ￾ent domains are presented. These include

• studying how the backrest structure affects the sitting comfort of a meeting

chair based on body pressures and contact areas between user and the chair;

• using affective values as a key factor to luxury brand building by focusing on

the Swiss watch industry;

Preface vii

• evaluating the transient signals of different button sounds by utilizing the wavelet

transform method;

• investigating the differences in the production processes of high-end garments

manufactured in Japan and Italy;

• adapting customers for online shopping of clothes by the ability to identify the

fabric used and the prior knowledge in fabric;

• administering self-report and physiological measures to understand color￾related emotions in different environments;

• gaining an insight into probable human-centered design trends by analyzing

movie scenes;

• analyzing the volatile compounds of white mother chrysanthemum flower on

sleep quality;

• examining emotional characteristics in response to various shades of white that

could help in designing white-based products;

• understanding the differences in skin physiology parameters and affective val￾ues in skincare products;

• devising a machine learning model to extract important information pertaining

to useful product features based on customers’ reviews.

We would like to express our sincere gratitude to all authors who have con￾tributed their works for inclusion in this book. We would also like to extend our

appreciation to the editorial team at Springer who have diligently helped in mak￾ing this book a reality. We hope that this book will serve as a useful reference for

readers to learn solid knowledge pertaining to different methodologies of affective

engineering and apply the acquired knowledge to undertake challenges in various

industrial domains.

31 December, 2013 Junzo Watada

Hisao Shiizuka

Kun-Pyo Lee

Tsuyoshi Otani

Chee-Peng Lim

ix

Contents

Part I Methodology

A Bio-Signal-Based Control Approach to Building

a Comfortable Space. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Junzo Watada, Chee Peng Lim and Yung-chin Hsiao

A New Social Emotion Estimating Method by Measuring

Micro-movement of Human Bust. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

Eui Chul Lee, Mincheol Whang, Deajune Ko, Sangin Park

and Sung-Teac Hwang

Affective Engineering in Application to Bi-Level Human

Migration Models. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

Vyacheslav V. Kalashnikov, Nataliya I. Kalashnykova,

Yazmín G. Acosta Sánchez and Vitaliy V. Kalashnikov Jr

Analysis and Evaluation of Business Signs Using Deviation Values. . . . . . 39

Masaaki Koyama, Yuki Takahashi and Hisao Shiizuka

Defining Design Subjects According to the Context

in Which Problems Occur. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

Masami Maekawa and Toshiki Yamaoka

Design Management Strategy: A Case Study of an Affective Product. . . . 67

Kana Sugimoto and Shin’ya Nagasawa

Kansei as a Function of Aesthetic Experience in Product Design. . . . . . . . 83

Oluwafemi S. Adelabu and Toshimasa Yamanaka

Kawaii Rules: Increasing Affective Value of Industrial Products. . . . . . . 97

Michiko Ohkura, Tsuyoshi Komatsu and Tetsuro Aoto

x Contents

Modeling Emotional Evaluation of Traditional Vietnamese Aodai

Clothes Based on Computer Vision and Machine Learning. . . . . . . . . . . . 111

Thang Cao, Hung T. Nguyen, Hien M. Nguyen and Yukinobu Hoshino

Near-Infrared Spectroscopy (NIRS) Analysis of Emotion

When Reading e-Books with Sound Effects. . . . . . . . . . . . . . . . . . . . . . . . . 123

Akira Nagai, Eric W. Cooper and Katsuari Kamei

The Effects of Culture on Users’ Perception of a Webpage:

A Comparative Study of the Cognitive Styles of Chinese,

Koreans, and Americans. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

Ying Dong and Kun-Pyo Lee

Part II Application

Backrest Designs in Meeting Chairs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155

Toshio Matsuoka, Hirokazu Kimura, Hiroyuki Kanai,

Fusao Yasuda and Masaki Matsumoto

Branding Luxury Through Affective Value Case of Swiss

Watch Industry. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167

Shinichiro Terasaki and Shin’ya Nagasawa

Button-Sound-Quality Evaluation for Car Audio Main Units. . . . . . . . . . 181

Shunsuke Ishimitsu

Characteristics of the Design and Production Process

for Italian- and Japanese-Made Tailored Jackets

in the Global Market. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193

Tsuyoshi Otani, KyoungOk Kim, Keiko Miyatake, Kimiko Sano

and Masayuki Takatera

Online Shopping and Individual Consumer Adaptation:

The Relationship Between Fabric-Identification Ability

and Prior Knowledge. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209

Tomoharu Ishikawa, Kazuya Sasaki, Hiroko Shimizu and Miyoshi Ayama

Reading Emotion of Color Environments: Computer Simulations

with Self-Reports and Physiological Signals. . . . . . . . . . . . . . . . . . . . . . . . . 219

So-Yeon Yoon and Kevin Wise

Reviewing the Role of the Science Fiction Special Interest

Group via User Interfaces: The Case of Science Fiction Movies. . . . . . . . 233

Shigeyoshi Iizuka, Jun Iio and Hideyuki Matsubara

Contents xi

Sleep Quality and Skin-Lightening Effects of White Mother

Chrysanthemum Aroma. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241

Se Jin Park, Murali Subramaniyam, Myung-Kug Moon,

Byeong-Bae Jeon, Eun-Ju Lee, Sang-Hoon Han

and Chang-Sik Woo

The Emotional Characteristics of White for Applications

of Product Color Design. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253

Nooree Na and Hyeon Jeong Suk

The Influence of Skincare Routines on Skin Physiology

Parameters and Affective Values. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265

Yuet Sim Chan, Yukiko Tamura, Misako Kuroda and Takao Someya

Understanding Product Features Using a Hybrid Machine

Learning Model. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281

Manjeevan Seera, Chee Peng Lim and Junzo Watada

Index. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293

Part I

Methodology

3

Abstract It is difficult to define a comfortable space for people. This is partly

because comfort relates to many attributes specifying a space, partly because all

people have different preferences and also because even the same person changes

his or her preference according to the state of health, body conditions, working

state, and so on. Various parameters and attributes should be controlled in order

to realize such a comfortable space according to the database of past usages.

Information obtained from human bodies such as temperature, blood pressure, and

alpha waves can be employed to adjust the space to the best condition. The objec￾tive of the paper is to present the possibility that a space is able to be adjusted to a

human condition based on human brainwaves.

Keywords  Affective engineering  •  Living body measurement  •  Fuzzy control  • 

Neural network  •  Comfortable space

A Bio-Signal-Based Control Approach

to Building a Comfortable Space

Junzo Watada, Chee Peng Lim and Yung-chin Hsiao

J. Watada et al. (eds.), Industrial Applications of Affective Engineering,

DOI: 10.1007/978-3-319-04798-0_1, © Springer International Publishing Switzerland 2014

J. Watada (*) · Y. Hsiao

Graduate School of Information, Production and Systems, Waseda University, 2-7 Hibikino,

Wakamatsu, Kitakyushu 808-0135, Japan

e-mail: [email protected]

Y. Hsiao

e-mail: [email protected]

C. P. Lim

Centre for Intelligent Systems Research, Deakin University, Geelong Waurn Ponds Campus

Locked Bag 20000, Geelong Victoria 3220, Australia

e-mail: [email protected]

4 J. Watada et al.

1 Introduction

In today’s stressfull society, comfortable space and life are important to eliminate

intense stress on us. We should create a suite of new technological tools to realize a

comfortable space. In this chapter, the comfortable space means one where a person

feels at ease and free from stress when he/she stays there although each person has a

different feeling about being comfortable. It is hard to realize a comfortable space for

all humans. Therefore, we should build a comfortable space from each person’s view￾point. It is hypothesized that a machine or a system is able to recognize and evaluate

the state of a person from his/her behavior, voice, and other measurements by auto￾matically gathering data and recognizing patterns from the collected data. If we can

measure an electroencephalogram (brainwaves, heart beats, sweat, saliva, etc.) by an

instrument, the obtained information can be employed to realize a comfortable space.

The objective of this chapter is to show the possibility that the measurement of

human senses enables us to realize a comfortable space for any person even if the

person changes his/her comfortable feeling toward the changing environment or

condition [1, 2].

2 Affective Engineering

Affective information means total information of human senses. Affective engineer￾ing is “a technology, method, or theory to translate human affective information or

image to production of real things or to design of objects.” It is vague and uncertain

that a customer has an image or expectation about some product. Again, affective

engineering is a technology to build such affective information or vague image in

product design in some way [3]. The objective of this paper is to employ brainwaves

obtained from a person to adjust the environment of a space to the most comfortable

state that he/she feels. There are many measurements from a human body which can

be used for the control of a space, for instance, heart beats, sweat, saliva, etc.

In [4] bio-potential signals, which included electroencephalographic (EEG),

electrooculargraphic (EOG), and electromyographic (EMG) signals from psycho￾logical experiments were collected. The EEG signals were analyzed in three dif￾ferent frequency bands, namely a low-frequency band including δ and θ waves,

a middle-frequency band including the alpha wave, and a high-frequency band

including the beta wave. The aim of the experiment was to recognize different

types of emotions based on bio-potential signals, which included joy, anger, sad￾ness, fear, and relax. To stimulate the emotions, a number of commercial films

were broadcasted on TV. The support vector machine (SVM) was used as the emo￾tion classifier. The results showed that multi-modal bio-potential signals were use￾ful for emotion recognition, and the SVM was deemed suitable as the underlying

classifier for the emotion recognition tasks. In another experiment [5] with a simi￾lar experimental setting, the aim was to recognize pleasure and unpleasure emo￾tions. To generate pleasure and unpleasure, A bio-signal-based control approach to

A Bio-Signal-Based Control Approach to Building a Comfortable Space 5

building comfortable space three stimuli, classical music as well as music mixed

with noise (e.g., industrial noise) were played. The SVM and a neural network

were used for the classification tasks. The experimental outcome showed that both

methods produce similar results.

A Bayesian network was deployed for emotion recognition using EEG signals

[6]. Audio and visual pictures were used to induce emotions, such as joy, neu￾tral, anger, sad, and surprise. EEG signals were transformed into power spectrum

using the fast Fourier transform method, while low-frequencies EEG artifacts were

eliminated. The results showed that, while the probability values for many differ￾ent emotions were different, those of anger and sadness were similar. On the other

hand, machine learning techniques were employed to predict a learner’s emotions

in an intelligent tutoring system based on EEG signals [7]. The emotion states that

were of interest included anger, boredom, confusion, contempt, curious, disgust,

eureka, and frustration. The best classification accuracy yielded by the k-nearest

neighbor algorithm was above 82 %.

A wavelet-chaos-based method was applied to detection of seizure and epilepsy

using EEG signals and sub-bands [8]. Specifically, the δ, θ, α, β, and γ sub-bands

of EEG signals were examined, and quantified in terms of correlation dimension

(CD) and the largest Lyapunov exponent (LLE). It was found that, subject to a

large number of EEG segments, the average values of CD were useful for differ￾entiating three groups of subjects (healthy subjects, epileptic subjects during a

seizurefree interval, and epileptic subjects during a seizure) based on β and γ sub￾bands, while those of LLE was useful for differentiating these three groups of sub￾jects using α sub-band. EEG signals were also used as a source to detect deceptive

and truthful responses [9]. The main objective was to extract joint time-frequency

EEG features through wavelet analysis. During the experiment, EEG signals were

recorded from four electrode sites when five subjects went through a modified ver￾sion of the guilty knowledge test. The results from the wavelet analysis revealed

significant differences between deceptive and truthful responses.

Another application of EEG signals as a source for biometric identification was

investigated [10]. Gaussian mixture models and the maximum a posteriori model

adaptation were deployed for person authentication, that is, accepting or rejecting a

person claiming an identity. A series of experimental simulations was performed to

demonstrate the potential of the proposed method. Nevertheless, the database used

was too small to render any conclusive lessons in regard to person authentication.

3 Brainwaves

In affective engineering, physiological measurement is widely employed such as

impression method, psychological measure, and so on to quantize affective infor￾mation. In physiological measurement, emotional quantity includes automatic

nerve responses or brainwaves against an external stimulus. In this chapter, we

employ the measurement of brainwaves.

6 J. Watada et al.

3.1 Electroencephalogram

The electroencephalogram is explained in its operations and functions shortly in

the following.

3.1.1 Spontaneous Electroencephalogram

It is not easy to interpret the cognitive meaning of an electroencephalogram (brain￾wave). Nevertheless, some characteristics of the electroencephalogram have been

explained quantitatively. The electroencephalogram is a kind of oscillated brain￾waves. Such brainwaves can be characterized using amplitude and frequency.

Specifically, in many researches, electroencephalograms are classified according the

difference of frequencies and compared with consciousness states (wake levels).

3.1.2 Electrical Voltage Related to Events

In order to find the meaning of an electroencephalogram, it is significant to ana￾lyze brainwaves recorded in an electroencephalograph when a stimulus is given,

for example, when an experimenter gives light flash or large sound to a test sub￾ject. Such brainwaves are named event-related potential (ERP) or evoked potential

since they come out from a specific stimulus or event. Comparing with the resulted

brainwaves, an electroencephalogram a brain produces spontaneously is named a

spontaneous or freely electroencephalogram or back brainwave.

The electroencephalograph is obtained by duplicating various amplitudes of

frequencies. It is hard to clarify their characteristics by observing and measuring

an electroencephalogram obtained by giving a stimulus sound to a test subject

because the effects would be buried under such a spontaneous electroencephalo￾gram. It is possible to abstract the effect from duplication of electroencephalo￾graphs at the same timing because the same kind of results by the same strength

of stimuli removes the influence of the spontaneous electroencephalograms. Such

spontaneous electroencephalograms are leveled by duplicating such noises. Then,

the ERP can be clearly obtained. The widely adapted interpretation of the electro￾encephalogram is to understand as electronic activities of a brain and the duplica￾tion of many small voltages from synapses.

The 10/20 method is widely employed as the positions of electrodes, which is

the standard of International Electroencephalogram Society. But it is not necessary

to measure all of them. Both sides of the frontal part, whole temporal, centered

temporal, and central portions are used widely [3].

3.1.3 Types of Electroencephalograms

Brainwaves are categorized into four groups such as δ brainwaves, θ brainwaves,

α brainwaves, and β brainwaves. Brainwaves with lower frequencies than α