Big-Data Analytics for Cloud, IoT and Cognitive Computing

www.ebook3000.com

Big-Data Analytics for Cloud, IoT and Cognitive Computing

www.ebook3000.com

Big-Data Analytics for Cloud, IoT

and Cognitive Computing

Kai Hwang

University of Southern California, Los Angeles, USA

Min Chen

Huazhong University of Science and Technology, Hubei, China

This edition first published 2017

© 2017 John Wiley & Sons Ltd

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or

transmitted, in any form or by any means, electronic, mechanical, photocopying, recording or otherwise,

except as permitted by law. Advice on how to obtain permission to reuse material from this title is available

at http://www.wiley.com/go/permissions.

The right of Kai Hwang and Min Chen to be identified as the authors of this work has been asserted in

accordance with law.

Registered Office

John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, USA

John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, UK

Editorial Office

The Atrium, Southern Gate, Chichester, West Sussex, PO19 8SQ, UK

For details of our global editorial offices, customer services, and more information about Wiley products

visit us at www.wiley.com.

Wiley also publishes its books in a variety of electronic formats and by print-on-demand. Some content that

appears in standard print versions of this book may not be available in other formats.

Limit of Liability/Disclaimer of Warranty

While the publisher and authors have used their best efforts in preparing this work, they make no

representations or warranties with respect to the accuracy or completeness of the contents of this work and

specifically disclaim all warranties, including without limitation any implied warranties of merchantability or

fitness for a particular purpose. No warranty may be created or extended by sales representatives, written

sales materials or promotional statements for this work. The fact that an organization, website, or product is

referred to in this work as a citation and/or potential source of further information does not mean that the

publisher and authors endorse the information or services the organization, website, or product may provide

or recommendations it may make. This work is sold with the understanding that the publisher is not

engaged in rendering professional services. The advice and strategies contained herein may not be suitable

for your situation. You should consult with a specialist where appropriate. Further, readers should be aware

that websites listed in this work may have changed or disappeared between when this work was written and

when it is read. Neither the publisher nor authors shall be liable for any loss of profit or any other

commercial damages, including but not limited to special, incidental, consequential, or other damages.

Library of Congress Cataloging-in-Publication Data

Names: Hwang, Kai, author. | Chen, Min, author.

Title: Big-Data Analytics for Cloud, IoT and Cognitive Computing/

Kai Hwang, Min Chen.

Description: Chichester, UK ; Hoboken, NJ : John Wiley & Sons, 2017. |

Includes bibliographical references and index.

Identifiers: LCCN 2016054027 (print) | LCCN 2017001217 (ebook) | ISBN

9781119247029 (cloth : alk. paper) | ISBN 9781119247043 (Adobe PDF) | ISBN

9781119247296 (ePub)

Subjects: LCSH: Cloud computing–Data processing. | Big data.

Classification: LCC QA76.585 .H829 2017 (print) | LCC QA76.585 (ebook) | DDC

004.67/82–dc23

LC record available at https://lccn.loc.gov/2016054027

Cover Design: Wiley

Cover Images: (Top Inset Image) © violetkaipa/Shutterstock;(Bottom Inset Image) ©

3alexd/Gettyimages;(Background Image) © adventtr/Gettyimages

Set in 10/12pt WarnockPro by Aptara Inc., New Delhi, India

Printed in Great Britain by TJ International Ltd, Padstow, Cornwall

10 9 8 7 6 5 4 3 2 1

www.ebook3000.com

v

Contents

About the Authors xi

Preface xiii

About the Companion Website xvii

Part  Big Data, Clouds and Internet of Things 1

 Big Data Science and Machine Intelligence 3

1.1 Enabling Technologies for Big Data Computing 3

1.1.1 Data Science and Related Disciplines 4

1.1.2 Emerging Technologies in the Next Decade 7

1.1.3 Interactive SMACT Technologies 13

1.2 Social-Media, Mobile Networks and Cloud Computing 16

1.2.1 Social Networks and Web Service Sites 17

1.2.2 Mobile Cellular Core Networks 19

1.2.3 Mobile Devices and Internet Edge Networks 20

1.2.4 Mobile Cloud Computing Infrastructure 23

1.3 Big Data Acquisition and Analytics Evolution 24

1.3.1 Big Data Value Chain Extracted from Massive Data 24

1.3.2 Data Quality Control, Representation and Database Models 26

1.3.3 Big Data Acquisition and Preprocessing 27

1.3.4 Evolving Data Analytics over the Clouds 30

1.4 Machine Intelligence and Big Data Applications 32

1.4.1 Data Mining and Machine Learning 32

1.4.2 Big Data Applications – An Overview 34

1.4.3 Cognitive Computing – An Introduction 38

1.5 Conclusions 42

Homework Problems 42

References 43

 Smart Clouds, Virtualization and Mashup Services 45

2.1 Cloud Computing Models and Services 45

2.1.1 Cloud Taxonomy based on Services Provided 46

2.1.2 Layered Development Cloud Service Platforms 50

2.1.3 Cloud Models for Big Data Storage and Processing 52

vi Contents

2.1.4 Cloud Resources for Supporting Big Data Analytics 55

2.2 Creation of Virtual Machines and Docker Containers 57

2.2.1 Virtualization of Machine Resources 58

2.2.2 Hypervisors and Virtual Machines 60

2.2.3 Docker Engine and Application Containers 62

2.2.4 Deployment Opportunity of VMs/Containers 64

2.3 Cloud Architectures and Resources Management 65

2.3.1 Cloud Platform Architectures 65

2.3.2 VM Management and Disaster Recovery 68

2.3.3 OpenStack for Constructing Private Clouds 70

2.3.4 Container Scheduling and Orchestration 74

2.3.5 VMWare Packages for Building Hybrid Clouds 75

2.4 Case Studies of IaaS, PaaS and SaaS Clouds 77

2.4.1 AWS Architecture over Distributed Datacenters 78

2.4.2 AWS Cloud Service Offerings 79

2.4.3 Platform PaaS Clouds – Google AppEngine 83

2.4.4 Application SaaS Clouds – The Salesforce Clouds 86

2.5 Mobile Clouds and Inter-Cloud Mashup Services 88

2.5.1 Mobile Clouds and Cloudlet Gateways 88

2.5.2 Multi-Cloud Mashup Services 91

2.5.3 Skyline Discovery of Mashup Services 95

2.5.4 Dynamic Composition of Mashup Services 96

2.6 Conclusions 98

Homework Problems 98

References 103

 IoT Sensing, Mobile and Cognitive Systems 105

3.1 Sensing Technologies for Internet of Things 105

3.1.1 Enabling Technologies and Evolution of IoT 106

3.1.2 Introducing RFID and Sensor Technologies 108

3.1.3 IoT Architectural and Wireless Support 110

3.2 IoT Interactions with GPS, Clouds and Smart Machines 111

3.2.1 Local versus Global Positioning Technologies 111

3.2.2 Standalone versus Cloud-Centric IoT Applications 114

3.2.3 IoT Interaction Frameworks with Environments 116

3.3 Radio Frequency Identification (RFID) 119

3.3.1 RFID Technology and Tagging Devices 119

3.3.2 RFID System Architecture 120

3.3.3 IoT Support of Supply Chain Management 122

3.4 Sensors, Wireless Sensor Networks and GPS Systems 124

3.4.1 Sensor Hardware and Operating Systems 124

3.4.2 Sensing through Smart Phones 130

3.4.3 Wireless Sensor Networks and Body Area Networks 131

3.4.4 Global Positioning Systems 134

3.5 Cognitive Computing Technologies and Prototype Systems 139

3.5.1 Cognitive Science and Neuroinformatics 139

3.5.2 Brain-Inspired Computing Chips and Systems 140

www.ebook3000.com

Contents vii

3.5.3 Google’s Brain Team Projects 142

3.5.4 IoT Contexts for Cognitive Services 145

3.5.5 Augmented and Virtual Reality Applications 146

3.6 Conclusions 149

Homework Problems 150

References 152

Part  Machine Learning and Deep Learning Algorithms 155

 Supervised Machine Learning Algorithms 157

4.1 Taxonomy of Machine Learning Algorithms 157

4.1.1 Machine Learning Based on Learning Styles 158

4.1.2 Machine Learning Based on Similarity Testing 159

4.1.3 Supervised Machine Learning Algorithms 162

4.1.4 Unsupervised Machine Learning Algorithms 163

4.2 Regression Methods for Machine Learning 164

4.2.1 Basic Concepts of Regression Analysis 164

4.2.2 Linear Regression for Prediction and Forecast 166

4.2.3 Logistic Regression for Classification 169

4.3 Supervised Classification Methods 171

4.3.1 Decision Trees for Machine Learning 171

4.3.2 Rule-based Classification 175

4.3.3 The Nearest Neighbor Classifier 181

4.3.4 Support Vector Machines 183

4.4 Bayesian Network and Ensemble Methods 187

4.4.1 Bayesian Classifiers 188

4.4.2 Bayesian Belief Networks 191

4.4.3 Random Forests and Ensemble Methods 195

4.5 Conclusions 200

Homework Problems 200

References 203

 Unsupervised Machine Learning Algorithms 205

5.1 Introduction and Association Analysis 205

5.1.1 Introduction to Unsupervised Machine Learning 205

5.1.2 Association Analysis and A priori Principle 206

5.1.3 Association Rule Generation 210

5.2 Clustering Methods without Labels 213

5.2.1 Cluster Analysis for Prediction and Forecasting 213

5.2.2 K-means Clustering for Classification 214

5.2.3 Agglomerative Hierarchical Clustering 217

5.2.4 Density-based Clustering 221

5.3 Dimensionality Reduction and Other Algorithms 225

5.3.1 Dimensionality Reduction Methods 225

5.3.2 Principal Component Analysis (PCA) 226

5.3.3 Semi-Supervised Machine Learning Methods 231

viii Contents

5.4 How to Choose Machine Learning Algorithms? 233

5.4.1 Performance Metrics and Model Fitting 233

5.4.2 Methods to Reduce Model Over-Fitting 237

5.4.3 Methods to Avoid Model Under-Fitting 240

5.4.4 Effects of Using Different Loss Functions 242

5.5 Conclusions 243

Homework Problems 243

References 247

 Deep Learning with Artificial Neural Networks 249

6.1 Introduction 249

6.1.1 Deep Learning Mimics Human Senses 249

6.1.2 Biological Neurons versus Artificial Neurons 251

6.1.3 Deep Learning versus Shallow Learning 254

6.2 Artificial Neural Networks (ANN) 256

6.2.1 Single Layer Artificial Neural Networks 256

6.2.2 Multilayer Artificial Neural Network 257

6.2.3 Forward Propagation and Back Propagation in ANN 258

6.3 Stacked AutoEncoder and Deep Belief Network 264

6.3.1 AutoEncoder 264

6.3.2 Stacked AutoEncoder 267

6.3.3 Restricted Boltzmann Machine 269

6.3.4 Deep Belief Networks 275

6.4 Convolutional Neural Networks (CNN) and Extensions 277

6.4.1 Convolution in CNN 277

6.4.2 Pooling in CNN 280

6.4.3 Deep Convolutional Neural Networks 282

6.4.4 Other Deep Learning Networks 283

6.5 Conclusions 287

Homework Problems 288

References 291

Part  Big Data Analytics for Health-Care and Cognitive Learning 293

 Machine Learning for Big Data in Healthcare Applications 295

7.1 Healthcare Problems and Machine Learning Tools 295

7.1.1 Healthcare and Chronic Disease Detection Problem 295

7.1.2 Software Libraries for Machine Learning Applications 298

7.2 IoT-based Healthcare Systems and Applications 299

7.2.1 IoT Sensing for Body Signals 300

7.2.2 Healthcare Monitoring System 301

7.2.3 Physical Exercise Promotion and Smart Clothing 304

7.2.4 Healthcare Robotics and Mobile Health Cloud 305

7.3 Big Data Analytics for Healthcare Applications 310

7.3.1 Healthcare Big Data Preprocessing 310

7.3.2 Predictive Analytics for Disease Detection 312

www.ebook3000.com

Contents ix

7.3.3 Performance Analysis of Five Disease Detection Methods 316

7.3.4 Mobile Big Data for Disease Control 320

7.4 Emotion-Control Healthcare Applications 322

7.4.1 Mental Healthcare System 323

7.4.2 Emotion-Control Computing and Services 323

7.4.3 Emotion Interaction through IoT and Clouds 327

7.4.4 Emotion-Control via Robotics Technologies 329

7.4.5 A 5G Cloud-Centric Healthcare System 332

7.5 Conclusions 335

Homework Problems 336

References 339

 Deep Reinforcement Learning and Social Media Analytics 343

8.1 Deep Learning Systems and Social Media Industry 343

8.1.1 Deep Learning Systems and Software Support 343

8.1.2 Reinforcement Learning Principles 346

8.1.3 Social-Media Industry and Global Impact 347

8.2 Text and Image Recognition using ANN and CNN 348

8.2.1 Numeral Recognition using TensorFlow for ANN 349

8.2.2 Numeral Recognition using Convolutional Neural Networks 352

8.2.3 Convolutional Neural Networks for Face Recognition 356

8.2.4 Medical Text Analytics by Convolutional Neural Networks 357

8.3 DeepMind with Deep Reinforcement Learning 362

8.3.1 Google DeepMind AI Programs 362

8.3.2 Deep Reinforcement Learning Algorithm 364

8.3.3 Google AlphaGo Game Competition 367

8.3.4 Flappybird Game using Reinforcement Learning 371

8.4 Data Analytics for Social-Media Applications 375

8.4.1 Big Data Requirements in Social-Media Applications 375

8.4.2 Social Networks and Graph Analytics 377

8.4.3 Predictive Analytics Software Tools 383

8.4.4 Community Detection in Social Networks 386

8.5 Conclusions 390

Homework Problems 391

References 393

Index 395

www.ebook3000.com

xi

About the Authors

Kai Hwang is Professor of Electrical Engineering and Computer Science at the Univer￾sity of Southern California (USC). He has also served as a visiting Chair Professor at

Tsinghua University, Hong Kong University, University of Minnesota and Taiwan Uni￾versity. With a PhD from the University of California, Berkeley, he specializes in com￾puter architecture, parallel processing, wireless Internet, cloud computing, distributed

systems and network security. He has published eight books, including Computer Archi￾tecture and Parallel Processing (McGraw-Hill 1983) and Advanced Computer Archi￾tecture (McGraw-Hill 2010). The American Library Association has named his book:

Distributed and Cloud Computing (with Fox and Dongarra) as a 2012 outstanding title

published by Morgan Kaufmann. His new book, Cloud Computing for Machine Learn￾ing and Cognitive Applications (MIT Press 2017) is a good companion to this book.

Dr Hwang has published 260 scientific papers. Google Scholars has cited his published

work 16,476 times with an h-index of 54 as of early 2017. An IEEE Life Fellow, he has

served as the founding Editor-in-Chief of the Journal of Parallel and Distributed Com￾puting (JPDC) for 28 years.

Dr Hwang has served on the editorial boards of IEEE Transactions on Cloud Comput￾ing (TCC), Parallel and Distributed Systems (TPDS), Service Computing (TSC) and the

Journal of Big Data Intelligence. He has received the Lifetime Achievement Award from

IEEE CloudCom 2012 and the Founder’s Award from IEEE IPDPS 2011. He received the

2004 Outstanding Achievement Award from China Computer Federation (CCF). Over

the years, he has produced 21 PhD students at USC and Purdue University, four of them

elevated to IEEE Fellows and one an IBM Fellow. He has chaired numerous interna￾tional conferences and delivered over 50 keynote speech and distinguished lectures in

IEEE/ACM/CCF conferences or at major universities worldwide. He has served as a con￾sultant or visiting scientist for IBM, Intel, Fujitsu Reach Lab, MIT Lincoln Lab, JPL at

Caltech, French ENRIA, ITRI in Taiwan, GMD in Germany, and the Chinese Academy

of Sciences.

Min Chen is a Professor of Computer Science and Technology at Huazhong University

of Science and Technology (HUST), where he serves as the Director of the Embedded

and Pervasive Computing (EPIC) Laboratory. He has chaired the IEEE Computer Soci￾ety Special Technical Communities on Big Data. He was on the faculty of the School

of Computer Science and Engineering at Seoul National University from 2009 to 2012.

Prior to that, he has worked as a postdoctoral fellow in the Department of Electrical and

Computer Engineering, University of British Columbia for 3 years.

xii About the Authors

Dr Chen received Best Paper Award from IEEE ICC 2012. He is a Guest Editor forIEEE

Network, IEEE Wireless Communications Magazine, etc. He has published 260 papers

including 150+ SCI-indexed papers. He has 20 ESI highly cited or hot papers. He has

published the book: OPNET IoT Simulation (2015) and Software Defined 5G Networks

(2016) with HUST Press, and another book on Big Data Related Technologies (2014) in

the Springer Series in Computer Science. As of early 2017, Google Scholars cited his

published work over 8,350 times with an h-index of 45. His top paper was cited more

than 900 times. He has been an IEEE Senior Member since 2009. His research focuses on

the Internet of Things, Mobile Cloud, Body Area Networks, Emotion-aware Computing,

Healthcare Big Data, Cyber Physical Systems, and Robotics.

www.ebook3000.com

xiii

Preface

Motivations and Objectives

In the past decade, the computer and information industry has experienced rapid

changes in both platform scale and scope of applications. Computers, smart phones,

clouds and social networks demand not only high performance but also a high degree

of machine intelligence. In fact, we are entering an era of big data analysis and cognitive

computing. This trendy movement is observed by the pervasive use of mobile phones,

storage and computing clouds, revival of artificial intelligence in practice, extended

supercomputer applications, and widespread deployment of Internet of Things (IoT)

platforms. To face these new computing and communication paradigm, we must

upgrade the cloud and IoT ecosystems with new capabilities such as machine learning,

IoT sensing, data analytics, and cognitive power that can mimic or augment human

intelligence.

In the big data era, successful cloud systems, web services and data centers must be

designed to store, process, learn and analyze big data to discover new knowledge or

make critical decisions. The purpose is to build up a big data industry to provide cogni￾tive services to offset human shortcomings in handling labor-intensive tasks with high

efficiency. These goals are achieved through hardware virtualization, machine learning,

deep learning, IoT sensing, data analytics, and cognitive computing. For example, new

cloud services appear as Learning as a Services (LaaS), Analytics as a Service (AaaS), or

Security as a Service (SaaS), along with the growing practices of machine learning and

data analytics.

Today, IT companies, big enterprises, universities and governments are mostly con￾verting their data centers into cloud facilities to support mobile and networked appli￾cations. Supercomputers having a similar cluster architecture as clouds are also under

transformation to deal with the large data sets or streams. Smart clouds become greatly

on demand to support social, media, mobile, business and government operations.

Supercomputers and cloud platforms have different ecosystems and programming envi￾ronments. The gap between them must close up towards big data computing in the

future. This book attempts to achieve this goal.

A Quick Glance of the Book

The book consists of eight Chapters, presented in a logic flow of three technical parts.

The three parts should be read or taught in a sequence, entirely or selectively.

xiv Preface

Part I has three chapters on data science, the roles of clouds, and IoT devices or frame￾works for big data computing. These chapters cover enabling technologies to explore

smart cloud computing with big data analytics and cognitive machine learning capa￾bilities.We cover cloud architecture, IoT and cognitive systems, and software support.

Mobile clouds and IoT interaction frameworks are illustrated with concrete system

design and application examples.

Part II has three chapters devoted to the principles and algorithms for machine learn￾ing, data analytics, and deep learning in big data applications. We present both super￾vised and unsupervised machine learning methods and deep learning with artificial

neural networks. The brain-inspired computer architectures, such as IBM SyNapse’s

TrueNorth processors, Google tensor processing unit used in Brain programs, and

China’s Cambricon chips are also covered here. These chapters lay the necessary foun￾dations for design methodologies and algorithm implementations.

Part III presents two chapters on big data analytics for machine learning for health￾care and deep learning for cognitive and social-media applications. Readers should

master themselves with the systems, algorithms and software tools such as Google’s

DeepMind projects in promoting big data AI applications on clouds or even on mobile

devices or any computer systems. We integrate SMACT technologies (Social, Mobile,

Analytics, Clouds and IoT) towards building an intelligent and cognitive computing

environments for the future.

Part I: Big Data, Clouds and Internet of Things

Chapter 1: Big Data Science and Machine Intelligence

Chapter 2: Smart Clouds, Virtualization and Mashup Services

Chapter 3: IoT Sensing, Mobile and Cognitive Systems

Part II: Machine Learning and Deep Learning Algorithms

Chapter 4: Supervised Machine Learning Algorithms

Chapter 5: Unsupervised Machine Learning Algorithms

Chapter 6: Deep Learning with Artificial Neural Networks

Part III: Big Data Analytics for Health-Care and Cognitive Learning

Chapter 7: Machine Learning for Big Data in Healthcare Applications

Chapter 8: Deep Reinforcement Learning and Social Media Analytics

Our Unique Approach

To promote effective big data computing on smart clouds or supercomputers, we take a

technological fusion approach by integrating big data theories with cloud design prin￾ciples and supercomputing standards. The IoT sensing enables large data collection.

Machine learning and data analytics help decision-making. Augmenting clouds and

supercomputers with artificial intelligence (AI) features is our fundamental goal. These

AI and machine learning tasks are supported by Hadoop, Spark and TensorFlow pro￾gramming libraries in real-life applications.

The book material is based on the authors’ research and teaching experiences over

the years. It will benefit those who leverage their computer, analytical and application

skills to push for career development, business transformation and scientific discovery

in the big data world. This book blends big data theories with emerging technologies on

www.ebook3000.com