Data Science Using Oracle Data Miner and Oracle R Enterprise

Transform Your Business Systems

into an Analytical Powerhouse

—

Sibanjan Das

Data Science

Using Oracle

Data Miner and

Oracle R Enterprise

Data Science Using

Oracle Data

Miner and Oracle R

Enterprise

Transform Your Business Systems

into an Analytical Powerhouse

Sibanjan Das

Data Science Using Oracle Data Miner and Oracle R Enterprise

Sibanjan Das

Pune, Maharashtra, India

ISBN-13 (pbk): 978-1-4842-2613-1 ISBN-13 (electronic): 978-1-4842-2614-8

10.1007/978-1-4842-2614-8

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole

or part of the material is concerned, specifically the rights of translation, reprinting, reuse of

illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical

way, and transmission or information storage and retrieval, electronic adaptation, computer

software, or by similar or dissimilar methodology now known or hereafter developed.

Trademarked names, logos, and images may appear in this book. Rather than use a trademark

symbol with every occurrence of a trademarked name, logo, or image we use the names, logos,

and images only in an editorial fashion and to the benefit of the trademark owner, with no

intention of infringement of the trademark.

The use in this publication of trade names, trademarks, service marks, and similar terms, even if

they are not identified as such, is not to be taken as an expression of opinion as to whether or not

they are subject to proprietary rights.

While the advice and information in this book are believed to be true and accurate at the

date of publication, neither the authors nor the editors nor the publisher can accept any legal

responsibility for any errors or omissions that may be made. The publisher makes no warranty,

express or implied, with respect to the material contained herein.

Managing Director: Welmoed Spahr

Lead Editor: Celestin Suresh John

Technical Reviewer: Ajit Jaokar

Editorial Board: Steve Anglin, Pramila Balan, Laura Berendson, Aaron Black,

Louise Corrigan, Jonathan Gennick, Robert Hutchinson, Celestin Suresh John,

Nikhil Karkal, James Markham, Susan McDermott, Matthew Moodie, Natalie Pao,

Gwenan Spearing

Coordinating Editor: Prachi Mehta

Copy Editor: Deanna Hegle

Compositor: SPi Global

Indexer: SPi Global

Artist: SPi Global

Distributed to the book trade worldwide by Springer Science+Business Media New York, 233

Spring Street, 6th Floor, New York, NY 10013. Phone 1-800-SPRINGER, fax (201) 348-4505,

e-mail [email protected], or visit www.springeronline.com. Apress Media, LLC is

a California LLC and the sole member (owner) is Springer Science + Business Media Finance

Inc (SSBM Finance Inc). SSBM Finance Inc is a Delaware corporation.

For information on translations, please e-mail [email protected], or visit www.apress.com.

Apress and friends of ED books may be purchased in bulk for academic, corporate, or promotional

use. eBook versions and licenses are also available for most titles. For more information, reference

our Special Bulk Sales–eBook Licensing web page at www.apress.com/bulk-sales.

Any source code or other supplementary materials referenced by the author in this text are

available to readers at www.apress.com. For detailed information about how to locate your book’s

source code, go to www.apress.com/source-code/. Readers can also access source code at

SpringerLink in the Supplementary Material section for each chapter.

Printed on acid-free paper

I dedicate this book

To my parents, grandparents, sisters, brothers, and brothers-in-law

who motivated me to do what I always wanted to do

To my Wife and six-month old son Sachit for their love and care

To my mentors and Coaches Karthik Krishna Jonnalagadha,

John Kopchak, Prabir Sen, Bibhudatta Ghadei, Harishchandra Rakh, and

Nirmal Babu who has enriched my life with guidance and lessons

that help me to be a better technocrat and person every day.

Contents at a Glance

About the Author �� xiii

About the Technical Reviewer �� xv

Acknowledgments�� xvii

Introduction�� xix

■Chapter 1: Getting Started With Oracle Advanced Analytics �� 1

■Chapter 2: Installation and Hello World�� 35

■Chapter 3: Clustering Methods�� 75

■Chapter 4: Association Rules�� 117

■Chapter 5: Regression Analysis�� 147

■Chapter 6: Classification Methods�� 189

■Chapter 7: Advanced Topics �� 239

■Chapter 8: Solutions Deployment �� 267

Index�� 285

vii

Contents

About the Author �� xiii

About the Technical Reviewer �� xv

Acknowledgments�� xvii

Introduction�� xix

■Chapter 1: Getting Started With Oracle Advanced Analytics �� 1

Data Science �� 2

Business Understanding��3

Data Understanding��3

Data Preparation��3

Modeling��3

Evaluation��3

Deployment��3

Machine Learning�� 4

Supervised Learning��5

Unsupervised Learning��5

Getting Started with Oracle Advanced Analytics �� 6

Oracle Data Miner��8

R Technologies in Oracle �� 15

Oracle R Enterprise��15

Advantages of using Oracle R Enterprise ��17

Analytical SQL and PLSQL Functions�� 18

Package DBMS_STATS_FUNC �� 18

SQL Functions��22

Summary�� 33

■ Contents

viii

■Chapter 2: Installation and Hello World�� 35

Booting Up Oracle Data Miner�� 36

Installation Prerequisites��36

Installation��36

Welcome to the World—Oracle Data Miner�� 48

SQL Developer Components for ODM�� 54

ODM Data Dictionary �� 59

Boot Up Oracle R Enterprise�� 59

Prerequisites��60

Oracle R Distribution��60

Oracle R Enterprise Server Installation��65

Oracle R Enterprise Client Installation��67

Install Supporting Packages for the ORE Client��70

Welcome to the World of Oracle R Enterprise�� 71

ORE Data Dictionary �� 73

Summary�� 74

■Chapter 3: Clustering Methods�� 75

Clustering Approaches �� 76

The k-means Algorithm�� 77

k-means in Oracle Advanced Analytics �� 80

Clustering Rules Evaluation Metrics��81

Parameters to Tune k-means Clustering �� 81

Creating a Cluster Model in Oracle Advanced Analytics�� 82

Clustering using SQL and PLSQL ��83

Case Study—Customer Segmentation�� 93

Business understanding ��94

Data understanding ��94

■ Contents

RFM Segmentation—Data Preparation �� 95

RFM Segmentation—DATA Modeling �� 97

Need-Based Segmentation—Data Preparation�� 100

Need-Based Segmentation—Data Modeling �� 106

Result Evaluation��108

Deployment—Storing the Results Back to the Database�� 110

Assigning Segments to New Customers�� 114

Summary�� 115

■Chapter 4: Association Rules�� 117

Introduction to Association Rules�� 118

Terminologies Associated with Association Rules�� 118

Working of an Apriori Algorithm �� 120

Identify Interesting Rules �� 122

Algorithm Settings�� 123

Model Settings �� 123

Association Rules Using SQL and PLSQL�� 124

Creating the Association Rules Model Using Oracle R Enterprise �� 127

Creating the Association Model Using SQL Developer �� 130

Case Study—Market Basket Analysis�� 134

Business Understanding��135

Data Understanding��135

Data Preparation��136

Data Modeling ��136

High-Level Technical Overview��137

Execution��143

Summary�� 145

■ Contents

■Chapter 5: Regression Analysis�� 147

Understanding Relationships �� 148

Regression Analysis �� 151

Working of OLS Regression�� 152

Assumptions of OLS �� 153

OLS Regression in Oracle Advanced Analytics �� 154

GLM Regression �� 160

Ridge Regression �� 161

Parameters to Tune the GLM Model��161

GLM and Ridge Regression in Oracle Advanced Analytics �� 162

GLM Regression Using SQL and PLSQL�� 162

Creating GLM Regression Using Oracle R Enterprise �� 166

Creating GLM and Ridge Regression from SQL Developer�� 168

Guidelines for Regression Modeling��174

Case Study: Sales Forecasting �� 175

Business Understanding��176

Data Understanding��176

Data Preparation��177

Data Modeling ��179

Summary�� 187

■Chapter 6: Classification Methods�� 189

Overview of Classification Techniques �� 190

Logistic Regression�� 190

Naïve Bayes�� 193

Creating a Naïve Bayes Model Using SQL and PLSQL �� 195

Creating a Naïve Bayes Classifier Using Oracle R Enterprise �� 198

Assessing the Model Quality for Classifiers �� 201

■ Contents

Decision Trees �� 203

Entropy and Information Gain��204

Information Gain ��205

Pruning Methods/Early Stopping Criteria for Decision Trees�� 206

Parameters to Tune a Decision Tree Model�� 207

Decision Tree Modeling Using SQL and PLSQL ��208

Decision Tree Modeling Using Oracle R Enterprise�� 210

SVM�� 211

Parameters to Tune the SVM Model��214

SVM Using SQL and PLSQL��215

SVM Using Oracle R Enterprise��218

A Few Other Important PLSQL APIs for Classifiers �� 219

Choosing a Classifier�� 221

Case Study: Customer Churn Prediction�� 222

Business Understanding��222

Data Understanding��222

Data Preparation��223

Data Modeling ��227

Selecting the Best Classifier��229

Results Interpretation ��236

Summary�� 237

■Chapter 7: Advanced Topics �� 239

Overview of Neural Networks�� 240

Function of a Hidden Layer Neuron ��241

Parameters for Neural Network��242

Neural Network Using Oracle Advanced Analytics �� 244

■ Contents

xii

Overview of Anomaly Detection �� 247

One-Class SVM��248

Anomaly Detection Using Oracle Advanced Analytics �� 248

Overview of Predictive Analytics in Oracle Advanced Analytics �� 251

Predictive Analytics Using SQL Developer GUI �� 251

DBMS_PREDICTIVE_ANALYTICS��255

EXPLAIN��255

PREDICT��256

PROFILE ��258

Overview of Product Recommendation Engine �� 259

Product Recommendation Engine Using Oracle Advanced Analytics�� 260

Overview of Random Forest�� 262

Random Forest Using Oracle Advanced Analytics �� 263

Summary�� 265

■Chapter 8: Solutions Deployment �� 267

Export and Import an ODM Workflow �� 267

Deploy Option for Data Miner Workflows in SQL Developer�� 273

Import and Export Data Mining Models�� 277

EXPORT_MODEL ��278

IMPORT_MODEL��279

Renaming Models��280

Drop Models ��280

Predictive Model Markup Language (PMML)�� 281

Importing and Exporting PMML models into Oracle Advanced Analytics �� 282

Summary�� 283

Index�� 285

xiii

About the Author

Sibanjan Das is a Business Analytics and Data Science

consultant. He has over six years of experience in the IT

industry working on enterprise resource planning

(ERP) systems, implementing predictive analytics

solutions in business systems and Internet of Things

(IoT). An enthusiastic and passionate professional

about technology and innovation, he has had the

passion for wrangling with data from the early days of

his career. He also enjoys reading, writing, and

networking. His writings have appeared in various

Analytics Magazines, and Klout (http://klout.com)

has rated him among the top 2% professionals in the

world talking about Artificial Intelligence, Machine

Learning, Data Science, and IoT.

Sibanjan holds a Masters in IT degree with a major in Business Analytics from

Singapore Management University, Singapore, and is a Computer Science Engineering

graduate from Institute of Technical Education and Research, India. He is a Six Sigma

Green Belt from the Institute of Industrial Engineers and also holds several industry

certifications such as OCA, OCP, CSCMS, and ITIL V3.

About the Technical

Reviewer

Ajit Jaokar is the Director of the AI/Deep Learning labs

for Future cities at UPM (Universidad Politécnica de

Madrid). Ajit's work spans research, entrepreneurship,

and academia relating to IoT, predictive analytics, and

mobility. His current research focus is on applying data

science algorithms to IoT applications. This research

underpins his teaching at Oxford University (Data

Science for IoT) and the “City Sciences” program at

UPM. Ajit lives in London and is a British citizen.

He has been nominated to the World Economic

Forum’s “Future of the Internet” council, board of

Connected Liverpool (Smart city), and World Smart

Capital program (Amsterdam). Ajit moderates/chairs

Oxford University’s Next generation mobile applications panel. Ajit has been involved

in IoT-based roles for the webinos project (Seventh Framework Program [Fp7]) and has

spoken at MobileWorld Congress, CTIA (The Wireless Association), CEBIT, Web20 expo,

European Parliament, Stanford University, MIT Sloan, Fraunhofer Institute for Open

Communication Systems (FOKUS), and the University of St. Gallen. He has been involved

in transatlantic technology policy discussions. Ajit is passionate about teaching Data

Science to young people through Space Exploration working with Ardusat.

xvii

Acknowledgments

I wish to express my gratitude to each and every one who has contributed to my career

and success. This is my first attempt at writing a book. I am equally excited and wish that

everyone who reads this book finds it worthy. The list of people who influence me is long.

Keeping the length of text in mind, I would like to thank a few of them who have made a

lot of difference to me and in my career.

I am grateful to my colleagues and friends in Oracle and Zensar for their

encouragement and support. Thank you Abhijit, Pravin, Dinesh, Sandhya, Anand, Sanjay,

Eulalio, Sanjiv, Rajesh G., Team Toshiba, and Team Co-Development for believing in me

and providing me an opportunity to work with you all.

Personal thanks to Pradeep, Sangeetha, Karthik, Charlie, Boriana, Ari, Vinayak,

Mohan, Vijay, Puskala, Swathi, Rajesh, Naresh, and VoF Team for their helping hand and

constant support for my work during my tenure at Oracle. I learned a lot from you all. The

experience gained from working alongside you has provided the grist for this book.

Huge thanks to my friends and professors at Singapore Management University—

Prof. Jing Jiang, Dr. Kam Tin Seong, Prof. Prabir Sen, Venkata Narayanan, Koo Ping Shung,

and Murphy Choy—for sharing their knowledge and experience with me. Thank you

Ankit, Mudit, Ai Bowen, Mugdha ,Edwin, Shantanu, Upkar, and Maruthi for being my

partners in projects and crime.

Many thanks to my friends Sarat, Itishree, Vikas Kumar, Biswajit, Hamid, and Trojan

Horses for your constant support, sharing drinks, and lending your ears to me when I

needed it most.

Final thanks to the Apress team who worked behind the scenes to create a book out

of my drafts of thoughts. Thank you, Suresh John Celestin, for reaching out to me to write

the book. The book was made better by Apress Editors Prachi Metha, Ajit Jaokar, and

Poonam Jain. Your comments and edits were valuable for the book.

xix

Introduction

We are now in an era where consumer preferences are changing at lightning speed,

social media is making the reviews transparent, and new competitors are appearing

out of nowhere. Digitalization is starting to rule the world, and employee attrition has

increased. To surf these new waves of business challenges, organizations are getting

driven by technologies and deploying a variety of new enterprise applications like ERP

Systems, CRM (customer relationship management) tools, Big Data, and Cloud-based

solutions. Also, organizations are relying on data science technologies and algorithms

to be intelligent and proactive. Data science, which is a relatively new term in the

Business and IT industry, has hit the main stream for investment in the last few years.

This is because it provides organizations the opportunity to gain insights about their

business and drive proactive planning. Although getting some information or insights

out of the data has been there since a decade ago, proactive planning is relatively new.

Organizations are now able to predict what might happen in the future, which helps them

to formulate strategies and plan in advance for the possible roadblocks. Statistical models

and quantitative analysis sits at the core of data science and measures past performance

to predict the future.

But this is not the end of the story. To keep up with everyone’s expectations,

organizations need to attend the customers as soon as they need service, retain

employees before they resign, or identify frauds before they happen. They also have to

be operationally efficient and at the same time cost-effective. This requires the ability

to apply analytical insights to business operations at the right time. The only way to stay

ahead of everyone is to be agile at data-driven, decision-making capabilities. This is

possible when algorithms are brought near to the origin of data such as databases and

data science routines are automated and integrated to business process workflows.

Data science involves a wide array of technologies and statistical algorithms. This

makes it difficult to automate each and every aspect of it. However, there are some areas

in data science that can be automated using scripts and workflows.

At a high level, we can classify data science automation into the following categories:

Repetitive tasks automation: Repetitive tasks are those that have to be done

every time while building models. Data extraction, data cleaning, and basic data

transformations such as imputing null values and algorithm-specific transformations are

some tasks that fall into this category. These are to be done even for the same set of data

every time. Automating these tasks would take some of the burdens off of a data scientist

so they can concentrate more on solving business problems.

Automated statistician: This is an area of data science automation where statistical

routines and machine learning are automated. The system executes the best algorithm

based on the provided data set. It hides the intricacies and mathematical complexity of

algorithms from the user. The user needs to provide automated statisticians with data.

Thư viện tri thức trực tuyến

Data Science Using Oracle Data Miner and Oracle R Enterprise

Nội dung xem thử

Mô tả chi tiết

Tài liệu tương tự (6)

Introducing data science big data, machine learning and more, using python tools (2016)

Big Data, Cloud Computing, Data Science & Engineering

Data Science & Big Data Analytics

Data Science and Predictive Analytics

Data science & big data analytics

Data Science For Dummies®, 2nd Edition