Big Data> Data Mining Predictive Analytics, Data Science Data Mining Big Data Predictive Data sci Add term Interest over time 7 □ News headlines Data minin Big data Google Trends search, Jan 2008-Sep 2013, Worldwide o KDnuggets 2013
Big Data > Data Mining > > Predictive Analytics , Data Science © KDnuggets 2013 11 Big Data Google Trends search, Jan 2008- Sep 2013, Worldwide Data mining Data Mining Big Data
Data Science before" Big Data Ancient astronomers Kepler laws of planetary motion (1609) derived from observations by tycho Brahe Genetics-Gregor Mendel found patterns in inheritance of pea plants parent generation Western medicine o KDnuggets 2013
Data Science before “Big Data” • Ancient astronomers • Kepler laws of planetary motion (1609), derived from observations by Tycho Brahe • Genetics – Gregor Mendel found patterns in inheritance of pea plants • Western Medicine • … © KDnuggets 2013 12
Data Science Basic Principles Ideas Focus on actionable patterns Build predictive models-supervised learning(train test, x-validate) Avoid overfitting Calculating similarity of objects -unsupervised learning Avoid information leakers Select important variables/features Model accuracy vs lift: how much more prevalent a pattern is than would be expected by chance Estimate probability and cost/gain of actions lelp optimize decisions o KDnuggets 2013
Data Science Basic Principles & Ideas • Focus on actionable patterns • Build predictive models - supervised learning (train, test, x-validate) • Avoid overfitting • Calculating similarity of objects - unsupervised learning • Avoid information leakers • Select important variables/features • Model accuracy vs lift: how much more prevalent a pattern is than would be expected by chance • Estimate probability and cost/gain of actions • Help optimize decisions © KDnuggets 2013 13
What Changes in Data Science with Big Data? Data munging becomes much more complex New algorithms technology needed to deal with Big data Volume, velocity variety NeW, effective algorithms that require big data e.g.:deep belief networks, recommendations Predictions become(somewhat )more accurate New things become visible: social networks recommendations mobility knowledge? However, basic principles remain o KDnuggets 2013
What Changes in Data Science with Big Data? • Data munging becomes much more complex • New algorithms, technology needed to deal with Big Data Volume, Velocity, & Variety • New, effective algorithms that require Big Data: e.g.: deep belief networks, recommendations • Predictions become (somewhat ) more accurate • New things become visible: social networks, recommendations, mobility, knowledge ? • However, basic principles remain © KDnuggets 2013 14
Outline The Era of Big Data 2 Big Data Characteristics 3 A Big Data Processing Framework 4 Streaming Data and Streaming Features 5 Concluding Remarks
2 Big Data Characteristics 3 A Big Data Processing Framework 4 Streaming Data and Streaming Features Outline 5 Concluding Remarks 1 The Era of Big Data 15