Guest Post by Scott Terry, Rapid Progress Marketing and Modeling, LLC
When your hear data mining automation, what do you usually think of? In general, the automation world has us thinking about the following process: you upload a dataset, select a multitude of modeling engines, then out comes the best model, and voilà, you're done. There are many positives in this scenario; you save time and it's generally reliable regarding the 'best model.' However, what about automating model building EXPERIMENTS? You ask: what do you mean by experiments?
Since the dawn of data mining (beginning with Bayes Theorem in the 1700s), there have been many successes and failures, even by the top experts in the field. No matter what job function or industry you work in, it is generally agreed on that on-the-job training is a far better learning tool than any classroom lecture. Learning from our mistakes is one of the ways we move forward and accomplish our goals. The same goes for data mining practitioners and data scientists; hands-on experience (or lack thereof) results in victories and blunders that set the foundation for advancements in the field.
Topics: data mining
Predictive accuracy is repeatedly cited by data scientists as one of the most important demands in modern data mining algorithms and software. It stands right along side the importance of model-building speed, missing value handling, and memory efficiency. So, if it is so important, how do the experts TEST the accuracy of their models?
Let's get right to it! You're a beginner, and you want to know what is needed to start data mining and become an experienced data scientist overnight. We get it - this is the world we live in - quick and dirty. So here we go, take notes!
This article is an update (or add-on) to The Elusive Quest to Define Big Data published on August 6, 2013.
Topics: big data
Staying informed and up-to-date on the latest industry news doesn't need to be a chore. Have the latest news and trending topics delivered right to your inbox through LinkedIn Groups.
A question we hear time and again is "how does your data mining software determine the best model?" In order to shed some light on this, a well as provide an in-depth analysis for those who want to take it a step farther, we have created a detailed video analysis of how the 'best model' is chosen. The simple answer is: it depends.
January is commonly a time to reflect on the past and make predictions about the future. To each his own, I suppose, but I’m confident that many of us will agree on a few common themes for 2014.