Progress, Fear, and Machine Automation
Ramesh Subramanian takes a look forward to a future where machine learning is automated.
Turns out, quite a lot.
This started with the anxiety shared by some co-workers a while ago that the world will end when robots start making robots. And I just happened to hear about auto-ML, the process of automating machine learning. I started to wonder if data scientists were going the way of the ‘web master’? This was somewhat counter to a recent article I read that Deep Learning is largely about fitting the best curve, albeit, a complex curve.
Assuming you have a good handle on the problem you are trying to solve, there are three major steps in the typical ML process:
1. Data acquisition and prep: Finding reliable sources of data, getting it, cleaning it, and selecting the features (training parameters) and hyper parameters (e.g. value of K in KNN, tree depth in Random Forest, number of layers in a neural network) that will help build a predictive model. And as this article says, this step alone can consume almost 50% of your project time.
2. Modeling: The next step is to pick a machine learning algorithm (classification or regression) that will yield the best predictions after training it on a data set, often referred to as supervised learning. In unsupervised learning, algorithms such as KNN and Neural networks are used to “learn and predict” given a lot of data, where the observations closely resemble each other (images, text).
3. Optimizing: This phase involves validating the results and fine tuning – experimenting with other algorithms, features, and hyper parameter tuning.
Key points include:
- Auto-ML packages
- Winning algorithms
- Data-science education
Read the full article, When Machine Learning is automated..What else is left?, on LinkedIn.
