The paper presents some key lessons and "folk wisdom" that machine learning researchers and practitioners have learnt from experience and which are hard to find in textbooks.
All machine learning algorithms have three components:
- Representation for a learner is the set of classifiers/functions that can be possibly learnt. This set is called hypothesis space. If a function is not in hypothesis space, it can not be learnt.
- Evaluation function tells how good the machine learning model is.
- Optimization is the method to search for the most optimal learning model.