Home » Machine Learning Books: A review

Machine Learning Books: A review

I often get questions about how to become a data scientist or machine learning expert. While many books are available on the topics of pattern recognition and machine learning, most of them focus on a small set of popular subjects such as neural networks or kernel methods. Personally, I believe that a firm fundamental understanding of Bayesian inference, probabilistic theory, and parameter estimation and optimalization is essential in becoming a data scientist. Once you start to see relations between these fundamental concepts, learning about a single machine learning topic such as neural networks is as easy as learning the syntax of a programming language is to a seasoned programmer.

The following is a shortlist of the top machine learning books currently available. Each of these books will help you gain both an intuitive insight into and a theoretical foundation of the field of machine learning. I personally believe these books to be invaluable to any data scientist, whether you are just getting started, or you have experience in the field.

Pattern Classification

Level: Intermediate
This is the book where it all started for me. The author, Richard O. Duda, is one of the most famous pioneers in pattern recognition and his book is regarded by many as the bible of pattern recognition and machine learning. If you would only want to read one book, then this should be the one. Duda starts by an introduction into probabilistics and gradually continues with more advanced topics. Although the mathematics are not avoided, they are used in a practical manner to help you gain a deeper understanding of the most important machine learning techniques. When teaching pattern recognition at the University of Antwerp, this is the textbook I recommended to my students.
buy_on_amazon

Machine Learning

Level: Beginner
Compared to ‘Pattern Classification‘ from Richard O. Duda, this book might give a slightly broader overview of the domain of machine learning, ranging from supervised learning to genetic algorithms and reinforcement learning. This book might be the best place to get started if you think you lack the mathematical background to start with more theoretical works such as ‘Pattern Classification‘. This book covers most of the major machine learning algorithms and explains them in a practical and intuitive manner. As such, it can be seen as an easy way into the world of machine learning, and I would recommend it to anyone who is new to the domain of pattern recognition and data science.
buy_on_amazon

Pattern Recognition and Machine Learning

Level: Intermediate+
The author of this book, Christopher Michael Bishop, leads the renowned machine learning and perception group at Microsoft Research, and is considered one of the big guys in the machine learning industry. Personally, I believe this to be one of the best books to get started with machine learning if you have a strong mathematical background. Whereas ‘Machine Learning‘ from Tom M. Mitchell is the perfect introduction into machine learning if you are a computer scientist, this book is the perfect introduction into the field if you are a PhD candidate or a Mathematics major. ‘Pattern Classification‘ from Duda can probably be considered as a compromise between both.
buy_on_amazon

Machine Learning: A Probabilistic Perspective

Level: Advanced
From all books in my shortlist, this one is the most recent and hence the most up-to-date, written by Kevin P. Murphy, a well known machine learning expert from Google. The book contains over 1000 pages and provides a unique and impressive overview of both traditional machine learning techniques such as kernel based methods, and recent advances in machine learning such as deep neural networks. The reason I put this book at the bottom of my list is not because it isn’t a great book -it definitely is- but simply because the book covers almost every important topic in machine learning and by doing so often lacks a bit of depth. The book should be considered as an impressive piece of work to be used as a reference book by those who have some experience in the field. If you think you have nothing more to learn from the books earlier up in this list, and if you want to start broadening your view of the machine learning world, then this is the book you want!
buy_on_amazon

That’s it! Obviously there are many more good books on machine learning and pattern recognition, but the ones listed above can probably be considered to be must-read pieces of art before continuing to more specific materials about topics such as deep learning, active learning, representation learning, NLP, computer vision, etc. Feel free to leave your comments, suggestions and thoughts below!
Summary
Reviewer
Vincent Spruyt
Review Date
Reviewed Item
Machine Learning Books
Author Rating
5

comments

  1. David says:

    I think Pattern Classification is great but was still difficult to understand for me. I first read Learning from data which I think is much more accessible for beginners. I’m going to try Duda’s book next

  2. xin.do says:

    I used Machine Learning from Mitchell during my master course on machine learning. It is a great book and I still use lots of what I learned from it in my current job. But the book is quite expensive, wish there was an e-book version.

    • Great to hear that, Xin.do! I agree that Mitchell’s book is more accessible than the work of Bishop or Duda, but I would still recommend these to deepen your knowledge and understanding of what is introduced in Mitchell’s book.

  3. jonathan says:

    Great list, what are the best books on python for machine learning?

    • Although there are quite some books about using python for machine learning, I believe you are better off with the infinite number of online python tutorials that are out there. Start with scikit-learn (http://scikit-learn.org/stable/tutorial/) and then add Pandas (http://pandas.pydata.org/) to your list.
      To really understand the fundamental concepts of the most important machine learning algorithms however, I would still recommend reading a book that focusses solely on machine learning.
      The programming language you choose to implement your ideas is merely a semantic detail, and learning such language is easy compared with learning about data science and pattern recognition.

      • Steve Miller says:

        Sebastian Raschka’s book, (published after the above post) is rapidly becoming very highly regarding in this respect. There is also Joel Grus “Data Science from Scratch2 (also Python). Both are based on the concept of using Python to aid in understanding (as opposed to just being an engineering type guide to “getting the job done in Python” ). There are more detailed explanations of relative merits of each on Amazon.

  4. Suraj Tata Prasad says:

    Very elegantly written, and easy to follow articles. Drove home the concepts of Eigenvalues and PCA really well. Thanks Vincent.

  5. Robin White says:

    After you study machine learning by yourself by reading book, you might need to study some practical exercises to be better. I would recommend to take the machine leaning courses that I took, because it covered almost everything that I wanted. http://www.thedevmasters.com/machine-learning-in-python/

  6. Josep Ma. says:

    Thanks for the recommendation! I just wanted to point out that the second and third “Buy Now” Amazon buttons do not redirect to the correct books page…

Comments are very welcome!