1 New Intelligence Compilation1
Source: iamwire.com
Author: Wale Akinfaderin
Translator: Liu Xiaoqin
[New Intelligence Guide]The main purpose of this article is to provide resources and give advice on the mathematics needed for machine learning. Beginners in mathematics need not be discouraged, as one does not need to master a large amount of mathematical knowledge to start learning machine learning. As mentioned in this article, the most basic requirement is data analysis, and you can continue to learn mathematics while mastering more techniques and algorithms.
In recent months, many people have contacted me to express their enthusiasm for data science and for exploring statistical patterns using machine learning techniques to develop data-driven products. However, I found that some of them actually lack the necessary mathematical intuition and framework to obtain useful results. This is the main reason I wrote this article.
Recently, many useful machine and deep learning software have become very accessible, such as scikit-learn, Weka, Tensorflow, etc. The theory of machine learning intersects with statistics, probability theory, computer science, algorithms, and other areas. It arises from iterative learning based on data, attempting to uncover hidden insights for developing intelligent applications. Despite the infinite possibilities of machine learning and deep learning, having a comprehensive mathematical understanding of these technologies is necessary for understanding the internal workings of algorithms and obtaining good results.
Why is math important in machine learning? I want to emphasize the following points:
-
Choosing the right algorithm involves considering algorithm accuracy, training time, model complexity, the number of parameters, and the number of features.
-
Choosing parameter settings and validation strategies.
-
Understanding the trade-off between bias and variance to determine underfitting and overfitting.
-
Estimating correct confidence intervals and uncertainties.
Trying to understand an interdisciplinary field like machine learning, the main question is the amount of necessary mathematical knowledge and the level of math required to understand these techniques. The answer to this question is multifaceted and depends on individual levels and interests. Research on mathematical formulas and theoretical developments in machine learning is ongoing, with some researchers studying more advanced techniques. Below, I will outline what I believe to be the minimum level of math required to become a machine learning scientist/engineer, as well as the importance of each mathematical concept.
1. Linear Algebra
Skyler Speakman once said, “Linear algebra is the mathematics of the 21st century,” and I completely agree with this statement. In the field of ML, linear algebra is everywhere. Concepts such as Principal Component Analysis (PCA), Singular Value Decomposition (SVD), eigen decomposition, LU decomposition, QR decomposition, symmetric matrices, orthogonalization & standard orthogonalization, matrix operations, projections, eigenvalues & eigenvectors, vector spaces, and norms are essential for understanding optimization methods in machine learning. One great thing about linear algebra is that there are abundant resources available online. I always say traditional classrooms should fade away because there is such a wealth of resources on the internet. My favorite linear algebra course is by Professor Gilbert Strang from MIT.
2. Probability and Mathematical Statistics
Machine learning and mathematical statistics are not entirely different fields. In fact, recently someone defined machine learning as “doing mathematical statistics on a Mac.” The foundational knowledge of mathematical statistics and probability theory required for ML includes combinatorial mathematics, probability rules & axioms, Bayes’ theorem, random variables, variance and mean, conditional and joint distributions, standard distributions (Bernoulli, binomial, multinomial, uniform, and Gaussian), moment-generating functions, maximum likelihood estimation (MLE), prior and posterior distributions, maximum a posteriori estimation (MAP), and sampling methods.
3. Multivariable Calculus
Essential concepts include calculus, partial derivatives, vector functions, directional gradients, Hessians, Jacobians, Laplacians, and Lagrangian distributions.
4. Algorithms and Complexity Optimization
This is important for understanding the computational efficiency and scalability of machine learning algorithms, as well as the sparsity of dataset development. Knowledge of data structures (binary trees, hashing, heaps, stacks, etc.) is required, along with knowledge of dynamic programming, random & sublinear algorithms, graphs, gradients/stochastic trends, and primal-dual methods.
5. Others
This includes other mathematical concepts not covered by the four main areas mentioned above. This includes real analysis and complex analysis (sets and sequences, topological structures, metric spaces, single-valued and continuous functions, limits), information theory (entropy, information gain), function spaces, and manifolds.
Below are some MOOCs and learning resources for the mathematical concepts required for machine learning:
-
Khan Academy’s Linear Algebra, Probability & Statistics, Multivariable Calculus and Optimization.
-
Coding the Matrix: Linear Algebra through Computer Science Applications by Philip Klein, Brown University.
-
Linear Algebra – Foundations to Frontiers by Robert van de Geijn, University of Texas.
-
Applications of Linear Algebra, Part 1 and Part 2. A newer course by Tim Chartier, Davidson College.
-
Joseph Blitzstein – Harvard Stat 110 lectures
-
Larry Wasserman’s book – All of statistics: A Concise Course in Statistical Inference.
-
Boyd and Vandenberghe’s course on Convex optimization from Stanford.
-
Linear Algebra – Foundations to Frontiers on edX.
-
Udacity’s Introduction to Statistics.
Finally, the main purpose of this article is to provide advice and useful resources regarding the important mathematical concepts needed for machine learning. However, some machine learning enthusiasts who may be beginners in mathematics might find this article discouraging (this is not my intention). For beginners, you do not need to master a large amount of mathematical knowledge before starting machine learning. As mentioned in this article, the most basic requirement is data analysis, and you can continue to learn mathematics while mastering more techniques and algorithms.
