MATHEMATICS
FOR
MACHINE LEARNING
Marc Peter Deisenroth
A. Aldo Faisal
Cheng Soon Ong
Contents
Foreword
1
Part I
1
1.1
1.2
1.3
2
2.1
2.2
2.3
2.4
2.5
2.6
2.7
2.8
2.9
Mathematical Foundations
9
11
12
13
16
17
19
22
27
35
40
44
48
61
63
64
70
71
72
75
76
78
79
80
81
91
94
96
98
99
i
Introduction and Motivation
Finding Words for Intuitions
Two Ways to Read This Book
Exercises and Feedback
Linear Algebra
Systems of Linear Equations
Matrices
Solving Systems of Linear Equations
Vector Spaces
Linear Independence
Basis and Rank
Linear Mappings
Affine Spaces
Further Reading
Exercises
Analytic Geometry
Norms
Inner Products
Lengths and Distances
Angles and Orthogonality
Orthonormal Basis
Orthogonal Complement
Inner Product of Functions
Orthogonal Projections
Rotations
Further Reading
Exercises
Matrix Decompositions
Determinant and Trace
3
3.1
3.2
3.3
3.4
3.5
3.6
3.7
3.8
3.9
3.10
4
4.1
This material is published by Cambridge University Press as
Mathematics for Machine Learning
by
Marc Peter Deisenroth, A. Aldo Faisal, and Cheng Soon Ong (2020). This version is free to view
and download for personal use only. Not for re-distribution, re-sale, or use in derivative works.
©by M. P. Deisenroth, A. A. Faisal, and C. S. Ong, 2021.
https://mml-book.com.
ii
4.2
4.3
4.4
4.5
4.6
4.7
4.8
Eigenvalues and Eigenvectors
Cholesky Decomposition
Eigendecomposition and Diagonalization
Singular Value Decomposition
Matrix Approximation
Matrix Phylogeny
Further Reading
Exercises
Vector Calculus
Differentiation of Univariate Functions
Partial Differentiation and Gradients
Gradients of Vector-Valued Functions
Gradients of Matrices
Useful Identities for Computing Gradients
Backpropagation and Automatic Differentiation
Higher-Order Derivatives
Linearization and Multivariate Taylor Series
Further Reading
Exercises
Probability and Distributions
Construction of a Probability Space
Discrete and Continuous Probabilities
Sum Rule, Product Rule, and Bayes’ Theorem
Summary Statistics and Independence
Gaussian Distribution
Conjugacy and the Exponential Family
Change of Variables/Inverse Transform
Further Reading
Exercises
Continuous Optimization
Optimization Using Gradient Descent
Constrained Optimization and Lagrange Multipliers
Convex Optimization
Further Reading
Exercises
Contents
105
114
115
119
129
134
135
137
139
141
146
149
155
158
159
164
165
170
170
172
172
178
183
186
197
205
214
221
222
225
227
233
236
246
247
5
5.1
5.2
5.3
5.4
5.5
5.6
5.7
5.8
5.9
6
6.1
6.2
6.3
6.4
6.5
6.6
6.7
6.8
7
7.1
7.2
7.3
7.4
Part II Central Machine Learning Problems
8
8.1
8.2
8.3
8.4
8.5
When Models Meet Data
Data, Models, and Learning
Empirical Risk Minimization
Parameter Estimation
Probabilistic Modeling and Inference
Directed Graphical Models
249
251
251
258
265
272
278
Draft (2022-01-11) of “Mathematics for Machine Learning”. Feedback:
https://mml-book.com.
Contents
8.6
9
9.1
9.2
9.3
9.4
9.5
10
10.1
10.2
10.3
10.4
10.5
10.6
10.7
10.8
11
11.1
11.2
11.3
11.4
11.5
12
12.1
12.2
12.3
12.4
12.5
12.6
Model Selection
Linear Regression
Problem Formulation
Parameter Estimation
Bayesian Linear Regression
Maximum Likelihood as Orthogonal Projection
Further Reading
Dimensionality Reduction with Principal Component Analysis
Problem Setting
Maximum Variance Perspective
Projection Perspective
Eigenvector Computation and Low-Rank Approximations
PCA in High Dimensions
Key Steps of PCA in Practice
Latent Variable Perspective
Further Reading
Density Estimation with Gaussian Mixture Models
Gaussian Mixture Model
Parameter Learning via Maximum Likelihood
EM Algorithm
Latent-Variable Perspective
Further Reading
Classification with Support Vector Machines
Separating Hyperplanes
Primal Support Vector Machine
Dual Support Vector Machine
Kernels
Numerical Solution
Further Reading
iii
283
289
291
292
303
313
315
317
318
320
325
333
335
336
339
343
348
349
350
360
363
368
370
372
374
383
388
390
392
395
References
©2021 M. P. Deisenroth, A. A. Faisal, C. S. Ong. Published by Cambridge University Press (2020).
微控制器 MCU
京公网安备 11010802033920号
Copyright © 2005-2022 EEWORLD.com.cn, Inc. All rights reserved
评论