AI AtlasStudy route for ML and AI

One finite route. Exact scope. Resources with jobs.

01
Foundational · Core route

Mathematical Foundations

The mathematical language needed to understand, implement and diagnose learning systems.

5 sections37 study units · about 167 hours73 required topics10 optional / depth / later
Passed0/37 units
x
0 Learned0 Practised0 Passed
Saved in this browser only.
01/05

Current section

Linear Algebra & Proof

Why

Models are compositions of linear maps. Geometry, rank and conditioning explain what a model can represent and when a correct formula still fails numerically.

Study units, in order

8 units · about 42 hours · covers 18 topics

Tick Learn when you finish the study list, Practice when the named practice passes, and Pass when you meet Stop when without notes.

Section mini-project · about 10 to 14 hours

Least squares and PCA with rank and conditioning failures

Uses U1, U2, U3, U4, U5, U6, U7, U8

Start the shared numlab/ repository with a linear-algebra package: a QR least-squares solver, power iteration, a cutoff-aware pseudoinverse and PCA. Each routine is checked against an independent NumPy oracle and shown failing on a case you construct, and the solver with its conditioning evidence carries into the capstone.

Steps

  1. Before coding, write notes/linalg.md: derive the normal equations, show the least-squares projector is symmetric and idempotent, and present PCA as both maximum variance and best rank approximation.
  2. Implement thin QR by modified Gram-Schmidt or Householder reflections and solve least squares by back substitution on Rx=Q⊤bRx=Q^\top b. Check a full-rank tall case against numpy.linalg.lstsq with projector identities, keeping the matrix and tolerances your choice.
  3. Make one column a combination of the others, show the small diagonal entry of R and the exploding solution, then detect it with a relative threshold on R entries that you justify, either raising an error or falling back to the minimum-norm solution.
  4. Add power iteration on a symmetric matrix reporting the Rayleigh quotient and the eigen-residual ∥Av−ρv∥\|Av-\rho v\| each step. Match numpy.linalg.eigh on a separated case, then construct a tied-magnitude case where the quotient settles at a non-eigenvalue and the residual rule reports failure.
  5. Build a pseudoinverse from numpy.linalg.svd with a cutoff relative to the largest singular value, checking it against numpy.linalg.pinv and the Penrose conditions, and show the no-cutoff behavior. Implement PCA on centered data and compare components through projectors or reconstructions so sign flips cannot fail.
  6. Assert shapes in every routine, including a batched case, and compare flop counts for the normal equations against QR. Then solve a rising-condition family from a known x∗x^*, tabulating conditioning, residual and error per solver and explaining the digit loss from the residual bound.

Deliverables

  • numlab/linalg/: QR least squares with rank detection, power iteration, the cutoff pseudoinverse and PCA.
  • Tests against NumPy oracles covering projector and Penrose checks, both failure cases, sign-invariant comparison and shapes.
  • notes/linalg.md with the derivations and notes/linalg-defense.md with the conditioning table and solver choice.

How it is checked

  • Rubric Self-grade with the rubric below

Rubric

CriterionPassStrong
Written derivationsThe notes derive the normal equations, projector properties and both PCA views without copying.Each derivation names the assumption it needs at the step that uses it.
Least squaresQR matches numpy.linalg.lstsq on a full-rank case, and the rank-deficient case is detected, not returned.The rank threshold is justified from the scale of R and rounding, with a minimum-norm fallback checked against the pseudoinverse.
Spectral methodsPower iteration matches numpy.linalg.eigh on the separated case and reports failure on the tied case.The note explains the settled quotient from the tied components, and the stopping rule uses the eigen-residual.
SVD and PCAThe pseudoinverse passes the Penrose conditions, and PCA reconstruction error matches the discarded singular values.Every comparison is sign-invariant, and the no-cutoff behavior is shown.
Conditioning evidenceThe table reports conditioning, residual and error for both solvers across the family.The observed digit loss is compared with the residual bound and the squared conditioning of the normal equations.

Feeds into: Capstone least squares and PCA: the QR solver with its rank failure, sign-invariant PCA checks and the conditioning table behind the solver choice.

Section checkpoint

Know when to leave

Derive least squares and PCA, implement both from a blank file, and diagnose one rank-deficient or ill-conditioned case.

Deliberately left for later
  • Advanced matrix perturbation bounds
  • Specialized sparse-direct solvers
  • Functional analysis beyond what later tracks require

Stage milestone

Numerical Learning Laboratory

Build a small NumPy project that connects least squares and PCA, reverse-mode differentiation, Monte Carlo uncertainty, experiment analysis and optimizer stability. Each component must include one failure case and one independent check.

1Derivations and assumption notes2Runnable implementation with tests and raw numerical comparisons3A short defense of solver, estimator and tolerance choices

Capstone rubric · about 10 to 14 hours

Built from the section mini-projects. Grade yourself against Pass first; Strong is the bar for research and engineering roles.

CriterionPassStrong
Least squares and PCAThe linear-algebra project meets Pass on every criterion, with its rank-deficient case and conditioning table runnable from the suite.The README links the conditioning table to the solver choice in the defense note.
Reverse-mode differentiationThe engine and harness pass, and the planted bug fails its directional check.One command reruns the step-size sweep and the planted-bug test.
Monte Carlo uncertaintyEvery simulation check in the repository reports its sample count and standard error and uses the stated acceptance rule.The heavy-tail failure and the drifting-predictive failure are both kept as tests.
Experiment analysisThe audit, the rerun and the decision memo are in the repository with the plan that preceded them.The memo stands on its own for a reader who has not seen the code.
Optimizer stabilityThe optimizer comparison, the divergence diagnosis and the finite-precision failure are reproduced by the suite.Each prediction sits beside its measurement in the notes.
Defense and rebuildA short defense note justifies each solver, estimator and tolerance choice, and a README lists the commands that rebuild every reported number.Every component names one failure case and one independent check in the README.