Skip to content
LearnMLByDoing
Free
02

Classical Machine Learning

Optimisation, regression, regularisation and model selection — the statistical toolkit that still wins most tabular problems.

Lessons
8
Study time
4.4 h
Access
First lesson free

Syllabus

  1. 2.1

    Gradient descent, saddle points and optimizer dynamics

    Race SGD, Momentum and Adam across a convex bowl, a saddle and the Rosenbrock valley, then write your own Adam and watch it roll.

    40 minFreeLive 3D lab and Python challengeintermediate
    • Derive the gradient descent update and the role of the learning rate
    • Explain how momentum accumulates velocity across a ravine
    • Implement Adam's bias-corrected first and second moment estimates
  2. 2.2

    Linear and logistic regression from first principles

    Closed-form least squares as a projection, then the sigmoid, cross-entropy and the convex surface it creates.

    30 minProLive 3D lab and Python challengeintermediate
    • Solve least squares with the normal equations and as a projection
    • Derive the logistic regression gradient from cross-entropy
    • Interpret coefficients, odds ratios and decision boundaries
  3. 2.3

    Bias, variance and regularisation

    Why more capacity can hurt, how ridge and lasso trade bias for variance, and how to choose λ with cross-validation.

    28 minProLive 3D lab and Python challengeintermediate
    • Decompose expected error into bias, variance and noise
    • Compare ridge and lasso geometrically
    • Select hyperparameters with k-fold cross-validation
  4. 2.4

    Decision trees, random forests and gradient boosting

    Axis-aligned partitions of feature space, bagging to cut variance, and boosting as gradient descent in function space.

    36 minProLive 3D lab and Python challengeintermediate
    • Grow a tree by greedy impurity reduction
    • Explain why bagging decorrelates and stabilises predictions
    • Read gradient boosting as functional gradient descent
  5. 2.5

    Support vector machines and the kernel trick

    Maximise the margin, then lift the data into a space where a plane can separate it, without ever computing that space.

    34 minProLive 3D lab and Python challengeintermediate
    • Derive the maximum-margin objective and identify support vectors
    • Trade margin width against violations with the C parameter
    • Use polynomial and RBF kernels as implicit feature maps
  6. 2.6

    k-means, Gaussian mixtures and EM

    Unsupervised structure: Lloyd's algorithm, k-means++ seeding, and soft clustering with expectation-maximisation.

    34 minProLive 3D lab and Python challengeintermediate
    • Run k-means and explain why it converges to a local optimum
    • Fit a Gaussian mixture with expectation-maximisation
    • Choose the number of clusters with inertia, silhouette and BIC
  7. 2.7

    PCA and dimensionality reduction

    Find the directions of greatest variance, project onto them, and see where a linear projection fails on a curled-up manifold.

    32 minProLive 3D lab and Python challengeintermediate
    • Derive principal components as eigenvectors of the covariance matrix
    • Compute PCA through the SVD and choose k from explained variance
    • Recognise when nonlinear methods such as kernel PCA, t-SNE or UMAP are needed
  8. 2.8

    Precision, recall, ROC and calibration

    Slide a decision threshold through two score distributions and watch the confusion matrix, ROC and precision-recall curves respond.

    32 minFree previewLive 3D lab and Python challengeintermediate
    • Compute precision, recall, F1 and specificity from a confusion matrix
    • Read ROC and precision-recall curves, and explain why AUC is a ranking probability
    • Pick a threshold from error costs and handle class imbalance