AS
All work

Project 04

Machine Learning Algorithms

Classic ML implemented from first principles

This project was designed to build a deep understanding of machine learning algorithms by implementing them from scratch, learning the inner workings, mathematical foundations and practical trade-offs of each one instead of treating them as black boxes.

PythonMachine LearningOOP
Source on GitHub
Abstract data visualization representing a decision boundary
Algorithms
4 from scratch
Best result
98.8% (RBF SVM)
Stack
NumPy · Pandas · OpenCV
Approach
Math first, library second

Why build them by hand

Coding these algorithms manually surfaced strengths, weaknesses and implementation pitfalls that stay invisible when you simply import them from a library. It solidified my understanding of the underlying principles and improved how I select and tune models for a given problem.

01 · Naïve Bayes classifier

Python · NumPy · Pandas

  • Implemented Naïve Bayes from scratch and applied it to BritishScottish.csv and other toy datasets
  • Achieved accurate classification and handled unseen data with Laplace smoothing
  • Built an understanding of probabilistic model foundations and the importance of smoothing

02 · Logistic regression on German Credit

Python · Pandas · scikit-learn

  • Applied logistic regression to the German Credit dataset to classify creditworthiness
  • Reached reliable accuracy and recall through cross-validation
  • Confirmed logistic regression as an interpretable baseline, and how much preprocessing and proper evaluation matter

03 · k-Nearest Neighbors

Python · NumPy · Pandas · Matplotlib · Plotly

  • Built both distance-weighted and unweighted KNN classifiers
  • Applied them to reviews_sentiment.csv to predict star ratings, with 3D visualizations of the feature space
  • Best accuracy at k = 4 to 5, with distance-weighted KNN outperforming majority voting
  • Normalization proved essential for distance-based methods, and distance weighting improved robustness

04 · Support Vector Machine for image classification

Python · NumPy · OpenCV · Matplotlib · Seaborn · scikit-learn

  • Implemented a linear SVM from scratch using SGD on the hinge loss
  • Extended it with One-vs-Rest for multi-class classification (cow, sky, grass)
  • Compared against scikit-learn's SVC with linear, polynomial and RBF kernels
  • Best performance with the RBF kernel at C = 10 → 98.8% accuracy
  • Used the trained model for pixel-level image classification, recoloring sky to light blue, grass to green and cow to brown

The section built up an understanding of hinge loss, SVM optimization, the kernel trick for non-linear separation, and practical image segmentation from pixel colors.

Next project

Bioinspired Robotics