Machine Learning Algorithms from Scratch

This project implements various machine learning algorithms from scratch using Python and NumPy, without relying on external libraries such as TensorFlow, Keras, or scikit-learn. The implemented algorithms include classification, regression, clustering, and basic neural network models.

Table of Contents

• Algorithms Implemented
  • Classification
    • K-Nearest Neighbors (KNN)
    • Logistic Regression
    • Naive Bayes
  • Regression
    • Linear Regression
  • Clustering
    • K-Means Clustering
  • Neural Networks
    • Perceptron
    • Neural Network
• Project Structure
• Getting Started
• Usage
• Contributions and Feedback
• Contact

Algorithms Implemented

Classification

1. K-Nearest Neighbors (KNN)

   • The KNN algorithm classifies data points based on the majority class of their k-nearest neighbors in the feature space.

2. Logistic Regression

   • Logistic Regression is a binary classification algorithm that models the probability of a data point belonging to a particular class using a logistic function.

3. Naive Bayes

   • The Naive Bayes algorithm is a probabilistic classifier based on Bayes' theorem with the assumption of independence between features.

Regression

4. Linear Regression
   • Linear Regression is used for predicting continuous numerical values by fitting a linear relationship between the input features and the target variable.

Clustering

5. K-Means Clustering
   • K-Means is an unsupervised clustering algorithm that partitions data points into 'k' clusters based on their similarity in the feature space.

Neural Networks

6. Perceptron

   • The Perceptron is a basic neural network unit that learns to classify inputs into two categories using a weighted sum and a threshold activation function.

7. Neural Network

   • This implementation represents a simple feedforward neural network with customizable architecture, including multiple layers and activation functions.

Project Structure

• /algorithms

  • Contains implementation files for each algorithm (e.g., KNN, KMeans, ...).

• Example.py

  • An example usage script that demonstrates how to use each implemented algorithm on sample datasets.

• README.md

  • The main documentation file (this file) that provides an overview of the project, algorithms, and instructions for usage.

Otherwise, this is the repository tree:

├───Classification
│   ├───KNN
│   │   └───KNN.py
│   │   └───Example.py
│   ├───LogisticRegression
│   │   └───LogisticRegression,py
│   │   └───Example.py
│   └───Naive Bayes
│       └───NaiveBayes.py
│   │   └───Example.py
├───Clustering
│   └───KMeans
│       └───KMeans.py
│   │   └───Exampl.py
├───Neural Network
│   ├───Neural Network
│   │   └───NeuralNetwork.py
│   │   └───train_dataset.csv
│   └───Perceptron
│       └───Perceptron.py
│   │   └───Example.py
└───Regression
    └───LinearRegression
│   │   └───LinearRegression.py
        └───Example.py