H&E Model Recommendation Tool

Built as a DATA3888 capstone project at the University of Sydney, this tool benchmarks four classifiers (HOG+SVM, ResNet18, DenseNet121, EfficientNet-B0) on H&E-stained breast histology image classification across four clinically meaningful cell types (Invasive Tumour, DCIS1, CD8+ T Cells, Macrophages), six imaging conditions (Full Colour, Harris, Low Resolution, Grayscale, Noise, Understained), and six dataset sizes (500-25,000 images), for a total of 144 runs.

The results are served through an interactive Streamlit tool: given an imaging condition and approximate dataset size, a computational researcher receives an evidence-based model recommendation with supporting metrics, training curves, confusion matrix, and per-class recall.

Quickstart

Python 3.10+ required.

pip install -r requirements.txt
streamlit run app.py

The app will open automatically, or navigate to http://localhost:8501 in your browser.

Project Structure

• app.py - Streamlit application entry point
• precomputed.py - Recommendation logic and data loading
• requirements.txt - Python dependencies
• ShinyAppData-*/ - Model results, curves, and confusion matrices
• iqsim_assets/ - Image quality simulator assets
• figures/ - Project figures and visualisations
• EDA/ - Exploratory data analysis scripts
• models/ - Training pipeline and classification scripts

Model Training

Setup Notes

• Dataset: The original cell images are not included in this repository and were provided as part of the DATA3888 course at the University of Sydney, derived from Taylor et al. (2023).

• Image data path: The pipeline scripts expect the original cell images to be stored at ../../50/, i.e. ../../50/Invasive_Tumor/, etc. Update IMAGE_ROOT in the relevant scripts if your path differs.

• Baseline exploration data path: The baseline-exploration/ scripts expect the original cell images at ../../../100/, i.e. ../../../100/Invasive_Tumor/, etc. Update the scripts if your path differs.

• Metadata file: The CBR metadata file is expected at ../../metadata_code/cbr.csv. Update CBR_PATH in generate_spatial_split.py if your path differs.

• Google Drive (local scripts): he_classification-local-svm.py and he_classification-local-summary.py expect Google Drive to be mounted as a local filesystem at G:/My Drive. Update the path in those scripts if your Drive is mounted elsewhere.

• Image type naming: The original Full Colour Mayer's stain images are referred to as Original throughout the scripts and cache folder structure, alongside the other augmentation types (Grayscale, Harris, Low_Resolution, Noise, Understained).

---

Pipeline

To go from the full raw dataset to a set of ready-to-train caches at all dataset sizes and image types, run the following scripts in order:

generate_spatial_split.py → select_subsample.py → preprocessing.py → generate_cache.py → subsample_cache.py

1. generate_spatial_split.py - generates split_data/split_assignments.csv, assigning every cell image to train/val/test based on spatial location to prevent data leakage.
2. select_subsample.py - copies a balanced subsample of the original Mayer's stain images to ../../50/subsample/Original/.
3. preprocessing.py - applies augmentations to the subsampled images and saves all image types to ../../50/subsample/.
4. generate_cache.py - builds .npy cache files for all image types and saves them to ../../50/cache/n25004/.
5. subsample_cache.py - subsamples the full cache (n=25,004) down to all required dataset sizes (500, 2500, 7500, 12500, 18000) and saves them to ../../50/cache.

Once complete, all model scripts can be run. After all model scripts have finished, run he_classification-local-summary.py to combine results and move them from Google Drive into results/ along with copies of the loss and accuracy curve PNGs.

---

Scripts

Data Preparation

Script	Purpose
generate_spatial_split.py	Produces split_data/split_assignments.csv - a mapping of every cell image filename to its train/val/test/excluded split. Uses a spatial grid of TILE_SIZE × TILE_SIZE tiles overlaid on the full H&E tissue, with a BUFFER-px exclusion strip at tile edges to prevent border overlap across splits. Cell image filenames are read from cell_names/ - one .txt file per cell type. Update CBR_PATH inside the script before use - it expects the metadata file at ../metadata_code/cbr.csv. Run this once before generate_cache.py.
select_subsample.py	Selects a balanced subsample of the original Mayer's stain images (same seed and MAX_PER_CLASS as the cache scripts) and copies them to ../50/subsample/Original/. Run this once before preprocessing so the preprocessing pipeline works on only the relevant images rather than the full dataset.
preprocessing.py	Applies augmentations to the subsampled Original images using torchstain (GPU) or a pure NumPy fallback, producing all image types (Grayscale, Harris, Low_Resolution, Noise, Understained) under ../50/subsample/. Also saves one before/after example PNG per image type to results/augmentations/.

Cache

Run these locally once before training.

Script	Purpose
generate_cache.py	Builds the cache for all image types (Original, Grayscale, Harris, Low_Resolution, Noise, Understained) from the subsampled images, using a spatial grid split (split_assignments.csv) to assign each cell to train/val/test. Eliminates data leakage caused by the 50px image borders shared between neighbouring cells. Produces cached_train.npy, cached_val.npy, cached_test.npy, label arrays, and HOG feature arrays per image type under ../50/cache/n25004/.
subsample_cache.py	Subsamples an existing full cache (n=25,004) to a smaller total image count (e.g. 500, 2500, 7500, 12500, 18000) to generate different dataset sizes. Operates across all image types (Original, Grayscale, Harris, Low_Resolution, Noise, Understained) in one pass, preserving class balance and train/val/test proportions exactly across all of them.
read_cache.py	Minimal script to inspect cache file contents, shape, and data type. Useful for sanity-checking cache output.

Classification

The dataset size and image type used for training can be set at the top of each model script. All other configuration (paths, hyperparameter search space, etc.) is also centralised at the top of each script.

Script	Run on	Models	Input
he_classification-colab-resnet18.py	Google Colab	ResNet18	NumPy cache from Google Drive
he_classification-colab-densenet121.py	Google Colab	DenseNet121	NumPy cache from Google Drive
he_classification-colab-efficientnetb0.py	Google Colab	EfficientNet-B0	NumPy cache from Google Drive
he_classification-local-svm.py	Local	SVM (HOG)	NumPy cache from local drive
he_classification-local-summary.py	Local	All models	Results CSVs + output TXTs from local Google Drive

All Colab scripts use Optuna (10 trials, median pruner) to tune hyperparameters against validation accuracy, with early stopping (patience 5, max 30 epochs) and light augmentation (random flipping and colour jitter). Training curves, a results CSV, and an output TXT are saved to Google Drive on completion. Run he_classification-local-summary.py after all model scripts have finished to combine results and move them from Google Drive into results/ along with copies of the loss and accuracy curve PNGs. Scripts were run on an A100 GPU runtime (80 GB VRAM, 175 GB RAM); A100 access requires Colab Pro+, and some runs may OOM on lower-tier runtimes (e.g. T4).

---

baseline-exploration/ - Feature & Model Selection Scripts

These scripts were used to justify the choice of HOG features over raw pixels and to select the best classifier, on the original 2-class binary task (Invasive Tumour vs CD4+ T Cells, 100 images per class). They are not part of the main training pipeline.

Script	Models	Purpose
pix_vs_hog.py	KNN, SVM, RF, XGBoost	Compares all four classifiers using both raw pixel and HOG features. Runs repeated 5-fold CV (20 repeats) and reports test accuracy for each model–feature combination. Primary script for justifying HOG over pixels.
rf_pix_vs_hog.py	Random Forest	RF-specific pixels vs HOG comparison. Selects ntree via OOB error curve for each feature type separately, then runs repeated CV and test evaluation. Saves OOB curves and a summary CSV to results/rf_pix_vs_hog/.
svm_pix_vs_hog.py	SVM	SVM-specific pixels vs HOG comparison. Tunes C, gamma, and kernel jointly via 5-fold grid search for each feature type, then runs repeated CV and test evaluation. Saves a summary CSV to results/svm_pix_vs_hog/.
hog_comparisons.py	KNN, SVM, RF, XGBoost	Compares all four classifiers using HOG features only. Tunes each model independently (KNN: grid search over k; SVM: grid search over C/gamma/kernel; RF: OOB curve; XGBoost: grid search over n_estimators/max_depth/lr), then runs repeated CV and full test evaluation. Saves a summary CSV and RF OOB curve to results/hog_comparisons/.

HOG features use an exact reimplementation of R's OpenImageR::HOG(image, cells=8, orientations=9) with a [-1, 0, 1] gradient kernel (not scikit-image's [-0.5, 0, 0.5]) to match the R pipeline.

---

Dependencies (Local)

cd models
pip install -r requirements.txt

For PyTorch with CUDA support, visit https://pytorch.org/get-started/locally/, input your specs, and retrieve the pip command for your setup.

Example:

pip install torch torchvision --index-url https://download.pytorch.org/whl/cu130 --force-reinstall

To check CUDA availability and version on NVIDIA GPUs:

nvidia-smi

Dependencies (Google Colab)

pip install optuna

---

Google Colab Setup

Mount Google Drive and clone the repo:

from google.colab import drive
drive.mount('/content/drive')

TOKEN = "your_token_here"
!git clone https://{TOKEN}@github.com/bbat2575/he-model-recommender.git
%cd he-model-recommender
!git config user.email "your@email.com"
!git config user.name "Your Name"
!git remote set-url origin https://{TOKEN}@github.com/bbat2575/he-model-recommender.git

Note: If the script fails to connect to your repo on the first attempt, just try again.

---

Generating a GitHub Personal Access Token

1. Sign into GitHub
2. Click your profile picture → Settings
3. Scroll to Developer settings (bottom of left sidebar)
4. Click Personal access tokens → Tokens (classic)
5. Click Generate new token (top right) → Generate new token (classic)
6. In the Note field, enter a name e.g. colab-access
7. Set Expiration to 60 days
8. Tick the repo scope checkbox (top level)
9. Click Generate token (bottom of page)
10. Copy it immediately - you cannot view it again after leaving the page

Paste the token in place of your_token_here in both {TOKEN} positions in the Colab setup cell each session.

Example:

https://jkl_swCeGpwlejWxcPLTSsfJFDURnbmKL93QWaNM@github.com/bbat2575/he-model-recommender.git

Don't share your token with anyone. Save it somewhere secure on your local machine.

Screenshots

---

---