feat: LoRA training pipeline + Colab notebook for free GPU fine-tuning

[oyi77]

Summary

Adds LoRA (Low-Rank Adaptation) support for parameter-efficient fine-tuning of OpenMythos models. Includes a complete training pipeline and Colab notebook for free GPU training.

Changes

open_mythos/lora.py

• LoRAConfig: Configuration (rank, alpha, dropout, target_modules)
• LoRALinear: Linear layer with low-rank adapter (A + B matrices)
• apply_lora(): Model-level LoRA application
• save/load_lora_adapter(): Lightweight adapter persistence (~1-10MB)
• merge_lora_weights(): Merge LoRA into base model for inference

training/lora_finetune.py

• Complete CLI training script
• Built-in finance demo dataset
• Mixed precision (FP16), gradient clipping, cosine LR scheduler
• Custom dataset support (JSONL/JSON/TXT)

notebooks/OpenMythos_LoRA_FineTune.ipynb

• Step-by-step Colab notebook (free T4 GPU)
• QLoRA mode for 8GB VRAM
• Finance/trading demo data

Usage

from open_mythos import OpenMythos, mythos_1b
from open_mythos.lora import LoRAConfig, apply_lora, save_lora_adapter

model = OpenMythos(mythos_1b())
model = apply_lora(model, LoRAConfig(rank=16, alpha=32))
# Train...
save_lora_adapter(model, 'my_adapter.pt')

CLI

# Standard LoRA (16GB VRAM)
python training/lora_finetune.py --variant 1b --dataset finance

# QLoRA (8GB VRAM, fits Colab free T4)
python training/lora_finetune.py --variant 1b --dataset finance --qlora

Key Features

• Only ~0.5% parameters trained (LoRA)
• Adapter file: ~1-10MB (shareable)
• QLoRA: INT4 + LoRA = 8GB VRAM
• Free GPU compatible (Colab T4, Kaggle)