Skip to content

CONTRIBUTING.md

Latest commit

 

History

History
317 lines (212 loc) · 7.41 KB

CONTRIBUTING.md

File metadata and controls

317 lines (212 loc) · 7.41 KB
Raw

CONTRIBUTING.md

Overview

RAGU is a local-first RAG application focused on question answering over structural biology PDFs. The project favors practical self-hosted dependencies and a compact Python codebase over deep abstraction.

This document explains the implemented architecture, the main technical decisions, and the coding/operational constraints contributors should preserve.

Current implementation summary

flowchart TD
    PDF[PDF upload] --> Loader[PyMuPDFLoader]
    Loader --> Splitter[RecursiveCharacterTextSplitter]
    Splitter --> Embed[Ollama embeddings]
    Embed --> Chroma[Chroma service]

    Prompt[Question] --> Retrieve[Chroma retrieval]
    Retrieve --> Rerank[Cross-encoder reranking]
    Rerank --> Chat[Ollama chat]
    Chat --> Stream[Streamlit streamed answer]
    Stream --> PDB[PDB extraction]
    PDB --> MolStar[Mol* viewer]
    Stream --> TTS[Piper TTS when available]
Loading

Core technical decisions

1. Single-file application entrypoint

The majority of product logic still lives in app.py.

Why:

  • the project is still small enough to keep iteration speed high
  • deployment and debugging remain straightforward
  • most changes still touch one vertical slice of the app

Tradeoff:

  • concerns are mixed
  • modular reuse and testing are harder than they should be

2. Self-hosted inference with Ollama

Ollama is used for:

  • chat generation
  • embedding generation

Why:

  • keeps the app local/self-hosted
  • avoids cloud inference dependencies
  • fits privacy-sensitive local workflows

Tradeoff:

  • heavier startup and runtime resource usage
  • model selection materially affects usability on low-end hardware

3. Dedicated Chroma service over HTTP

The Docker deployment no longer uses embedded Chroma persistence inside the app container. The app talks to a dedicated Chroma service via chromadb.HttpClient(...).

Why:

  • avoids embedded SQLite permission issues inside the app container
  • aligns with Chroma’s supported client/server deployment model
  • separates vector storage concerns from the app runtime

Tradeoff:

  • adds another service to the stack
  • local development becomes more operationally involved

4. Cross-encoder reranking

Retrieved chunks are reranked with cross-encoder/ms-marco-MiniLM-L-6-v2.

Why:

  • improves answer relevance over plain nearest-neighbor retrieval
  • keeps the retrieval pipeline simple while improving precision

Tradeoff:

  • extra CPU and model-loading overhead

5. Optional Piper text-to-speech

Piper is treated as an optional enhancement.

Why:

  • improves usability when audio playback is desired
  • remains fully optional so the app still works in text-only environments

Current behavior:

  • the app checks whether Piper is available
  • if Piper is unavailable, the app shows a visible status message and remains text-only
  • if Piper synthesis succeeds, Streamlit renders an audio player in the browser

Tradeoff:

  • architecture-specific install behavior
  • extra build-time and image-size cost

6. Separate observability for Chroma and Ollama

The stack has two distinct monitoring/observability paths:

  • Chroma -> OTEL Collector -> Zipkin (+ Prometheus scrape of collector metrics)
  • Ollama -> community exporter -> Prometheus -> Grafana

Why:

  • Chroma already documents an OTEL-based observability path
  • Ollama does not expose a first-party monitoring stack, so exporter-based monitoring is the practical option

Tradeoff:

  • more containers
  • one monitoring dependency is community-maintained, not official

Runtime architecture

Application layer

Implemented in Streamlit inside app.py.

Responsibilities:

  • PDF upload
  • answer prompt input
  • streamed answer rendering
  • PDB visualization
  • optional TTS playback
  • retrieval debugging visibility

Ingestion layer

Implemented with:

  • PyMuPDFLoader
  • RecursiveCharacterTextSplitter

Responsibilities:

  • temporary file handling
  • PDF parsing
  • chunk generation

Vector layer

Implemented through Chroma.

Responsibilities:

  • embedding-backed storage
  • retrieval
  • collection lifecycle

Generation layer

Implemented through Ollama.

Responsibilities:

  • embedding inference
  • chat inference
  • streamed response delivery

Reranking layer

Implemented through sentence_transformers.CrossEncoder.

Responsibilities:

  • rerank retrieved chunks
  • improve context selection before generation

Visualization layer

Implemented with Mol* in a Streamlit HTML embed.

Responsibilities:

  • detect PDB code
  • render structure when possible

Monitoring layer

Implemented through:

  • OTEL Collector
  • Zipkin
  • community Ollama exporter
  • Prometheus
  • Grafana
  • cAdvisor

Service topology

flowchart LR
    App[app]
    Chroma[chroma]
    Ollama[ollama]
    Redis[redis]
    OTel[otel-collector]
    Zipkin[zipkin]
    Exporter[ollama-exporter]
    Prom[prometheus]
    Grafana[grafana]
    CAdvisor[cadvisor]

    App --> Chroma
    App --> Ollama
    App -. future .-> Redis

    Chroma --> OTel
    OTel --> Zipkin
    OTel --> Prom

    Ollama --> Exporter
    Exporter --> Prom
    CAdvisor --> Prom
    Prom --> Grafana
Loading

Dependency rationale

Application/runtime

  • streamlit: fast local UI
  • ollama: self-hosted LLM/embedding inference
  • chromadb: vector store client

Document processing

  • langchain_community
  • langchain_core
  • langchain_text_splitters
  • PyMuPDF

Reranking/ML

  • sentence_transformers
  • torchvision

Operations

  • Docker Compose
  • Chroma server
  • OTEL Collector
  • Zipkin
  • Prometheus
  • Grafana
  • cAdvisor
  • Redis

Optional TTS

  • Piper binary and voice model installed in the app image when supported

Coding decisions to preserve

  • Use env-driven configuration instead of hardcoded hostnames, models, file paths, or binary paths.
  • Keep optional integrations optional. Piper and Redis should never block the core Q&A path.
  • Prefer graceful degradation over hard failure for non-core features.
  • Avoid “hidden” server-side UX behavior. If a feature is unavailable, tell the user.
  • Keep the Chroma HTTP mode working as the default Docker path.
  • Do not reintroduce direct ollama.chat(...) calls that bypass OLLAMA_BASE_URL.

Development workflow

Local

  1. Create a Python virtual environment.
  2. Install requirements.txt.
  3. Start Ollama locally and pull the required models.
  4. Optionally configure local Piper.
  5. Run streamlit run app.py.

Docker Compose

  1. Review .env.
  2. Build the app image.
  3. Start the stack with docker compose up --build.
  4. Wait for Ollama model bootstrap on first start.
  5. Use Grafana/Zipkin if you need monitoring or tracing.

To run against an Ollama service on the Docker host, use the host override instead of the bundled ollama service:

docker compose -f docker-compose.yml -f docker-compose.host-ollama.yml --profile minimal up --build

Contribution rules

  • Keep changes simple.
  • Prefer root-cause fixes over workaround patches.
  • Minimize blast radius.
  • Keep docs synchronized with code and Compose.
  • If you add or remove services, update the architecture diagrams.
  • If you change model defaults, update .env and README.md.
  • If you change TTS behavior, document the fallback mode explicitly.

Current gaps

  • app.py is still monolithic
  • no automated tests yet
  • Redis not yet integrated in the Python runtime
  • frontend styling is still mostly Streamlit-default
  • monitoring dashboard is intentionally conservative and could be improved with richer exporter-specific panels