Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CONTEXT.md
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@ examples/
ask_interaction.py # ASK KI with a typed BindingModel
post_measurement.py # POST KI with argument and result BindingModels
11-custom_datatypes.py # Custom literal datatypes with Datatype and RdfLiteral
12-dockerized/ # Dockerfile + compose.yaml for running a KB next to an SC/KD
custom-settings/
custom_settings.py # KnowledgeBaseSettings subclass + ki_from_settings pattern
settings.yaml # Example YAML config for all four KI types
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,7 @@ The [`examples/`](./examples/) directory contains runnable examples covering all
| [09-sparql-store/](./examples/09-sparql-store/) | Connecting a SPARQL store as a knowledge base |
| [10-cli.py](./examples/10-cli.py) | Start a KB via the `knowledge-mapper run` CLI |
| [11-custom_datatypes.py](./examples/11-custom_datatypes.py) | Literals with custom (non-XSD) datatypes in binding models |
| [12-dockerized/](./examples/12-dockerized/) | Package a KB as a Docker image and run it with a Smart Connector via Docker Compose |

See the [examples README](./examples/README.md) for prerequisites and setup instructions.

Expand Down
5 changes: 5 additions & 0 deletions examples/12-dockerized/.dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
*
!pyproject.toml
!uv.lock
!app.py
!asker.py
44 changes: 44 additions & 0 deletions examples/12-dockerized/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# syntax=docker/dockerfile:1.7
#
# Packages the example knowledge base app as a Docker image. Build context is
# this folder:
#
# docker build -t km-dockerized-example .

# ---------- Builder: install dependencies into /opt/venv with uv

FROM ghcr.io/astral-sh/uv:python3.13-bookworm-slim AS builder

ENV UV_COMPILE_BYTECODE=1 \
UV_LINK_MODE=copy \
UV_PYTHON_DOWNLOADS=never \
UV_PROJECT_ENVIRONMENT=/opt/venv

WORKDIR /src

# Only the dependency manifests, so this layer stays cached until they change.
COPY pyproject.toml uv.lock ./

RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --locked --no-dev

# ---------- Runtime: slim Python image with the venv, the app and a non-root user

FROM python:3.13-slim-bookworm AS runtime

RUN groupadd --system app && useradd --system --gid app --home-dir /app --create-home app

COPY --from=builder /opt/venv /opt/venv

ENV PATH="/opt/venv/bin:$PATH" \
PYTHONDONTWRITEBYTECODE=1 \
PYTHONUNBUFFERED=1

WORKDIR /app
COPY app.py asker.py ./

USER app

# The KB is a client of its Smart Connector, so no port is exposed. Exec form
# keeps the CLI as PID 1, so it receives SIGTERM and unregisters cleanly.
CMD ["knowledge-mapper", "run", "app.py:kb"]
90 changes: 90 additions & 0 deletions examples/12-dockerized/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# 12 — Dockerized knowledge base

Shows how to package a knowledge base (KB) application as a Docker image and run it
next to a Smart Connector (SC) and a Knowledge Directory (KD) using Docker Compose.
The KB logic is kept minimal; the focus is on the Docker workflow. You can apply this
pattern to any of the other examples.

## Files

| File | Purpose |
|------|---------|
| `app.py` | ANSWER KB that answers greeting requests. Started with `knowledge-mapper run app.py:kb`. |
| `asker.py` | One-shot ASK KB that queries `app.py` once to show the interaction works, then exits. |
| `pyproject.toml` / `uv.lock` | The app's dependencies (`knowledge-mapper` pinned to a release from PyPI). |
| `Dockerfile` | Multi-stage build: uv installs the locked dependencies into a venv; a slim runtime image runs the app as a non-root user. |
| `.dockerignore` | Keeps the build context limited to the files the image needs. |
| `compose.yaml` | Starts the KD, the SC, the answering KB and the asker. |

## Run it

From this folder:

```bash
docker compose up --build
```

After a few seconds, the asker logs the answers it received from the dockerized KB
and exits:

```text
asker-1 | ... [dockerized-asker] Received answer: {'greeting': '<http://example.org/knowledge-mapper/dockerized#hello>', 'text': '"Hello from a container!"'}
asker-1 | ... [dockerized-asker] Received answer: {'greeting': '<http://example.org/knowledge-mapper/dockerized#goedemorgen>', ...}
asker-1 exited with code 0
```

The answering KB keeps running. Press `Ctrl+C`, or run `docker compose down` to stop
everything. On `SIGTERM`, the `knowledge-mapper` CLI unregisters the KB from the SC
before it exits.

To run the asker again while the rest is up:

```bash
docker compose run --rm asker
```

## How it works

### Configuration through environment variables

The image contains no configuration. Both scripts build their KB with
`KnowledgeBase.from_settings(KnowledgeBaseSettings())`. That reads the KB identity and
the SC endpoint from environment variables, with `__` as the separator for nested
fields:

| Variable | Example |
|----------|---------|
| `KNOWLEDGE_BASE__ID` | `http://example.org/knowledge-mapper/dockerized#answer-kb` |
| `KNOWLEDGE_BASE__NAME` | `dockerized-answer-kb` |
| `KNOWLEDGE_BASE__DESCRIPTION` | `A dockerized KB that answers greeting requests.` |
| `KNOWLEDGE_ENGINE_ENDPOINT` | `http://smart-connector:8280/rest` |

This lets you deploy the same image to different environments. The knowledge
interactions themselves are defined in code.

### Networking

All services share the default compose network and reach each other by service
name:

- The KBs call the SC's REST API at `http://smart-connector:8280/rest`.
- The SC registers with the KD at `http://knowledge-directory:8282`. It announces
`KE_RUNTIME_EXPOSED_URL` (`http://smart-connector:8081`) as the address where other
SCs can reach it.

A KB only makes outgoing requests (it long-polls its SC for incoming requests), so the
KB containers expose no ports. The SC's REST API is not published to the host either.
Add `ports: ["8280:8280"]` to the `smart-connector` service if you want to reach it
from the host, for example to run the other examples against it.

### Startup order

`depends_on` only waits for a container to start, not for the SC to be ready. If the
SC is not up yet, `knowledge-mapper run` exits with an error, and
`restart: unless-stopped` starts the KB again until registration succeeds. The asker
retries in code instead, both for connecting and until the answering KB responds.

### Using a different `knowledge-mapper` version

The version is pinned in `pyproject.toml`. To change it, edit the dependency and run
`uv lock` in this folder, then rebuild with `docker compose up --build`.
61 changes: 61 additions & 0 deletions examples/12-dockerized/app.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
"""Dockerized example: an ANSWER knowledge base that runs inside a container.

The KB logic is deliberately minimal (it answers with a fixed list of greetings);
the point of this example is the Docker workflow around it. See README.md.

Nothing environment-specific is hard-coded: the KB identity and the Smart Connector
URL are read from environment variables by ``KnowledgeBaseSettings`` (nested fields
use ``__`` as delimiter), so the same image can be deployed anywhere:

KNOWLEDGE_BASE__ID, KNOWLEDGE_BASE__NAME, KNOWLEDGE_BASE__DESCRIPTION,
KNOWLEDGE_ENGINE_ENDPOINT

The container starts it with ``knowledge-mapper run app.py:kb``; the CLI handles
connect, register, the handling loop and unregistering on SIGTERM (``docker stop``).
"""

import logging

from knowledge_mapper import (
BindingSet,
KnowledgeBase,
KnowledgeBaseSettings,
KnowledgeInteraction,
)

# This file is copied into the image on its own, so it mirrors the logging helper in
# examples/shared.py instead of importing it.
logger = logging.getLogger("dockerized-example")
_handler = logging.StreamHandler()
_handler.setFormatter(
logging.Formatter("%(asctime)s [%(levelname)s] [dockerized-example] %(message)s")
)
logger.addHandler(_handler)
logger.setLevel(logging.INFO)

settings = KnowledgeBaseSettings() # type: ignore[call-arg]
kb = KnowledgeBase.from_settings(settings).build()


@kb.answer_ki(
name="greeting-answer-ki",
graph_pattern="""
?greeting a ex:Greeting ;
ex:hasText ?text .
""",
prefixes={"ex": "http://example.org/knowledge-mapper/dockerized#"},
)
def greeting_answer_ki(
binding_set: BindingSet, info: KnowledgeInteraction
) -> BindingSet:
logger.info(f"Answering a greeting request with bindings: {binding_set}")
return [
{
"greeting": "<http://example.org/knowledge-mapper/dockerized#hello>",
"text": '"Hello from a container!"',
},
{
"greeting": "<http://example.org/knowledge-mapper/dockerized#goedemorgen>",
"text": '"Goedemorgen vanuit een container!"',
},
]
70 changes: 70 additions & 0 deletions examples/12-dockerized/asker.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
"""Dockerized example: a one-shot ASK knowledge base that queries ``app.py``.

Runs from the same image as the answering KB (see compose.yaml) to show that the
dockerized KB is reachable on the knowledge network. It asks once, logs the
answers, unregisters and exits.

Configured through the same environment variables as ``app.py``.
"""

import asyncio
import logging

from knowledge_mapper import KnowledgeBase, KnowledgeBaseSettings

logger = logging.getLogger("dockerized-asker")
_handler = logging.StreamHandler()
_handler.setFormatter(
logging.Formatter("%(asctime)s [%(levelname)s] [dockerized-asker] %(message)s")
)
logger.addHandler(_handler)
logger.setLevel(logging.INFO)

ATTEMPTS = 30
RETRY_DELAY_SECONDS = 2

settings = KnowledgeBaseSettings() # type: ignore[call-arg]
kb = KnowledgeBase.from_settings(settings).build()

kb.ask_ki(
name="greeting-ask-ki",
graph_pattern="""
?greeting a ex:Greeting ;
ex:hasText ?text .
""",
prefixes={"ex": "http://example.org/knowledge-mapper/dockerized#"},
)


async def wait_for_smart_connector() -> None:
for _ in range(ATTEMPTS):
try:
await kb.connect()
return
except Exception as error:
logger.info(f"Smart Connector not ready yet ({error}), retrying...")
await asyncio.sleep(RETRY_DELAY_SECONDS)
raise RuntimeError("Smart Connector did not become available.")


async def main() -> None:
await wait_for_smart_connector()
await kb.register()
try:
# The answering KB may still be starting, so retry until it answers.
for _ in range(ATTEMPTS):
result = await kb.ask([], "greeting-ask-ki")
if result:
for binding in result:
logger.info(f"Received answer: {binding}")
return
logger.info("No answers yet, retrying...")
await asyncio.sleep(RETRY_DELAY_SECONDS)
raise RuntimeError("No knowledge base answered the greeting request.")
finally:
await kb.unregister()
await kb.close()


if __name__ == "__main__":
asyncio.run(main())
45 changes: 45 additions & 0 deletions examples/12-dockerized/compose.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# Runs the dockerized knowledge base next to a Smart Connector and a Knowledge
# Directory. Start with `docker compose up --build`; see README.md.

services:
knowledge-directory:
image: ghcr.io/tno/knowledge-engine/knowledge-directory:1.3.2

smart-connector:
image: ghcr.io/tno/knowledge-engine/smart-connector:1.3.2
environment:
# Address other Smart Connectors use to reach this one, via the KD.
KE_RUNTIME_EXPOSED_URL: http://smart-connector:8081
KE_RUNTIME_PORT: 8081
KD_URL: http://knowledge-directory:8282
depends_on:
- knowledge-directory
# The REST API (port 8280) is only used by containers on this network, so it
# is not published. Add `ports: ["8280:8280"]` to reach it from the host.

answer-kb:
build: .
image: km-dockerized-example:local
environment:
KNOWLEDGE_BASE__ID: http://example.org/knowledge-mapper/dockerized#answer-kb
KNOWLEDGE_BASE__NAME: dockerized-answer-kb
KNOWLEDGE_BASE__DESCRIPTION: A dockerized KB that answers greeting requests.
# Containers reach each other by service name on the compose network.
KNOWLEDGE_ENGINE_ENDPOINT: http://smart-connector:8280/rest
depends_on:
- smart-connector
# The KB exits if the Smart Connector is not up yet; restarting retries.
restart: unless-stopped

asker:
# Same image as answer-kb, started with a different command.
build: .
image: km-dockerized-example:local
command: ["python", "asker.py"]
environment:
KNOWLEDGE_BASE__ID: http://example.org/knowledge-mapper/dockerized#asker
KNOWLEDGE_BASE__NAME: dockerized-asker
KNOWLEDGE_BASE__DESCRIPTION: A one-shot KB that asks the answer-kb for greetings.
KNOWLEDGE_ENGINE_ENDPOINT: http://smart-connector:8280/rest
depends_on:
- answer-kb
12 changes: 12 additions & 0 deletions examples/12-dockerized/pyproject.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
[project]
name = "knowledge-mapper-dockerized-example"
version = "0.1.0"
description = "A knowledge base application packaged as a Docker image."
requires-python = ">=3.13"
dependencies = [
"knowledge-mapper==0.1.0rc1",
]

[tool.uv]
# The app is a set of scripts, not an installable package.
package = false
Loading
Loading