Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -38,3 +38,6 @@ demos/grid-glb-demo/.forge.resolved.yaml

# Helm chart packaging artifacts
*.tgz

# Demo-local tool caches (e.g. fetched istioctl)
demos/**/.cache/
8 changes: 8 additions & 0 deletions demos/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,14 @@ consumer gateway, a separately secured provider gateway, and a private
inference endpoint. The directory currently records the standalone demo
contract in a validated, runnable environment.

## MaaS IPP lab

The [MaaS IPP lab](maas-ipp/README.md) is a single-cluster Forge environment
that brings up the stock Models-as-a-Service Kind path (Istio Gateway, Kuadrant
auth, controller-owned IPP EnvoyFilters, llm-d sim). Use it to develop and
validate MaaS dataplane changes before swapping IPP for Praxis in the
controller.

## Existing-Cluster Installation

For deploying Grid onto existing Kubernetes clusters (rather than disposable
Expand Down
195 changes: 195 additions & 0 deletions demos/maas-ipp/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,195 @@
# MaaS IPP lab (Forge)

Single-cluster Kind environment that reproduces the **stock MaaS** Kind datapath:

```text
Client → Istio Gateway → IPP-pre → Kuadrant Auth → IPP-post → HTTPRoute → LLM sim

Check warning on line 6 in demos/maas-ipp/README.md

View workflow job for this annotation

GitHub Actions / Detect hidden unicode characters

Unicode Safety [non-ascii-identifier]

U+2192 <unnamed U+2192> -- Non-ASCII U+2192 <unnamed U+2192> in identifier '→' (policy: ascii-only)

Check warning on line 6 in demos/maas-ipp/README.md

View workflow job for this annotation

GitHub Actions / Detect hidden unicode characters

Unicode Safety [non-ascii-identifier]

U+2192 <unnamed U+2192> -- Non-ASCII U+2192 <unnamed U+2192> in identifier '→' (policy: ascii-only)

Check warning on line 6 in demos/maas-ipp/README.md

View workflow job for this annotation

GitHub Actions / Detect hidden unicode characters

Unicode Safety [non-ascii-identifier]

U+2192 <unnamed U+2192> -- Non-ASCII U+2192 <unnamed U+2192> in identifier '→' (policy: ascii-only)

Check warning on line 6 in demos/maas-ipp/README.md

View workflow job for this annotation

GitHub Actions / Detect hidden unicode characters

Unicode Safety [non-ascii-identifier]

U+2192 <unnamed U+2192> -- Non-ASCII U+2192 <unnamed U+2192> in identifier '→' (policy: ascii-only)

Check warning on line 6 in demos/maas-ipp/README.md

View workflow job for this annotation

GitHub Actions / Detect hidden unicode characters

Unicode Safety [non-ascii-identifier]

U+2192 <unnamed U+2192> -- Non-ASCII U+2192 <unnamed U+2192> in identifier '→' (policy: ascii-only)

Check warning on line 6 in demos/maas-ipp/README.md

View workflow job for this annotation

GitHub Actions / Detect hidden unicode characters

Unicode Safety [non-ascii-identifier]

U+2192 <unnamed U+2192> -- Non-ASCII U+2192 <unnamed U+2192> in identifier '→' (policy: ascii-only)
```

Forge owns cluster lifecycle and infra stacks. `maas-controller` **owns EnvoyFilter / IPP deployment** — this demo never authors Praxis or IPP filter YAML.

Check warning on line 9 in demos/maas-ipp/README.md

View workflow job for this annotation

GitHub Actions / Detect hidden unicode characters

Unicode Safety [non-ascii-identifier]

U+2014 <unnamed U+2014> -- Non-ASCII U+2014 <unnamed U+2014> in identifier '—' (policy: ascii-only)

This profile is for MaaS + Praxis integration work. It is **not** the Grid multi-cluster GLB demo, and it intentionally diverges from issue #2’s “CRDs-only / skip Authorino” simulation table.

Check warning on line 11 in demos/maas-ipp/README.md

View workflow job for this annotation

GitHub Actions / Detect hidden unicode characters

Unicode Safety [non-ascii-identifier]

U+201D <unnamed U+201D> -- Non-ASCII U+201D <unnamed U+201D> in identifier 'Authorino”' (policy: ascii-only)

Check warning on line 11 in demos/maas-ipp/README.md

View workflow job for this annotation

GitHub Actions / Detect hidden unicode characters

Unicode Safety [non-ascii-identifier]

U+201C <unnamed U+201C> -- Non-ASCII U+201C <unnamed U+201C> in identifier '“CRDs' (policy: ascii-only)

Check warning on line 11 in demos/maas-ipp/README.md

View workflow job for this annotation

GitHub Actions / Detect hidden unicode characters

Unicode Safety [non-ascii-identifier]

U+2019 <unnamed U+2019> -- Non-ASCII U+2019 <unnamed U+2019> in identifier '’s' (policy: ascii-only)

## Pins

Version and namespace pins live in `forge.yaml` cluster `properties`. Stacks
template them into URL steps and `exec.env`; `scripts/lib.sh` defaults are only
fallbacks for running scripts outside Forge.


| Component | Property / env |
| ------------- | -------------------------------------- |
| MetalLB | `metallbVersion` (+ `metallbSha256`) |
| Gateway API | `gatewayApiVersion` (+ sha256) |
| GIE CRDs | `gieVersion` → `GIE_VERSION` |
| Istio | `istioVersion` → `ISTIO_VERSION` |
| cert-manager | `certManagerVersion` (+ sha256) |
| Kuadrant Helm | `kuadrantVersion` → `KUADRANT_VERSION` |
| Namespaces | `maasNamespace`, `gatewayNamespace` |




## Prerequisites

- Docker, `kind`, `kubectl`, `kustomize` (≥5.7), `helm`, `jq`, `curl` (or `wget`), `openssl`, `python3`
- `istioctl` is **not** required on PATH — `scripts/install-istio.sh` fetches
`cluster.properties.istioVersion` (via `ISTIO_VERSION`) into `demos/maas-ipp/.cache/`
- Optional for arm64 LLMIS build: `gh`, `docker buildx`
- A local checkout of [models-as-a-service](https://github.com/opendatahub-io/models-as-a-service):

```bash
export MAAS_ROOT=/path/to/models-as-a-service
```



## Bring up

From the grid repo root (paths in `forge.yaml` are relative to the config file):

```bash
export MAAS_ROOT=/path/to/models-as-a-service

# Create Kind cluster
cargo run -p praxis-forge -- up --config demos/maas-ipp/forge.yaml

# Apply stacks (metallb → … → maas-fixtures). Required after `up`.
cargo run -p praxis-forge -- apply local --config demos/maas-ipp/forge.yaml
```

Cluster context: `kind-maas-ipp-local`.

Re-apply a single stack if needed:

```bash
cargo run -p praxis-forge -- apply local --stack maas-platform --config demos/maas-ipp/forge.yaml
cargo run -p praxis-forge -- apply local --stack maas-fixtures --config demos/maas-ipp/forge.yaml
```

Optional overrides (consumed by `scripts/install-maas-platform.sh`):

```bash
export MAAS_CONTROLLER_IMAGE=quay.io/you/maas-controller:dev
export MAAS_API_IMAGE=quay.io/you/maas-api:dev
export IPP_IMAGE=quay.io/opendatahub/odh-ai-gateway-payload-processing:odh-stable
# Default for this lab is praxis (requires MAAS_ROOT with MAAS_IPP_PROFILE support).
export MAAS_IPP_PROFILE=praxis
export PRAXIS_EXTPROC_IMAGE=praxis-extproc:dev
# Stock llm-d IPP instead:
# export MAAS_IPP_PROFILE=llm-d
```



## API key

```bash
API_KEY=$(kubectl --context kind-maas-ipp-local -n maas-system exec deploy/maas-api -- \
curl -sk https://localhost:8443/v1/api-keys \
-H "X-MaaS-Username: demo-user" \
-H 'X-MaaS-Group: ["system:authenticated"]' \
-H "Content-Type: application/json" \
-d '{"name":"demo"}' | jq -r '.key')
echo "$API_KEY"
```



## Call models

Gateway LB (MetalLB on the Kind docker network — reachable from the Kind host):

```bash
GW=$(kubectl --context kind-maas-ipp-local -n istio-system \
get svc maas-default-gateway-istio -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
```

**Internal llm-d sim** (`LLMInferenceService` `sim-internal`):

```bash
curl -sk "https://${GW}/llm-internal/sim-internal/v1/chat/completions" \
-H "Authorization: Bearer ${API_KEY}" \
-H "Content-Type: application/json" \
-d '{"model":"facebook/opt-125m","messages":[{"role":"user","content":"hi"}],"max_tokens":8}'
```

**External model** (`ExternalModel` `llm-katan-openai` — remote simulator must be reachable):

```bash
curl -sk "https://${GW}/llm/llm-katan-openai/v1/chat/completions" \
-H "Authorization: Bearer ${API_KEY}" \
-H "Content-Type: application/json" \
-d '{"model":"llm-katan-openai","messages":[{"role":"user","content":"hi"}],"max_tokens":8}'
```

Port-forward instead of LB:

```bash
kubectl --context kind-maas-ipp-local -n istio-system \
port-forward svc/maas-default-gateway-istio 19090:80
# then http://localhost:19090/... with the same paths
```



## Validate

```bash
./demos/maas-ipp/scripts/validate.sh kind-maas-ipp-local
```

Expect:

- unauthenticated `/v1/models` → **401**
- API key + internal llm-d sim chat completion → **200** with a choices body



## Tear down

```bash
cargo run -p praxis-forge -- down --config demos/maas-ipp/forge.yaml
```



## Rebuild with local maas-controller changes

EnvoyFilter YAML stays in the controller. After editing under `$MAAS_ROOT/maas-controller`:

```bash
export MAAS_ROOT=/path/to/models-as-a-service
# rebuild.sh retags quay.io/opendatahub/maas-controller:* → localhost/…
# so imagePullPolicy=Always cannot re-pull over the kind-loaded image.
./demos/maas-ipp/scripts/rebuild.sh kind-maas-ipp-local maas-controller
```

Builds from `MAAS_ROOT`, `kind load`s, sets the Deployment image, patches
`imagePullPolicy` to `IfNotPresent`, and rolls out.

Also: `./demos/maas-ipp/scripts/rebuild.sh kind-maas-ipp-local maas-api`

Load `praxis-extproc:dev` separately if `MAAS_IPP_PROFILE=praxis`:

```bash
kind load docker-image praxis-extproc:dev --name maas-ipp-local
```

Forge must **not** apply a competing Praxis EnvoyFilter; the controller reconcile owns IPP.

## Stacks


| Stack | Role |
| --------------- | --------------------------------------------------------------- |
| `metallb` | LB + Kind docker-network pool |
| `gateway-api` | GW API 1.5.1 + GIE CRDs |
| `istio` | 1.30.3 minimal + GIE pilot flags |
| `cert-manager` | cert-manager + maas-api CA chain |
| `kuadrant` | Helm Kuadrant + Authorino trust |
| `maas-platform` | Postgres, CRDs, KServe/llmisvc, Gateway, stock controller → IPP |
| `maas-fixtures` | ExternalModel + LLMInferenceService sim + subscription |


GIE is enabled so an InferencePool backend can be attached later without reinstalling Istio. v1 validate uses the stock MaaS HTTPRoute → LLMInferenceService path.
150 changes: 150 additions & 0 deletions demos/maas-ipp/forge.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,150 @@
apiVersion: forge.praxis.dev/v1alpha1
kind: Environment

metadata:
name: maas-ipp

spec:
runtime:
provider: docker
clusterPrefix: maas-ipp

network:
# Single-cluster lab. MetalLB pool is configured via scripts/install-metallb-pool.sh
# (forge metallb-auto-pool requires crossCluster: true).
crossCluster: false
dnsZone: maas-ipp.test

clusters:
- name: local
stacks:
- metallb
- gateway-api
- istio
- cert-manager
- kuadrant
- maas-platform
- maas-fixtures
properties:
# Single source of truth for pin versions / namespaces.
# Stacks template these into URL steps and exec env; lib.sh defaults
# only apply when scripts are run outside Forge.
metallbVersion: "v0.14.9"
metallbSha256: "951065e85692aa106f1bb5d5a487d9306154923a794ab1d82122881cbaf588e4"
gatewayApiVersion: "1.5.1"
gatewayApiSha256: "751002b3b91a87f7ae3bd2517c79a47a8d7ed6702901808a1cf9bd97d284f9b8"
gieVersion: "v1.5.0"
istioVersion: "1.30.3"
certManagerVersion: "1.17.2"
certManagerSha256: "dab2794176629d8042ce97b885f378dfe4ef7b5bed5859e9f06616ead9980f4b"
kuadrantVersion: "1.3.1"
maasNamespace: maas-system
gatewayNamespace: istio-system

stacks:
metallb:
description: MetalLB load balancer with Kind docker-network IP pool
steps:
- type: url
url: https://raw.githubusercontent.com/metallb/metallb/{{ cluster.properties.metallbVersion }}/config/manifests/metallb-native.yaml
sha256: "{{ cluster.properties.metallbSha256 }}"
- type: wait
resource: deployment/controller
namespace: metallb-system
condition: available
timeout: "180s"
- type: exec
command:
- bash
- scripts/install-metallb-pool.sh
- kind-maas-ipp-{{ cluster.name }}

gateway-api:
description: Gateway API CRDs + Inference Extension (GIE) CRDs
steps:
- type: url
url: https://github.com/kubernetes-sigs/gateway-api/releases/download/v{{ cluster.properties.gatewayApiVersion }}/standard-install.yaml
sha256: "{{ cluster.properties.gatewayApiSha256 }}"
- type: exec
env:
GIE_VERSION: "{{ cluster.properties.gieVersion }}"
command:
- bash
- scripts/install-gie-crds.sh
- kind-maas-ipp-{{ cluster.name }}

istio:
description: Istio minimal with Gateway API Inference Extension enabled
steps:
- type: exec
env:
ISTIO_VERSION: "{{ cluster.properties.istioVersion }}"
command:
- bash
- scripts/install-istio.sh
- kind-maas-ipp-{{ cluster.name }}
- type: wait
resource: deployment/istiod
namespace: istio-system
condition: available
timeout: "180s"

cert-manager:
description: cert-manager + Kind TLS CA chain for maas-api
steps:
- type: url
url: https://github.com/cert-manager/cert-manager/releases/download/v{{ cluster.properties.certManagerVersion }}/cert-manager.yaml
sha256: "{{ cluster.properties.certManagerSha256 }}"
- type: wait
resource: deployment/cert-manager
namespace: cert-manager
condition: available
timeout: "180s"
- type: wait
resource: deployment/cert-manager-webhook
namespace: cert-manager
condition: available
timeout: "180s"
- type: exec
env:
MAAS_NAMESPACE: "{{ cluster.properties.maasNamespace }}"
command:
- bash
- scripts/install-maas-tls.sh
- kind-maas-ipp-{{ cluster.name }}

kuadrant:
description: Kuadrant + Authorino CA trust for maas-api
steps:
- type: exec
env:
KUADRANT_VERSION: "{{ cluster.properties.kuadrantVersion }}"
MAAS_NAMESPACE: "{{ cluster.properties.maasNamespace }}"
command:
- bash
- scripts/install-kuadrant.sh
- kind-maas-ipp-{{ cluster.name }}

maas-platform:
description: Postgres, MaaS/KServe CRDs, Gateway, stock maas-controller (deploys IPP)
steps:
- type: exec
env:
MAAS_NAMESPACE: "{{ cluster.properties.maasNamespace }}"
GATEWAY_NAMESPACE: "{{ cluster.properties.gatewayNamespace }}"
command:
- bash
- scripts/install-maas-platform.sh
- kind-maas-ipp-{{ cluster.name }}

maas-fixtures:
description: ExternalModel + llm-d sim LLMInferenceService + subscription/auth fixtures
steps:
- type: exec
env:
MAAS_NAMESPACE: "{{ cluster.properties.maasNamespace }}"
GATEWAY_NAMESPACE: "{{ cluster.properties.gatewayNamespace }}"
command:
- bash
- scripts/install-maas-fixtures.sh
- kind-maas-ipp-{{ cluster.name }}
16 changes: 16 additions & 0 deletions demos/maas-ipp/scripts/install-gie-crds.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
#!/usr/bin/env bash
# Install Gateway API Inference Extension CRDs (GIE).
set -euo pipefail
# shellcheck source=lib.sh
source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/lib.sh"
parse_context "$@"
require_cmd kubectl

if kc get crd inferencepools.inference.networking.k8s.io &>/dev/null; then
ok "GIE CRDs already installed"
exit 0
fi

kustomize_build "github.com/kubernetes-sigs/gateway-api-inference-extension/config/crd?ref=${GIE_VERSION}" \
| kc apply -f -
ok "GIE CRDs ${GIE_VERSION} installed"
Loading
Loading