Mount versioned LakeFS repositories into computing-unit pods (FUSE / GeeseFS)

[aicam]

Feature Summary

Add the ability to mount a versioned LakeFS repository directly into a computing-unit pod's file system, so operator code can read the repository's files from a local path instead of downloading them over HTTP first. LakeFS backs both datasets and models in Texera, so this works for either. The initial consumer is the Python UDF operator, which gets a new "Mount dataset" property; the mounted directory is exposed to user code as a path variable.

Problem it solves: today, if an operator needs a repository's files (e.g. a large ML model with multiple shards), the options are to stream them through the storage API or download the whole repository into the pod before use. For large repositories (multi-GB models) this is slow, uses a lot of pod disk, and forces a full transfer even when the code only reads part of the data.

Proposed Solution or Design

Texera stores datasets and models as LakeFS repositories on MinIO/S3. LakeFS exposes an S3-compatible gateway, so a LakeFS repository at a specific commit can be mounted as a read-only file system using a FUSE S3 client. We use GeeseFS for this.

Key properties:

• Lazy / on-demand — the mount transfers zero bytes; file contents are fetched with ranged reads only when the code actually read()s them, with read-ahead for throughput.
• Version-pinned & immutable — a repository version maps to a LakeFS commit, so the mounted content is content-addressed and never changes underneath the reader; no cache invalidation concerns.
• Storage-backend agnostic — GeeseFS always talks to the LakeFS S3 gateway, so it works identically whether LakeFS is backed by in-cluster MinIO or an external S3 bucket.

Scope (Kubernetes only): mounting requires FUSE inside the pod, which needs a privileged computing-unit pod and CAP_SYS_ADMIN on the mount binary. The capability is always on (no feature flag): the GeeseFS binary ships in every computing-unit image and mounting happens whenever a repository is requested. Pods run privileged so the mount can be performed.

Authorization (no global credentials in the pod). Computing-unit pods are user-accessible — Python UDF code runs as the user and can read the pod's environment — so the mount must not carry the global LakeFS admin identity (that would expose every user's repositories). Instead, the mount reuses the per-user JWT the pod already holds (USER_JWT_TOKEN) and the same authorization file-service already applies to repository reads (userHasReadAccess). A JWT-authenticated S3 proxy in file-service sits in front of the LakeFS S3 gateway: GeeseFS is given the pod's JWT as its S3 access key, so it rides in every request's signature; the proxy verifies the JWT, checks the user's read access to the requested repository, and re-signs to LakeFS with the global credentials held only server-side. Because the credential is the JWT the pod already has, no separate mount credential is minted or stored — file-service holds no per-mount state and stays horizontally scalable. Nothing that grants access beyond the user's own repositories is ever placed in the pod.

End-to-end flow:

1. Python UDF gets a new "Mount dataset" property — a dataset + version picker that resolves to /ownerEmail/datasetName/versionName.
2. At compile time the path is resolved to a repository:commitHash locator via LakeFS/DB lookup.
3. The locator flows through PhysicalOp → WorkerConfig to the Python worker.
4. Before the Python worker starts, DatasetMountManager runs GeeseFS against the file-service S3 proxy — using the pod's JWT as the S3 credential — mounting repository:commitHash read-only at a local path in the pod.
5. The mounted path is exposed to UDF code as the variable MOUNTED_DATASET_PATH.

This has been prototyped and validated end-to-end: a ~2 GB sharded PyTorch model in one repository, loaded inside a Python UDF via torch.load() from the mounted path, with bit-exact outputs — against both in-cluster MinIO and a remote AWS S3 backing store.

Implementation. The whole feature is delivered in a single PR, #6608, with these pieces:

• Computing-unit image — install GeeseFS + FUSE (fuse3, CAP_SYS_ADMIN on the binary) in the computing-unit image and run computing-unit pods privileged (always on, no feature flag).
• JWT-authenticated S3 proxy (file-service) — a scope-enforcing, re-signing reverse proxy in front of the LakeFS S3 gateway that authenticates the pod's JWT, checks userHasReadAccess for the requested repository, and re-signs to LakeFS with global credentials held only server-side.
• Engine mount manager — DatasetMountManager runs GeeseFS against the proxy with the pod's JWT; the repository locator flows through PhysicalOp/WorkerConfig to the Python worker.
• Repository-version resolution — resolve /ownerEmail/datasetName/versionName to a repository:commitHash locator.
• Python UDF "Mount dataset" property — a dataset + version picker on the operator, with the mounted path exposed to UDF code as MOUNTED_DATASET_PATH.

Future consumers (other operators, other languages) can reuse the same mount plumbing.

Affected Area: Deployment / Infrastructure, Workflow Engine (Amber), Storage / Metadata