Skip to content

Fix auto device falling back when a provider fails to load#1674

Open
Suh0161 wants to merge 1 commit into
huggingface:mainfrom
Suh0161:fix/auto-device-fallback
Open

Fix auto device falling back when a provider fails to load#1674
Suh0161 wants to merge 1 commit into
huggingface:mainfrom
Suh0161:fix/auto-device-fallback

Conversation

@Suh0161
Copy link
Copy Markdown

@Suh0161 Suh0161 commented May 4, 2026

Closes #1642

Fix: auto device falls back gracefully when a provider fails to load

When device: 'auto' is used and a high-priority execution provider (e.g. CUDA on Linux without libcuda.so installed) is unavailable, createInferenceSession would throw instead of trying the next provider in the list.

The fallback now lives in constructSessions in session.js, where the device selection context is available. Each provider is tried individually — on failure a warning is logged and the next one is attempted. createInferenceSession itself is unchanged.

// Before: hard crash on Linux x64 without libcuda.so
// After: warns and automatically falls back to the next available provider
await pipeline('text-generation', 'onnx-community/LFM2-1.2B-ONNX', { device: 'auto' });

When device='auto' and a high-priority execution provider (e.g. CUDA on
Linux without libcuda.so installed) is not available, createInferenceSession
would throw instead of trying the next provider in the list.

The fallback now lives in constructSessions, where the device selection
context is available. Each provider is tried individually. On failure a
warning is logged and the next provider is attempted. createInferenceSession
itself is unchanged.

Closes huggingface#1642.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Auto device on Linux x64 fails hard when CUDA shared library is unavailable instead of falling back

1 participant