[Bugfix] Enforce strict timeout for Posix AIO

[dante159753]

Summary

Enforce hard timeouts for Posix AIO cache operations so a stuck mount no longer leaves vLLM requests waiting forever on external KV cache loads.

Changes

• Add task-level terminal timeout handling for Posix AIO tasks.
• Make AIO submit/completion, task wait/check, and timeout cleanup tolerate stuck kernel IO.
• Add timeout metrics and vLLM integration handling so load failures can be rescheduled through kv_load_failure_policy=recompute.
• Keep AIO timeout configuration simple by using the existing timeout_ms default instead of separate AIO timeout knobs.
• Add Posix store tests for open timeout, IO timeout, repeated timeout, and saturated stuck-task behavior.

Why

When a mounted cache backend stops responding, existing AIO requests can be submitted successfully but never complete. Before this change, requests that hit external KV cache could remain blocked until the mount recovered, making inference requests hang under storage incidents.

Impact

With the default timeout_ms, stuck AIO load tasks are force-completed as timeout failures. In vLLM, deployments using kv_load_failure_policy=recompute can continue serving by recomputing the affected KV blocks instead of waiting indefinitely. If the storage backend remains stuck, cache loads may be slower because they time out and fall back to recompute, but the service remains available.

Verification

• A2 online vLLM+UCM hang simulation:
  • Setup: loop-backed ext4 mount through device mapper, UCM storage_backends on that mount, posix_io_engine: aio, io_direct: true, no explicit timeout_ms.
  • Current branch: dmsetup suspend for ~75s during external cache load; /v1/models stayed responsive; 8 concurrent completion requests returned 200 OK in ~60.8s; post-resume requests completed in ~0.64s.
  • Baseline origin/develop: same setup; 8 concurrent completion requests all hit client timeout at ~55s while UCM repeatedly logged Task(1) has not finished after (2000) ms; post-resume requests completed after the mount recovered.