feat: retain diffusion tuning samples in GPU cache

[changwangss]

Description

Use the remaining diffusion tuning cache budget to retain samples(cached block input) on GPU and reduce repeated CPU-to-GPU transfers.

The existing prefetch implementation (#2340 )allocates two staging buffers and optional GPU best-parameter snapshots. Even when auto allows a much larger budget, the remaining space is unused and repeated samples are transferred again.

the log collected base on #2340, still about 120g gpu memory unused.

2026-09-11 03:55:10 INFO tuning_cache.py L92: Diffusion tuning auto cache budget after warmup: 120.15 GiB.
2026-09-11 03:55:10 INFO tuning_cache.py L122: Diffusion tuning prefetch enabled with 2.68 GiB of GPU buffers; best parameters on GPU.

This change retains samples on first use within the remaining budget:

• Account for actual tensor sizes, including block inputs, conditioning, and reference outputs.
• Reuse resident samples on subsequent accesses; stop growing when the budget is exhausted or allocation encounters OOM.
• Continue using the existing prefetch path for uncached samples.
• Copy resident data into the existing GPU staging buffers before forward to protect cached contents from in-place modifications.

Caches remain local to each block and are released after tuning. Wan's transformer and transformer_2 still collect calibration inputs separately and use independent tuning caches.

No new option is required:

--low_gpu_mem_usage --diffusion_tuning_cache_size auto

Type of Change

Bug fix

Related Issues

Fixes or relates to #

Checklist Before Submitting

• My code has been tested locally.
• Documentation has been updated as needed.
• New or updated tests are included where applicable.
• The CUDA CI has passed. You can trigger it by commenting /azp run Unit-Test-CUDA-AutoRound.