TensorFlow image classification tests running out of memory on Linux x64 #7836

System Information (please complete the following information):

• OS & Version: Ubuntu 22.04 and CentOS Stream 9 x64 Helix containers.
• ML.NET Version: PR Run existing integration tests against published packages #7833, commit 100af589f.
• .NET Version: .NET 8, Debug and Release.

Describe the bug
Accurate native-library detection enables previously skipped TensorFlow tests. The process exits with code 137 during TensorFlowImageClassificationWithExponentialLRScheduling in Release and TensorFlowImageClassificationBadImages in Debug. Debug logs report peak memory up to 7.2 GB. An OOM kill is suspected, not kernel-confirmed.

Screenshots, Code, Sample Projects
Ubuntu Debug log · Ubuntu Release log

Additional context
Temporarily quarantine the six image-classification training methods (10 test cases) as a group on Linux x64, including local runs and both build configurations. This covers default training, architecture variants, learning-rate scheduling, early stopping, and bad images. Keep non-training TensorFlow tests and other platforms enabled.

Reproduced outside Helix in Ubuntu 22.04 WSL x64 with the original Release binaries, .NET 8.0.16 and two CPUs: at a 16 GiB limit, 36 tests passed with 2 existing skips (peak RSS about 14.2 GiB). At an experimental 8 GiB limit, the kernel OOM-killed dotnet during BadImages. This does not establish the actual Helix memory limit. Memory retention across tests may contribute, so individual skips may only move the failure later.

The two initial skips were honored, but build 1643911 was killed later in TensorFlowImageClassification or TensorFlowImageClassificationEarlyStopping (Ubuntu Release log). The latest payload also OOM-kills at 8 GiB in WSL. Enabling the existing per-test GC option did not prevent the OOM, so that experiment was reverted.

With the training-group quarantine, the same 8 GiB WSL reproduction completes successfully: 26 passed and 8 reported skips (6 training methods plus 2 existing model-unavailable skips), with peak RSS about 1.26 GiB. The two quarantined theories cover multiple data rows. Linux CI still needs to rerun; this is a temporary coverage reduction, not a fix for the memory retention.