CORS-4336: Add CI jobs for AWS European Sovereign Cloud (EUSC) by liweinan · Pull Request #75568 · openshift/release

liweinan · 2026-03-02T18:04:40Z

Implement continuous integration support for AWS EUSC partition (aws-eusc) in eusc-de-east-1 region. Includes cluster profile definition, service endpoints configuration, custom AMI handling, and periodic test jobs.

This enables OpenShift testing on AWS's new European Sovereign Cloud infrastructure, which requires explicit endpoint configuration and custom RHCOS AMIs not available in public regions.

openshift-ci-robot · 2026-03-02T18:04:45Z

@liweinan: This pull request references CORS-4336 which is a valid jira issue.

Details

In response to this:

Implement continuous integration support for AWS EUSC partition (aws-eusc) in eusc-de-east-1 region. Includes cluster profile definition, service endpoints configuration, custom AMI handling, and periodic test jobs.

This enables OpenShift testing on AWS's new European Sovereign Cloud infrastructure, which requires explicit endpoint configuration and custom RHCOS AMIs not available in public regions.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci-robot · 2026-03-02T18:06:05Z

@liweinan, pj-rehearse: unable to determine affected jobs. This could be due to a branch that needs to be rebased. ERROR:

could not load configuration from candidate revision of release repo: failed to load ci-operator configuration from release repo: invalid ci-operator config: configuration has 2 errors:

 * tests[125]: invalid cluster profile "aws-eusc-qe"
 * tests[126]: invalid cluster profile "aws-eusc-qe"

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

...t/openshift-tests-private/openshift-openshift-tests-private-release-4.22__amd64-nightly.yaml

ci-operator/step-registry/cluster-profiles/cluster-profiles-config.yaml

...ipi/private/provision/cucushift-installer-rehearse-aws-eusc-ipi-private-provision-chain.yaml

ci-operator/step-registry/ipi/conf/aws/eusc-ami/ipi-conf-aws-eusc-ami-commands.sh

yunjiang29 · 2026-03-05T02:26:01Z

@liweinan as we discussed offline, for the new partition we need three types of cluster:

common IPI cluster
private cluster
disconnected (private) cluster
and based on above basic cluster, we also need to cover STS, custom KMS key, FIPS and minimum permission, you can refer to existing jobs.

...t/openshift-tests-private/openshift-openshift-tests-private-release-4.22__amd64-nightly.yaml

liweinan · 2026-03-05T02:38:24Z

@yunjiang29 Thanks for the review! I'll refactor this PR today.

openshift-ci-robot · 2026-03-05T12:18:26Z

@liweinan, pj-rehearse: unable to determine affected jobs. This could be due to a branch that needs to be rebased. ERROR:

could not load configuration from candidate revision of release repo: failed to load ci-operator configuration from release repo: invalid ci-operator config: configuration has 2 errors:

 * tests[126]: invalid cluster profile "aws-eusc"
 * tests[127]: invalid cluster profile "aws-eusc"

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

liweinan · 2026-03-06T03:16:23Z

@yunjiang29 Thanks for the detailed review! I'll update the PR recordingly.

openshift-ci-robot · 2026-03-06T06:37:31Z

@liweinan, pj-rehearse: unable to determine affected jobs. This could be due to a branch that needs to be rebased. ERROR:

could not load configuration from candidate revision of release repo: failed to load ci-operator configuration from release repo: invalid ci-operator config: configuration has 2 errors:

 * tests[126]: invalid cluster profile "aws-eusc"
 * tests[127]: invalid cluster profile "aws-eusc"

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

Address yunfei's review comments on PR openshift#75568: 1. Job naming convention: - Rename jobs from -f60 to -f7 suffix (non-destructive tests) - Update cron schedule to standard f7 pattern: 7,14,23,30 2. Private cluster configuration: - Add complete private cluster setup with bastion host - Add VPC, security groups, and proxy configuration - Set PUBLISH=Internal for private cluster access - Add minimal IAM permission provisioning - Follow pattern from cucushift-installer-rehearse-aws-ipi-private-provision 3. AMI configuration fix: - Replace deprecated compute.platform.aws.amiID field - Use platform.aws.defaultMachinePlatform.amiID instead

openshift-ci-robot · 2026-03-06T06:40:37Z

@liweinan, pj-rehearse: unable to determine affected jobs. This could be due to a branch that needs to be rebased. ERROR:

could not load configuration from candidate revision of release repo: failed to load ci-operator configuration from release repo: invalid ci-operator config: configuration has 2 errors:

 * tests[126]: invalid cluster profile "aws-eusc"
 * tests[127]: invalid cluster profile "aws-eusc"

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

1. Job naming convention: - Rename jobs from -f60 to -f7 suffix (non-destructive tests) - Update cron schedule to standard f7 pattern: 7,14,23,30 2. Private cluster configuration: - Add complete private cluster setup with bastion host - Add VPC, security groups, and proxy configuration - Set PUBLISH=Internal for private cluster access - Add minimal IAM permission provisioning - Follow pattern from cucushift-installer-rehearse-aws-ipi-private-provision 3. AMI configuration fix: - Replace deprecated compute.platform.aws.amiID field - Use platform.aws.defaultMachinePlatform.amiID instead 4. Generalize step registry components for reusability: - Enhance ipi-conf-aws-custom-endpoints to support multiple AWS partitions * Add AWS_DOMAIN_SUFFIX env var (defaults to amazonaws.com) * Support amazonaws.eu for EUSC, amazonaws.com.cn for China * Allow full URLs for maximum flexibility - Make ipi-conf-aws-eusc-ami more generic * Support AWS_CUSTOM_AMI_ID for general use * Maintain AWS_EUSC_AMI_ID for backward compatibility * Can be used for EUSC, China, GovCloud, or custom AMI scenarios - Use generic steps in EUSC provision chain with partition-specific config - Remove obsolete ipi-conf-aws-eusc-endpoints (replaced by generic version)

openshift-ci-robot · 2026-03-06T07:00:15Z

@liweinan, pj-rehearse: unable to determine affected jobs. This could be due to a branch that needs to be rebased. ERROR:

could not load configuration from candidate revision of release repo: failed to load ci-operator configuration from release repo: invalid ci-operator config: configuration has 2 errors:

 * tests[126]: invalid cluster profile "aws-eusc"
 * tests[127]: invalid cluster profile "aws-eusc"

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

...erator/step-registry/ipi/conf/aws/custom-endpoints/ipi-conf-aws-custom-endpoints-commands.sh

1. Job naming convention: - Rename jobs from -f60 to -f7 suffix (non-destructive tests) - Update cron schedule to standard f7 pattern: 7,14,23,30 2. Private cluster configuration: - Add complete private cluster setup with bastion host - Add VPC, security groups, and proxy configuration - Set PUBLISH=Internal for private cluster access - Add minimal IAM permission provisioning - Follow pattern from cucushift-installer-rehearse-aws-ipi-private-provision 3. Generalize step registry components for maximum reusability: a) Enhance ipi-conf-aws-custom-endpoints for all AWS partitions: - Add AWS_DOMAIN_SUFFIX env var (defaults to amazonaws.com) - Support amazonaws.eu (EUSC), amazonaws.com.cn (China) - Allow full URLs for maximum flexibility - Remove obsolete ipi-conf-aws-eusc-endpoints step b) Extend ipi-conf-aws to support custom AMI configuration: - Add AWS_AMI_ID env var for custom RHCOS AMI - Useful for EUSC, China, GovCloud, or any partition without public AMIs - Fix deprecated amiID field -> defaultMachinePlatform.amiID - Auto-detection still works for C2S/SC2S - Remove obsolete ipi-conf-aws-eusc-ami step c) EUSC provision chain now uses only generic steps with env config This refactoring reduces code duplication (net -59 lines) and makes step components reusable across all AWS partitions.

openshift-ci-robot · 2026-03-06T07:12:44Z

@liweinan, pj-rehearse: unable to determine affected jobs. This could be due to a branch that needs to be rebased. ERROR:

could not load configuration from candidate revision of release repo: failed to load ci-operator configuration from release repo: invalid ci-operator config: configuration has 2 errors:

 * tests[126]: invalid cluster profile "aws-eusc"
 * tests[127]: invalid cluster profile "aws-eusc"

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

liweinan · 2026-03-10T13:59:36Z

Relative PRs merged: #75441 / openshift/ci-tools#4973

ci-operator/step-registry/ipi/conf/aws/ipi-conf-aws-ref.yaml

@yunjiang29

This refactors the AWS European Sovereign Cloud (EUSC) CI configuration to maximize reuse of standard AWS workflows and reduce maintenance burden. Changes based on @yunjiang29's review feedback: - Reduced from 9 to 6 jobs following the pattern: 3 cluster types × 2 test types - Improved FIPS coverage from 1/9 (11%) to 2/6 (33%) jobs: * aws-eusc-ipi-fips-f7 (IPI + FIPS) * aws-eusc-ipi-private-sts-fips-f7 (Private + STS + FIPS) - Combined features across jobs: * aws-eusc-ipi-f28-destructive (destructive testing) * aws-eusc-ipi-private-mini-perm-f28 (Private + minimal permissions) * aws-eusc-ipi-disc-priv-kms-f7 (Disconnected + KMS) * aws-eusc-ipi-disc-priv-f28 (Disconnected destructive) - All jobs cover: FIPS, STS, KMS, minimal permissions across 3 cluster types - Deleted 15 EUSC-specific files, created 8 new ones (net reduction: -7 files) - Maximized reuse of standard AWS workflows: * Basic IPI: reuses cucushift-installer-rehearse-aws-ipi-deprovision * Private: reuses cucushift-installer-rehearse-aws-ipi-private-deprovision * Disconnected: reuses cucushift-installer-rehearse-aws-ipi-disconnected-private-provision * Private-STS: reuses cucushift-installer-rehearse-aws-ipi-private-cco-manual-security-token-service - EUSC-specific changes limited to: * Inserting ipi-conf-aws-custom-endpoints ref for service endpoint configuration * Custom provision chain for disconnected-private-kms (combines disconnected + KMS) - Deleted all EUSC-specific deprovision chains (reuse standard chains) - Removed unnecessary byo-kms and STS specific directory structures 1. Custom endpoints (ipi-conf-aws-custom-endpoints-commands.sh): - Removed auto-detection logic for AWS_DOMAIN_SUFFIX - Simplified to use environment variable or default to "amazonaws.com" - Removed Route53 endpoint configuration (global service) - Designed for easy removal when installer adds native EUSC support 2. AMI configuration (ipi-conf-aws-commands.sh): - Simplified from split variables (CONTROL_PLANE_AMI/COMPUTE_AMI) to single CONTROL_PLANE_AMI - Preserved C2S/SC2S auto-detection logic - Removed complex heredoc patching, kept simple approach - Updated documentation for clarity 1. **Minimize EUSC-specific code**: Only 8 workflow files vs 15 previously 2. **Maximize standard workflow reuse**: Follows USGov pattern, not C2S pattern 3. **Prepare for future evolution**: Custom endpoints easy to remove when installer supports EUSC natively 4. **FIPS coverage aligned with USGov**: 33% vs USGov's 18%, not C2S's 100% - make update completed successfully - All 6 jobs generated in ci-operator/jobs/.../periodics.yaml - Step registry validation passed Addresses: openshift#75568

@yunjiang29

This refactors the AWS European Sovereign Cloud (EUSC) CI configuration to maximize reuse of standard AWS workflows and reduce maintenance burden. Changes based on @yunjiang29's review feedback: - Reduced from 9 to 6 jobs following the pattern: 3 cluster types × 2 test types - Improved FIPS coverage from 1/9 (11%) to 2/6 (33%) jobs: * aws-eusc-ipi-fips-f7 (IPI + FIPS) * aws-eusc-ipi-private-sts-fips-f7 (Private + STS + FIPS) - Combined features across jobs: * aws-eusc-ipi-f28-destructive (destructive testing) * aws-eusc-ipi-private-mini-perm-f28 (Private + minimal permissions) * aws-eusc-ipi-disc-priv-kms-f7 (Disconnected + KMS) * aws-eusc-ipi-disc-priv-f28 (Disconnected destructive) - All jobs cover: FIPS, STS, KMS, minimal permissions across 3 cluster types - Deleted 15 EUSC-specific files, created 8 new ones (net reduction: -7 files) - Maximized reuse of standard AWS workflows: * Basic IPI: reuses cucushift-installer-rehearse-aws-ipi-deprovision * Private: reuses cucushift-installer-rehearse-aws-ipi-private-deprovision * Disconnected: reuses cucushift-installer-rehearse-aws-ipi-disconnected-private-provision * Private-STS: reuses cucushift-installer-rehearse-aws-ipi-private-cco-manual-security-token-service - EUSC-specific changes limited to: * Inserting ipi-conf-aws-custom-endpoints ref for service endpoint configuration * Custom provision chain for disconnected-private-kms (combines disconnected + KMS) - Deleted all EUSC-specific deprovision chains (reuse standard chains) - Removed unnecessary byo-kms and STS specific directory structures 1. Custom endpoints (ipi-conf-aws-custom-endpoints-commands.sh): - Removed auto-detection logic for AWS_DOMAIN_SUFFIX - Simplified to use environment variable or default to "amazonaws.com" - Removed Route53 endpoint configuration (global service) - Designed for easy removal when installer adds native EUSC support 2. AMI configuration (ipi-conf-aws-commands.sh): - Simplified from split variables (CONTROL_PLANE_AMI/COMPUTE_AMI) to single CONTROL_PLANE_AMI - Preserved C2S/SC2S auto-detection logic - Removed complex heredoc patching, kept simple approach - Updated documentation for clarity 1. **Minimize EUSC-specific code**: Only 8 workflow files vs 15 previously 2. **Maximize standard workflow reuse**: Follows USGov pattern, not C2S pattern 3. **Prepare for future evolution**: Custom endpoints easy to remove when installer supports EUSC natively 4. **FIPS coverage aligned with USGov**: 33% vs USGov's 18%, not C2S's 100% - make update completed successfully - All 6 jobs generated in ci-operator/jobs/.../periodics.yaml - Step registry validation passed Addresses: openshift#75568

liweinan · 2026-03-16T13:45:55Z

Related PR with Feature Gate: openshift/api#2740

liweinan · 2026-03-16T14:04:57Z

@yunjiang29 @tthvo

FYI I have just added FEATURE_SET: TechPreviewNoUpgrade to all 6 EUSC jobs to enable the feature gate support.

This will allow the installer to handle EUSC-specific ARN formats (arn:aws-eusc:*) and service endpoints once openshift/api#2740 merges (already approved).

yunjiang29 · 2026-03-18T08:08:26Z

@liweinan let's add followings jobs:

ci-operator/config/openshift/openshift-tests-private/openshift-openshift-tests-private-release-4.22__multi-nightly.yaml

aws-eusc-ipi-byo-kms-etcd-encryption-fips-tp-arm-f7
aws-eusc-ipi-byo-kms-etcd-encryption-fips-tp-amd-f28-destructive
* cucushift-installer-rehearse-aws-ipi-byo-kms-etcd-encryption

aws-eusc-ipi-disc-priv-tp-arm-f7
aws-eusc-ipi-disc-priv-tp-amd-f28-destructive
* cucushift-installer-rehearse-aws-ipi-disconnected-private

aws-eusc-ipi-private-sts-tp-arm-f7
aws-eusc-ipi-private-sts-tp-amd-f28-destructive
* cucushift-installer-rehearse-aws-ipi-private-cco-manual-security-token-service

aws-eusc-ipi-custom-dns-mini-perm-tp-arm-f7
aws-eusc-ipi-custom-dns-mini-perm-tp-arm-f28-destructive
* cucushift-installer-rehearse-aws-ipi-custom-dns


ci-operator/config/openshift/installer/openshift-installer-release-4.22.yaml
ci-operator/config/openshift/installer/openshift-installer-release-4.23.yaml
ci-operator/config/openshift/installer/openshift-installer-release-5.0.yaml
ci-operator/config/openshift/installer/openshift-installer-main.yaml

e2e-aws-eusc-techpreview
* openshift-e2e-aws

...erator/step-registry/ipi/conf/aws/custom-endpoints/ipi-conf-aws-custom-endpoints-commands.sh

ci-operator/step-registry/ipi/conf/aws/ipi-conf-aws-commands.sh

Implement comprehensive CI infrastructure for AWS EUSC partition in eusc-de-east-1 region. Job coverage (9 jobs): - Common IPI: aws-eusc-ipi-f7, aws-eusc-ipi-f28-destructive, aws-eusc-ipi-fips-f7 - Private: aws-eusc-ipi-private-f7, aws-eusc-ipi-private-f28-destructive, aws-eusc-ipi-private-fips-f7 - Disconnected: aws-eusc-ipi-disconnected-private-f7 - STS: aws-eusc-ipi-sts-f7 - KMS: aws-eusc-ipi-byo-kms-f7 Key features: - Dynamic service endpoint auto-detection from AWS API - Split AMI variables (CONTROL_PLANE_AMI, COMPUTE_AMI) for flexible configuration - Complete private cluster deprovision cleanup (bastion, security groups, stacks, S3) - Support for FIPS-enabled clusters - Disconnected (air-gapped) private cluster support - STS (Security Token Service) authentication with OIDC - Custom KMS key encryption for etcd - Both non-destructive (f7) and destructive (f28) test variants Technical implementation: - Cluster profile: aws-eusc with automatic region detection - Custom RHCOS AMI support for control plane and compute nodes separately - Endpoint auto-detection from AWS API (no hardcoded values) - Manual credentials mode for CCO - Minimal IAM permissions - Mirror registry for disconnected environments - Backward compatible with existing AWS partitions Workflows created: - cucushift-installer-rehearse-aws-eusc-ipi (common IPI) - cucushift-installer-rehearse-aws-eusc-ipi-private (private cluster) - cucushift-installer-rehearse-aws-eusc-ipi-disconnected-private (disconnected) - cucushift-installer-rehearse-aws-eusc-ipi-sts (STS authentication) - cucushift-installer-rehearse-aws-eusc-ipi-byo-kms (custom KMS key) Signed-off-by: Wei Li <weli@redhat.com>

@yunjiang29

This refactors the AWS European Sovereign Cloud (EUSC) CI configuration to maximize reuse of standard AWS workflows and reduce maintenance burden. Changes based on @yunjiang29's review feedback: - Reduced from 9 to 6 jobs following the pattern: 3 cluster types × 2 test types - Improved FIPS coverage from 1/9 (11%) to 2/6 (33%) jobs: * aws-eusc-ipi-fips-f7 (IPI + FIPS) * aws-eusc-ipi-private-sts-fips-f7 (Private + STS + FIPS) - Combined features across jobs: * aws-eusc-ipi-f28-destructive (destructive testing) * aws-eusc-ipi-private-mini-perm-f28 (Private + minimal permissions) * aws-eusc-ipi-disc-priv-kms-f7 (Disconnected + KMS) * aws-eusc-ipi-disc-priv-f28 (Disconnected destructive) - All jobs cover: FIPS, STS, KMS, minimal permissions across 3 cluster types - Deleted 15 EUSC-specific files, created 8 new ones (net reduction: -7 files) - Maximized reuse of standard AWS workflows: * Basic IPI: reuses cucushift-installer-rehearse-aws-ipi-deprovision * Private: reuses cucushift-installer-rehearse-aws-ipi-private-deprovision * Disconnected: reuses cucushift-installer-rehearse-aws-ipi-disconnected-private-provision * Private-STS: reuses cucushift-installer-rehearse-aws-ipi-private-cco-manual-security-token-service - EUSC-specific changes limited to: * Inserting ipi-conf-aws-custom-endpoints ref for service endpoint configuration * Custom provision chain for disconnected-private-kms (combines disconnected + KMS) - Deleted all EUSC-specific deprovision chains (reuse standard chains) - Removed unnecessary byo-kms and STS specific directory structures 1. Custom endpoints (ipi-conf-aws-custom-endpoints-commands.sh): - Removed auto-detection logic for AWS_DOMAIN_SUFFIX - Simplified to use environment variable or default to "amazonaws.com" - Removed Route53 endpoint configuration (global service) - Designed for easy removal when installer adds native EUSC support 2. AMI configuration (ipi-conf-aws-commands.sh): - Simplified from split variables (CONTROL_PLANE_AMI/COMPUTE_AMI) to single CONTROL_PLANE_AMI - Preserved C2S/SC2S auto-detection logic - Removed complex heredoc patching, kept simple approach - Updated documentation for clarity 1. **Minimize EUSC-specific code**: Only 8 workflow files vs 15 previously 2. **Maximize standard workflow reuse**: Follows USGov pattern, not C2S pattern 3. **Prepare for future evolution**: Custom endpoints easy to remove when installer supports EUSC natively 4. **FIPS coverage aligned with USGov**: 33% vs USGov's 18%, not C2S's 100% - make update completed successfully - All 6 jobs generated in ci-operator/jobs/.../periodics.yaml - Step registry validation passed Addresses: openshift#75568

Address review feedback to simplify configuration scripts: - Remove AWS_DOMAIN_SUFFIX from step parameters (use cluster profile) - Support separate CONTROL_PLANE_AMI and COMPUTE_AMI configuration - Replace yq-go with yq v4 for YAML manipulation - Eliminate unnecessary fallback logic, rely on correct parameter passing - Remove intermediate variables (RHCOS_AMI) in C2S auto-detection These changes follow existing script patterns and maintain compatibility with C2S/SC2S auto-detection while enabling flexible AMI configuration for partitions like EUSC.

Fixes CI check error that enforces OWNERS files for all component configuration directories.

- Update BASE_DOMAIN from qe.devcluster.openshift.com to ci-eusc.devcluster.openshift.com for all AWS EUSC CI jobs to use the dedicated delegated subdomain for CI/QE account - Add 8 multi-arch EUSC CI jobs in openshift-tests-private release-4.22 multi-nightly: * BYO KMS encryption with FIPS (ARM f7, AMD f28-destructive) * Disconnected private (ARM f7, AMD f28-destructive) * Private STS (ARM f7, AMD f28-destructive) * Custom DNS with minimal permissions (ARM f7, AMD f28-destructive) - Add e2e-aws-eusc-techpreview jobs to openshift/installer configs: * release-4.22, release-4.23, release-5.0, and main - Add installer repo to aws-eusc cluster profile owners - Restore version info comments in ipi-conf-aws-commands.sh All jobs use cluster_profile: aws-eusc with BASE_DOMAIN: ci-eusc.devcluster.openshift.com and FEATURE_SET: TechPreviewNoUpgrade.

The installer now configures service endpoints implicitly for EUSC partition, so manual endpoint configuration via ipi-conf-aws-custom-endpoints is no longer needed. Changes: - Remove ipi-conf-aws-custom-endpoints from all 5 EUSC workflow files - Update documentation to reflect implicit endpoint configuration - Simplify workflow by relying on installer's built-in EUSC support This addresses review feedback from yunjiang29 that the installer handles endpoints automatically for special AWS partitions like EUSC.

openshift-ci · 2026-03-18T14:41:19Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: liweinan
Once this PR has been reviewed and has the lgtm label, please assign liangxia, patrickdillon for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Update generated Prow job configurations after rebasing to the latest origin/main. Changes include: - Updated cluster assignments to match current build cluster distribution - EUSC jobs properly integrated with latest job generation logic

openshift-ci-robot · 2026-03-18T15:01:10Z

[REHEARSALNOTIFIER]
@liweinan: the pj-rehearse plugin accommodates running rehearsal tests for the changes in this PR. Expand 'Interacting with pj-rehearse' for usage details. The following rehearsable tests have been affected by this change:

Test name	Repo	Type	Reason
pull-ci-openshift-node_exporter-main-e2e-aws	openshift/node_exporter	presubmit	Registry content changed
pull-ci-openshift-node_exporter-main-okd-scos-e2e-aws-ovn	openshift/node_exporter	presubmit	Registry content changed
pull-ci-openshift-node_exporter-release-5.0-e2e-aws	openshift/node_exporter	presubmit	Registry content changed
pull-ci-openshift-node_exporter-release-4.23-e2e-aws	openshift/node_exporter	presubmit	Registry content changed
pull-ci-openshift-node_exporter-release-4.22-e2e-aws	openshift/node_exporter	presubmit	Registry content changed
pull-ci-openshift-node_exporter-release-4.21-e2e-aws	openshift/node_exporter	presubmit	Registry content changed
pull-ci-openshift-node_exporter-release-4.21-okd-scos-e2e-aws-ovn	openshift/node_exporter	presubmit	Registry content changed
pull-ci-openshift-node_exporter-release-4.20-e2e-aws	openshift/node_exporter	presubmit	Registry content changed
pull-ci-openshift-node_exporter-release-4.19-e2e-aws	openshift/node_exporter	presubmit	Registry content changed
pull-ci-openshift-node_exporter-release-4.18-e2e-aws	openshift/node_exporter	presubmit	Registry content changed
pull-ci-openshift-node_exporter-release-4.17-e2e-aws	openshift/node_exporter	presubmit	Registry content changed
pull-ci-openshift-node_exporter-release-4.16-e2e-aws	openshift/node_exporter	presubmit	Registry content changed
pull-ci-openshift-node_exporter-release-4.15-e2e-aws	openshift/node_exporter	presubmit	Registry content changed
pull-ci-openshift-node_exporter-release-4.14-e2e-aws	openshift/node_exporter	presubmit	Registry content changed
pull-ci-openshift-node_exporter-release-4.13-e2e-aws	openshift/node_exporter	presubmit	Registry content changed
pull-ci-openshift-node_exporter-release-4.12-e2e-aws	openshift/node_exporter	presubmit	Registry content changed
pull-ci-openshift-node_exporter-release-4.11-e2e-aws	openshift/node_exporter	presubmit	Registry content changed
pull-ci-openshift-node_exporter-release-4.10-e2e-aws	openshift/node_exporter	presubmit	Registry content changed
pull-ci-openshift-node_exporter-release-4.9-e2e-aws	openshift/node_exporter	presubmit	Registry content changed
pull-ci-openshift-node_exporter-release-4.8-e2e-aws	openshift/node_exporter	presubmit	Registry content changed
pull-ci-openshift-node_exporter-release-4.7-e2e-aws	openshift/node_exporter	presubmit	Registry content changed
pull-ci-openshift-node_exporter-release-4.6-e2e-aws	openshift/node_exporter	presubmit	Registry content changed
pull-ci-openshift-node_exporter-main-e2e-aws-upgrade	openshift/node_exporter	presubmit	Registry content changed
pull-ci-openshift-node_exporter-release-5.0-e2e-aws-upgrade	openshift/node_exporter	presubmit	Registry content changed
pull-ci-openshift-node_exporter-release-4.23-e2e-aws-upgrade	openshift/node_exporter	presubmit	Registry content changed

A total of 16213 jobs have been affected by this change. The above listing is non-exhaustive and limited to 25 jobs.

A full list of affected jobs can be found here

Interacting with pj-rehearse

Comment: /pj-rehearse to run up to 5 rehearsals
Comment: /pj-rehearse skip to opt-out of rehearsals
Comment: /pj-rehearse {test-name}, with each test separated by a space, to run one or more specific rehearsals
Comment: /pj-rehearse more to run up to 10 rehearsals
Comment: /pj-rehearse max to run up to 25 rehearsals
Comment: /pj-rehearse auto-ack to run up to 5 rehearsals, and add the rehearsals-ack label on success
Comment: /pj-rehearse list to get an up-to-date list of affected jobs
Comment: /pj-rehearse abort to abort all active rehearsals
Comment: /pj-rehearse network-access-allowed to allow rehearsals of tests that have the restrict_network_access field set to false. This must be executed by an openshift org member who is not the PR author

Once you are satisfied with the results of the rehearsals, comment: /pj-rehearse ack to unblock merge. When the rehearsals-ack label is present on your PR, merge will no longer be blocked by rehearsals.
If you would like the rehearsals-ack label removed, comment: /pj-rehearse reject to re-block merging.

openshift-ci · 2026-03-18T15:05:20Z

@liweinan: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Mar 2, 2026

openshift-ci bot requested review from neisw and xingxingxia March 2, 2026 18:05