Skip to content

stage kops/kubernetes to s3 buckets when running ec2 jobs - #18922

Merged
kubernetes-prow[bot] merged 1 commit into
kubernetes:masterfrom
upodroid:ec2-build-fix
Oct 31, 2026
Merged

kubernetes-prow[bot] merged 1 commit into
kubernetes:masterfrom
upodroid:ec2-build-fix

Conversation

@upodroid

Copy link
Copy Markdown
Member

No description provided.

@kubernetes-prow kubernetes-prow Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 25, 2026
@kubernetes-prow kubernetes-prow Bot added area/documentation cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Oct 25, 2026
@kubernetes-prow kubernetes-prow Bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Oct 26, 2026
@upodroid upodroid changed the title [wip] stage kops/kubernetes to s3 buckets when running ec2 jobs stage kops/kubernetes to s3 buckets when running ec2 jobs Oct 26, 2026
@kubernetes-prow kubernetes-prow Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 26, 2026
@upodroid

Copy link
Copy Markdown
Member Author

This is working correctly, we need the linked PR in k/release to be merged first.

@hakman

hakman commented Oct 26, 2026

Copy link
Copy Markdown
Member

This is working correctly, we need the linked PR in k/release to be merged first.

I am having some trouble understanding what's happening here. Was there a discussion that I missed somewhere?

Comment thread docs/contributing/test_versions.md Resolved
@upodroid

upodroid commented Oct 26, 2026 •

Copy link
Copy Markdown
Member Author

This is working correctly, we need the linked PR in k/release to be merged first.

I am having some trouble understanding what's happening here. Was there a discussion that I missed somewhere?

It's a continuation of #18662 but it also works on the build-kubernetes flag which your PR didn't modify.

Also, getting kops e2e jobs to use ephemeral S3 buckets instead of gs://k8s-staging-kops/pulls has been a long-standing to-do item. (Eliminating the GCP egress fee as well as blocking presubmits from writing to the staging bucket.)

@upodroid

upodroid commented Oct 27, 2026 •

Copy link
Copy Markdown
Member Author

The k/release PR has been merged. We can merge this PR now.

@hakman

hakman commented Oct 27, 2026

Copy link
Copy Markdown
Member

This is working correctly, we need the linked PR in k/release to be merged first.

I am having some trouble understanding what's happening here. Was there a discussion that I missed somewhere?

It's a continuation of #18662 but it also works on the build-kubernetes flag which your PR didn't modify.

Also, getting kops e2e jobs to use ephemeral S3 buckets instead of gs://k8s-staging-kops/pulls has been a long-standing to-do item. (Eliminating the GCP egress fee as well as blocking presubmits from writing to the staging bucket.)

Please try to understand, I am not aware of your intentions.
The PR description is BLANK, and this is a non-trivial change to review.

@hakman hakman left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, the S3 path works end to end in the calico and ec2-build logs. A few issues inline.

For the description: besides S3 staging, this fixes --build-kubernetes on AWS. Master rejects the default gs://k8s-staging-kops/pulls/... path, which is what fails pull-kubernetes-ec2-master-scale-correctness-2000 today. It also makes pull-kops-kubernetes-e2e-ubuntu-ec2-build actually run on AWS.

Comment thread tests/e2e/kubetest2-kops/aws/s3.go Resolved
@upodroid

Copy link
Copy Markdown
Member Author

I applied all the suggestions.

@hakman hakman left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One more small change, please.

Comment on lines 44 to 49
}
results, err := d.BuildOptions.Build()
if err != nil {
return err
// kubetest2 does not call Down() when Build() fails, so the staging bucket would leak.
return errors.Join(err, d.deleteStagingStore())
}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This only covers the builder failing. A defer catches all cases.

func (d *deployer) Build() (err error) {
	// kubetest2 does not call Down() when Build() fails, so delete the staging bucket on
	// every error, not only when the builder fails.
	defer func() {
		if err != nil {
			err = errors.Join(err, d.deleteStagingStore())
		}
	}()

	if err := d.init(); err != nil {
		return err
	}
	results, err := d.BuildOptions.Build()
	if err != nil {
		return err
	}

@kubernetes-prow kubernetes-prow Bot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 31, 2026
@kubernetes-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: hakman

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubernetes-prow kubernetes-prow Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Oct 31, 2026
@hakman

hakman commented Oct 31, 2026

Copy link
Copy Markdown
Member

/test pull-kops-e2e-kubernetes-azure
/test pull-kops-e2e-kubernetes-do-dns-none
/test pull-kops-aws-upgrade-dns-none
/test pull-kops-e2e-aws-ipv6-external-dns
/test pull-kops-ec2-master-scale-performance-100
/hold for tests to run

@kubernetes-prow kubernetes-prow Bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 31, 2026
@hakman

hakman commented Oct 31, 2026

Copy link
Copy Markdown
Member

/test pull-kops-e2e-k8s-aws-amazonvpc
/test pull-kops-aws-upgrade-dns-none

@upodroid

Copy link
Copy Markdown
Member Author

/hold cancel
/retest

@kubernetes-prow kubernetes-prow Bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 31, 2026
@upodroid

Copy link
Copy Markdown
Member Author

/hold

@kubernetes-prow kubernetes-prow Bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 31, 2026
@kubernetes-prow kubernetes-prow Bot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 31, 2026
@upodroid

Copy link
Copy Markdown
Member Author

/test pull-kops-e2e-kubernetes-azure
/test pull-kops-e2e-kubernetes-do-dns-none
/test pull-kops-aws-upgrade-dns-none
/test pull-kops-e2e-aws-ipv6-external-dns
/test pull-kops-ec2-master-scale-performance-100

@upodroid

Copy link
Copy Markdown
Member Author

/hold cancel

All the tests are passing

@kubernetes-prow kubernetes-prow Bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 31, 2026
@hakman

hakman commented Oct 31, 2026

Copy link
Copy Markdown
Member

/lgtm

@kubernetes-prow kubernetes-prow Bot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Oct 31, 2026
@hakman

hakman commented Oct 31, 2026

Copy link
Copy Markdown
Member

Thanks!

@kubernetes-prow
kubernetes-prow Bot merged commit a17cd15 into kubernetes:master Oct 31, 2026
33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. area/documentation cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants