Repository navigation
Add Shipwright Build and Shared Resource extension - #4130
sayan-biswas wants to merge 1 commit into
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #4130 +/- ##
==========================================
- Coverage 30.82% 30.78% -0.04%
==========================================
Files 478 526 +48
Lines 23418 25387 +1969
==========================================
+ Hits 7218 7815 +597
- Misses 16200 17572 +1372
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
7191617 to
7b1fd61
Compare
7b1fd61 to
53a146f
Compare
jkhelil
left a comment
There was a problem hiding this comment.
Add shipwrightbuildinformer + sharedresourceinformer event handlers with FilterController(&v1alpha1.TektonConfig{}) exactly like TektonChain. in pkg/reconciler/shared/tektonconfig/controller.go
|
@sayan-biswas you probably need to add network policies also to your implementation |
53a146f to
fab1478
Compare
fab1478 to
3778b78
Compare
3778b78 to
b48a54f
Compare
Network policies were there in the manifests, but I've removed them and added from the code like other components. |
cf11817 to
811de00
Compare
811de00 to
fccf62c
Compare
There was a problem hiding this comment.
🟡 Changes recommended
Critical RBAC gaps, missing Kubernetes Helm CRDs, and webhook TLS defects block installation or operation, while incomplete namespace validation permits adoption of protected namespaces.
25 open issues
Add Shipwright permissions to the effective OpenShift role · New Grant CSIDriver access for SharedResource installation and cleanup · New Clear stale component versions before fetching the configured version · New Include required CRDs in the Kubernetes chart · New Validate SharedResource namespace and network policy configuration · New Validate SharedResource namespace and network policy configuration · New Use the webhook's expected TLS Secret key names · New Renew webhook certificates before expiry and update the CA bundle · New Use the webhook's expected TLS Secret key names · New Inject the generated CA into Shipwright CRD conversion configs · New Grant CSIDriver permissions in the installed operator RBAC · New Wait for Shared Resource DaemonSets to become ready · New Apply TektonConfig scheduling settings to Shipwright workloads · New Continue reconciling Shipwright after Dashboard cleanup · New Apply image overrides to Shared Resource DaemonSets · New Apply SharedResource scheduling configuration to its workloads · New Propagate platform data changes to ShipwrightBuild · New Avoid shadowing the last observed resource state · New Check the Get error before dereferencing SharedResource · New Verify SharedResource main InstallerSets using the correct labels · New
And 5 more that still need to be addressed.
2 files not reviewed
pkg/apis/operator/v1alpha1/zz_generated.deepcopy.go: Generated filepkg/client/clientset/versioned/typed/operator/v1alpha1/fake/fake_operator_client.go: Generated file
What changed in this PR
Adds operator-managed Shipwright Build support on Kubernetes and OpenShift, plus an OpenShift-only SharedResource CSI component, integrated with TektonConfig and InstallerSets.
Changes:
- Adds component APIs, lifecycle helpers, validation, and generated clients and informers.
- Wires platform controllers, manifest transformations, certificates, and NetworkPolicies.
- Extends release fetching, CRD/RBAC packaging, deployment configuration, and unit/e2e tests.
This pull request requires changes. Unresolved findings affect installation permissions and packaging, namespace validation, webhook TLS, reconciliation and readiness, configuration propagation, release selection, and test reliability.
| File | Description |
|---|---|
test/utils/names.go |
Adds resource names for both components. |
test/utils/e2e_flags.go |
Adds component deployment label selectors. |
test/utils/clients.go |
Exposes typed component test clients. |
test/utils/cleanup.go |
Adds component teardown helpers. |
test/resources/shipwrightbuilds.go |
Adds Shipwright lifecycle test helpers. |
test/resources/sharedresources.go |
Adds SharedResource lifecycle test helpers. |
test/e2e/openshift/shipwrightbuild_test.go |
Adds OpenShift Shipwright installation and deletion tests. |
test/e2e/openshift/sharedresource_test.go |
Adds SharedResource lifecycle and deployment-recovery tests. |
test/e2e/kubernetes/shipwrightbuild_test.go |
Adds Kubernetes Shipwright lifecycle and recovery tests. |
pkg/webhook/webhook.go |
Registers both admission API types. |
pkg/reconciler/shared/tektonconfig/shipwrightbuild/shipwrightbuild_test.go |
Tests TektonConfig-driven Shipwright management. |
pkg/reconciler/shared/tektonconfig/sharedresource/sharedresource_test.go |
Tests TektonConfig-driven SharedResource management. |
pkg/reconciler/shared/tektonconfig/controller.go |
Watches both component CRs. |
pkg/reconciler/platform/const.go |
Defines component controller names. |
pkg/reconciler/openshift/tektonconfig/extension.go |
Adds reconciliation and teardown for both components. |
pkg/reconciler/openshift/shipwrightbuild/networkpolicies.go |
Defines Shipwright workload NetworkPolicies. |
pkg/reconciler/openshift/shipwrightbuild/networkpolicies_test.go |
Tests Shipwright policy construction. |
pkg/reconciler/openshift/shipwrightbuild/controller.go |
Connects the OpenShift Shipwright extension. |
pkg/reconciler/openshift/shipwrightbuild/common.go |
Defines webhook TLS flag constants. |
pkg/reconciler/openshift/sharedresource/transformer.go |
Adds SharedResource manifest transformations. |
pkg/reconciler/openshift/sharedresource/reconciler.go |
Implements SharedResource installation, status, and finalization. |
pkg/reconciler/openshift/sharedresource/networkpolicies.go |
Defines CSI driver and webhook NetworkPolicies. |
pkg/reconciler/openshift/sharedresource/networkpolicies_test.go |
Tests SharedResource policy construction. |
pkg/reconciler/openshift/sharedresource/extension.go |
Adds OpenShift TLS, metrics, and policy hooks. |
pkg/reconciler/openshift/sharedresource/extension_test.go |
Tests extension hooks and transformations. |
pkg/reconciler/openshift/sharedresource/controller.go |
Initializes SharedResource manifests and watches. |
pkg/reconciler/openshift/sharedresource/common.go |
Defines workload, metrics, and TLS identifiers. |
pkg/reconciler/openshift/openshiftplatform/config.go |
Registers both OpenShift component controllers. |
pkg/reconciler/openshift/common/serviceca.go |
Adds service-CA annotation transformers. |
pkg/reconciler/openshift/common/serviceca_test.go |
Tests service-CA annotation behavior. |
pkg/reconciler/kubernetes/tektonconfig/extension.go |
Adds TektonConfig-driven Shipwright management. |
pkg/reconciler/kubernetes/shipwrightbuild/transformer.go |
Adds Shipwright payload and webhook-argument transformations. |
pkg/reconciler/kubernetes/shipwrightbuild/transformer_test.go |
Tests manifest filtering and argument injection. |
pkg/reconciler/kubernetes/shipwrightbuild/extension.go |
Adds Kubernetes webhook certificate provisioning. |
pkg/reconciler/kubernetes/shipwrightbuild/controller.go |
Initializes Shipwright payloads, strategies, and watches. |
pkg/reconciler/kubernetes/shipwrightbuild/common.go |
Defines workload constants and webhook Secret provisioning. |
pkg/reconciler/kubernetes/shipwrightbuild/common_test.go |
Tests webhook Secret creation and reuse. |
pkg/reconciler/kubernetes/kubernetesplatform/config.go |
Registers the Kubernetes Shipwright controller. |
pkg/reconciler/common/transformers.go |
Adds image prefixes and container-argument injection. |
pkg/reconciler/common/transformer_add_container_args_test.go |
Tests container-argument transformation behavior. |
pkg/reconciler/common/testdata/kodata/shipwright-build/0.21.0/release.yaml |
Adds a Shipwright release-discovery fixture. |
pkg/reconciler/common/testdata/kodata/shipwright-build/0.20.0/release.yaml |
Adds an older Shipwright release fixture. |
pkg/reconciler/common/testdata/kodata/shared-resource/0.2.0/release.yaml |
Adds a SharedResource release-discovery fixture. |
pkg/reconciler/common/testdata/kodata/shared-resource/0.1.0/release.yaml |
Adds an older SharedResource release fixture. |
pkg/reconciler/common/releases.go |
Maps component types to payload directories. |
pkg/reconciler/common/releases_test.go |
Tests release discovery for both components. |
pkg/reconciler/common/initcontroller.go |
Registers component payload loading. |
pkg/reconciler/common/common.go |
Adds manifest resource-kind constants. |
pkg/reconciler/common/certificates.go |
Adds configurable-duration certificate Secret generation. |
pkg/reconciler/common/certificates_test.go |
Tests generated certificate Secret contents. |
pkg/common/scc.go |
Uses the injected client configuration for security clients. |
pkg/client/listers/operator/v1alpha1/shipwrightbuild.go |
Adds the generated Shipwright lister. |
pkg/client/listers/operator/v1alpha1/sharedresource.go |
Adds the generated SharedResource lister. |
pkg/client/listers/operator/v1alpha1/expansion_generated.go |
Adds component lister expansion interfaces. |
pkg/client/injection/reconciler/operator/v1alpha1/shipwrightbuild/state.go |
Adds generated Shipwright reconciliation state handling. |
pkg/client/injection/reconciler/operator/v1alpha1/shipwrightbuild/controller.go |
Adds generated Shipwright controller wiring. |
pkg/client/injection/reconciler/operator/v1alpha1/sharedresource/state.go |
Adds generated SharedResource reconciliation state handling. |
pkg/client/injection/reconciler/operator/v1alpha1/sharedresource/controller.go |
Adds generated SharedResource controller wiring. |
pkg/client/injection/informers/operator/v1alpha1/shipwrightbuild/shipwrightbuild.go |
Adds Shipwright informer injection. |
pkg/client/injection/informers/operator/v1alpha1/shipwrightbuild/filtered/shipwrightbuild.go |
Adds filtered Shipwright informer injection. |
pkg/client/injection/informers/operator/v1alpha1/shipwrightbuild/filtered/fake/fake.go |
Adds filtered fake Shipwright informer injection. |
pkg/client/injection/informers/operator/v1alpha1/shipwrightbuild/fake/fake.go |
Adds fake Shipwright informer injection. |
pkg/client/injection/informers/operator/v1alpha1/sharedresource/sharedresource.go |
Adds SharedResource informer injection. |
pkg/client/injection/informers/operator/v1alpha1/sharedresource/filtered/sharedresource.go |
Adds filtered SharedResource informer injection. |
pkg/client/injection/informers/operator/v1alpha1/sharedresource/filtered/fake/fake.go |
Adds filtered fake SharedResource informer injection. |
pkg/client/injection/informers/operator/v1alpha1/sharedresource/fake/fake.go |
Adds fake SharedResource informer injection. |
pkg/client/informers/externalversions/operator/v1alpha1/interface.go |
Exposes both typed component informers. |
pkg/client/informers/externalversions/generic.go |
Registers generic component informer lookup. |
pkg/client/clientset/versioned/typed/operator/v1alpha1/shipwrightbuild.go |
Adds the generated Shipwright API client. |
pkg/client/clientset/versioned/typed/operator/v1alpha1/sharedresource.go |
Adds the generated SharedResource API client. |
pkg/client/clientset/versioned/typed/operator/v1alpha1/operator_client.go |
Exposes both component clients. |
pkg/client/clientset/versioned/typed/operator/v1alpha1/generated_expansion.go |
Adds component client expansion interfaces. |
pkg/client/clientset/versioned/typed/operator/v1alpha1/fake/fake_shipwrightbuild.go |
Adds the fake Shipwright API client. |
pkg/client/clientset/versioned/typed/operator/v1alpha1/fake/fake_sharedresource.go |
Adds the fake SharedResource API client. |
pkg/client/clientset/versioned/typed/operator/v1alpha1/fake/fake_operator_client.go |
Exposes both fake component clients. |
pkg/apis/operator/v1alpha1/shipwrightbuild_validation.go |
Adds Shipwright singleton-name validation. |
pkg/apis/operator/v1alpha1/shipwrightbuild_validation_test.go |
Tests Shipwright name and deletion validation. |
pkg/apis/operator/v1alpha1/shipwrightbuild_types.go |
Defines the Shipwright component API and configuration. |
pkg/apis/operator/v1alpha1/shipwrightbuild_lifecycle.go |
Adds Shipwright condition and status helpers. |
pkg/apis/operator/v1alpha1/shipwrightbuild_lifecycle_test.go |
Tests Shipwright lifecycle status transitions. |
pkg/apis/operator/v1alpha1/shipwrightbuild_defaults.go |
Adds the Shipwright defaulting hook. |
pkg/apis/operator/v1alpha1/sharedresource_validation.go |
Adds SharedResource singleton-name validation. |
pkg/apis/operator/v1alpha1/sharedresource_validation_test.go |
Tests SharedResource name and deletion validation. |
pkg/apis/operator/v1alpha1/sharedresource_types.go |
Defines the SharedResource component API and configuration. |
pkg/apis/operator/v1alpha1/sharedresource_lifecycle.go |
Adds SharedResource condition and status helpers. |
pkg/apis/operator/v1alpha1/sharedresource_lifecycle_test.go |
Tests SharedResource lifecycle status transitions. |
pkg/apis/operator/v1alpha1/sharedresource_defaults.go |
Adds the SharedResource defaulting hook. |
pkg/apis/operator/v1alpha1/register.go |
Registers both component API kinds and lists. |
pkg/apis/operator/v1alpha1/openshift_platform.go |
Exposes both components in OpenShift configuration. |
pkg/apis/operator/v1alpha1/kubernetes_platform.go |
Exposes Shipwright in Kubernetes configuration. |
pkg/apis/operator/v1alpha1/const.go |
Defines singleton component resource names. |
Makefile |
Simplifies progress-message punctuation. |
hack/sync-helm-crds.sh |
Adds both component CRDs to OpenShift Helm synchronization. |
hack/openshift/fetch-build-strategies.sh |
Adds semantic-version tag-selection tooling. |
hack/fetch-releases.sh |
Adds component and platform-specific payload fetching. |
config/openshift/base/operator.yaml |
Enables both OpenShift component controllers. |
config/kubernetes/base/operator.yaml |
Enables the Kubernetes Shipwright controller. |
config/base/role.yaml |
Adds Shipwright build-strategy permissions. |
config/base/kustomization.yaml |
Includes both generated component CRDs. |
config/base/generated-crds/operator.tekton.dev_tektonconfigs.yaml |
Adds platform configuration schema fields. |
components.yaml |
Defines component repositories and release versions. |
Review effort: Balanced
| if !ok { | ||
| t.Fatal("Failed to get caller information") | ||
| } | ||
| m, err := mfc.NewManifest(filepath.Join((filepath.Dir(b)+"/.."), "manifests/"), clients.Config) |
|
|
||
| // AssertShipwrightBuildInstallerSets verifies if the TektonInstallerSets are created. | ||
| func AssertShipwrightBuildInstallerSets(t *testing.T, clients *utils.Clients) { | ||
| assertInstallerSets(t, clients, v1alpha1.ShipwrightBuildResourceName) |
| if !ok { | ||
| t.Fatal("Failed to get caller information") | ||
| } | ||
| m, err := mfc.NewManifest(filepath.Join((filepath.Dir(b)+"/.."), "manifests/"), clients.Config) |
There was a problem hiding this comment.
same as Jawed's comment below.
| if !ok { | ||
| t.Fatal("Failed to get caller information") | ||
| } | ||
| m, err := mfc.NewManifest(filepath.Join((filepath.Dir(b)+"/.."), "manifests/"), clients.Config) |
There was a problem hiding this comment.
filepath.Dir(b) resolves to test/resources/, so the path becomes test/manifests/. That directory only contains tekton-pipeline, there are no shipwright-build/ or shared-resource/ subdirectories
There was a problem hiding this comment.
The other components also does the same, so I was assuming manifests are pulled in during the test workflow runtime. Is there anything like that?
|
Please
// sharedresource_types.go We need to turn also shareresource default to false
|
0de4233 to
13c5345
Compare
13c5345 to
a3b3cbc
Compare
| sr.Status.MarkPreReconcilerComplete() | ||
|
|
||
| // Create Main InstallerSet | ||
| if err := r.installerSetClient.MainSet(ctx, sr, r.releaseManifest, FilterAndTransform(r)); err != nil { |
There was a problem hiding this comment.
For a standalone CR, should enabled: false disable the component, or is deleting the CR the intended way to remove it? Both SharedResource and ShipwrightBuild currently ignore this flag, while their docs say setting it to false removes the component.
There was a problem hiding this comment.
I don't see the other controllers using this property either. It's better to remove it from the Docs. Considering this would need sync the property from TektonConfig property and this CR's for edge cases.
| args: | ||
| - "-controllers" | ||
| - "tektonconfig,tektonpipeline,tektontrigger,tektonchain,tektonresult,tektondashboard,manualapprovalgate,tektonpruner,tektonkueue,tektonmulticlusterproxyaae,openshiftpipelinesascode" | ||
| - "tektonconfig,tektonpipeline,tektontrigger,tektonchain,tektonresult,tektondashboard,manualapprovalgate,tektonpruner,tektonkueue,tektonmulticlusterproxyaae,openshiftpipelinesascode,shipwrightbuild" |
There was a problem hiding this comment.
Are Helm installations meant to support these components as part of this PR? New CRDs are being added to the chart, but its controller lists and operator roles are still missing the corresponding entries. If Helm support is included, those need updating too.
There was a problem hiding this comment.
For kubernetes, only shipwrightbuild controller will be supported.
the roles includes "roles.yaml" under base and in openshift includes "shipwright.io" group.
Is there any other places they needs to be included?
| transformers = append(transformers, | ||
| openshiftcommon.AnnotateMetricsServingCert(shipwrightbuild.ControllerDeploymentName), | ||
| openshiftcommon.RenameServicePort(shipwrightbuild.ControllerDeploymentName, openshiftcommon.MetricsHTTPPort, openshiftcommon.MetricsHTTPSPort), | ||
| openshiftcommon.ApplyMetricsTLS(common.KindDeployment, shipwrightbuild.ControllerDeploymentName, |
There was a problem hiding this comment.
Where is shipwright-build-controller-metrics-tls expected to be created? This mounts it as a required Secret, but the certificate annotation targets a shipwright-build-controller Service that is not in the bundled release. Without that Secret, a fresh controller Pod cannot start when metrics mTLS is enabled.
There was a problem hiding this comment.
Created an issue upstream to include the service in the release manifests.
shipwright-io/build#2411
I'll comment out the metric tls for now.
… component Changes: Adds two new operator-managed components: ShipwrightBuild and SharedResource. New CRDs & API types — ShipwrightBuild and SharedResource added to pkg/apis/operator/v1alpha1/ (types, defaults, lifecycle, validation), registered in the scheme (register.go) with resource-name constants and the one-CR-per-type Validate() check. ShipwrightBuild component — Kubernetes reconciler + transformer and an OpenShift extension/controller, wired through TektonConfig on both platforms. Installs the Shipwright Build controller. SharedResource component — OpenShift-only reconciler, extension, controller, and transformer, plus NetworkPolicies and service-CA handling. OpenShift service-CA helper — new pkg/reconciler/openshift/common/serviceca.go for injecting the service CA bundle. Platform wiring — registered both components in TektonConfig extensions (Kubernetes & OpenShift), platform constants, the webhook, and the shared/init controllers. TektonInstallerSet DaemonSet support — adds a DaemonSet installer to the InstallerSet reconcile flow and client (new DaemonSetReady condition), mirroring the StatefulSet handling for apply, readiness wait, and cleanup, so components can ship DaemonSet workloads. RBAC / config — updates to config/base/role.yaml, kustomization, and operator manifests. Tests — unit tests for reconcilers, transformers, validation, lifecycle, NetworkPolicies, service-CA, and the DaemonSet installer; e2e tests and test/resources / test/utils helpers for both new components. Installs cert-manager on Kubernetes (skipped on OpenShift) on E2E workflow, so the ShipwrightBuild webhook certificate can be provisioned. Signed-off-by: Sayan Biswas <sabiswas@redhat.com>
a3b3cbc to
3ab2bdc
Compare
| | `catalog-{INDEX}-url` | URL endpoint of the hub API | Yes | | ||
|
|
||
| `catalog-{INDEX}-type` and `hub-catalog-type` were removed in Pipelines as Code 0.51.0. On upgrade, the operator deletes `hub-catalog-type` from TektonConfig, the OpenShiftPipelinesAsCode CR, and the `pipelines-as-code` ConfigMap. | ||
| | `catalog-{INDEX}-type` | Type of hub: `artifacthub` (for Artifact Hub) or `tektonhub` (for Tekton Hub) | Yes | |
There was a problem hiding this comment.
is this change required for the Shipwright component ?
|
|
||
| metrics, err := common.NoMetrics() | ||
| if err != nil { | ||
| logger.Errorf("Failed to create trigger metrics recorder %v", err) |
There was a problem hiding this comment.
| logger.Errorf("Failed to create trigger metrics recorder %v", err) | |
| logger.Errorf("Failed to create shipwrightbuild metrics recorder %v", err) |
| } | ||
|
|
||
| // On TektonConfig teardown, remove the shipwright-build CRs it owns. | ||
| if err := shipwrightbuild.DeleteIfExists(ctx, oe.operatorClientSet.OperatorV1alpha1().ShipwrightBuilds()); err != nil { |
There was a problem hiding this comment.
why do we want to remove shipwright-build CR?


Changes
Adds two new operator-managed components: ShipwrightBuild and SharedResource.
Submitter Checklist
These are the criteria that every PR should meet, please check them off as you
review them:
make test lintbefore submitting a PRSee the contribution guide for more details.
Release Notes