Skip to content

Commit 371c533

Browse files
labrenbelabrenbe
authored
feat: Allow specifying Helm values for operator installations (#425)
* allow specifying a values file that will be used for operator installations * minor refactor * rename FileTransfer error variant * Remove common key in yaml and deep merge logic; recommend yaml anchors * Improve helm value file error output * Fix values file example * explain why serialization statement won't fail * fix pre-commit
1 parent 4bab412 commit 371c533

34 files changed

Lines changed: 1903 additions & 65 deletions

‎docs/modules/stackablectl/partials/commands/cache.adoc‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,31 @@ File options:
6161
Use "stackablectl [OPTIONS] <COMMAND> -r path/to/releases1.yaml -r path/to/releases2.yaml"
6262
to provide multiple additional release files.
6363
64+
-f, --operator-values <VALUES_FILE>
65+
Path to a Helm values file that will be used for the installation of operators
66+
67+
The file is a YAML file containing Helm values used to deploy operators.
68+
Operator-specific keys (e.g. 'airflow-operator', 'zookeeper-operator') map
69+
to the Helm values for that operator. Use YAML anchors and aliases to share
70+
values across operators.
71+
72+
Example values file:
73+
74+
airflow-operator:
75+
tolerations: &default-tolerations
76+
- key: "example"
77+
operator: "Exists"
78+
effect: "NoSchedule"
79+
podAnnotations:
80+
example.com/team: "data-engineering"
81+
zookeeper-operator:
82+
tolerations: *default-tolerations
83+
podAnnotations:
84+
example.com/team: "platform"
85+
86+
Use "stackablectl [OPTIONS] <COMMAND> -f path/to/values.yaml" to provide a
87+
values file.
88+
6489
Helm repository options:
6590
--helm-repo-stable <URL>
6691
Provide a custom Helm stable repository URL
@@ -90,6 +115,8 @@ Operator specific configurations:
90115
--listener-class-preset <LISTENER_CLASS_PRESET>
91116
Choose the ListenerClass preset (`none`, `ephemeral-nodes` or `stable-nodes`).
92117
118+
This takes precedence over values provided via the '-f/--operator-values' flag.
119+
93120
This maps to the listener-operator Helm Chart preset value, see [the listener-operator documentation](https://docs.stackable.tech/home/nightly/listener-operator/listenerclass/#presets) for details.
94121
95122
[possible values: none, stable-nodes, ephemeral-nodes]

‎docs/modules/stackablectl/partials/commands/completions.adoc‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,31 @@ File options:
6464
Use "stackablectl [OPTIONS] <COMMAND> -r path/to/releases1.yaml -r path/to/releases2.yaml"
6565
to provide multiple additional release files.
6666
67+
-f, --operator-values <VALUES_FILE>
68+
Path to a Helm values file that will be used for the installation of operators
69+
70+
The file is a YAML file containing Helm values used to deploy operators.
71+
Operator-specific keys (e.g. 'airflow-operator', 'zookeeper-operator') map
72+
to the Helm values for that operator. Use YAML anchors and aliases to share
73+
values across operators.
74+
75+
Example values file:
76+
77+
airflow-operator:
78+
tolerations: &default-tolerations
79+
- key: "example"
80+
operator: "Exists"
81+
effect: "NoSchedule"
82+
podAnnotations:
83+
example.com/team: "data-engineering"
84+
zookeeper-operator:
85+
tolerations: *default-tolerations
86+
podAnnotations:
87+
example.com/team: "platform"
88+
89+
Use "stackablectl [OPTIONS] <COMMAND> -f path/to/values.yaml" to provide a
90+
values file.
91+
6792
Helm repository options:
6893
--helm-repo-stable <URL>
6994
Provide a custom Helm stable repository URL
@@ -93,6 +118,8 @@ Operator specific configurations:
93118
--listener-class-preset <LISTENER_CLASS_PRESET>
94119
Choose the ListenerClass preset (`none`, `ephemeral-nodes` or `stable-nodes`).
95120
121+
This takes precedence over values provided via the '-f/--operator-values' flag.
122+
96123
This maps to the listener-operator Helm Chart preset value, see [the listener-operator documentation](https://docs.stackable.tech/home/nightly/listener-operator/listenerclass/#presets) for details.
97124
98125
[possible values: none, stable-nodes, ephemeral-nodes]

‎docs/modules/stackablectl/partials/commands/demo.adoc‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,31 @@ File options:
6565
Use "stackablectl [OPTIONS] <COMMAND> -r path/to/releases1.yaml -r path/to/releases2.yaml"
6666
to provide multiple additional release files.
6767
68+
-f, --operator-values <VALUES_FILE>
69+
Path to a Helm values file that will be used for the installation of operators
70+
71+
The file is a YAML file containing Helm values used to deploy operators.
72+
Operator-specific keys (e.g. 'airflow-operator', 'zookeeper-operator') map
73+
to the Helm values for that operator. Use YAML anchors and aliases to share
74+
values across operators.
75+
76+
Example values file:
77+
78+
airflow-operator:
79+
tolerations: &default-tolerations
80+
- key: "example"
81+
operator: "Exists"
82+
effect: "NoSchedule"
83+
podAnnotations:
84+
example.com/team: "data-engineering"
85+
zookeeper-operator:
86+
tolerations: *default-tolerations
87+
podAnnotations:
88+
example.com/team: "platform"
89+
90+
Use "stackablectl [OPTIONS] <COMMAND> -f path/to/values.yaml" to provide a
91+
values file.
92+
6893
Helm repository options:
6994
--helm-repo-stable <URL>
7095
Provide a custom Helm stable repository URL
@@ -94,6 +119,8 @@ Operator specific configurations:
94119
--listener-class-preset <LISTENER_CLASS_PRESET>
95120
Choose the ListenerClass preset (`none`, `ephemeral-nodes` or `stable-nodes`).
96121
122+
This takes precedence over values provided via the '-f/--operator-values' flag.
123+
97124
This maps to the listener-operator Helm Chart preset value, see [the listener-operator documentation](https://docs.stackable.tech/home/nightly/listener-operator/listenerclass/#presets) for details.
98125
99126
[possible values: none, stable-nodes, ephemeral-nodes]

‎docs/modules/stackablectl/partials/commands/experimental-debug.adoc‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -78,6 +78,31 @@ File options:
7878
Use "stackablectl [OPTIONS] <COMMAND> -r path/to/releases1.yaml -r path/to/releases2.yaml"
7979
to provide multiple additional release files.
8080
81+
-f, --operator-values <VALUES_FILE>
82+
Path to a Helm values file that will be used for the installation of operators
83+
84+
The file is a YAML file containing Helm values used to deploy operators.
85+
Operator-specific keys (e.g. 'airflow-operator', 'zookeeper-operator') map
86+
to the Helm values for that operator. Use YAML anchors and aliases to share
87+
values across operators.
88+
89+
Example values file:
90+
91+
airflow-operator:
92+
tolerations: &default-tolerations
93+
- key: "example"
94+
operator: "Exists"
95+
effect: "NoSchedule"
96+
podAnnotations:
97+
example.com/team: "data-engineering"
98+
zookeeper-operator:
99+
tolerations: *default-tolerations
100+
podAnnotations:
101+
example.com/team: "platform"
102+
103+
Use "stackablectl [OPTIONS] <COMMAND> -f path/to/values.yaml" to provide a
104+
values file.
105+
81106
Helm repository options:
82107
--helm-repo-stable <URL>
83108
Provide a custom Helm stable repository URL
@@ -107,6 +132,8 @@ Operator specific configurations:
107132
--listener-class-preset <LISTENER_CLASS_PRESET>
108133
Choose the ListenerClass preset (`none`, `ephemeral-nodes` or `stable-nodes`).
109134
135+
This takes precedence over values provided via the '-f/--operator-values' flag.
136+
110137
This maps to the listener-operator Helm Chart preset value, see [the listener-operator documentation](https://docs.stackable.tech/home/nightly/listener-operator/listenerclass/#presets) for details.
111138
112139
[possible values: none, stable-nodes, ephemeral-nodes]

‎docs/modules/stackablectl/partials/commands/index.adoc‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -68,6 +68,31 @@ File options:
6868
Use "stackablectl [OPTIONS] <COMMAND> -r path/to/releases1.yaml -r path/to/releases2.yaml"
6969
to provide multiple additional release files.
7070
71+
-f, --operator-values <VALUES_FILE>
72+
Path to a Helm values file that will be used for the installation of operators
73+
74+
The file is a YAML file containing Helm values used to deploy operators.
75+
Operator-specific keys (e.g. 'airflow-operator', 'zookeeper-operator') map
76+
to the Helm values for that operator. Use YAML anchors and aliases to share
77+
values across operators.
78+
79+
Example values file:
80+
81+
airflow-operator:
82+
tolerations: &default-tolerations
83+
- key: "example"
84+
operator: "Exists"
85+
effect: "NoSchedule"
86+
podAnnotations:
87+
example.com/team: "data-engineering"
88+
zookeeper-operator:
89+
tolerations: *default-tolerations
90+
podAnnotations:
91+
example.com/team: "platform"
92+
93+
Use "stackablectl [OPTIONS] <COMMAND> -f path/to/values.yaml" to provide a
94+
values file.
95+
7196
Helm repository options:
7297
--helm-repo-stable <URL>
7398
Provide a custom Helm stable repository URL
@@ -97,6 +122,8 @@ Operator specific configurations:
97122
--listener-class-preset <LISTENER_CLASS_PRESET>
98123
Choose the ListenerClass preset (`none`, `ephemeral-nodes` or `stable-nodes`).
99124
125+
This takes precedence over values provided via the '-f/--operator-values' flag.
126+
100127
This maps to the listener-operator Helm Chart preset value, see [the listener-operator documentation](https://docs.stackable.tech/home/nightly/listener-operator/listenerclass/#presets) for details.
101128
102129
[possible values: none, stable-nodes, ephemeral-nodes]

‎docs/modules/stackablectl/partials/commands/operator.adoc‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,31 @@ File options:
6464
Use "stackablectl [OPTIONS] <COMMAND> -r path/to/releases1.yaml -r path/to/releases2.yaml"
6565
to provide multiple additional release files.
6666
67+
-f, --operator-values <VALUES_FILE>
68+
Path to a Helm values file that will be used for the installation of operators
69+
70+
The file is a YAML file containing Helm values used to deploy operators.
71+
Operator-specific keys (e.g. 'airflow-operator', 'zookeeper-operator') map
72+
to the Helm values for that operator. Use YAML anchors and aliases to share
73+
values across operators.
74+
75+
Example values file:
76+
77+
airflow-operator:
78+
tolerations: &default-tolerations
79+
- key: "example"
80+
operator: "Exists"
81+
effect: "NoSchedule"
82+
podAnnotations:
83+
example.com/team: "data-engineering"
84+
zookeeper-operator:
85+
tolerations: *default-tolerations
86+
podAnnotations:
87+
example.com/team: "platform"
88+
89+
Use "stackablectl [OPTIONS] <COMMAND> -f path/to/values.yaml" to provide a
90+
values file.
91+
6792
Helm repository options:
6893
--helm-repo-stable <URL>
6994
Provide a custom Helm stable repository URL
@@ -93,6 +118,8 @@ Operator specific configurations:
93118
--listener-class-preset <LISTENER_CLASS_PRESET>
94119
Choose the ListenerClass preset (`none`, `ephemeral-nodes` or `stable-nodes`).
95120
121+
This takes precedence over values provided via the '-f/--operator-values' flag.
122+
96123
This maps to the listener-operator Helm Chart preset value, see [the listener-operator documentation](https://docs.stackable.tech/home/nightly/listener-operator/listenerclass/#presets) for details.
97124
98125
[possible values: none, stable-nodes, ephemeral-nodes]

‎docs/modules/stackablectl/partials/commands/release.adoc‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -64,6 +64,31 @@ File options:
6464
Use "stackablectl [OPTIONS] <COMMAND> -r path/to/releases1.yaml -r path/to/releases2.yaml"
6565
to provide multiple additional release files.
6666
67+
-f, --operator-values <VALUES_FILE>
68+
Path to a Helm values file that will be used for the installation of operators
69+
70+
The file is a YAML file containing Helm values used to deploy operators.
71+
Operator-specific keys (e.g. 'airflow-operator', 'zookeeper-operator') map
72+
to the Helm values for that operator. Use YAML anchors and aliases to share
73+
values across operators.
74+
75+
Example values file:
76+
77+
airflow-operator:
78+
tolerations: &default-tolerations
79+
- key: "example"
80+
operator: "Exists"
81+
effect: "NoSchedule"
82+
podAnnotations:
83+
example.com/team: "data-engineering"
84+
zookeeper-operator:
85+
tolerations: *default-tolerations
86+
podAnnotations:
87+
example.com/team: "platform"
88+
89+
Use "stackablectl [OPTIONS] <COMMAND> -f path/to/values.yaml" to provide a
90+
values file.
91+
6792
Helm repository options:
6893
--helm-repo-stable <URL>
6994
Provide a custom Helm stable repository URL
@@ -93,6 +118,8 @@ Operator specific configurations:
93118
--listener-class-preset <LISTENER_CLASS_PRESET>
94119
Choose the ListenerClass preset (`none`, `ephemeral-nodes` or `stable-nodes`).
95120
121+
This takes precedence over values provided via the '-f/--operator-values' flag.
122+
96123
This maps to the listener-operator Helm Chart preset value, see [the listener-operator documentation](https://docs.stackable.tech/home/nightly/listener-operator/listenerclass/#presets) for details.
97124
98125
[possible values: none, stable-nodes, ephemeral-nodes]

‎docs/modules/stackablectl/partials/commands/stack.adoc‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -65,6 +65,31 @@ File options:
6565
Use "stackablectl [OPTIONS] <COMMAND> -r path/to/releases1.yaml -r path/to/releases2.yaml"
6666
to provide multiple additional release files.
6767
68+
-f, --operator-values <VALUES_FILE>
69+
Path to a Helm values file that will be used for the installation of operators
70+
71+
The file is a YAML file containing Helm values used to deploy operators.
72+
Operator-specific keys (e.g. 'airflow-operator', 'zookeeper-operator') map
73+
to the Helm values for that operator. Use YAML anchors and aliases to share
74+
values across operators.
75+
76+
Example values file:
77+
78+
airflow-operator:
79+
tolerations: &default-tolerations
80+
- key: "example"
81+
operator: "Exists"
82+
effect: "NoSchedule"
83+
podAnnotations:
84+
example.com/team: "data-engineering"
85+
zookeeper-operator:
86+
tolerations: *default-tolerations
87+
podAnnotations:
88+
example.com/team: "platform"
89+
90+
Use "stackablectl [OPTIONS] <COMMAND> -f path/to/values.yaml" to provide a
91+
values file.
92+
6893
Helm repository options:
6994
--helm-repo-stable <URL>
7095
Provide a custom Helm stable repository URL
@@ -94,6 +119,8 @@ Operator specific configurations:
94119
--listener-class-preset <LISTENER_CLASS_PRESET>
95120
Choose the ListenerClass preset (`none`, `ephemeral-nodes` or `stable-nodes`).
96121
122+
This takes precedence over values provided via the '-f/--operator-values' flag.
123+
97124
This maps to the listener-operator Helm Chart preset value, see [the listener-operator documentation](https://docs.stackable.tech/home/nightly/listener-operator/listenerclass/#presets) for details.
98125
99126
[possible values: none, stable-nodes, ephemeral-nodes]

‎docs/modules/stackablectl/partials/commands/stacklet.adoc‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -66,6 +66,31 @@ File options:
6666
Use "stackablectl [OPTIONS] <COMMAND> -r path/to/releases1.yaml -r path/to/releases2.yaml"
6767
to provide multiple additional release files.
6868
69+
-f, --operator-values <VALUES_FILE>
70+
Path to a Helm values file that will be used for the installation of operators
71+
72+
The file is a YAML file containing Helm values used to deploy operators.
73+
Operator-specific keys (e.g. 'airflow-operator', 'zookeeper-operator') map
74+
to the Helm values for that operator. Use YAML anchors and aliases to share
75+
values across operators.
76+
77+
Example values file:
78+
79+
airflow-operator:
80+
tolerations: &default-tolerations
81+
- key: "example"
82+
operator: "Exists"
83+
effect: "NoSchedule"
84+
podAnnotations:
85+
example.com/team: "data-engineering"
86+
zookeeper-operator:
87+
tolerations: *default-tolerations
88+
podAnnotations:
89+
example.com/team: "platform"
90+
91+
Use "stackablectl [OPTIONS] <COMMAND> -f path/to/values.yaml" to provide a
92+
values file.
93+
6994
Helm repository options:
7095
--helm-repo-stable <URL>
7196
Provide a custom Helm stable repository URL
@@ -95,6 +120,8 @@ Operator specific configurations:
95120
--listener-class-preset <LISTENER_CLASS_PRESET>
96121
Choose the ListenerClass preset (`none`, `ephemeral-nodes` or `stable-nodes`).
97122
123+
This takes precedence over values provided via the '-f/--operator-values' flag.
124+
98125
This maps to the listener-operator Helm Chart preset value, see [the listener-operator documentation](https://docs.stackable.tech/home/nightly/listener-operator/listenerclass/#presets) for details.
99126
100127
[possible values: none, stable-nodes, ephemeral-nodes]

0 commit comments

Comments
 (0)