Skip to content

[BUG]: Empty Topology= causes Worker CPU and memory registration failure #20

Description

@crliu3227

Version

1.2.1

Which installation method(s) does this occur on?

Helm / Kubernetes manifests

Describe the bug.

Description

When a NodeSet does not configure a Slurm topology, the operator still injects an empty Topology= entry into slurmd --conf.

The container image entrypoint then appends CoreSpecCount and MemSpecLimit. Slurm parses CoreSpecCount as the value of Topology, causing node registration to fail and the Worker to enter DRAIN+INVALID_REG.

Reproduction

Use a NodeSet with CPU and memory limits, without a topology configuration:

apiVersion: slinky.slurm.net/v1beta1
kind: NodeSet
spec:
  slurmd:
    resources:
      limits:
        cpu: "8"
        memory: 16Gi

Do not configure spec.extraConf: Topology=..., and do not provide a valid topology for the Kubernetes node.
Actual behavior
The resource environment variables are correct:

POD_CPUS=8
POD_MEMORY=16384

However, the final slurmd invocation contains:

--conf Features=slinky Topology= CoreSpecCount=36 MemSpecLimit=175607

The controller reports:

Reason=Failed to set topology 'CoreSpecCount=36'
State=IDLE+DRAIN+DYNAMIC_NORM+INVALID_REG
CPUEfctv=80

CoreSpecCount is not applied, so Slurm incorrectly treats all host CPUs as schedulable.
Expected behavior
When topology is not configured, the operator must not generate Topology=.
The final configuration should be:

--conf Features=slinky CoreSpecCount=36 MemSpecLimit=175607

Minimum reproducible example

Relevant log output

Additional environment details

Other/Misc.

No response

Code of Conduct

  • I agree to follow Slurm Operator's Code of Conduct
  • I have searched the open bugs and have found no duplicates for this bug report

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions