Skip to content

Commit 6ee51c0

Browse files
committed
feat(topology): support variable fabric and accelerated tiers
Signed-off-by: Dmitry Shmulevich <dshmulevich@nvidia.com>
1 parent 241235a commit 6ee51c0

45 files changed

Lines changed: 473 additions & 489 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.claude/CLAUDE.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -55,8 +55,8 @@ These structures propagate across every provider and engine. Changing them in a
5555
| Surface | Why it's load-bearing |
5656
|---|---|
5757
| `pkg/topology/``Graph`, the `Vertex` tree, and topology constants | Every provider returns it; every engine consumes it. A shape change ripples to all of them. |
58-
| Helm `provider.name` / `engine.name` / `topologyNodeLabels` | External contract for operators deploying Topograph. |
59-
| The variable label families `network.topology.nvidia.com/level-N` and `accelerated.topology.nvidia.com/level-N` | Consumed by downstream projects (KAI Scheduler, NVSentinel, Kueue); level 0 is closest to the node. |
58+
| Helm `provider.name` / `engine.name` | External contract for operators deploying Topograph. |
59+
| The variable label families `network.topology.nvidia.com/tier-N` and `accelerated.topology.nvidia.com/domain-N` | Consumed by downstream projects (KAI Scheduler, NVSentinel, Kueue); fabric tier 0 is closest to the node and accelerated domain 0 is narrowest. |
6060

6161
## 2. Setup and Installation
6262

@@ -169,7 +169,7 @@ Engines are much rarer (four exist: slurm, k8s, nfd, slinky). Follow the same re
169169

170170
### Label and annotation reference
171171

172-
Label keys written by the Kubernetes and Slinky engines are documented in `docs/reference/node-labels.md`. Do not invent new keys in provider or engine code — values flow through the canonical graph; keys are configured via Helm `topologyNodeLabels`.
172+
Label keys written by the Kubernetes engine are documented in `docs/reference/node-labels.md`. Do not invent new keys in provider code — values flow through the canonical graph. Optional custom keys are configured independently through the k8s engine's closest-first `fabricLabels` and narrowest-first `acceleratedLabels` arrays; when an array is provided, only explicitly listed entries in that family are labeled.
173173

174174
## 5. Pull Request Guidelines
175175

AGENTS.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -55,8 +55,8 @@ These structures propagate across every provider and engine. Changing them in a
5555
| Surface | Why it's load-bearing |
5656
|---|---|
5757
| `pkg/topology/``Graph`, the `Vertex` tree, and topology constants | Every provider returns it; every engine consumes it. A shape change ripples to all of them. |
58-
| Helm `provider.name` / `engine.name` / `topologyNodeLabels` | External contract for operators deploying Topograph. |
59-
| The variable label families `network.topology.nvidia.com/level-N` and `accelerated.topology.nvidia.com/level-N` | Consumed by downstream projects (KAI Scheduler, NVSentinel, Kueue); level 0 is closest to the node. |
58+
| Helm `provider.name` / `engine.name` | External contract for operators deploying Topograph. |
59+
| The variable label families `network.topology.nvidia.com/tier-N` and `accelerated.topology.nvidia.com/domain-N` | Consumed by downstream projects (KAI Scheduler, NVSentinel, Kueue); fabric tier 0 is closest to the node and accelerated domain 0 is narrowest. |
6060

6161
## 2. Setup and Installation
6262

@@ -169,7 +169,7 @@ Engines are much rarer (four exist: slurm, k8s, nfd, slinky). Follow the same re
169169

170170
### Label and annotation reference
171171

172-
Label keys written by the Kubernetes and Slinky engines are documented in `docs/reference/node-labels.md`. Do not invent new keys in provider or engine code — values flow through the canonical graph; keys are configured via Helm `topologyNodeLabels`.
172+
Label keys written by the Kubernetes engine are documented in `docs/reference/node-labels.md`. Do not invent new keys in provider code — values flow through the canonical graph. Optional custom keys are configured independently through the k8s engine's closest-first `fabricLabels` and narrowest-first `acceleratedLabels` arrays; when an array is provided, only explicitly listed entries in that family are labeled.
173173

174174
## 5. Pull Request Guidelines
175175

CHANGELOG.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
1919

2020
### Changed
2121

22-
- **BREAKING:** Topology now uses variable, closest-first level families: `network.topology.nvidia.com/level-N` for fabric tiers and `accelerated.topology.nvidia.com/level-N` for accelerated tiers. The canonical `InstanceTopology` and graph conversion support arbitrary depths; the fixed `accelerator`, `leaf`, `spine`, and `core` label keys and their Helm overrides are replaced by `topologyNodeLabels.fabricPrefix` and `topologyNodeLabels.acceleratedPrefix`.
22+
- **BREAKING:** Topology now uses variable, closest-first fabric tiers and narrowest-first accelerated domains: `network.topology.nvidia.com/tier-N` and `accelerated.topology.nvidia.com/domain-N`. The canonical `InstanceTopology` and graph conversion support arbitrary depths. The fixed `accelerator`, `leaf`, `spine`, and `core` keys and process-wide Helm/CLI overrides are replaced by optional `fabricLabels` and `acceleratedLabels` arrays on the `k8s` engine. Each omitted array defaults every discovered entry in its family; when provided, only explicitly configured entries in that family are labeled.
2323
- Simulation model node names are now treated as hostnames; the model-backed test provider generates their instance IDs with an `i-` prefix.
2424
- The node-observer and node-data-broker are now rendered directly by the main Topograph Helm chart instead of local subcharts. Their existing `node-observer.*` and `node-data-broker.*` values paths are unchanged.
2525
- **BREAKING (Helm chart `0.5.0``0.6.0`):** the chart now ships a hardened default security context across the API server, node-observer, and node-data-broker: non-root (`runAsNonRoot`, UID/GID `65532`), `seccompProfile: RuntimeDefault`, `allowPrivilegeEscalation: false`, `readOnlyRootFilesystem: true`, and all capabilities dropped — satisfying the Kubernetes `restricted` Pod Security Standard out of the box. This changes the default runtime posture of every workload; operators who relied on root, a writable rootfs, or added capabilities must override the relevant keys (see the migration note below). `appVersion` is unchanged (`v0.5.0`; no binary change).

charts/topograph/templates/NOTES.txt

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -48,9 +48,6 @@
4848
{{ .Values.nfdNamespace }}. This must be the namespace where NFD master runs.
4949
{{- end }}
5050

51-
Topology label families use closest-first numeric levels:
52-
fabric: {{ .Values.topologyNodeLabels.fabricPrefix }}N
53-
accelerated: {{ .Values.topologyNodeLabels.acceleratedPrefix }}N
5451
{{- if not .Values.serviceAccount.create }}
5552

5653
2. Topograph is configured to use the existing ServiceAccount "{{ include "topograph.serviceAccountName" . }}".

charts/topograph/templates/deployment.yaml

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -50,12 +50,6 @@ spec:
5050
- /usr/local/bin/topograph
5151
args:
5252
- -v={{ .Values.verbosity }}
53-
{{- with .Values.topologyNodeLabels.fabricPrefix }}
54-
- -k8s-topology-key-fabric-prefix={{ . }}
55-
{{- end }}
56-
{{- with .Values.topologyNodeLabels.acceleratedPrefix }}
57-
- -k8s-topology-key-accelerated-prefix={{ . }}
58-
{{- end }}
5953
{{- if or .Values.nodeDataBroker.enabled $hasGcpCredentials $isNFDEngine .Values.env }}
6054
env:
6155
{{- if .Values.nodeDataBroker.enabled }}

charts/topograph/tests/__snapshot__/render_snapshot_test.yaml.snap

Lines changed: 0 additions & 48 deletions
Original file line numberDiff line numberDiff line change
@@ -8,10 +8,6 @@ renders default values.yaml:
88
kubectl --namespace topograph port-forward $POD_NAME 8080:$CONTAINER_PORT
99

1010
NOTE: node-data-broker applies node annotations once when each broker pod starts.
11-
12-
Topology label families use closest-first numeric levels:
13-
fabric: network.topology.nvidia.com/level-N
14-
accelerated: accelerated.topology.nvidia.com/level-N
1511
2: |
1612
apiVersion: v1
1713
data:
@@ -63,8 +59,6 @@ renders default values.yaml:
6359
containers:
6460
- args:
6561
- -v=3
66-
- -k8s-topology-key-fabric-prefix=network.topology.nvidia.com/level-
67-
- -k8s-topology-key-accelerated-prefix=accelerated.topology.nvidia.com/level-
6862
command:
6963
- /usr/local/bin/topograph
7064
env:
@@ -557,10 +551,6 @@ renders values.k8s.gateway-api-example.yaml:
557551
it and (if needed) published its listener address in DNS.
558552

559553
NOTE: node-data-broker applies node annotations once when each broker pod starts.
560-
561-
Topology label families use closest-first numeric levels:
562-
fabric: network.topology.nvidia.com/level-N
563-
accelerated: accelerated.topology.nvidia.com/level-N
564554
2: |
565555
apiVersion: v1
566556
data:
@@ -612,8 +602,6 @@ renders values.k8s.gateway-api-example.yaml:
612602
containers:
613603
- args:
614604
- -v=3
615-
- -k8s-topology-key-fabric-prefix=network.topology.nvidia.com/level-
616-
- -k8s-topology-key-accelerated-prefix=accelerated.topology.nvidia.com/level-
617605
command:
618606
- /usr/local/bin/topograph
619607
env:
@@ -1128,10 +1116,6 @@ renders values.k8s.gcp-federated-workload-identity-example.yaml:
11281116
kubectl --namespace topograph port-forward $POD_NAME 8080:$CONTAINER_PORT
11291117

11301118
NOTE: node-data-broker applies node annotations once when each broker pod starts.
1131-
1132-
Topology label families use closest-first numeric levels:
1133-
fabric: network.topology.nvidia.com/level-N
1134-
accelerated: accelerated.topology.nvidia.com/level-N
11351119
2: |
11361120
apiVersion: v1
11371121
data:
@@ -1183,8 +1167,6 @@ renders values.k8s.gcp-federated-workload-identity-example.yaml:
11831167
containers:
11841168
- args:
11851169
- -v=3
1186-
- -k8s-topology-key-fabric-prefix=network.topology.nvidia.com/level-
1187-
- -k8s-topology-key-accelerated-prefix=accelerated.topology.nvidia.com/level-
11881170
command:
11891171
- /usr/local/bin/topograph
11901172
env:
@@ -1709,10 +1691,6 @@ renders values.k8s.gcp-service-account-example.yaml:
17091691
kubectl --namespace topograph port-forward $POD_NAME 8080:$CONTAINER_PORT
17101692

17111693
NOTE: node-data-broker applies node annotations once when each broker pod starts.
1712-
1713-
Topology label families use closest-first numeric levels:
1714-
fabric: network.topology.nvidia.com/level-N
1715-
accelerated: accelerated.topology.nvidia.com/level-N
17161694
2: |
17171695
apiVersion: v1
17181696
data:
@@ -1764,8 +1742,6 @@ renders values.k8s.gcp-service-account-example.yaml:
17641742
containers:
17651743
- args:
17661744
- -v=3
1767-
- -k8s-topology-key-fabric-prefix=network.topology.nvidia.com/level-
1768-
- -k8s-topology-key-accelerated-prefix=accelerated.topology.nvidia.com/level-
17691745
command:
17701746
- /usr/local/bin/topograph
17711747
env:
@@ -2278,10 +2254,6 @@ renders values.k8s.ib-example.yaml:
22782254
kubectl --namespace topograph port-forward $POD_NAME 8080:$CONTAINER_PORT
22792255

22802256
NOTE: node-data-broker applies node annotations once when each broker pod starts.
2281-
2282-
Topology label families use closest-first numeric levels:
2283-
fabric: network.topology.nvidia.com/level-N
2284-
accelerated: accelerated.topology.nvidia.com/level-N
22852257
2: |
22862258
apiVersion: v1
22872259
data:
@@ -2333,8 +2305,6 @@ renders values.k8s.ib-example.yaml:
23332305
containers:
23342306
- args:
23352307
- -v=3
2336-
- -k8s-topology-key-fabric-prefix=network.topology.nvidia.com/level-
2337-
- -k8s-topology-key-accelerated-prefix=accelerated.topology.nvidia.com/level-
23382308
command:
23392309
- /usr/local/bin/topograph
23402310
env:
@@ -2855,10 +2825,6 @@ renders values.slinky.block-example.yaml:
28552825
kubectl --namespace topograph port-forward $POD_NAME 8080:$CONTAINER_PORT
28562826

28572827
NOTE: node-data-broker applies node annotations once when each broker pod starts.
2858-
2859-
Topology label families use closest-first numeric levels:
2860-
fabric: network.topology.nvidia.com/level-N
2861-
accelerated: accelerated.topology.nvidia.com/level-N
28622828
2: |
28632829
apiVersion: v1
28642830
data:
@@ -2910,8 +2876,6 @@ renders values.slinky.block-example.yaml:
29102876
containers:
29112877
- args:
29122878
- -v=3
2913-
- -k8s-topology-key-fabric-prefix=network.topology.nvidia.com/level-
2914-
- -k8s-topology-key-accelerated-prefix=accelerated.topology.nvidia.com/level-
29152879
command:
29162880
- /usr/local/bin/topograph
29172881
env:
@@ -3436,10 +3400,6 @@ renders values.slinky.partition-example.yaml:
34363400
kubectl --namespace topograph port-forward $POD_NAME 8080:$CONTAINER_PORT
34373401

34383402
NOTE: node-data-broker applies node annotations once when each broker pod starts.
3439-
3440-
Topology label families use closest-first numeric levels:
3441-
fabric: network.topology.nvidia.com/level-N
3442-
accelerated: accelerated.topology.nvidia.com/level-N
34433403
2: |
34443404
apiVersion: v1
34453405
data:
@@ -3491,8 +3451,6 @@ renders values.slinky.partition-example.yaml:
34913451
containers:
34923452
- args:
34933453
- -v=3
3494-
- -k8s-topology-key-fabric-prefix=network.topology.nvidia.com/level-
3495-
- -k8s-topology-key-accelerated-prefix=accelerated.topology.nvidia.com/level-
34963454
command:
34973455
- /usr/local/bin/topograph
34983456
env:
@@ -4034,10 +3992,6 @@ renders values.slinky.tree-example.yaml:
40343992
kubectl --namespace topograph port-forward $POD_NAME 8080:$CONTAINER_PORT
40353993

40363994
NOTE: node-data-broker applies node annotations once when each broker pod starts.
4037-
4038-
Topology label families use closest-first numeric levels:
4039-
fabric: network.topology.nvidia.com/level-N
4040-
accelerated: accelerated.topology.nvidia.com/level-N
40413995
2: |
40423996
apiVersion: v1
40433997
data:
@@ -4089,8 +4043,6 @@ renders values.slinky.tree-example.yaml:
40894043
containers:
40904044
- args:
40914045
- -v=3
4092-
- -k8s-topology-key-fabric-prefix=network.topology.nvidia.com/level-
4093-
- -k8s-topology-key-accelerated-prefix=accelerated.topology.nvidia.com/level-
40944046
command:
40954047
- /usr/local/bin/topograph
40964048
env:

charts/topograph/tests/deployment_test.yaml

Lines changed: 1 addition & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -272,25 +272,14 @@ tests:
272272
name: NODE_DATA_BROKER_NAMESPACE
273273
template: templates/deployment.yaml
274274

275-
- it: passes the verbosity flag and topology key args
275+
- it: passes the verbosity flag
276276
set:
277277
verbosity: 5
278-
topologyNodeLabels:
279-
fabricPrefix: example.com/fabric-
280-
acceleratedPrefix: example.com/accelerated-
281278
asserts:
282279
- contains:
283280
path: spec.template.spec.containers[0].args
284281
content: -v=5
285282
template: templates/deployment.yaml
286-
- contains:
287-
path: spec.template.spec.containers[0].args
288-
content: -k8s-topology-key-fabric-prefix=example.com/fabric-
289-
template: templates/deployment.yaml
290-
- contains:
291-
path: spec.template.spec.containers[0].args
292-
content: -k8s-topology-key-accelerated-prefix=example.com/accelerated-
293-
template: templates/deployment.yaml
294283

295284
- it: renders user-supplied environment variables, initContainers, and lifecycle
296285
set:

charts/topograph/values.schema.json

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@
5050
},
5151
"params": {
5252
"type": "object",
53-
"description": "Engine-specific parameters. For slinky, useGpuCliqueLabel=true reads nvidia.com/gpu.clique as the topology/block domain source."
53+
"description": "Engine-specific parameters. The k8s engine accepts closest-first fabricLabels and narrowest-first acceleratedLabels arrays. Each omitted array defaults every discovered entry in its family; when provided, additional entries in that family are omitted. For slinky, useGpuCliqueLabel=true reads nvidia.com/gpu.clique as the topology/block domain source."
5454
}
5555
},
5656
"required": ["name"]
@@ -119,7 +119,6 @@
119119
"initContainers": { "type": "array" },
120120
"lifecycle": { "type": "object" },
121121
"config": { "type": "object" },
122-
"topologyNodeLabels": { "type": "object" },
123122
"podAnnotations": { "type": "object" },
124123
"podLabels": { "type": "object" },
125124
"podSecurityContext": { "type": "object" },

charts/topograph/values.yaml

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -67,11 +67,6 @@ config:
6767
# Optional secret with API credentials
6868
# credentialsSecret:
6969

70-
topologyNodeLabels:
71-
# Level numbers are appended to these prefixes. Level 0 is closest to a node.
72-
fabricPrefix: network.topology.nvidia.com/level-
73-
acceleratedPrefix: accelerated.topology.nvidia.com/level-
74-
7570
podAnnotations: {}
7671
podLabels: {}
7772

cmd/topograph/main.go

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -28,17 +28,13 @@ import (
2828

2929
"github.com/NVIDIA/topograph/internal/version"
3030
"github.com/NVIDIA/topograph/pkg/config"
31-
"github.com/NVIDIA/topograph/pkg/engines/k8s"
3231
"github.com/NVIDIA/topograph/pkg/server"
3332
)
3433

3534
func main() {
3635
var cfg string
37-
var fabricLabelPrefix, acceleratedLabelPrefix string
3836
var ver bool
3937
flag.StringVar(&cfg, "c", "/etc/topograph/topograph-config.yaml", "config file")
40-
flag.StringVar(&fabricLabelPrefix, "k8s-topology-key-fabric-prefix", k8s.DefaultFabricLabelPrefix, "K8s node-label prefix for fabric levels")
41-
flag.StringVar(&acceleratedLabelPrefix, "k8s-topology-key-accelerated-prefix", k8s.DefaultAcceleratedLabelPrefix, "K8s node-label prefix for accelerated-network levels")
4238
flag.BoolVar(&ver, "version", false, "show the version")
4339

4440
klog.InitFlags(nil)
@@ -50,8 +46,6 @@ func main() {
5046
os.Exit(0)
5147
}
5248

53-
k8s.InitLabels(fabricLabelPrefix, acceleratedLabelPrefix)
54-
5549
if err := mainInternal(cfg); err != nil {
5650
klog.Error(err.Error())
5751
os.Exit(1)

0 commit comments

Comments
 (0)