Pod is not scheduled with istio ambient mode

[Patafix]

Environmental Info:
K3s Version:
k3s version v1.31.2+k3s1 (6da2042)
go version go1.22.8

Node(s) CPU architecture, OS, and Version:
17-Ubuntu SMP Mon Sep 2 13:48:07 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Ubuntu 24.04.1 LTS

Cluster Configuration:
Single node

Describe the bug:
After installing istio Ambient mode with specific instruction for k3s, all new pod are stuck in creating state.

Steps To Reproduce:

• Installed K3s
• Installed istio chart Ambient mode

Additional context / logs:

plugin type="istio-cni" name="istio-cni" failed (add): failed to find plugin "istio-cni" in path [/var/lib/rancher/k3s/data/cni]

Workaround:
ln -s /var/lib/rancher/k3s/data/current/bin/istio-cni /var/lib/rancher/k3s/data/cni/istio-cni

After this command everything work again.

[brandond]

This sounds like an issue with Istio's k3s-specific install instructions? I'm not sure what we need to fix here.

[brandond]

See also: https://docs.k3s.io/networking/multus-ipams

K3s looks at $DATA_DIR/data/cni for CNI plugin binaries. By default this is /var/lib/rancher/k3s/data/cni. Additional CNI plugins should be installed to this location.

Prior to the October 2024 releases, CNI binaries were part of the K3s userspace bundle at $DATA_DIR/data/$HASH/bin, where the hash is unique to each release of K3s. This made it difficult to deploy additional CNI plugins, as the path would change every time K3s was upgraded. If deploying Multus to an older release of K3s, you should use /var/lib/rancher/k3s/data/current/bin/ as the CNI bin dir, but expect that the plugins will need to be re-deployed whenever K3s is upgraded.

[dereknola]

Yeah considering istio builds specific install functionality around K3s "platform" they need to fix their install location for their cni plugin
https://istio.io/latest/docs/ambient/install/platform-prerequisites/#k3s

[Patafix]

I guess this path are wrong right ?
https://github.com/istio/istio/blob/master/manifests/charts/istio-cni/files/profile-platform-k3s.yaml

This line :
cniBinDir: /var/lib/rancher/k3s/data/current/bin/

Must be: /var/lib/rancher/k3s/data/cni/

[brandond]

Yes. As noted above, the "current" bin dir changes every time K3s is upgraded. That is no longer the correct place to install CNI binaries.

[Patafix]

For someone using the ambient helm chart, looking for a solution :

--set cni.cniConfDir=/var/lib/rancher/k3s/agent/etc/cni/net.d --set cni.cniBinDir=/var/lib/rancher/k3s/data/cni/

It's work