netplugin failed panic runtime error when trying to add route-override plugin to multus

[davidjveer]

Hey there, running into this issue and was hoping you could help.

We're trying to use route-override to delete a route added by dhcp plugin via Multus/macvlan and receiving an error. Note that when the route override portion of the multus config is removed this works with just macvlan/dhcp.

Here is some environment info:

• containerd v1.7.11-k3s2
• Multus v4.0.2
• Ubuntu 20.04 LTS (fips) host OS.
  Release notes for release of k8s used

Here's the multus config we're trying to use.

{
    "cniVersion": "0.3.1",
    "name": "sw-lan",
    "type": "multus",
    "delegates": [
        {
            "type": "route-override",
            "name": "sw-lan",
            "delroutes": [
                {
                    "dst": "10.10.95.0/24"
                }
            ]
        },
        {
            "type": "macvlan",
            "name": "sw-lan",
            "mode": "bridge",
            "ipam": {
                "type": "dhcp"
            }
        }
    ]
}

Here's the error received on the pod trying to start:

error adding container to network "sw-lan": netplugin failed: "panic: runtime error: invalid memory address or nil pointer dereference
  [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x4e3900]
  
  goroutine 1 [running]:
  github.com/containernetworking/cni/pkg/types/100.(*Result).Version(0x7f062ffc0c40?)
    /usr/src/cni-route-override/vendor/github.com/containernetworking/cni/pkg/types/100/types.go:202
  github.com/containernetworking/cni/pkg/types/internal.Convert({0x55fa50, 0x0}, {0x5301e9, 0x5})
    /usr/src/cni-route-override/vendor/github.com/containernetworking/cni/pkg/types/internal/convert.go:60 +0x4f
  github.com/containernetworking/cni/pkg/types/100.NewResultFromResult(...)
    /usr/src/cni-route-override/vendor/github.com/containernetworking/cni/pkg/types/100/types.go:77
  main.processRoutes({0xc00001c05a, 0x37}, 0xc000162000)
    /usr/src/cni-route-override/cmd/route-override/route-override.go:225 +0x14d
  main.cmdAdd(0xc00013c310)
    /usr/src/cni-route-override/cmd/route-override/route-override.go:304 +0x66
  github.com/containernetworking/cni/pkg/skel.(*dispatcher).checkVersionAndCall(0xc000149ee8, 0xc00013c310, {0x55f418, 0xc00011a9f0}, 0x53b230)
    /usr/src/cni-route-override/vendor/github.com/containernetworking/cni/pkg/skel/skel.go:166 +0x20a
  github.com/containernetworking/cni/pkg/skel.(*dispatcher).pluginMain(0xc000149ee8, 0xa?, 0x41a480a575894b61?, 0x1885983171979e?, {0x55f418, 0xc00011a9f0}, {0x530085, 0x4})
    /usr/src/cni-route-override/vendor/github.com/containernetworking/cni/pkg/skel/skel.go:219 +0x2ca
  github.com/containernetworking/cni/pkg/skel.PluginMainWithError(...)
    /usr/src/cni-route-override/vendor/github.com/containernetworking/cni/pkg/skel/skel.go:273
  github.com/containernetworking/cni/pkg/skel.PluginMain(0x406d19?, 0x404cf1?, 0x401140?, {0x55f418?, 0xc00011a9f0?}, {0x530085?, 0xc0000061a0?})
    /usr/src/cni-route-override/vendor/github.com/containernetworking/cni/pkg/skel/skel.go:288 +0xd1
  main.main()
    /usr/src/cni-route-override/cmd/route-override/route-override.go:410 +0x49

Let me know if you can help at all, or need any other information.

Otherwise, if you know of a better way to remove this route for 10.10.95.0/24 -- that would allow us to work around this issue. Currently we're using an initContainer to delete the route:

Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
0.0.0.0         169.254.1.1     0.0.0.0         UG        0 0          0 eth0
10.10.95.0      0.0.0.0         255.255.255.0   U         0 0          0 sw0
169.254.1.1     0.0.0.0         255.255.255.255 UH        0 0          0 eth0

Thank you!

[AWildBeard]

I've run into the same issue. Out of curiosity, did you identify an alternative?

[davidjveer]

@AWildBeard The best solution we could come up with was creating an initContainer with alpine/another small container image to remove the routes on pod start. Seems to work pretty well, but this was the only alternative I could come up with presently.