Static route funkiness when adding the same route via the same next-hop but with different vlans

[taspelund]

Running mgadm add-v4-route against the same prefix multiple times SEEMS to result in a no-op when using an existing next-hop but with different vlan settings (either changing the VID or removing the VID):

treyaspelund@Tallon-IV 12:04:33 PM | ~/git/maghemite  trey/ipv6_static_routing ✓ ▶
‣ ./target/debug/mgadm static add-v4-route 100.64.10.0/22 4.4.4.4 --vlan-id 4            <<<<<<<<<<  add path via VID Some(4)

treyaspelund@Tallon-IV 12:04:36 PM | ~/git/maghemite  trey/ipv6_static_routing ✓ ▶
‣ ./target/debug/mgadm static get-v4-routes
{
    "100.64.10.0/22": [
        Path {
            bgp: None,
            nexthop: 4.4.4.4,
            rib_priority: 1,
            shutdown: false,
            vlan_id: Some(
                4,
            ),
        },
    ],
}

treyaspelund@Tallon-IV 12:04:38 PM | ~/git/maghemite  trey/ipv6_static_routing ✓ ▶
‣ ./target/debug/mgadm static add-v4-route 100.64.10.0/22 4.4.4.4 --vlan-id 5               <<<<<<<<<  new VID Some(5)

treyaspelund@Tallon-IV 12:04:40 PM | ~/git/maghemite  trey/ipv6_static_routing ✓ ▶
‣ ./target/debug/mgadm static get-v4-routes
{
    "100.64.10.0/22": [
        Path {
            bgp: None,
            nexthop: 4.4.4.4,
            rib_priority: 1,
            shutdown: false,
            vlan_id: Some(
                4,          <<<<<<<<<<  old VID 4
            ),
        },
    ],
}

treyaspelund@Tallon-IV 12:04:41 PM | ~/git/maghemite  trey/ipv6_static_routing ✓ ▶
‣ ./target/debug/mgadm static add-v4-route 100.64.10.0/22 4.4.4.4                   <<<<<<<<<<  new VID None

treyaspelund@Tallon-IV 12:06:11 PM | ~/git/maghemite  trey/ipv6_static_routing ✓ ▶
‣ ./target/debug/mgadm static get-v4-routes
{
    "100.64.10.0/22": [
        Path {
            bgp: None,
            nexthop: 4.4.4.4,
            rib_priority: 1,
            shutdown: false,
            vlan_id: Some(
                4,
            ),
        },
    ],
}

Attempting to delete the VID None path shows no difference in output:

treyaspelund@Tallon-IV 12:06:13 PM | ~/git/maghemite  trey/ipv6_static_routing ✓ ▶
‣ ./target/debug/mgadm static remove-v4-route 100.64.10.0/22 4.4.4.4

treyaspelund@Tallon-IV 12:11:59 PM | ~/git/maghemite  trey/ipv6_static_routing ✓ ▶
‣ ./target/debug/mgadm static get-v4-routes
{
    "100.64.10.0/22": [
        Path {
            bgp: None,
            nexthop: 4.4.4.4,
            rib_priority: 1,
            shutdown: false,
            vlan_id: Some(
                4,
            ),
        },
    ],
}

However, deleting the VID 4 path reveals the VID 5 path is still present:

treyaspelund@Tallon-IV 12:12:00 PM | ~/git/maghemite  trey/ipv6_static_routing ✓ ▶
‣ ./target/debug/mgadm static remove-v4-route 100.64.10.0/22 4.4.4.4 --vlan-id 4

treyaspelund@Tallon-IV 12:12:04 PM | ~/git/maghemite  trey/ipv6_static_routing ✓ ▶
‣ ./target/debug/mgadm static get-v4-routes
{
    "100.64.10.0/22": [
        Path {
            bgp: None,
            nexthop: 4.4.4.4,
            rib_priority: 1,
            shutdown: false,
            vlan_id: Some(
                5,
            ),
        },
    ],
}

Interestingly, there are no static routes shown in the imported table (loc_rib) after this:

treyaspelund@Tallon-IV 12:12:21 PM | ~/git/maghemite  trey/ipv6_static_routing ✓ ▶
‣ ./target/debug/mgadm bgp status imported
failed to parse prefix [::1/128]: malformed ip addr           <<<<<<  ignore this, this is from my working branch

treyaspelund@Tallon-IV 12:12:21 PM | ~/git/maghemite  trey/ipv6_static_routing ✓ ▶
‣

Another example from main:

treyaspelund@Tallon-IV 12:21:22 PM | ~/git/maghemite  main
‣ ./target/debug/mgadm static get-v4-routes
{}

treyaspelund@Tallon-IV 12:21:24 PM | ~/git/maghemite  main
‣ ./target/debug/mgadm static add-v4-route 100.64.10.0/22 4.4.4.4

treyaspelund@Tallon-IV 12:21:41 PM | ~/git/maghemite  main
‣ ./target/debug/mgadm static add-v4-route 100.64.10.0/22 4.4.4.4 --vlan-id 4

treyaspelund@Tallon-IV 12:21:47 PM | ~/git/maghemite  main
‣ ./target/debug/mgadm static add-v4-route 100.64.10.0/22 4.4.4.4 --vlan-id 5

treyaspelund@Tallon-IV 12:21:48 PM | ~/git/maghemite  main
‣ ./target/debug/mgadm static get-v4-routes
{
    "100.64.10.0/22": [
        Path {
            bgp: None,
            nexthop: 4.4.4.4,
            rib_priority: 1,
            shutdown: false,
            vlan_id: Some(
                4,
            ),
        },
    ],
}

treyaspelund@Tallon-IV 12:21:50 PM | ~/git/maghemite  main
‣ ./target/debug/mgadm static remove-v4-route 100.64.10.0/22 4.4.4.4 --vlan-id 5

treyaspelund@Tallon-IV 12:21:53 PM | ~/git/maghemite  main
‣ ./target/debug/mgadm static get-v4-routes
{
    "100.64.10.0/22": [
        Path {
            bgp: None,
            nexthop: 4.4.4.4,
            rib_priority: 1,
            shutdown: false,
            vlan_id: Some(
                4,
            ),
        },
    ],
}

treyaspelund@Tallon-IV 12:21:54 PM | ~/git/maghemite  main
‣ ./target/debug/mgadm static remove-v4-route 100.64.10.0/22 4.4.4.4 --vlan-id 4

treyaspelund@Tallon-IV 12:22:06 PM | ~/git/maghemite  main
‣ ./target/debug/mgadm static get-v4-routes
{
    "100.64.10.0/22": [
        Path {
            bgp: None,
            nexthop: 4.4.4.4,
            rib_priority: 1,
            shutdown: false,
            vlan_id: None,
        },
    ],
}

It seems like the add/remove here is not always consistent, or at least not in the way you'd expect.
I think we should make an explicit decision around how we want this to be handled.
i.e.
Do we want to allow the same next-hop to be used with different Option settings?

Regardless of which direction we go, it seems like this could use some polishing to make sure it consistently behaves inline with our choice.