Feature Request - Device Discovery: Add Site tag support to VLAN section

[Kramdin-thinklogistics]

Currently, it seems all VLAN data collected by NAPALM (at least for IOS) is ingested into Netbox without the site metadata attached (even if site is defined in the default section). I'm not sure if it's supposed to work that way, or if the data model doesn't yet support this.

It makes a lot of work having to cleanup the data by scripting/manually assigning each imported VLAN to its respective site

[leoparente]

Hi @Kramdin-thinklogistics it is actually a missing property in the discovery implementation, so we will address it in the agent side.

[leoparente]

hi @Kramdin-thinklogistics. We have added this feature. Can you try to validate it using latest docker develop image?

docker pull netboxlabs/orb-agent:develop and then run agent using this image.

[Kramdin-thinklogistics]

ok I pulled the develop image and ran a scan.

I'm assuming it is necessary to add the site tag under vlan default section as follows?:

nelric_access:
config:
schedule: "30 2 * * *"
defaults:
site: nelric
role: Access Router
vlan:
site: nelric

With it defined, I get the following error:
ERROR:device_discovery.policy.runner:Policy nelric_access, Hostname xxxx: 'NoneType' object is not iterable

With it not defined, the error goes away, but the VLANs are still imported into Netbox with site set to null (can it not inherit the site default at the config level?):

vid: 1005
name: "trnet-default"
role: null
site: null
tags: []
group: null
status: "active"
tenant: null
comments: ""
qinq_role: null
qinq_svlan: null
description: ""
custom_fields: {}

[leoparente]

hi @Kramdin-thinklogistics. It supposed to inherit from site default at config level. Can you share the version for device discovery in orb agent logs?
there will be something like this in your logs

{"time":"2025-05-21T14:00:41.046997742Z","level":"INFO","msg":"device-discovery readiness ok, got version ","device_discovery_version":"1.1.1"}

[Kramdin-thinklogistics]

Well, that may explain it, I have this entry which says 1.0.0:
{"time":"2025-05-21T16:49:58.157921275Z","level":"INFO","msg":"device-discovery readiness ok, got version ","device_discovery_version":"1.0.0"}

But I did recreate the container using the devlop image...strange

[Kramdin-thinklogistics]

I found a mistake in my script - I pulled the devlop image, but my container creation still referenced the latest image. I'll fix it and let you know

[Kramdin-thinklogistics]

Ok, so I'm using the correct image now (got version 1.1.1) and it appears to have updated the site in Netbox which is great.

The only problem I see now is that we run multiple policies to properly ingest different sites & device roles, and it appears that the agent is only using the site def in the last policy it encounters and assigning things there (devices in each of the policies are being placed in the right sites, so I don't think it's a policy def issue); screenshot of one of the offending change logs from Netbox :

[leoparente]

Hi @Kramdin-thinklogistics here we have a problem as multiple devices have the same vid and name for a VLAN and netbox does not ensure uniqueness in that case. What would you expect to happen in you use case regarding VLANs?

[Kramdin-thinklogistics]

hi @leoparente - Yes, I agree that is currently a limitation in Netbox since in reality multiple devices in the same site can have the same vlan defined, but that doesn't mean they are logically/physically connected and therefore one in the same. Until that is adjusted, as a partial mitigation, when you pull data from a device, can all relevant device-specific info be applied to sub-artifacts (like vlan, interface, address, etc)?

So, in the case of vlan, partial mitigation would look like:

• Device A
  • Site A
  • Vlan 1 -> inherit Site A
  • Vlan 2 -> inherit Site A

• Device B
  • Site A
  • Vlan 1 -> inherit Site A (unfortunately, overwritten as it can't be differentiated from vlans in Device A)
  • Vlan 2 -> inherit Site A (unfortunately, overwritten as it can't be differentiated from vlans in Device A)

• Device C
  • Site B
  • Vlan 1 -> inherit Site B (differentiated by device site membership)
  • Vlan 2 -> inherit Site B (differentiated by device site membership)

I think you may have intended it to work this way; it just seems like maybe an implementation error, where only Site B from the targets yaml is being attributed in the above example?

[leoparente]

Hi @Kramdin-thinklogistics thanks for the explanation. It is not a simply implementation error as Site does not help to determine uniquiness in VLANs. Diode tries to figure out if you are updating a existing Entity or creating a new one. As VLAN site is like a MTU for A interface (a field that could be updated) . When there is the same VID and Name, Diode simply update the site; For instance the following code you would expect to perform what you said:

from netboxlabs.diode.sdk import DiodeClient
from netboxlabs.diode.sdk.ingester import (
    VLAN,
    Entity,
)


def main():
    with DiodeClient(
        target="grpc://localhost:8080/diode",
        app_name="my-test-app",
        app_version="0.0.1",
        client_id="diode-ingest",
        client_secret="<my_secret>",
    ) as client:
        entities = []

        vlan1 = VLAN(
            name="default",
            vid=1,
            site="siteA",
        )

        entities.append(Entity(vlan=vlan1))
        
        vlan2 = VLAN(
            name="default",
            vid=1,
            site="siteB",
        )

        entities.append(Entity(vlan=vlan2))

        response = client.ingest(entities=entities)
        if response.errors:
            print(f"Errors: {response.errors}")


if __name__ == "__main__":
    main()

However, In the first moment Diode will create the entity because it does not exist, and then update it.

[leoparente]

That said, in your opinion, do you think that the partial mitigation of having VLANs associated properly with a site would be useful? Because if that is the case, we can discuss it internally the best approach for Diode to handle this use case.

[leoparente]

For now, I will remove the implemented feature of adding site to VLANs to not cause confusion regarding this and will keep this ticket open

[Kramdin-thinklogistics]

Hi @leoparente - This software is great so thanks to everyone who has contributed to building it and for really trying to understand how to make it better.

Yes, I think a mitigation would be very useful, especially for anyone running enterprise networks with multiple (potentially overlapping VIDs, Q-in-Q, etc.)

Thanks for the explanation of what Diode is doing behind the scenes. If I understand correctly though, can it not query for vlan id + site to determine whether to perform a create/update operation? I can simulate this idea in Netbox by using a vlan filter for site where even with 20 entries with vlan 1, it only returns the one with the site that was specified in the filter. So, if no entry exists, create a new VLAN entry with site, name, description, etc, otherwise update the existing vid entry with newly polled values (name, description, etc)?