Data Isolation: Methods to sync data between Netbox instances

[vyas-n]

My organization has a couple of regulatory requirements that prevent us from using a single Netbox instance as a centralized source of truth for all of our environments.

We've identified that we need about 3–4 instances of Netbox to be compliant and cover them all, but now need to figure out how to send data from one instance to another.

With some objects (Companies, Device Roles, Device Types, …) we require the data to be in-sync across all the instances.

With other objects (Devices, IP Addresses, VLANs, …) we require the data to be promoted from one environment to another (For Example: Dev → Prod).

Is there a set of methods that we can use to implement this behavior?

[mtinberg]

Others can chime in with suggestions, but you have to write some in-house automation software to accomplish this. If you search for "netbox sync" you can find other people who have done similar things and reuse or refactor their work to fit your situation. There isn't a built in webUI facility to promote records between Netbox instances, but all the infrastructure is there for you to straightforwardly make your own if needed. — Mark Tinberg ***@***.***> Division of Information Technology-Network Services University of Wisconsin-Madison

…

________________________________ From: Vyas ***@***.***> Sent: Thursday, September 7, 2023 9:43 AM To: netbox-community/netbox ***@***.***> Cc: Subscribed ***@***.***> Subject: [netbox-community/netbox] Data Isolation: Methods to sync data between Netbox instances (Discussion #13711) My organization has a couple of regulatory requirements that prevent us from using a single Netbox instance as a centralized source of truth for all of our environments. We've identified that we need about 3–4 instances of Netbox to be compliant and cover them all, but now need to figure out how to send data from one instance to another. With some objects (Companies, Device Roles, Device Types, …) we require the data to be in-sync across all the instances. With other objects (Devices, IP Addresses, VLANs, …) we require the data to be promoted from one environment to another (For Example: Dev → Prod). Is there a set of methods that we can use to implement this behavior? — Reply to this email directly, view it on GitHub<#13711>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAS7UM2OWSPFKL53MR7NRG3XZHMRBANCNFSM6AAAAAA4PASTVU>. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[adparis99]

There really is a plugin for everything, hadn't heard of that one before. My first thought was using webhooks and some combination of Ansible or Python, but I imagine there's a lot of room for error going that route if a script doesn't execute properly or a webhook gets hung.

[DanSheps]

This is actually something I want to write a plugin for.

[vyas-n]

That'd be awesome!

And to be honest, this might be a good starting point for a Netbox-native Backup & restore utility.

[talha700]

We use postgres streaming replication to run active/standby instances of Netbox in our environment, but it replicates the database in its entirety.
Maybe have a look at postgres Logical replication ?

[vyas-n]

Thanks @talha700,

That is something we're considering using for running Netbox with HA, but I don't know how it would keep data isolated between the instances.

Is there a way to selectively sync Netbox objects within Postgres?

[zmaster7]

I've only used streaming replication myself, but I wonder if the new-ish "event rules" would be able to accomplish some of the functionality you need, via webhooks or custom scripts (https://docs.netbox.dev/en/stable/features/event-rules/). For example, you may have a condition to see that the status is set to "active" when an object gets updated, which then pushes the changes from dev over to prod.

[xeor]

Did anyone start tackling this issue or wrote something that they are able to share?

[antoinekh]

I'm also interested if someone has already started something.
I haven't found anything that exists.