Is it possible not to increment dashboard and connection ids on restoring?

[maxout123]

Is your feature request related to a problem? Please describe.

Now, on dashboards (or connections) upload, all dashboards/connections are just deleted in Grafana and then uploaded from scratch, thats's why their IDs are changed on every upload.

If we are keeping them in GIT, we are getting new changes on every download after upload.
So it becomes an infinite loop: we do upload -> ids changed in grafana, we do download -> ids changed in git -> we do upload, and so on..

Describe the solution you'd like

IDs of already existing resources in Grafana should be kept on upload.

If there is a way in grafana to specify IDs on resource creation - that's would be great.
If not, as a first approach you can match resources by UID and do PATCH instead of DELETE+PUT.

Describe alternatives you've considered
As an alternative, if IDs in export files don't mean anything as of now, maybe just fill them with some constants in export files? So export files won't change after every upload.

[safaci2000]

Above are all the supported endpoints for dashboards. I'm not a maintainer of grafana, so adding a PATCH to the API is something that needs to be requested upstream.

The main reason I'm doing a delete/create is because I can't really tell if something has changed or not. The easiest way for the 'upload' behavior is to delete all dashboards under the watched folders and re-create them. The assumptions here for any folder under 'watched' is that VCS is the source of truth, anything left is cleared to be wiped. That's why it has the concept of whitelisting folders.

I can look at trying to do a GET, compare and only delete/re-create if they changed. I have a feeling you'll get a good bit of false positives, but I'll see if that can be done.

The point of GDG is to let you have a VCS version. You should only be pulling data when you have changes you want to incorporate and your VCS is the source of truth, so everything on grafana is ready to be wiped. Anything that follows a different pattern I suggest putting a non-watched folder.

[safaci2000]

I wrote some hacky code to implement the ask above.

One of the issues is that even if the UID is preserved, the ID has to be wiped out.

There are two endpoints that are supported: api/dashboards/db POST which creates a new entity and has no ID field to set. It's also only a partial import, ignoring library panel references last I checked.

The import endpoint: api/dashboards/import will only work if the ID is removed. So no matter what patter you use, you will still end up with something like this for the diff.

+++ b/test/data/org_1/dashboards/General/top-talkers-over-time.json
@@ -16,7 +16,7 @@
"editable": true,
"gnetId": null,
"graphTooltip": 0,

• "id": 69,

• "id": 80,

Since this might be valuable I'll change this to a conversation, but at this point, there is no pattern to fix this unless upstream changes.

[maxout123]

Looks like api/dashboards/db endpoint also allows to update existing dashboards?
If so, maybe this workflow will be available:

1. getDashboardByUid
2. if exists, update existing dashboard by id received in step 1
3. if not exist, create dashboard

Seems there will be no id changes. Also, no comparison of dashboards jsons required.
Quite complex but looks working :)

[safaci2000]

I think I can overwrite with the import as well, the db endpoint isn't an option since it breaks existing features that the /db does not support.

I think there is a patter I can add that will preserve the behavior you like, but there's other facts I need to address, like removing any existing dashboards that do not match the existing file system and so on.

I'll create some pseudo code and see about adding this feature in a future release.

[safaci2000]

Here's the hacky implementation, I yanked some of the code from testify for the JSON comparison.

https://github.com/esnet/gdg/tree/feature/preserveIDs

[maxout123]

As an option, if IDs in export files don't mean anything (id field is lost anyway on upload in current implementation), maybe you can provide an option to just fill them with some constants in export files?
Replace id in every export file of dashboard and connection to 1, as an example. Or even strip id field completely?

So export files won't change after every upload, because there will be id: 1 every time. It will help to break push-pull endless loop )

[safaci2000]

#211 Issue to track the requested behavior change.

[safaci2000]

@maxout123 if you are explicitly watching for changes in git, I don't think the preservIDs will fix this issue.

Even if the IDs don't change, I have a partial implementation of this, you end up with the version incrimenting. Even if there are no changes if you push a new dashboard, and pull it back down you'll get diffs like this:

-  "version": 2
+  "version": 3