Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add support for exporting/importing an org and all its data from db #890

Open
11 of 16 tasks
tw4l opened this issue May 25, 2023 · 4 comments · May be fixed by #1394
Open
11 of 16 tasks

Add support for exporting/importing an org and all its data from db #890

tw4l opened this issue May 25, 2023 · 4 comments · May be fixed by #1394
Assignees
@ikreymer @tw4l
Labels
enhancement New feature or request

Comments

@tw4l
Copy link
Contributor

tw4l commented May 25, 2023 •
edited

Export

Data

  • Org (filtered by id)
  • Crawls (filtered by oid)
  • Crawl files
  • Crawl configs (filtered by oid)
  • Crawl config revisions (filtered by crawl configs returned in step above)
  • Collections (filtered by oid)
  • Profiles (filtered by oid)
  • Profile files
  • Limited user information (id, role, and email included in org export)
  • Database version - for now import fails if export version doesn't match current db version

Files

Separate process?

  • Crawl files
  • Profile files

Import

Data

  • Import data from JSON
  • Generate new passwords for users?

Files

Separate process?

  • Crawl files
  • Profile files
@tw4l tw4l self-assigned this May 25, 2023
@tw4l tw4l added the enhancement New feature or request label May 25, 2023
@tw4l tw4l changed the title Add support for serializing database to file and importing an existing database Add support for exporting/importing an org and all its data from db Aug 23, 2023
@tw4l
Copy link
Contributor Author

tw4l commented Nov 14, 2023

May also need to export/import configmaps if we expect imported workflows to be able to run crawls

@tw4l
Copy link
Contributor Author

tw4l commented Nov 15, 2023

Re-create configmaps on workflow import

This was referenced Nov 21, 2023
Draft
Closed
@tw4l tw4l mentioned this issue Dec 13, 2023
Open
@SuaYoo SuaYoo added this to the v1.9.0 milestone Dec 13, 2023
@ikreymer ikreymer modified the milestones: v1.9.0, v1.10.0 Feb 7, 2024
@tw4l tw4l mentioned this issue Apr 4, 2024
Open
@ikreymer ikreymer removed this from the v1.10.0 milestone Apr 30, 2024
@SuaYoo
Copy link
Collaborator

SuaYoo commented May 20, 2024

@tw4l does this need frontend work?

@tw4l
Copy link
Contributor Author

tw4l commented May 20, 2024 •
edited

@tw4l does this need frontend work?

We had previously talked about this being an API-only feature, since only we would run it, at least for now. But happy to revisit that!

@ikreymer ikreymer self-assigned this May 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ikreymer ikreymer

@tw4l tw4l

Labels
enhancement New feature or request
Projects
Webrecorder Projects
Status: In Review
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Add superuser API endpoints to export and import org data webrecorder/browsertrix
3 participants
@ikreymer @SuaYoo @tw4l