Easy-to-use Databricks Notebooks for Admin Tasks.
Made with ❤️ by Dotlas Inc
This repository contains a directory of Databricks notebooks that assists with administrative tasks for Databricks, or otherwise helps as a supporting utility.
For example, consider the following use-cases:
- 📆 View a calendar of scheduled jobs to resolve conflicts in Databricks workflows.
- 🐼 Upload a Pandas DataFrame to Delta Lake
- 📑 Update Delta Lake table Documentation using Pydantic Models
- ➿ Migrate Jobs between Databricks workspaces
- ⚙️ Mass-edit Job Clusters in Existing Jobs
| Notebook | Description |
|---|---|
| Workflow Calendar | Visualize scheduled Jobs on a calendar, eyeball conflicts and view historic runs as a Gantt chart |
| Delta Docs with Pydantic | If you have pydantic models with fields containing description and tags that are used as data models, transfer these field descriptions to Delta lake columns as comments and tags. |
| Pandas to Delta | Use databricks-sql-python and SQLAlchemy to upload a Pandas DataFrame to Delta Lake from outside a Databricks environment |
| Workspace Jobs Migration | Migrate Workflows from one Databricks workspace to another |
| Job Cluster Update | Use the Databricks API to mass-update Job and Task configs |
| Workflow Config Exporter | Export existing workflow configuration and save it for future consumption |
- Check out the launch discussion on this LinkedIn Release Post, with a highlight from Databricks CEO, Ali Ghodsi.
- Feel free to raise an issue on this repository to start a discussion about new features, bug fixes or enhancements.
- See CONTRIBUTING.md for guidelines when adding or modifying notebooks in this repository.