This is an example application which uses OpenAI, Databricks Unity Catalog and DBSQL to convert natural language queries into SQL and execute it against DBSQL.
- local
Read more about this example project here.
- Databricks Unity Catalog provides metadata about catalogs, schemas and tables
- OpenAI API provides API to generate SQL statements from the natural language
- Databricks SQL is the execution engine to run the generated SQL statements
- React for the frontend application
- React Bootstrap for layout and components style
- React Syntax Highlighter for SQL syntax highlighting
- Axios - HTTP client, used for backend communication
- React Bootstrap Table - for table visualization
- FastAPI for the backend server
- Databricks SQL Connector to communicate with DBSQL endpoint
- OpenAI Python API - for API calls from backend
git clone https://github.com/renardeinside/databricks-uc-semantic-layer.git- Get your DBSQL endpoint coordinates from the UI
- Get your OpenAI API key
- Generate the data using the job defined in
./uc-semantic-layer:
- Configure the catalog and schema in
./uc-semantic-layer/conf/data_preparation.yml- Run the job (either on the interactive or as a job):
cd uc-semantic-layer
# install dbx and other relevant libraries
pip install -r unit-requirements.txt
# optional - configure dbx to use another profile (by default it uses the DEFAULT one)
dbx configure -e default --profile=<some-other-profile-name>
# this is to execute on an interactive cluster
dbx execute --job=semantic-layer-data-preparation --cluster-name=<interactive-cluster-name>
# this is to launch on automated one, please configure your node_type_id in conf/deployment.yml
dbx deploy --job=semantic-layer-data-preparation --files-only
dbx launch --job=semantic-layer-data-preparation --as-run-submit --trace
cd .. # back to the project root- Setup the relevant variables in
.envfile (check.env.examplefor a reference). - Start the services:
make launch- Open http://localhost:3000 and enjoy the app!