-
Notifications
You must be signed in to change notification settings - Fork 217
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Lutra roadmap #4177
Comments
Regarding WASM support: it is limited by upstream library support. Currently no data source libraries used by connector_arrow compile for wasm32-unknown-unknown, but it seems like rusqlite is close. When that is done, lutra still won't compile as it needs access to a file system to discover the project. I've specifically made sure that |
Overall, great!
Yes! Or a separate function could do the collection and pass to |
I'm not sure how lutra works, but am I correct in assuming that it automatically recognizes the schema of tables? |
It does have this capability (see https://github.com/aljazerzen/connector_arrow/blob/main/connector_arrow/src/api.rs for what connector_arrow supports) and my approach of passing this information to the prqlc is to generate type definitions in PRQL. Naive approach would be for lutra to pass schema information to prqlc directly in some internal representation, but that would couple lutra and prqlc very tightly. Instead, I added TLDR; lutra will allow pulling schema into PRQL source, which will avoid some compiler problems, which will allow us to say "in this case, compiler may error out and say it needs more schema info". |
I think that approach seems like the best choice given the current PRQL behavior of working without a schema and compiling to substrait, etc., which would not be possible without schema information. Great job! |
What's up?
After the initial implementation in #4134 has merged, there is still a lot of things to implement for Lutra. Tasks in order of priority:
@lutra.duckdb
(medium),@lutra.postgres
(medium),The text was updated successfully, but these errors were encountered: