Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

create turbo-static for compile time graph analysis #8037

Merged
merged 1 commit into from
Jun 4, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
101 changes: 81 additions & 20 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ resolver = "2"
members = [
"crates/node-file-trace",
"crates/tower-uds",
"crates/turbo-static",
"crates/turbo-tasks*",
"crates/turbo-prehash",
"crates/turbopack*",
Expand Down
2 changes: 2 additions & 0 deletions crates/turbo-static/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
call_resolver.bincode
graph.cypherl
25 changes: 25 additions & 0 deletions crates/turbo-static/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
[package]
name = "turbo-static"
version = "0.1.0"
edition = "2021"
license = "MPL-2.0"

[dependencies]
bincode = "1.3.3"
clap = { workspace = true, features = ["derive"] }
ctrlc = "3.4.4"
ignore = "0.4.22"
itertools.workspace = true
lsp-server = "0.7.6"
lsp-types = "0.95.1"
proc-macro2 = { workspace = true, features = ["span-locations"] }
serde = { workspace = true, features = ["derive"] }
serde_json.workspace = true
serde_path_to_error = "0.1.16"
syn = { version = "2", features = ["parsing", "full", "visit", "extra-traits"] }
tracing-subscriber = { version = "0.3.18", features = ["env-filter"] }
tracing.workspace = true
walkdir = "2.5.0"

[lints]
workspace = true
33 changes: 33 additions & 0 deletions crates/turbo-static/readme.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Turbo Static

Leverages rust-analyzer to build a complete view into the static dependency
graph for your turbo tasks project.

## How it works

- find all occurences of #[turbo_tasks::function] across all the packages you
want to query
- for each of the tasks we find, query rust analyzer to see which tasks call
them
- apply some very basis control flow analysis to determine whether the call is
made 1 time, 0/1 times, or 0+ times, corresponding to direct calls,
conditionals, or for loops
- produce a cypher file that can be loaded into a graph database to query the
static dependency graph

## Usage

This uses an in memory persisted database to cache rust-analyzer queries.
To reset the cache, pass the `--reindex` flag. Running will produce a
`graph.cypherl` file which can be loaded into any cypher-compatible database.

```bash
# pass in the root folders you want to analyze. the system will recursively
# parse all rust code looking for turbo tasks functions
cargo run --release -- ../../../turbo ../../../next.js
# now you can load graph.cypherl into your database of choice, such as neo4j
docker run \
--publish=7474:7474 --publish=7687:7687 \
--volume=$HOME/neo4j/data:/data \
neo4j
```