Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(cli): --filter flag #8185

Merged
merged 84 commits into from Apr 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
84 commits
Select commit Hold shift + click to select a range
ad920cb
Skeleton code for `bun run --workspace`
Jarred-Sumner Jan 13, 2024
6be006e
Update run_command.zig
Jarred-Sumner Jan 13, 2024
a734de8
implement directory traversal to find workspace root
gvilums Jan 14, 2024
c83507b
finish --workspace implementation
gvilums Jan 15, 2024
1868ad3
clean up changes in run_command.zig
gvilums Jan 15, 2024
496bac0
add workspace tests, update harness to handle nested dirs
gvilums Jan 15, 2024
1b659b3
[autofix.ci] apply automated fixes
autofix-ci[bot] Jan 15, 2024
2f33677
basic filtering
gvilums Jan 16, 2024
ea8ea8d
[autofix.ci] apply automated fixes
autofix-ci[bot] Jan 16, 2024
d8d0ffb
working filter without patterns
gvilums Jan 16, 2024
0c9c555
update tests, filter mostly working
gvilums Jan 17, 2024
f74aad9
simplify package name parsing, commit tests
gvilums Jan 17, 2024
4adbf68
support filter even without workspace setup
gvilums Jan 17, 2024
44ea630
move filter arg handling to separate source file
gvilums Jan 17, 2024
9d17118
use bun.sys.chdir, match root package for scripts
gvilums Jan 17, 2024
e68e7b0
fix exit code handling
gvilums Jan 18, 2024
5cb2d58
Merge branch 'main' into georgijs/bun-run--workspace
Jarred-Sumner Jan 18, 2024
af5728c
ignore node_modules and directories starting with . in --filter
gvilums Jan 18, 2024
ad18255
progress converting --filter to use iterators
gvilums Jan 18, 2024
5b19f98
convert filtering to use iterators
gvilums Jan 18, 2024
a4d1b17
cleanup
gvilums Jan 18, 2024
024d86e
implement DirEntry access method for glob (currently crashing)
gvilums Jan 19, 2024
9819a13
cleanup and fixes
gvilums Jan 19, 2024
c8c21bf
run js files in subprocess when filter flag passed
gvilums Jan 19, 2024
2557763
clean up dead code
gvilums Jan 20, 2024
5e467cc
fix fd leak in run_command.zig
gvilums Jan 20, 2024
7f01fae
Merge branch 'main' into georgijs/bun-run--workspace
gvilums Jan 20, 2024
1a3fbc1
[autofix.ci] apply automated fixes
autofix-ci[bot] Jan 20, 2024
9a7ba61
fix issues after merge
gvilums Jan 20, 2024
63124c3
use posix-spawn in runBinary, fix resource PATH variable resource leak
gvilums Jan 20, 2024
9af23ad
move filter argument to runtime category
gvilums Jan 20, 2024
8ab8941
Merge branch 'main' into georgijs/bun-run--workspace
Jarred-Sumner Jan 21, 2024
26a720d
fix test harness
gvilums Jan 21, 2024
3426873
add js and binary tests to filter-workspace
gvilums Jan 22, 2024
a363f48
[autofix.ci] apply automated fixes
autofix-ci[bot] Jan 22, 2024
5637a7d
Merge remote-tracking branch 'origin/main' into georgijs/bun-run--wor…
gvilums Jan 22, 2024
2c48289
Merge remote-tracking branch 'origin/main' into georgijs/bun-run--wor…
gvilums Jan 22, 2024
03037de
Merge branch 'main' into georgijs/bun-run--workspace
gvilums Apr 4, 2024
5b91ca3
fix compile after merge
gvilums Apr 4, 2024
63d5cd6
[autofix.ci] apply automated fixes
autofix-ci[bot] Apr 4, 2024
127a599
clean up filter-workspace test
gvilums Apr 4, 2024
c39e61d
[autofix.ci] apply automated fixes
autofix-ci[bot] Apr 4, 2024
f1db7ad
fixes to running binaries
gvilums Apr 4, 2024
7d122b3
fix actually setting cwd_override
gvilums Apr 4, 2024
ca4c96d
windows fixes
gvilums Apr 5, 2024
4faf32d
address some review comments
gvilums Apr 5, 2024
7e7e55d
handle malformed JSON
gvilums Apr 5, 2024
101a893
add various tests
gvilums Apr 5, 2024
b4300bb
[autofix.ci] apply automated fixes
autofix-ci[bot] Apr 5, 2024
08af77a
update docs for filter
gvilums Apr 5, 2024
a3f6998
[autofix.ci] apply automated fixes
autofix-ci[bot] Apr 5, 2024
a26eca6
reset tinycc commit
gvilums Apr 5, 2024
494fa3d
filtered run prototype
gvilums Apr 8, 2024
a9b9ce0
make pretty
gvilums Apr 8, 2024
d8f76f7
implement abort handler (not working)
gvilums Apr 8, 2024
8e17407
make prettier
gvilums Apr 8, 2024
75864c7
prep for windows
gvilums Apr 9, 2024
000d513
windows path and printing fixes
gvilums Apr 9, 2024
78814e6
implement log-style output (not tui)
gvilums Apr 9, 2024
b8abffd
fix issues when logging to file
gvilums Apr 9, 2024
3f40cc6
Merge remote-tracking branch 'origin/main' into georgijs/bun-run--wor…
gvilums Apr 9, 2024
22dfc45
revert a bunch of unecessary changes
gvilums Apr 9, 2024
ac3ad16
cleanup
gvilums Apr 9, 2024
da3485f
implement dependency order execution
gvilums Apr 9, 2024
6ba7eb3
detect circular dependencies, fix cancel hang
gvilums Apr 10, 2024
defd08f
Fix `$PATH`
Jarred-Sumner Apr 10, 2024
7d6e443
ignore dep order on loop, stream on linux, sort pkgs
gvilums Apr 10, 2024
f781e51
support pre and post scripts
gvilums Apr 10, 2024
4ddddb9
add more filter tests, print elapsed time
gvilums Apr 10, 2024
a5fcbfa
enable 'bun --filter' without run
gvilums Apr 10, 2024
4046074
Merge remote-tracking branch 'origin/main' into georgijs/bun-run--wor…
gvilums Apr 10, 2024
e0726a3
fix harness after merge
gvilums Apr 10, 2024
473a2d4
[autofix.ci] apply automated fixes
autofix-ci[bot] Apr 10, 2024
ccfebfb
print number of scripts we're waiting for
gvilums Apr 10, 2024
f7e54d8
update docs, fix windows build
gvilums Apr 11, 2024
c178624
fix tests on windows
gvilums Apr 11, 2024
22846af
[autofix.ci] apply automated fixes
autofix-ci[bot] Apr 11, 2024
c9ff770
fix uninitialized memory
gvilums Apr 11, 2024
5083cf6
use terminal synchronized update sequences
gvilums Apr 11, 2024
beb3d24
Merge branch 'main' into georgijs/bun-run--workspace
Jarred-Sumner Apr 12, 2024
32bfdb1
Merge branch 'main' into georgijs/bun-run--workspace
Jarred-Sumner Apr 12, 2024
66b8a2d
Add skip list
Jarred-Sumner Apr 12, 2024
9c0f1d6
Preallocate
Jarred-Sumner Apr 12, 2024
63c4f73
Use current bun in tests
Jarred-Sumner Apr 12, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
58 changes: 58 additions & 0 deletions docs/cli/filter.md
@@ -0,0 +1,58 @@
Use the `--filter` flag to execute lifecycle scripts in multiple packages at once:

```bash
bun --filter <pattern> <script>
```

Say you have a monorepo with two packages: `packages/api` and `packages/frontend`, both with a `dev` script that will start a local development server. Normally, you would have to open two separate terminal tabs, cd into each package directory, and run `bun dev`:

```bash
cd packages/api
bun dev

# in another terminal
cd packages/frontend
bun dev
```

Using `--filter`, you can run the `dev` script in both packages at once:

```bash
bun --filter '*' dev
```

Both commands will be run in parallel, and you will see a nice terminal UI showing their respective outputs:
![Terminal Output](https://github.com/oven-sh/bun/assets/48869301/2a103e42-9921-4c33-948f-a1ad6e6bac71)


## Matching

`--filter` accepts a pattern to match specific packages, either by name or by path. Patterns have full support for glob syntax.

### Package Name `--filter <pattern>`

Name patterns select packages based on the package name, as specified in `package.json`. For example, if you have packages `pkga`, `pkgb` and `other`, you can match all packages with `*`, only `pkga` and `pkgb` with `pkg*`, and a specific package by providing the full name of the package.

### Package Path `--filter ./<glob>`

Path patterns are specified by starting the pattern with `./`, and will select all packages in directories that match the pattern. For example, to match all packages in subdirectories of `packages`, you can use `--filter './packages/**'`. To match a package located in `pkgs/foo`, use `--filter ./pkgs/foo`.

## Workspaces

Filters respect your [workspace configuration](/docs/install/workspaces.md): If you have a `package.json` file that specifies which packages are part of the workspace,
`--filter` will be restricted to only these packages. Also, in a workspace you can use `--filter` to run scripts in packages that are located anywhere in the workspace:

```bash
# Packages
# src/foo
# src/bar

# in src/bar: runs myscript in src/foo, no need to cd!
bun run --filter foo myscript
```

## Dependency Order
Bun will respect package dependency order when running scripts. Say you have a package `foo` that depends on another package `bar` in your workspace, and both packages have a `build` script. When you run `bun --filter '*' build`, you will notice that `foo` will only start running once `bar` is done.

### Cyclic Dependencies

13 changes: 13 additions & 0 deletions docs/cli/run.md
Expand Up @@ -151,6 +151,19 @@ By default, Bun respects this shebang and executes the script with `node`. Howev
$ bun run --bun vite
```

### Filtering

in monorepos containing multiple packages, you can use the `--filter` argument to execute scripts in many packages at once.

Use `bun run --filter <name_pattern> <script>` to execute `<script>` in all packages whose name matches `<name_pattern>`.
For example, if you have subdirectories containing packages named `foo`, `bar` and `baz`, running
```bash
bun run --filter 'ba*' <script>
```
will execute `<script>` in both `bar` and `baz`, but not in `foo`.

Find more details in the docs page for [filter](/docs/cli/filter.md).

## `bun run -` to pipe code from stdin

`bun run -` lets you read JavaScript, TypeScript, TSX, or JSX from stdin and execute it without writing to a temporary file first.
Expand Down
1 change: 1 addition & 0 deletions docs/install/workspaces.md
Expand Up @@ -61,6 +61,7 @@ Workspaces have a couple major benefits.

- **Code can be split into logical parts.** If one package relies on another, you can simply add it as a dependency in `package.json`. If package `b` depends on `a`, `bun install` will install your local `packages/a` directory into `node_modules` instead of downloading it from the npm registry.
- **Dependencies can be de-duplicated.** If `a` and `b` share a common dependency, it will be _hoisted_ to the root `node_modules` directory. This reduces redundant disk usage and minimizes "dependency hell" issues associated with having multiple versions of a package installed simultaneously.
- **Run scripts in multiple pacakges.** You can use the [`--filter` flag](/docs/cli/filter.md) to easily run `package.json` scripts in multiple packages in your workspace.

{% callout %}
⚡️ **Speed** — Installs are fast, even for big monorepos. Bun installs the [Remix](https://github.com/remix-run/remix) monorepo in about `500ms` on Linux.
Expand Down
3 changes: 3 additions & 0 deletions docs/nav.ts
Expand Up @@ -180,6 +180,9 @@ export default {
page("install/lifecycle", "Lifecycle scripts", {
description: "How Bun handles package lifecycle scripts with trustedDependencies",
}),
page("cli/filter", "Filter", {
description: "Run scripts in multiple packages in parallel",
}),
page("install/lockfile", "Lockfile", {
description:
"Bun's binary lockfile `bun.lockb` tracks your resolved dependency tree, making future installs fast and repeatable.",
Expand Down
6 changes: 6 additions & 0 deletions src/bun.js/api/bun/process.zig
Expand Up @@ -81,6 +81,7 @@ pub const Rusage = if (Environment.isWindows) win_rusage else std.os.rusage;
const Subprocess = JSC.Subprocess;
const LifecycleScriptSubprocess = bun.install.LifecycleScriptSubprocess;
const ShellSubprocess = bun.shell.ShellSubprocess;
const ProcessHandle = @import("../../../cli/filter_run.zig").ProcessHandle;
// const ShellSubprocessMini = bun.shell.ShellSubprocessMini;
pub const ProcessExitHandler = struct {
ptr: TaggedPointer = TaggedPointer.Null,
Expand All @@ -93,6 +94,7 @@ pub const ProcessExitHandler = struct {
Subprocess,
LifecycleScriptSubprocess,
ShellSubprocess,
ProcessHandle,

SyncProcess,
},
Expand All @@ -116,6 +118,10 @@ pub const ProcessExitHandler = struct {
const subprocess = this.ptr.as(LifecycleScriptSubprocess);
subprocess.onProcessExit(process, status, rusage);
},
.ProcessHandle => {
const subprocess = this.ptr.as(ProcessHandle);
subprocess.onProcessExit(process, status, rusage);
},
@field(TaggedPointer.Tag, bun.meta.typeBaseName(@typeName(ShellSubprocess))) => {
const subprocess = this.ptr.as(ShellSubprocess);
subprocess.onProcessExit(process, status, rusage);
Expand Down
26 changes: 26 additions & 0 deletions src/cli.zig
Expand Up @@ -34,6 +34,7 @@ const bundler = bun.bundler;
const DotEnv = @import("./env_loader.zig");
const RunCommand_ = @import("./cli/run_command.zig").RunCommand;
const CreateCommand_ = @import("./cli/create_command.zig").CreateCommand;
const FilterRun = @import("./cli/filter_run.zig");

const fs = @import("fs.zig");
const Router = @import("./router.zig");
Expand Down Expand Up @@ -186,6 +187,7 @@ pub const Arguments = struct {
};

const auto_or_run_params = [_]ParamType{
clap.parseParam("--filter <STR>... Run a script in all workspace packages matching the pattern") catch unreachable,
clap.parseParam("-b, --bun Force a script or package to use Bun's runtime instead of Node.js (via symlinking node)") catch unreachable,
clap.parseParam("--shell <STR> Control the shell used for package.json scripts. Supports either 'bun' or 'system'") catch unreachable,
};
Expand Down Expand Up @@ -432,6 +434,10 @@ pub const Arguments = struct {
cwd = try bun.getcwdAlloc(allocator);
}

if (cmd == .RunCommand or cmd == .AutoCommand) {
ctx.filters = args.options("--filter");
}

if (cmd == .TestCommand) {
if (args.option("--timeout")) |timeout_ms| {
if (timeout_ms.len > 0) {
Expand Down Expand Up @@ -1154,6 +1160,8 @@ pub const Command = struct {
bundler_options: BundlerOptions = BundlerOptions{},
runtime_options: RuntimeOptions = RuntimeOptions{},

filters: []const []const u8 = &[_][]const u8{},

preloads: []const string = &[_]string{},
has_loaded_global_config: bool = false,

Expand Down Expand Up @@ -1703,6 +1711,13 @@ pub const Command = struct {
if (comptime bun.fast_debug_build_mode and bun.fast_debug_build_cmd != .RunCommand) unreachable;
const ctx = try Command.Context.create(allocator, log, .RunCommand);

if (ctx.filters.len > 0) {
FilterRun.runScriptsWithFilter(ctx) catch |err| {
Output.prettyErrorln("<r><red>error<r>: {s}", .{@errorName(err)});
Global.exit(1);
};
}

if (ctx.positionals.len > 0) {
if (try RunCommand.exec(ctx, false, true, false)) {
return;
Expand All @@ -1725,6 +1740,7 @@ pub const Command = struct {
},
.AutoCommand => {
if (comptime bun.fast_debug_build_mode and bun.fast_debug_build_cmd != .AutoCommand) unreachable;

var ctx = Command.Context.create(allocator, log, .AutoCommand) catch |e| {
switch (e) {
error.MissingEntryPoint => {
Expand All @@ -1737,6 +1753,13 @@ pub const Command = struct {
}
};

if (ctx.filters.len > 0) {
FilterRun.runScriptsWithFilter(ctx) catch |err| {
Output.prettyErrorln("<r><red>error<r>: {s}", .{@errorName(err)});
Global.exit(1);
};
}

if (ctx.runtime_options.eval.script.len > 0) {
const trigger = bun.pathLiteral("/[eval]");
var entry_point_buf: [bun.MAX_PATH_BYTES + trigger.len]u8 = undefined;
Expand Down Expand Up @@ -1821,6 +1844,9 @@ pub const Command = struct {
}

if (ctx.positionals.len > 0 and extension.len == 0) {
if (ctx.filters.len > 0) {
Output.prettyln("<r><yellow>warn<r>: Filters are ignored for auto command", .{});
}
if (try RunCommand.exec(ctx, true, false, true)) {
return;
}
Expand Down