feat(dbt): divest from fqn in a subsetted execution

[rexledesma]

Summary & Motivation

More thoughts on the line of #21851: we don't actually need
to rely on selecting by fqn, which can be very long.

Instead, we can use other markers of uniqueness, if we assume that a project is standalone. This is a reasonable assumption, considering that multi-project setups are only supported by dbt Cloud.

---

From https://docs.getdbt.com/reference/node-selection/methods:

Selector methods return all resources that share a common property, using the syntax method:value. While it is recommended to explicitly denote the method, you can omit it (the default value will be one of path, file or fqn).

And https://docs.getdbt.com/faqs/models/unique-model-names:

Within one project: yes! To build dependencies between models, you need to use the ref function, and pass in the model name as an argument. dbt uses that model name to uniquely resolve the ref to a specific model. As a result, these model names need to be unique, even if they are in distinct folders.

So, for models and snapshots, we can rely on running tests by their file name.

---

And from: https://docs.getdbt.com/reference/node-selection/test-selection-examples#more-complex-selection

Through the combination of direct and indirect selection, there are many ways to accomplish the same outcome. Let's say we have a data test named assert_total_payment_amount_is_positive that depends on a model named payments. All of the following would manage to select and execute that test specifically:

dbt test --select "assert_total_payment_amount_is_positive" # directly select the test by name

We can select tests using their test name.

---

After applying these optimizations when constructing the subset selection, the selection string is reduced in size by about 30-40%.

How I Tested These Changes

pytest

[rexledesma]

• feat(dbt): divest from fqn in a subsetted execution #21857 👈
• master

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @rexledesma and the rest of your teammates on Graphite

[cmpadden]

This lgtm.

To re-iterate my understanding, path doesn't include project_name, which when repeated adds up, and you explained is only necessary in multi-project environments. Any concern with a file moving that would impact path would also impact fqn, so it's a non-issue.

[rexledesma]

@cmpadden Yeah that's right.

cc @johannkm for your eyes as well, since this is of interest to asset checks. It doesn't solve the problem entirely, but this would give us more leeway when hitting the subprocess character limit.

I might gate this on a setting in DagsterDbtTranslator so that it doesn't unintentionally break folks who are using the current subsetted selection string successfully.