Breaking changes proposal

[pd93]

Context

This document proposes a new approach to breaking changes in Task. Currently, many features that require a breaking change have been blocked from being added to Task because of they would require a major version bump. Major version bumps can be disruptive to users, so we want to make them infrequently.

Unfortunately, due to the time commitments required, it is not very practical for us to make a large major release with a lot of breaking changes in one go. We need a better approach that allows us to slowly build up a series of breaking changes over time that can then be released in a major version.

Proposal

We propose a new process for introducing breaking changes into minor version releases of Task. The default behavior in these minor release must remain unchanged so that we don't break Task for existing users. However, users will be able to enable experimental flags and/or environment variables that allow them to try new behaviors before they are fully released.

This will allow us to slowly introduce and iterate on breaking changes in a way that gives users a chance to give feedback and the maintainers more time to introduce the breaking changes for a major release.

What do experimental environment variables look like?

TASK_X_<FEATURE>=1

• The TASK_ prefix is to avoid collisions with other environment variables.
• The X_ is to indicate that this is an experimental feature.
• The <FEATURE> should be replaced with a concise description of the feature
  being enabled.
• The =1 is to indicate that the feature is enabled. If the environment
  variable is unset or set to any other value, the feature is disabled. Using a
  number avoids inconsistency with casing (e.g. true vs True vs TRUE etc).

How do we deprecate old behaviors?

When new experimental flags are introduced, we should mark the old behaviors as deprecated. This means that we can remove the old behaviors in the next major release and the experimental features will become the default behavior.

How do we track breaking changes and deprecations?

Experimental flags will track breaking changes to an extent. However, we should also maintain a new document on taskfile.dev that
tracks breaking changes and deprecations. This page will also allow us to provide guidance on how users should migrate to the new features.

What is the workflow/process for adding experimental features?

1. Proposal:

   All experimental features start with a proposal in the form of a GitHub issue. If the maintainers decide that an issue has enough support and is a breaking change or complex/controversial enough to require user feedback, then the issue will be marked with the new experiment:proposal label. At this point, the issue becomes a proposal and a period of consultation begins. During this period, we request that users provide feedback on the proposal and how it might effect their use of Task. It is up to the discretion of the maintainers to decide how long this period lasts.

2. Draft:

   Once a proposal's consultation ends, a contributor may pick up the work and begin the initial implementation. Once a PR is opened the maintainers will ensure that it meets the requirements for an experimental feature (i.e. flags are in the right format etc) and merge the feature. Once this code is released, the experiment:proposal label may be removed and replaced byexperiment:draft. This indicates that an implementation is now available for use in a release and the experiment is open for feedback from real users.

   During the draft period, major changes to the implementation may be made based on the feedback received from users. There are no stability guarantees and experimental features may be abandoned at any time.

3. Candidate:

   Once the community reaches an acceptable level of consensus on the experiment and changes are less frequent and significant, the experiment:draft label may be replaced by experiment:candidate. This indicates that a proposal is likely to accepted and will enter a period for final comments and minor changes.

4. Stable:

   Once a suitable amount of time has passed with no changes or feedback, an experiment will be given the experiment:stable label. At this point, the functionality will be treated like any other feature in Task and any changes must be backward compatible. This allows users to migrate to the new functionality without having to worry about anything breaking in future releases. This provides the best experience for users migrating to a new major version.

5. Released:

   When making a new major release of Task, all experiments marked as experiment:stable will move to experiment:released and their behaviors will become the new default in Task. Experiments in an earlier stage (i.e. not stable) cannot be released and so will continue to be experiments in the new version.

6. Abandoned/Superseeded:

   If an experiment is unsuccessful at any point then it will be given the experiment:abandoned or experiment:superseeded labels depending on which is more suitable. These experiments will be removed from Task.

Considerations (Request for comment)

What if users want to enable an experimental feature for all users of a distributed Taskfile?

Since environment variables are not distributed with Taskfiles, it's not feasible to commit Taskfiles that use experimental features with environment variables alone. If you want to enable experimental features at a repository level, you can create a .env file in the same directory as the root Taskfile. Experimental environment variables will then be loaded from this file. e.g:

# .env
TASK_X_FEATURE1=1

Only variables starting with TASK_X_ will be loaded from this file. This allows you to use the .env file for other purposes as well and shouldn't interfere with other tools or Task's dotenv field.

v2 schema support

I personally feel like it is time to drop support for v2. v3 was released nearly 4 years ago and users have had a lot of time to update. Any users who still use v2 will be able to continue using released versions of Task, but I don't see a huge requirement to continue supporting it in future versions. This will allow us to tidy up a bunch of code before thinking about introducing potential v4 features as experiments.

[andreynering]

Good work @pd93, I think this is a good proposal! 👏

I think a good start is to have a draft of the page that describe how deprecations work on the project, and then @jaedle will be unblocked to work on #1038, that will probably be the first change to use this pattern.

[pd93]

Replying to some of @zbindenren's comments here to keep everything together:

how long is a flag experimental? Maybe we should do it like kubernetes does. 3-6 months and then it is dropped or added.

Generally, I would suggest that flags stay experimental until a major version is released and that behavior becomes the default. If an experimental feature doesn't fit in Task or needs changing somehow, then it will be removed/changed once we've received enough feedback to make that decision. This whole proposal is about not rushing decisions, and instead slowly iterating based on user feedback.

Perhaps experimental features could have a status so that users can easily see their progress. Something like:

• proposed - Idea submitted, no work done yet. Pending feedback on implementation plan.
• draft - Early implementation created. Pending user feedback.
• candidate - User support is generally positive. Late implementation created. Small changes only.
• stable - Implementation is finialised. Will be default behavior in next major release.
• released - Implementation is now the default behavior.
• abandoned - Experiment is abandoned due to negative/mixed feedback.
• superseeded - Experiment is abandoned due to a better proposal.

EDIT: I've updated the proposal with a new section on the workflow of an experiment

how do we introduce breaking changes to the taskfile yaml itself. Maybe define when and how and under what circumstances to relaease a v4 etc...

I think this can be done in the same way. The YAML parsing library we use if flexible enough to handle variable YAML schemas (we still support v2 atm). We just have to make sure that when we parse a file containing an experimental feature that isn't enabled, we display an appropriate error message so that it isn't confusing to the users when something doesn't work.

Abstracting this logic in a nice way is going to be the challenge! I have some ideas about how we can nicely wrap up this functionality that I will include in a future PR.

[pd93]

I've done a bit of experimenting (pun intended) with the Task CLI in an attempt to get some initial code to handle experimental flags etc working and immediately ran into some issues. I explained this briefly on Discord, but for convenience, I'll post a summary here...

Basically, the --x-<name> flags are a pain. If a normal flag is dependent on an experiment being enabled, then we have to do weird things like parsing the flags twice (once to load experiment flags and once to load the normal flags depending on which experiments are enabled). It leads to some pretty ugly/complex code which pflag (and pretty much all flag parsers) aren't designed to handle.

I did have a think about building something custom to do Task's flags since we do things a bit differently anyway (using flags instead of commands), but this is a rabbit hole I don't want to go down now. Anyway - my point is, I think we should drop support for --x-<name> flags for now and just rely on environment variables for experiments. Users will still be able to run one off commands with experiments enabled via TASK_X_NAME=1 task <mytask> anyway so I don't think we're losing much by dropping them.

Edit: Updated proposal with these comments.