Future of alchemlyb

[orbeckst]

In alchemistry/flamel#8 we started discussing the future of alchemlyb, its CLI flamel, and in particular where "workflows" would find a place.

We had a virtual meeting on Wed 25 August 2021, 21:00 UTC = 2pm AZ = 2pm Los Angeles = 22:00 London (UK) = 23:00 Berlin (Germany).

Agenda

introductions & interest in alchemlyb

@dotsdl @mrshirts @xiki-tempula @harlor @orbeckst introduced themselves.

review of current guiding principles of alchemlyb

• Core Philosophy and API Principles
• simple functions, only use classes with states for estimators
• use pandas dataframes as data format (with attributes)
• test everything because correctness is the highest priority

open discussion of the need to expand/change the mission

Gather opinions on what alchemlyb ( = sum of alchemlyb, alchemtest, flamel, ...) should be. What is its mission? Do we need to update core philosophy and API principles?

Most of the previous roadmap has been achieved. What's missing is a fully functional CLI (successor to alchemical-analysis.py) and better user documentation.

Discussion centered on what functionality (scientific drivers) should be in alchemlyb.

1. uncertainty estimation by bootstrapping (@mrshirts ); @dotsdl pointed out PR WIP: bootstrapping submodule, parameter for estimators #94 ; but it is possible that for more complicated data sets (replica exchange etc) this cannot be easily formulated in the standard alchemlyb framwework and would be better handled as a "workflow"
2. calculation of decorrelation times (@mrshirts ); @dotsdl pointed out PR WIP: Refactor of subsampling module #98 ; there's also a question if the decorrelation functions in pymbar should be in a separate package where they could be improved
3. convergence analysis (@xiki-tempula ): currently part of PR Implement the workflow pipeline #114 (ABFE workflow) but should be a separate building block in a alchemlyb.convergence module. This would fit into the standard schema of alchemlyb as a function that takes data and an estimator as input.

Decided that we needed to discuss "workflows" before deciding on mission changes.

Where should code for "workflows" live?

Workflows are script-like code blocks that combined multiple alchemlyb functions to perform a partial or complete (end-to-end) free energy analysis. As an example, PR #114 implements the analysis of absolute binding free energy calculations (ABFE) with GROMACS data.

Decide where "workflows" should find a home, e.g., as a module in alchemlyb or flamel or a separate repository and package?

@xiki-tempula explained his ABFE workflow PR #114. It replicates the functionality in alchemical-analysis.py using alchemlyb building blocks. He opted to write it as script-like code because the plugin architecture of flamel was not easily to adapt to his use case. The workflow reads data, decorrelates it, runs an estimator, does convergence analysis, and plots results.

@dotsdl considered that such code could be bundled with alchemlyb and provide a "best practice" example implementation of a general free energy application. @harlor agreed.

@orbeckst was concerned to include code that only supports a specific MD package (here, GROMACS) but after quick review of the code it was agreed that this could be made more general.

It was decided that we add a alchemlyb.workflows module.

1. We need to update the Mission and Core Philosophy and API Principles to include workflows that are "essential" or "common". They should be generic and use building blocks as much as possible. Code needs to match the high standards of the rest of alchemlyb.
2. We make an effort to include the ABFE workflow, provided that we can (1) abstract as much code into generic building blocks (e.g., convergence analysis) and (2) that the code is not MD package specific. The code has to be well-tested (as is currently the case for the PR).

Develop a rough roadmap

How should the development of

• alchemlyb
• flamel

move forward?

Decided to revisit after we resolved the integration of workflows in alchemlyb. No resolution.

Action items

If any work needs to be done, find volunteers to do it.

• @xiki-tempula will raise a PR in alchemlyb to update the mission and API principles to include workflows
• @dotsdl and @orbeckst will review the mission PR
• @orbeckst and @dotsdl will review PR Implement the workflow pipeline #114 with a view towards (1) which code can be moved into new alchemlyb building blocks and (2) how to make it MD code agnostic. The review will follow the principles laid down in the update mission/core principles.
• @mrshirts will move forward with work on pymbar and raise issues in alchemlyb to alert us of any breaking changes that will need to be accounted for when pymbar 4.0 is released.

[orbeckst]

Issue #160 was completed with the update to the Mission. See Mission statement and API Principles.

[orbeckst]

Roadmap for flamel

Based on alchemistry/flamel#8 (comment) and echoed in the flamel README.

Mission

The aim of the flamel project is to develop a command line interface (CLI) to alchemlyb, the well-tested and actively developed library for alchemical free energy calculations.

It is supposed to become the successor of the now unsupported alchemical-analysis script.

Objectives

We decided to add "robust and general workflows" to alchemlyb (see Mission and API principles) as discussed in #159 .

Following up on @xiki-tempula 's original alchemistry/flamel#8 (comment) and alchemistry/flamel#8 (comment) , the likely direction is to turn flamel into a thin CLI. I suggest the following, summarizing the discussion and adding my own take:

1. Because flamel can use the soon-to-be-available alchemlyb.workflows module (in particular PR Implement the workflow pipeline #114 ), we don't need the current plugin structure anymore. Instead, workflows just have the appropriate kwargs to choose decorrelation etc and flamel's job is to pass the correct commandline arguments to the workflow functions.
2. I would prefer if output functionality were still be handled by flamel because alchemlyb should remain simple in that it only deals with DataFrames as data.
3. flamel will need some integration tests (add tests flamel#9 ).
4. flamel will remain in its own repository and just install alchemlyb as a dependency (once we have an installable package make flamel pip installable flamel#3 ). We do not want to bundle flamel with alchemlyb so that alchemlyb can be easily installed as a pure library (and so that we can keep the testing separate).

[xiki-tempula]

I agree. The current #114 would output a pandas data frame when I finished it. The flamel will then transform it into a text output.