LineageFlow is a suite of libraries, executables and tools for the manipulation and exploration of spatio-temporal cell lineages and measurements over them. It has been developed for the study of reconstructed cell lineages extracted from 3D+time microscopy images of developing organisms. Its main features are:
- a modular and composable ecosystem of algorithms;
- a declarative interface for algorithms and the generation of executables;
- an uniform graphical interface for all algorithms following the interface;
- integrated tools for the visual, statistical and computational exploration of measurements over temporal lineages.
More details on the ideas underlying this system can be found here.
LineageFlow has been developed for Linux, and is written in Haskell.
In order to build it, nix must be installed in your computer.
This can be done with:
curl https://nixos.org/nix/install | sh
and by setting the following options in /etc/nix/nix.conf:
substituters = https://cache.nixos.org https://nixcache.reflex-frp.org
trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= ryantrinkle.com-1:JJiAKaRv9mWgpVAz8dwewnZe0AzzEAzPkagE9SP5NWI=
which enables the appropriated binary caches for Nix, which greatly reduces the installation time of the system. Afterwards, build the server, algorithms and client:
make server
make client
These can be deployed to a virtual machine if desired.
In first place, a location for the database must be chosen.
An empty folder suffices and it can be populated with tracking data using lf-import.
The server can be launched from the terminal with:
export LINEAGEFLOW_DATABASE=some-path
make run-server
which will start a session, and log the requests received by the client.
The graphical client can be launched from a separate terminal window with:
make run-client
which will open a chromium window containing the app and a short tutorial.
To enter a shell with all executables in the environment, simply type:
nix-shell
For the usage of algorithms from the command line, see the description of the lineageflow-algorithm package.
The code structure of all the algorithms is the same, and can be used as a model for new ones.
In general lines, one implements algorithms in the src folder, and declares them in the app folder.
The file app/Main.hs declares the collection of algorithms that the executable contains.
During the development, one configures the system with:
nix-shell --command "cabal configure"
which works thanks to the shell.nix file included in all algorithm folders.
Afterwards, one may develop using cabal repl and cabal build as usual.
LineageFlow has been conceived as an ecosystem of roughly independent parts that interact through declarative interfaces. Internally, this interface is defined by the Haskell types modelling the object of study and the data being manipulated. Externally, this interface is defined by a model of the interaction of users with algorithms, centered on the description of algorithms as entities that:
- takes parameters as input;
- takes sets of named measurements as input;
- outputs a set of named measurements.
This base definition has two derivations:
- algorithm declarations, that allow users to be informed on what are its parameters, inputs and outputs ;
- algorithm queries, allows one to instantiate these parameters, inputs and outputs with actual data, making it possible to run the algorithm.
All interaction with algorithms happens through these interfaces, and for this reason, tools have been created to make the definition and use of these more practical and precise for both developers and users. For developers, this is made by leveraging the type system of Haskell in order to automatically generate interfaces from type definitions. For users, this is given by a graphical user interface based on a server-client communication model, which automatizes the interpretation of algorithm definitions and generates forms guaranteeing that only well-formed queries can be made.
Other design decisions include the possibility of multiple databases for storing measurements, and multiple io methods for reading measurements from files.
The packages of the suite have the following dependency graph:
Each package has a well determined function:
types: types related to temporal cell lineages and measurements over it;prelude: custom prelude for a safer manipulation of measurements;io: interface for reading and writing measurements from files;io-*: particular implementation of IO method;base: types for the manipulation of algorithms;declaration: types for the declaration of algorithms;algorithm: generation of executables from algorithm declarations;query: types for querying measurements from a database;database: interface for measurement database interactions;database-*: particular implementation of a measurement database;server-api: API for server-client interaction;server: server for querying databases, launching algorithms and running exploration tools;client: graphical interface for the server API.
The following packages define algorithms both in library and executable form:
tracking: generation of spatio-temporal lineages from cell trackings;statistics: global statistics for measurements;derivatives: discrete derivatives for cell trajectories;homogenization: homogenization of measurements on time, space, etc.;triangulations: generation and manipulation of triangulations;trajectories: trajectories, genealogical trajectories and path integrals;clustering: clustering of cell trajectories;deviations: statistics of deviations of neighboring cells;forces: estimation of forces using least-squares methods;
The following tools are also included in the suite:
import: importing cell trackings from other file formats;export: exporting cell selections to other file formats;viewer: visualization of measurements in 3D+time;plot: plot generation for standard statistics on measurements;playground: utilities for interactive calculation on GHCi;script: scripts for the automation of multi-step algorithms;
with the following interface packages:
viewer-interface: interface for thelf-viewerexecutable;plot-interface: interface for thelf-plotexecutable;
This project is licensed under the AGPL-3.
This project has been developed by Juan Raphael Diaz Simões during a PhD thesis, in collaboration between the Condensed Matter Physics Laboratory at École Polytechnique and CNRS, and the BioEmergences Laboratory at CNRS.
The project has been directed by Denis Grebenkov, Paul Bourgine and Nadine Peyriéras.