Develop mode architecture
The parts of FlowDeploy develop mode, and how they fit together.
There are four primary FlowDeploy Develop components:
- A workflow manager running on a macOS or Linux machine
- A shared file system
- The FlowDeploy API
- Execution environments
A workflow manager is a pipelining tool geared towards running bioinformatics pipelines. The three most commonly used pipeline languages are Nextflow, Snakemake, and WDL.
The workflow manager sets:
- The order of execution of tasks (usually a DAG)
- Code to be executed per task
- Input and output file locations
- Docker containers for each task
Check what your local workflow manager is logging. Sometimes, they include environment variables and keys.
Shared file system
See the shared file system guide for more.
Workflow managers communicate with the FlowDeploy API over HTTPS using the Task Execution Service (TES) standard from the Global Alliance for Genomics and Health (GA4GH) non-profit. Nextflow, Snakemake, and WDL (via Cromwell) have a built-in TES integration.
By default, FlowDeploy executes in our managed AWS environment. You can set specific steps to run on your machine, or disable the integration completely to run your pipelines locally.
FlowDeploy can execute in other cloud environments. Contact us to learn more.