Flows

Certain pipelines can have different code paths that require different parameters. For example, an RNAseq workflow can have a choice between running different aligners, and might have different parameter sets for each choice of aligner. If a user wants to use the STAR aligner, it doesn’t make sense for the user to also have to provide parameters specific to salmon.

For this reason, the Latch SDK provides composable Flow constructs, that together enable a developer to create mutually exclusive parameter groups and more. These can be wired up readily, and allow for a much more intuitive user experience running the pipeline.

Flows are defined by a list of pre-defined FlowBase objects (see below for more information about these) provided to the flow argument in the LatchMetadata constructor. Flow objects are rendered one on top of the other on the Latch Console in order.

​

FlowBase objects

Every flow element is a subclass of a common FlowBase super class. FlowBase is used as a blanket term to cover every one of its subsclasses, and a FlowBase should never be instantiated directly.

​

Primitives

Primitive FlowBase objects are standalone and can’t be composed with any other flow elements. They are used to either provide text information or parameter widgets.

Text objects allow a developer to display Markdown rendered text in the parameter interface. Text objects take a single string argument containing the text to display.

text = Text(
    "Sample provided has to include an identifier for the sample (Sample name) and one or two files corresponding to the reads (single-end or paired-end, respectively)"
)

Title objects allow a developer to add a title element to the current flow. Title objects again take a single string argument, with the desired title text.

title = Title(
    "STAR Aligner Parameters"
)

Finally, Params objects dictate where to display the input components for specific parameters. Params objects take a variable number of string arguments, each of which are a parameter to the workflow.

params = Params(
    "index",
    "genome"
)

...

@workflow(metadata=metadata)
def workflow(index: LatchFile, genome: LatchFile):
    ...

These primitive elements become powerful when used in the composable elements below.

​

Composable Elements

Sections / Spoilers are the main way to group flow elements together. They can be used to delineate a set of parameters that all apply to a certain part of a pipeline.

Sections / Spoilers take as first argument a string representing the title of the section. Both then take a variable number of additional FlowBase arguments which represent the constituent parts of the section. These can be primitives like Text or Params, but they can also be Sections or Forks.

section = Section(
    "Samples",
    Text(
        "Sample provided has to include an identifier for the sample (Sample name)"
        " and one or two files corresponding to the reads (single-end or paired-end, respectively)"
    ),
    Params("single_end", "paired_end", "genome"),
)

The only difference between Section and Spoiler elements is that in the console, Spoilers are collapsible, while Sections are not.

Forks are the main way to separate parameters that are mutually exclusive. These can be used to, for example, separate parameters that correspond to one aligner from another aligner. Forks render in the Console as a set of different sections which can be selected using a toggle.

To use a Fork, first add a fork parameter to the main workflow. This parameter may be called anything, but it must be of type string. This parameter allows downstream tasks to know which branch of the Fork was chosen when running the pipeline.

The first argument to a Fork object is the name of the fork parameter as a string. The next argument to the Fork object is a string representing the title of the Fork element in the Console. Thereafter, Fork objects take a variable number of keyword arguments of type ForkBranch.

ForkBranches are essentially Sections from a developer standpoint. Both take a string as first argument with the title / display name and both take a variable number of flow elements as additional arguments. ForkBranches can only be used in Forks however. ForkBranches’ display_names are used as their title in the outer Fork toggle.

The keyword used for each ForkBranch is propagated to the fork parameter specified - that way at runtime, tasks can differentiate which Fork branch was taken and select logic accordingly.

fork = Fork(
    "sample_fork",
    "Choose read type",
    single_end=ForkBranch("Single-end", Params("single_end")),
    paired_end=ForkBranch("Paired-end", Params("paired_end")),
),

...

@workflow(metadata=metadata)
def workflow(
    sample_fork: str # <-- can be either "paired_end" or "single_end" depending on which branch is selected
    single_end: Optional[LatchFile] = None # <-- note the default value
    single_end: Optional[Tuple[LatchFile, LatchFile]] = None # <-- note the default value
): ...