Get started with your first workflow

LatchBio

Platform

Documentation

API References

Blog

Website

Sign Up

Contact Us

Log In

The Latch SDK is an open-source toolchain to define serverless bioinformatics workflows with plain Python and deploy associated no-code interfaces using single commands.

Introduction

What is the Latch SDK?

Latch is a platform for you to store, analyze, and visualize multiomics data.

What is LatchBio?

Quickstart

Latch Data is a cloud based file storage system built for storing biological data.

Overview

What is Latch Data?

Basic Uploading and Downloading

Latch allows you to mount your own AWS S3 Buckets and use them the same as you would any data on Latch. All you need is to connect AWS account with Latch which will allow you to mount any buckets in that account onto Latch.

Mounting S3 Buckets

Latch allows you to mount your own GCP Buckets and use them the same as you would any data on Latch.

Mounting GCP Buckets

This tutorial explains how to get an API key from BaseSpace that you can use to automatically import your sequencing runs into Latch.

BaseSpace Downloader

Command Line Interface

Command Line Interface Data Upload/Download

Data Sharing

FastQC aims to provide a simple way to do some quality control checks on raw sequence data coming from high throughput sequencing pipelines. It provides a modular set of analyses which you can use to give a quick impression of whether your data has any problems of which you should be aware before doing any further analysis.

FastQC

The Integrative Genomics Viewer (IGV) is a high-performance, easy-to-use, interactive tool for the visual exploration of genomic data. BAM, Fasta, etc. files can be opened in an IGV Browser natively within the Latch Platform. 

Using IGV on Latch

Learn how to set default viewer settings when opening files in IGV on Latch

Setting Viewer Defaults for IGV on Latch

The IGV browser comes with many hosted options for common Reference Genomes to view Alignment Files to.

IGV Hosted Reference Genomes

Connect your sample sheets, metadata, and analysis — all in one place.

What is Latch Registry?

Learn how to set up your registry to manage your data.

Registry Basics

Manually Create Single Records

Bulk Import Data Using a CSV

Bulk Link Sequencing Files to Existing Records

Benchling Integration

As you centralize your biological data in Latch Registry, you will often find that some records are related or even dependent on one another. Linked records offer a powerful way for you to define relationships between records from different tables, enhancing your data analysis and enabling new insights that may have been difficult to discover without linked records.

Connect Records Across Tables

Import Registry Records in Latch SDK Workflows

Latch's Workflow Manager comes with out-of-the-box community workflows and a Python SDK that allows uploading of custom workflows.

What are Latch Workflows?

Bulk RNA Seq

End-to-end bulk RNA-seq analysis using LVBRS

Every workflow in Latch allows you to import a CSV containing values for its parameters. This is another way to import data when doing bulk runs of a workflow.

CSV Parameter Import

Access the scale of the cloud with the flexibility of your personal computer.

What are Latch Pods?

In this tutorial, we will walk through the basics of creating your first Pod.

Latch Pods Basics

Latch Pods provides direct access to all files stored on Latch Data.

Accessing Latch Data in a Pod

Best Practices for Working with Latch Data inside Pods

Pods allow easy customization of auto shutdown interval when inactive.

Setting Up Auto Shutdown from Latch

Configure Pods for scheduled shutdown using systemd.

Configuring Auto Shutdown

Latch Pods make it easy to host any custom application, such as Dash Apps, RShiny, Streamlit, and more, on Latch.

Host a Custom App

Pod Templates provide an easy way to take a snapshot of a Pod’s dependencies and files, and save it as a template that is reusable in future Pods for your organization. 

Pod Templates

You may want to set up SSH access for Pod if you want to access Pod from your local terminal or open your favorite IDE, such as Visual Studio Code, inside a Pod.

Set up SSH Access

Follow our tutorial to use Visual Studio Code inside your Pod and navigate the filesystem on your favorite IDE.

Integration with VSCode

If you want to add a GitHub repository to Pod when Pod is created for the first time or commit to a GitHub repository in the future, you have to first set up your GitHub personal access token (PAT).

Integration with GitHub

Set up dashboards with interactive visualizations and data transformations for scientists to explore their data.

What are Latch Plots?

Learn how to bring data into a plotting layout and chart it.

Plots Basics

Latch Plots Basics

Layouts

Create interactive and customizable plots.

Plotting Overview

Learn about the plot types supported in a plotting layout.

 Plot Types

Supported Plot Types

Facet your plot based on a categorical column.

Faceting

Learn what options are available for customizing the appearance of your plots.

Appearance

Error Bars

Learn how to make a selection of points and use that selection to filter the source table.

Selecting Points

Learn how to filter the values displayed in a plot.

Filtering

The Data Transformation component functions similarly to a Python cell in a Jupyter Notebook, allowing users to input, modify, and execute Python code interactively.

Widgets can be used to create transforms that users can update based on user input. You're able to define a widget and use its user input in your analysis code.

Widgets

Learn how to view and use dataframe outputs from a transfrom.

Transform Outputs

Write descriptions using markdown within a transform.

Description Text

Learn how to use a table display to view data and filter it for use downstream.

Table Display/Filter

Learn how to add description text to your layout.

Text Display

The guide below walks through how you can perform qPCR analysis on Latch using Plot Templates. 

qPCR

The Ultimate Guide to Relative qPCR Analysis on Latch

The Notebook View is where you bring in data, perform transformations in  Python, and visualize your data.

Develop your first Plot Notebook

Develop your First Plot Notebook

Learn how to collaborate with team members, manage access, share data and more with workspaces on Latch

Learn about billing for Latch workspaces and how to add your credit card information.

Billing

Workspace Billing

Learn how to create a new workspace on Latch

Creating a Workspace

Learn how to invite users to an existing workspace

Inviting Members to a Workspace

Learn about workspace roles and permissions

Workspace Roles

Learn how to manage multiple workspaces under a single organization on Latch

About

Learn how to add and transfer workspaces to an organization on Latch

Adding Workspaces to an Organization

Learn how to invite members to the organization and accept invitations

Inviting Members to an Organization

Learn about roles and permissions for members of an organization.

Organization Roles

Frequently Asked Questions

This Privacy Policy describes how Latch Bio, Inc. (“we”, “us”, “our”, or the “Company”) handles personal information that we collect through our digital properties that link to this Privacy Policy, including our website (collectively, the “Service”), as well as through social media, our marketing activities, and other activities described in this Privacy Policy.

Privacy Policy

Latch Privacy Policy

Terms of Service

Latch Terms of Service

Quick Start

In this demonstration, we will examine a workflow which sorts and assembles COVID sequencing data.

Authoring your Own Workflow

A workflow is an analysis that takes in some input, processes it in one or more steps and produces some output.

What is a Workflow?

When building a workflow with multiple tasks, it can be difficult to decide when to split larger tasks into smaller tasks. Some of the tradeoffs are listed below to guide this decision:

When to Split Up Tasks

When to split up a task into multiple tasks

Files and directories on Latch can be referred to in code or through the CLI using **Latch URLs**.

Latch URLs

Working with Remote Files

When working with bioinformatics workflows, we are often passing around large files or directories between our tasks. These files are usually located in cloud object stores and are copied to the file systems of the machines on which the task is scheduled.

LatchFile / LatchDir

Task executions produce logs, displayed on the Latch console to provide users visibility into their workflows. However, these logs tend to be terribly verbose. It's tedious to sift through piles of logs looking for useful signals; instead, important information, warnings, and errors should be prominently displayed. This is accomplished through the Latch SDK's new messaging feature.

Messages

When a workflow is executed and tasks are scheduled, the machines needed to run the task are provisioned automatically and managed for the user until task completion. Tasks can be annotated with the resources they are expected to consume (eg. CPU, RAM, GPU) at runtime and these requests will be fullfilled during the scheduling process.

Defining Cloud Resources

Caching allows workflow developers to reuse the results of previously run tasks to prevent wasted time and computation. This is helpful when running large batches of workflows with redundant inputs or when debugging errors in the middle of a workflow where upstream state can be reused.

Caching

In order to support the functionality of an `if-elif-else` clause within the body of a workflow, we introduce the method `create_conditional_section`. This method creates a new conditional section in a workflow, allowing a user to conditionally execute a task based on the value of a task result.

Conditional Sections

There are many pipelines in bioinformatics that require running a processing step in parallel and aggregating their outputs at the end for downstream analysis. A prominent example of this is bulk RNA-sequencing, where alignment is performed to produce transcript abundances per sample, and gene counts of all samples are subsequently merged. Having a single count matrix makes it convenient to use in downstream steps, such as differential gene expression analysis. Another example is performing FastQC on multiple samples and summarizing the results in a MultiQC report.

Map Task

Often a workflow can depend on _secret data_, such as an API key, to function correctly. To make storing and using secret data easy, the Latch SDK comes with special utilities that handle this securely.

Storing and Using Secrets

When developing a workflow, it's helpful to run the task functions before executing the entire workflow in the cloud to debug the environment and logical issues. Since tasks run in a different environment than your local computer (namely, the one defined in your Dockerfile(s)), there may be discrepancies when running your code locally.To address this, the Latch SDK comes with a command that allows you to run tasks and debug your environment without having to reregister your workflow and run it through the UI.

Development and Debugging

We will walk through a quick end-to-end flow for testing and debugging a variant calling workflow to demonstrate how to use latch develop.

Learning through An Example

It is often desirable to be able to access a shell from within a running task of a workflow, to debug a misbehaving program or inspect some files for example.

Remote Execution

Tutorial

Execution Profiles

Caching and Resuming

Private Registries

About Page

Publishing Results

Advanced Usage

Snakemake Workflow Cloud Compatibility

Caching Snakemake Tasks

In this guide, we will walk through how to upload a simple Snakemake workflow to Latch.

Debugging Snakemake

The following page outlines common problems with uploading Snakemake workflows and solutions.

Get Started

Python SDK

Nextflow

Snakemake

Automation

Registry

What is the Latch SDK?

Instant no-code interfaces for accessibility and publication

First class static typing

Containerization and versioning of every registered change

Reliable and scalable managed cloud infrastructure

Single line definition of arbitrary resource requirements (eg. CPU, GPU) for serverless execution

Get started with your first workflow

Python SDK

Snakemake Integration

Get Started

Python SDK

Nextflow

Snakemake

Automation

Registry

Instant no-code interfaces for accessibility and publication

First class static typing

Containerization and versioning of every registered change

Reliable and scalable managed cloud infrastructure

Single line definition of arbitrary resource requirements (eg. CPU, GPU) for serverless execution

​Get started with your first workflow

Python SDK

Snakemake Integration

Get started with your first workflow