Skip to main content
Available in latch >= 2.65.7
Workflows on Latch run inside a container built from your Dockerfile. This environment can differ significantly from your local machine (different OS, Python version, system packages, or installed dependencies). These differences can cause issues that don’t appear during local testing. To avoid surprises after registration, it’s best to debug inside the exact environment your workflow will use. The latch develop command lets you build your workflow image and drop into an interactive shell within that container. From there, you can:
  • Run individual tasks or workflows directly
  • Check installed packages and dependencies
  • Test imports, data access, and other environment-specific behavior
This approach ensures that the code you test is running in the same conditions it will see when executed on Latch, helping you catch both environment and logic issues before running the workflow end-to-end.

​
Setup

  • Install latch version 2.65.7 or later.
  • Start a development shell for your workflow by using the following commands:
$ cd test-wf # Navigate to your workflow directory
$ latch register --staging . # The `--staging` flag builds image without releasing a new version to Latch Console
$ latch develop . # Open interactive shell inside the workflow environment
This opens an interactive shell on a remote instance running the environment built from your Dockerfile.

​
Example Usage

On your local machine, create test scripts first, then run them from within the latch develop interactive shell to validate your workflow. Below is a sample test script for the nf-core/atac-seq workflow. This script runs nf_nf_core_atacseq function with multiple input samples stored in public S3 buckets:
nf-core/atacseq Test Example
from wf.__init__ import *

nf_nf_core_atacseq(
    input=[
        SampleSheet(
            sample="Sample_6",
            fastq_1=LatchFile("s3://latch-public/test-data/35929/Test_Dataset_Verified/Sample_6/Sample_6.Rep_1.R1.fastq.gz"),
            fastq_2=LatchFile("s3://latch-public/test-data/35929/Test_Dataset_Verified/Sample_6/Sample_6.Rep_1.R2.fastq.gz"),
            replicate=1,
        ),
        SampleSheet(
            sample="Sample_6",
            fastq_1=LatchFile("s3://latch-public/test-data/35929/Test_Dataset_Verified/Sample_6/Sample_6.Rep_2.R1.fastq.gz"),
            fastq_2=LatchFile("s3://latch-public/test-data/35929/Test_Dataset_Verified/Sample_6/Sample_6.Rep_2.R2.fastq.gz"),
            replicate=2,
        ),
        SampleSheet(
            sample="Sample_1",
            fastq_1=LatchFile("s3://latch-public/test-data/35929/Test_Dataset_Verified/Sample_1/Sample_1.Rep_1.R1.fastq.gz"),
            fastq_2=LatchFile("s3://latch-public/test-data/35929/Test_Dataset_Verified/Sample_1/Sample_1.Rep_1.R2.fastq.gz"),
            replicate=1,
        ),
        SampleSheet(
            sample="Sample_1",
            fastq_1=LatchFile("s3://latch-public/test-data/35929/Test_Dataset_Verified/Sample_1/Sample_1.Rep_2.R1.fastq.gz"),
            fastq_2=LatchFile("s3://latch-public/test-data/35929/Test_Dataset_Verified/Sample_1/Sample_1.Rep_2.R2.fastq.gz"),
            replicate=2,
        ),
    ],
    genome_source="latch_genome_source",
    run_name="Test-1",
    read_length=50,
    aligner=Aligner.bowtie2,
    fasta=None,
    gtf=None,
    outdir=LatchOutputDir("latch:///ATAC_Seq_Test"),
    latch_genome=Reference.hg19,
    with_control=False,
    skip_trimming=False,
    skip_qc=False,
)
Inside latch-develop shell
python3 ~/atacseq/tests/main.py
See the full GitHub repository here.
If you are using local files while testing with latch develop, the workflow function signature still requires a LatchFile object. Pass the local file path to LatchFile, for example LatchFile("/root/path/to/local_test_file.fastq.gz")

​
latch develop Sync Behavior

  • latch develop syncs files from your local workflow directory using rsync. Files outside this directory are not synced. If you need test data, place it in a subfolder inside your workflow directory.
  • Updated or new local files overwrite matching files in the container. All code changes must be made locally. Edits made inside the latch develop container are not saved back to your machine and may be overwritten during sync.
  • Deleted local files are not removed from the container.
  • If you change your Dockerfile, run latch register --staging to rebuild the image, then run latch develop again to enter the updated environment.

​
Advanced Notes for Nextflow Users

In a normal Latch production run, the initialize task provisions a shared POSIX filesystem volume (OFS) and returns an identifier (usually passed as pvc_name) so downstream Nextflow tasks can read and write to a common mount. This works because the task is running inside Latch’s execution environment, which has access to the internal dispatcher service that provisions storage and Kubernetes infrastructure to mount the volumeWhen you run latch develop, none of those services exist — you’re just inside an interactive shell in the workflow container. As a result, any attempt to call initialize will fail.The solution is to either run the runtime_task directly using "" as the value of pvc_name or adding a conditional if check in the initialize task as below:
@custom_task(...)
def initialize(run_name: str) -> str:
    if os.environ.get("LATCH_NF_DEBUG") is not None:
        return ""

    ...
Full GitHub repository example here.
In your latch.config, set the executor to local in debug mode so tasks can run without k8s. This ensures you can still test task logic inside latch develop without depending on infrastructure that only exists in production.
    process {
    executor = 'k8s'
}

if (System.getenv("LATCH_NF_DEBUG") == "true") {
    process {
        executor = 'local'
    }
}
Full GitHub repository example here
In the workflow’s Dockerfile, ensure that you are using either latch-base-nextflow versions >= v3.0.5 or >= v2.5.6 (See example)
When running tasks, the Python interpreter must have the latch SDK. If your container uses another environment (e.g., mamba/conda), it may:
  • Not include latch package
  • Override $PATH so the wrong Python is used
Best practice:
  • Use /usr/local/bin/python3 (default in Latch base images)
  • Install extra packages into this env instead of creating a separate one
  • Check with:
which python3
Make sure that your process config requests fewer resources than normal. This is to avoid running into issues where processes won’t run because they request more resources than are available in the latch develop’s instance.
If necessary, modify the -XX:ActiveProcessorCount in your Nextflow command in the entrypoint.py file to cap Java’s perceived cores.(See example here)

​
Choosing Instance Size

This feature is available in latch versions 2.59.0 and later.
Use the --instance-size flag to run latch develop on a specific instance type. This is useful when debugging code that depends on certain hardware specs. Example: 
$ latch develop . --instance-size small_gpu_task
runs on an instance with 1 NVIDIA T4 GPU. Supported instance sizes:
Instance SizeCoresRAMGPU
small_task24 GiB–
medium_task30100 GiB–
large_task90170 GiB–
small_gpu_task730 GiB1× NVIDIA T4
large_gpu_task63245 GiB1× NVIDIA A10G
v100_x1_task748 GiB1× NVIDIA V100
g6e_xlarge_task432 GiB1× NVIDIA L40S
Note: Larger instances may take 5+ minutes to start.