> ## Documentation Index
> Fetch the complete documentation index at: https://wiki.latch.bio/llms.txt
> Use this file to discover all available pages before exploring further.

# Development and Debugging

> Run and debug workflow tasks before registration

<Note>Available in `latch >= 2.65.7`</Note>

Workflows on Latch run inside a container built from your Dockerfile. This environment can differ significantly from your local machine (different OS, Python version, system packages, or installed dependencies). These differences can cause issues that don't appear during local testing.

To avoid surprises after registration, it's best to debug inside the exact environment your workflow will use.

The `latch develop` command lets you build your workflow image and drop into an interactive shell within that container. From there, you can:

* Run individual tasks or workflows directly
* Check installed packages and dependencies
* Test imports, data access, and other environment-specific behavior

This approach ensures that the code you test is running in the same conditions it will see when executed on Latch, helping you catch both environment and logic issues before running the workflow end-to-end.

## Setup

* Install `latch` version `2.65.7` or later.
* Start a development shell for your workflow by using the following commands:

```console theme={null}
$ cd test-wf # Navigate to your workflow directory
$ latch register --staging . # The `--staging` flag builds image without releasing a new version to Latch Console
$ latch develop . # Open interactive shell inside the workflow environment
```

This opens an interactive shell on a remote instance running the environment built from your Dockerfile.

## Example Usage

On your local machine, create test scripts first, then run them from within the `latch develop` interactive shell to validate your workflow.

Below is a sample test script for the `nf-core/atac-seq` workflow. This script runs nf\_nf\_core\_atacseq function with multiple input samples stored in public S3 buckets:

```python nf-core/atacseq Test Example  expandable theme={null}
from wf.__init__ import *

nf_nf_core_atacseq(
    input=[
        SampleSheet(
            sample="Sample_6",
            fastq_1=LatchFile("s3://latch-public/test-data/35929/Test_Dataset_Verified/Sample_6/Sample_6.Rep_1.R1.fastq.gz"),
            fastq_2=LatchFile("s3://latch-public/test-data/35929/Test_Dataset_Verified/Sample_6/Sample_6.Rep_1.R2.fastq.gz"),
            replicate=1,
        ),
        SampleSheet(
            sample="Sample_6",
            fastq_1=LatchFile("s3://latch-public/test-data/35929/Test_Dataset_Verified/Sample_6/Sample_6.Rep_2.R1.fastq.gz"),
            fastq_2=LatchFile("s3://latch-public/test-data/35929/Test_Dataset_Verified/Sample_6/Sample_6.Rep_2.R2.fastq.gz"),
            replicate=2,
        ),
        SampleSheet(
            sample="Sample_1",
            fastq_1=LatchFile("s3://latch-public/test-data/35929/Test_Dataset_Verified/Sample_1/Sample_1.Rep_1.R1.fastq.gz"),
            fastq_2=LatchFile("s3://latch-public/test-data/35929/Test_Dataset_Verified/Sample_1/Sample_1.Rep_1.R2.fastq.gz"),
            replicate=1,
        ),
        SampleSheet(
            sample="Sample_1",
            fastq_1=LatchFile("s3://latch-public/test-data/35929/Test_Dataset_Verified/Sample_1/Sample_1.Rep_2.R1.fastq.gz"),
            fastq_2=LatchFile("s3://latch-public/test-data/35929/Test_Dataset_Verified/Sample_1/Sample_1.Rep_2.R2.fastq.gz"),
            replicate=2,
        ),
    ],
    genome_source="latch_genome_source",
    run_name="Test-1",
    read_length=50,
    aligner=Aligner.bowtie2,
    fasta=None,
    gtf=None,
    outdir=LatchOutputDir("latch:///ATAC_Seq_Test"),
    latch_genome=Reference.hg19,
    with_control=False,
    skip_trimming=False,
    skip_qc=False,
)
```

```bash Inside latch-develop shell theme={null}
python3 ~/atacseq/tests/main.py
```

See the full GitHub repository [here](https://github.com/latchbio-nfcore/atacseq/tree/harihara-testscases/tests).

<Tip>
  If you are using local files while testing with latch develop, the workflow function signature still requires a LatchFile object. Pass the local file path to LatchFile, for example `LatchFile("/root/path/to/local_test_file.fastq.gz")`
</Tip>

## `latch develop` Sync Behavior

* `latch develop` syncs files from your local workflow directory using rsync. Files outside this directory are not synced. If you need test data, place it in a subfolder inside your workflow directory.
* Updated or new local files overwrite matching files in the container. All code changes must be made locally. Edits made inside the latch develop container are not saved back to your machine and may be overwritten during sync.
* Deleted local files are not removed from the container.
* If you change your Dockerfile, run `latch register --staging` to rebuild the image, then run `latch develop` again to enter the updated environment.

## Advanced Notes for Nextflow Users

<AccordionGroup>
  <Accordion title="1. The `initialize` task will always fail in `latch develop`">
    In a normal Latch production run, the `initialize` task provisions a shared POSIX filesystem volume (OFS) and returns an identifier (usually passed as `pvc_name`) so downstream Nextflow tasks can read and write to a common mount. This works because the task is running inside Latch's execution environment, which has access to the internal dispatcher service that provisions storage and Kubernetes infrastructure to mount the volume

    When you run `latch develop`, none of those services exist — you're just inside an interactive shell in the workflow container. As a result, any attempt to call initialize will fail.

    The solution is to either run the `runtime_task` directly using `""` as the value of `pvc_name` or adding a conditional `if` check in the `initialize` task as below:

    ```python theme={null}
    @custom_task(...)
    def initialize(run_name: str) -> str:
        if os.environ.get("LATCH_NF_DEBUG") is not None:
            return ""

        ...
    ```

    Full GitHub repository example [here](https://github.com/latchbio-nfcore/atacseq/blob/37c28a0c62d2463eec5f8038fc251efc4ba58e24/wf/entrypoint.py#L70).
  </Accordion>

  <Accordion title="2. Update `latch.config` default executor">
    In your `latch.config`, set the executor to `local` in debug mode so tasks can run without k8s. This ensures you can still test task logic inside `latch develop` without depending on infrastructure that only exists in production.

    ```latch.config theme={null}
        process {
        executor = 'k8s'
    }

    if (System.getenv("LATCH_NF_DEBUG") == "true") {
        process {
            executor = 'local'
        }
    }
    ```

    Full GitHub repository example [here](https://github.com/latchbio-nfcore/atacseq/blob/37c28a0c62d2463eec5f8038fc251efc4ba58e24/latch.config#L5)
  </Accordion>

  <Accordion title="3. Use a compatible Nextflow base image">
    In the workflow's Dockerfile, ensure that you are using either `latch-base-nextflow` versions `>= v3.0.5`  or `>= v2.5.6` (See [example](https://github.com/latchbio-nfcore/atacseq/blob/37c28a0c62d2463eec5f8038fc251efc4ba58e24/Dockerfile#L2))
  </Accordion>

  <Accordion title="4. Use the Python environment with Latch installed">
    When running tasks, the Python interpreter must have the latch SDK. If your container uses another environment (e.g., mamba/conda), it may:

    * Not include `latch` package
    * Override \$PATH so the wrong Python is used

    **Best practice:**

    * Use `/usr/local/bin/python3` (default in Latch base images)
    * Install extra packages into this env instead of creating a separate one
    * Check with:

    ```bash theme={null}
    which python3
    ```
  </Accordion>

  <Accordion title="5. Reduce resource requests">
    Make sure that your process config requests fewer resources than normal. This is to avoid running into issues where processes won't run because they request more resources than are available in the `latch develop`'s instance.
  </Accordion>

  <Accordion title="6. Optionally, lower Java CPU usage">
    If necessary, modify the `-XX:ActiveProcessorCount` in your Nextflow command in the `entrypoint.py` file to cap Java's perceived cores.(See example [here](https://github.com/latchbio-nfcore/atacseq/blob/37c28a0c62d2463eec5f8038fc251efc4ba58e24/wf/entrypoint.py#L316))
  </Accordion>
</AccordionGroup>

## Choosing Instance Size

<Tip>This feature is available in `latch` versions `2.59.0` and later.</Tip>

Use the `--instance-size` flag to run latch develop on a specific instance type. This is useful when debugging code that depends on certain hardware specs.

Example:

```console theme={null}
$ latch develop . --instance-size small_gpu_task
```

runs on an instance with 1 NVIDIA T4 GPU.

**Supported instance sizes:**

| Instance Size     | Cores | RAM     | GPU            |
| ----------------- | ----- | ------- | -------------- |
| `small_task`      | 2     | 4 GiB   | –              |
| `medium_task`     | 30    | 100 GiB | –              |
| `large_task`      | 90    | 170 GiB | –              |
| `small_gpu_task`  | 7     | 30 GiB  | 1× NVIDIA T4   |
| `large_gpu_task`  | 63    | 245 GiB | 1× NVIDIA A10G |
| `v100_x1_task`    | 7     | 48 GiB  | 1× NVIDIA V100 |
| `g6e_xlarge_task` | 4     | 32 GiB  | 1× NVIDIA L40S |

Note: Larger instances may take 5+ minutes to start.