Remote Files - LatchBio

The Latch SDK provides a convenient means of referencing files or directories hosted on Latch within task functions.

Downloading Files

To download files locally:

from latch.ldata.path import LPath

if __name__ == '__main__':
  latch_path = LPath("latch:///welcome/deseq2/design.csv")
  local_path = latch_path.download()
  print(local_path.read_text())

To download files to a pre-determined local path:

from latch.ldata.path import LPath
from pathlib import Path

if __name__ == '__main__':
  local_path = Path("./test.txt")
  latch_path = LPath("latch:///welcome/deseq2/design.csv")
  latch_path.download(local_path)
  print(local_path.read_text())

Caching File Downloads

By default, the download method will not cache the file. To enable caching, pass cache=True.

from latch.ldata.path import LPath
from pathlib import Path

latch_path = LPath("latch:///welcome/deseq2/design.csv")
local_path = Path(f"{latch_path.node_id()}.csv") # Provides local destination path to download to
latch_path.download(local_path, cache=True) # Enable caching

If you do not provide a local destination path, the file will be downloaded to the default ~/.latch/ directory. Without caching enabled, this can result in multiple downloads of the same file, unnecessarily increasing storage usage. To avoid this, it is strongly recommended to set a destination path and enable caching, especially when using LPath in Latch Pods or Latch Plots.

Uploading Files

To upload files from a local path to a Latch path:

from latch.ldata.path import LPath

if __name__ == '__main__':
  latch_path = LPath("latch:///welcome/deseq2/design.csv")
  latch_path.upload_from("./test.txt")
  print(local_path.read_text())

To upload to a remote path, the parent directory of the remote path must already exist. To ensure that the parent directory exists, use LPath("latch:///parent_directory").mkdirp().

Copying Between Latch Paths

Copy data from one Latch path to another:

from latch.ldata.path import LPath

src_path = LPath("latch:///welcome/deseq2/design.csv")
dst_path = LPath("latch:///design_copy.csv")
src_path.copy_to(dst_path)

Deleting Latch Files

Recursively remove a remote file or directory:

from latch.ldata.path import LPath

latch_path = LPath("latch:///welcome/deseq2/design.csv")
latch_path.rmr()

Fetching File Metadata

Use the following methods to fetch metadata of your remote files without downloading the file first:

node_id()
name()
content_type()
size()

LPath objects will cache metadata information for the lifetime of the object after any LPath method is invoked. To re-populate the cache, use the LPath().fetch_metadata() method. For example:

from latch.ldata.path import LPath

latch_path = LPath('latch:///dir/file.txt')
name = latch_path.name() # metadata is fetched, name = "file.txt"
# rename the file in the Latch Console to file2.txt
latch_path.name() # returns "file.txt"
latch_path.fetch_metadata()
latch_path.name() # returns "file2.txt"

Working with Directories

LPath also provides the methods for working with remote directories. For example:

from latch.ldata.path import LPath

latch_path = LPath('latch:///welcome')
latch_path.is_dir() # returns True
paths = latch_path.iterdir() # returns an Iterator; does not traverse nested directories
next(paths) # returns LPath(path='latch:///welcome/CRISPResso2')
latch_path.size_recursive() # returns 10467327039

The size_recursive method can be very slow on large directories and should only be used if absolutely necessary.

​Downloading Files

​Caching File Downloads

​Uploading Files

​Copying Between Latch Paths

​Deleting Latch Files

​Fetching File Metadata

​Working with Directories

Downloading Files

Caching File Downloads

Uploading Files

Copying Between Latch Paths

Deleting Latch Files

Fetching File Metadata

Working with Directories