Working with Remote Files
The Latch SDK provides a convenient means of referencing files or directories hosted on Latch within task functions.
Downloading Files
To download files locally:
from latch.ldata.path import LPath
if __name__ == '__main__':
latch_path = LPath("latch:///welcome/deseq2/design.csv")
local_path = latch_path.download()
print(local_path.read_text())
To download files to a pre-determined local path:
from latch.ldata.path import LPath
from pathlib import Path
if __name__ == '__main__':
local_path = Path("./test.txt")
latch_path = LPath("latch:///welcome/deseq2/design.csv")
latch_path.download(local_path)
print(local_path.read_text())
Uploading Files
To upload files from a local path to a Latch path:
from latch.ldata.path import LPath
if __name__ == '__main__':
latch_path = LPath("latch:///welcome/deseq2/design.csv")
latch_path.upload_from("./test.txt")
print(local_path.read_text())
To upload to a remote path, the parent directory of the
remote path must already exist. To ensure that the parent directory
exists, use LPath("latch:///parent_directory").mkdirp()
.
Copying Between Latch Paths
Copy data from one Latch path to another:
from latch.ldata.path import LPath
src_path = LPath("latch:///welcome/deseq2/design.csv")
dst_path = LPath("latch:///design_copy.csv")
src_path.copy_to(dst_path)
Deleting Latch Files
Recursively remove a remote file or directory:
from latch.ldata.path import LPath
latch_path = LPath("latch:///welcome/deseq2/design.csv")
latch_path.rmr()
Fetching File Metadata
Use the following methods to fetch metadata of your remote files without downloading the file first:
node_id()
name()
content_type()
size()
LPath objects will cache metadata information for the lifetime of the object
after any LPath
method is invoked. To re-populate the cache, use the
LPath().fetch_metadata()
method. For example:
from latch.ldata.path import LPath
latch_path = LPath('latch:///dir/file.txt')
name = latch_path.name() # metadata is fetched, name = "file.txt"
# rename the file in the Latch Console to file2.txt
latch_path.name() # returns "file.txt"
latch_path.fetch_metadata()
latch_path.name() # returns "file2.txt"
Working with Directories
LPath also provides the methods for working with remote directories. For example:
from latch.ldata.path import LPath
latch_path = LPath('latch:///welcome')
latch_path.is_dir() # returns True
paths = latch_path.iterdir() # returns an Iterator; does not traverse nested directories
next(paths) # returns LPath(path='latch:///welcome/CRISPResso2')
latch_path.size_recursive() # returns 10467327039
The size_recursive
method can be very slow on large directories and should only be used
if absolutely necessary
Was this page helpful?