Table Objects

Table objects describe Registry Tables. A Table object can either be instantiated via a call to Project.list_tables() or directly using its ID.

Tables are for the most part lazy, in that they don’t perform any network requests without an explicit call to Table.load() or to a property getter. Two exceptions to this are Table.list_records() and Table.update(), both of which are discussed below.

​

Instance Methods

• Table.load(), if called, will perform a network request and cache values for each of the Table’s properties.
• Table.list_records() will return a generator that yields a paginated dictionary of Records that are present in the calling Table. The keys of this dictionary are Record IDs and the values are the corresponding Record objects. This function also takes an optional keyword-only page_size argument that dictates the size of the returned page. The value of this argument must be a postive integer. If not provided, the default is a page size of 100. Pages are ordered by Record ID, with lower IDs being yielded first.

An example of typical usage of Table.list_records() is below.

from latch.registry.table import Table

tbl = Table(id="1234")

for page in tbl.list_records():
    for record_id, record in page.items():
        # do stuff with the Record `record`.
        ...

Unlike the rest of this API, Table.list_records() will always perform a network request for each returned page.

​

Property Getters

All property getters have an optional load_if_missing boolean argument which, if True, will call Account.load() if the requested property has not been loaded already. This defaults to True.

• Table.get_display_name() will return the display_name of the calling Table as a string
• Table.get_columns() will return a dictionary containing the columns of the calling Table. The keys of the dictionary are column names, and its values are Column objects. Column is a convenience dataclass with the properties
  • Column.key: the key of the column.
  • Column.type: the (python) type of the column.
  • Column.upstream_type: similar to Column.type. However, this is a dataclass which contains an internal representation of the column’s data type, and should not be accessed or modified directly.

​

Updater

A Table can be modified by using the Table.update() function. Table.update() returns a context manager (and hence must be called using with syntax) with the following methods:

• upsert_record(record_name: str, column_data: Dict[str, Any]) will either (up)date or in(sert) a record with name record_name with the column values prescribed in column_data. Each key of column_data must be a valid column key (meaning that there must be a column in the calling Table with the same key), and the value corresponding to that key must be same type as the column (meaning that it is an instance of the column’s (python) type).
• delete_record(name: str) will delete the record with name name. If no such record exists, the method call will be a noop.
• upsert_column(key: str, type: RegistryPythonType, *, required: bool = False) will create a column with key key and type type. For now, updating column types (i.e. calling upsert_column with a key that already exists, and a type that differs from the type of the column) is not allowed, and attempting to do so will raise an exception.

​

Export to pandas

A Table can be exported as a pandas.DataFrame using the Table.get_dataframe() function. This requires pandas to be installed. Doing so will load the entire table from the network into memory, so it can be costly for larger tables.

>>> t = Table(id=345)
>>> t.get_dataframe()
   experiment_accession  ...  total_size  total_spots         Name
0           SRX17527395  ...   383793732     12300558  SRR21524988
1           SRX17527394  ...   414766909     13238380  SRR21524989
2           SRX17527393  ...   406432082     13029979  SRR21524990
3           SRX17527392  ...   445083594     14249689  SRR21524991
4           SRX17527391  ...   392081937     12545386  SRR21524992
5           SRX17527390  ...   438156525     14020607  SRR21524993
6           SRX17527389  ...   434284549     13945947  SRR21524994
7           SRX17527388  ...   450489579     14452049  SRR21524995
8           SRX17527387  ...   404992915     12876927  SRR21524996

[9 rows x 24 columns]

​

Planned Methods (Not Implemented Yet)

• delete_column(column_name: str)

The following is an example for how to update a Table.

from latch.registry.table import Table

t = Table(id="1234")

with t.update() as updater:
    updater.upsert_record(
        name="record 1",
        Size=10
    )
    updater.upsert_record(
        name="record 2",
        Size=15
    )

The code above will upsert two records, called record 1 and record 2, with the provided values for the column Size.

When using an updater, no network requests are made until the end of the with block. This mimics transactions in relational database systems, and has several similar behaviors, namely that

• If any exception is thrown inside the with block, none of the updates made inside the with block will be sent over the network, so no changes will be made to the Table.
• All updates are made at once in a single network request at the end of the with block. This significantly boosts performance when a large amount of updates are made at once.