2023-10-27 - Spanner Housekeeping¶

Release¶

Status: Available

Type: DataOps

Date: 2023-10-27

Problem¶

Over time the number of redundant columns start to build up in our spanner backend database. These are usually a result of a attributes for a feature that is prototyped then maybe discarded, or persisted else where in the database

Solution¶

Remove the report_key attribute from the catalog table, this was originally for the information apps which have since been setup as standalone catalog objects. Drop the redundant reports table. Remove the clustering, partitioning and matched_on attributes from the config table. These attributes have been simplied into load_type (change_data, event_data) and load_pattern (insert, upsert, replace)

Leverage the Magic¶

Our amazing backend spanner database which we use to power our config driven architecture

ADI¶

Jolly good! keeping the tech debt in check and making sure the backend is continually optimised for speed

Last Refreshed¶

Doc Refreshed: 2024-03-02