Table Of Contents
Table Of Contents

2023-10-27 - Spanner Housekeeping

Release

Status: Available

Type: DataOps

Date: 2023-10-27

Problem

Over time the number of redundant columns start to build up in our spanner backend database. These are usually a result of a attributes for a feature that is prototyped then maybe discarded, or persisted else where in the database

Solution

Remove the report_key attribute from the catalog table, this was originally for the information apps which have since been setup as standalone catalog objects. Drop the redundant reports table. Remove the clustering, partitioning and matched_on attributes from the config table. These attributes have been simplied into load_type (change_data, event_data) and load_pattern (insert, upsert, replace)

Leverage the Magic

Our amazing backend spanner database which we use to power our config driven architecture

ADI

Jolly good! keeping the tech debt in check and making sure the backend is continually optimised for speed

Last Refreshed

Doc Refreshed: 2024-05-20