2023-09-18 - Consume - Archive Tiles¶
Problem¶
When a destructive Consume rebuild happens it breaks any dependent Info Apps, for example Looker Studio reports.
Depending on the last mile tool, when the Consume tile schema changes, the report breaks and there is no way to recover the widget definition or the metrics defined in the report.
If you have not exported a copy of the the report definition, and you can’t remember how each widget or metric was defined, you cant rebuild it.
Destructive Consume Tile rebuilds happen regular;y, when magicians want to update a tiles schema or make changes to a rule that rebuilds upstream tiles.
We need a way to preserve an archive copy of the affected Consume Tiles and their data for reference.
Solution¶
When a user (or automagic process) deletes a landing or consume tile, automatically create an archive copy of the tile and the underlying physical table before we drop and rebuild it.
At the same time, if the tile we are deleting is a Consume also create an archive consume view pointing to the archived data so users can still reference the archived data in the last mile tool, such as Looker Studio.
Leverage the Magic¶
You will see new Archive Tiles appear in the Catalog, when a destructive change has been invoked in the AgileData Platform.
When you view the Consume tables in BigQuery via your last mile tool, you will see a copy of the consume tile with a date appended to the end of it. this tile/table is a copy of the Consume Tile, before the destructive rebuild was done.
Point your original report at it, or clone your original report and point it at this archived tile/table.
The archived tile/table will not be updated with any new data collection.
And when the archived tile.table will be automatically deleted after a period of time.
ADI¶
Cowabunga!
Good starter for 10 to manage the impact of the destructive rule changes, but think we will end up iterating this pattern a bit more before it’s truely magical.
Customer¶
Kudos!
That process will help me manage my dashboards, reports, data services and models when a destructive change has happened.
Last Refreshed¶
Doc Refreshed: 2024-03-02