2023-03-27 - Impact Assessment (Blast Radius) for Upstream Config¶
Problem¶
Changing a rule can affect other rules and have an unintended consequence.
Solution¶
Check and notify of the upstream impacts of rule changes to identify potential issues before the affected rules are run.
Leverage the Magic¶
Data Magicians get a notification.
A tag (update pending) is associated with rules that lets them see which rules have been marked for change.
ADI¶
Phenomenal!, Data Magicians are notified and marking impacted rules means we can check the rules before they are run and let Data Magicians know if there are going to be any down stream issues.
Customer¶
Hip-hip hooray!, it used to be very confusing when an existing rule suddenly failed because I had made a change else where. Now I get notified and can see which rules have been impacted.
Magician Partner¶
When a change rule is updated, the new blast radius pattern automatically checks for all the upstream config that uses that object and flags it with a status of ‘update pending’ in the gui and the backend config. A notification is also sent that upstream config will be updated when next executed. This is important so we can see the objects that are affected and are now in a pending state and may change in the future when rules are run next. Example - updating a chnage rule to introduce a new column into a detail table. Any consumes that use that detail are marked for update, and any rules that use those consumes are marked for updated - eg we may have built a metric rule or a union rule on an affected consume. When the the affected consume is next run, the pending flag means it will be checked and recreated if required. When the rule that uses that consume as a source is next run, the pending flag means it will be re-validated and the change rule updated automatically. If successful it will be loaded, but if the rule can not be validated, a notification is sent that the change rule needs user intervention and the rule will not run, this prevents upstream failures from occuring. The pattern is designed to prevent errors by keeping tighter control over changed and flagging config that needs re-validating at runtime. We can’t validate at the time the downstream rule has changed because its impact doesn’t occur until it runs, at that point changes are made in the database.
Last Refreshed¶
Doc Refreshed: 2024-03-02