2023-05-22 - Rule Templates for Given Tiles Iteration¶
Problem¶
Creating rules which use a shared tile for the Given doesn’t always work because of missing expected columns like effective date, or row key etc.
Solution¶
Use a simple fall back pattern … if effective_date is in the Given tile then use it, if not check for user effective date rule and use that, else fall back to current_date() as the effective date.
Check for expiry_date in the Given tile, same pattern as above.
Do the same for consume tiles which are used for the Given.
Leverage the Magic¶
A magical feature, no action by Data Magicians required.
ADI¶
Woohoo!, love it when we reuse patterns to provide more simplciity for the Data Magician.
Customer¶
Yippee!, now I can use any tile as a Given tile in a rule.
Magician Partner¶
Historically we only had patterns for shared_history and shared_analytics, everything else that was from a shared source wasn’t specifically catered for, becasue … Our patterns all rely on a number of attributes to work - effective_date, expiry_date, unique_record_key, source_partition. When a source doesn’t have one or more of these attributes, either the user needs to supply it or we need to infer it.. Currently the templates are using a mixture of these approaches, they need to be rebuilt where every shared_ input is automatically checked for what it does or does not contain, then use what the user has passed in the rules to decide whether to infer or use the users inputs.
Last Refreshed¶
Doc Refreshed: 2024-03-02