Change Rules¶
Change Rules is the logic we create to change the way data behaves. Change Rules are created using the standard AgileData Natural Language Rules format.
Why Change Data¶
Often data in the Systems Of Record is not stored in a way that enables it to be easily used for making business decisions.
To make is easier for the user to access and reuse this data we change the way data is stored in AgileData.
Don’t confuse AgileData’s Change Rules with common-data-warehouse-terms-term:change-data-capture-cdc, which is a pattern commonly used in the data warehouse domain.
Reasons to Change Data¶
There are a number of examples where changing data adds value:
Changing Data Structures;
Creating a Single View
Creating Golden Records
Cleansing data;
Inferring data;
Calculating new data
Changing Data Structures¶
Systems Of Record are configured to store data in a way that makes it quick and easy for the user to enter the data, and efficient for the underlying data storage technology (database) to store that data.
The downside of this approach is that accessing and querying this data requires the user to join multiple pieces of data (tables and fields) together to view the data in a way that is useful and that makes sense.
In AgileData we use Business Events, Business Concepts and Business Details as a way of representing data to match the way the users view the organisations core business processes.
This requires you to change the way data is stored in the systems-of-record to being stored in a way that matches these core business events.
Luckily this is made easy in AgileData.
Creating a Single View¶
Creating Golden Records¶
Cleansing Data¶
Sometimes the Systems Of Record allow users to enter data that does not meet our expected business context. We need to change this data to meet our expectations.
For example they may enter Wellington, Wlg and Welly to represent the city of Wellington in New Zealand. When using this data the users may expect these to be automatically grouped together as the Business Context of these three examples is the same.
Typical terms you may hear for this are:
Standardising data;
Formatting data;
Standardising Data¶
This is where we change data to ensure like data aligns.
This may include:
Standardising
Standardising all date formats and time zones to be the same (where appropriate);
Standardising addresses to be of the same format;
Removing start and end white spaces;
Splitting data out into multiple columns
Formating Data¶
This is where we change the way data is stored or viewed, but don’t change the actual data.
This may include:
Rounding numbers where appropriate to fewer decimal places;
Changing text to alter capitalisation;
Changing text to become a number so that you BI/AI tool can automatically sum or average the numbers.
Parsing data from a semi-structured format such as JSON and storing it in a structured format (Concepts, Details or Events)
Inferring Data¶
We often change data to infer something that the data does not physically store. For example movement of a service desk ticket from a underway queue to a done queue could infer that the ticket status has changed.
Calculating Additional Measures¶
We often change data to calculate new data. For example subtracting expenditure from revenue to store a value for profit.