2023-05-22 - Version Dropped Files Automatically¶

Problem¶

When Data Magicians drop multiple files with no version or timestamp in the name the files end up being overwriten in the archived bucket (because they have the same name).

When Data Magicians drop an initial load and backfill load files at the same time there is no pattern to safely ‘rack and stack’ them into the same landing tile because the file name was used to name the landing tile, and if multiple versioned files were dropped they would end up in different target landing tiles.

Solution¶

Check for and append the date the file was droppped to the dropped file name, will automatically provide file versioning for landing and history data, and versioning of archived files.

Leverage the Magic¶

A magical feature, no action by Data Magicians required.

ADI¶

Eureka!, no more file drop errors when multiple files are dropped at the same time.

Customer¶

Jolly good!, I can drop files until my heart is content and AgileData will magically make my data turn up.