Table Of Contents
Table Of Contents

2023-05-22 - Version Dropped Files Automatically

Problem

When Data Magicians drop multiple files with no version or timestamp in the name the files end up being overwriten in the archived bucket (because they have the same name).

When Data Magicians drop an initial load and backfill load files at the same time there is no pattern to safely ‘rack and stack’ them into the same landing tile because the file name was used to name the landing tile, and if multiple versioned files were dropped they would end up in different target landing tiles.

Solution

Check for and append the date the file was droppped to the dropped file name, will automatically provide file versioning for landing and history data, and versioning of archived files.

Leverage the Magic

A magical feature, no action by Data Magicians required.

ADI

Eureka!, no more file drop errors when multiple files are dropped at the same time.

Customer

Jolly good!, I can drop files until my heart is content and AgileData will magically make my data turn up.