Table Of Contents
Table Of Contents

2023-05-22 - Filedrop Header Parser Iteration

Release

Status: Available

Type: DataOps

Date: 2023-05-22

Problem

Filedrop files containing multiple windows character underscores in the file header are being picked up as delimiters.

Solution

Update the file header parser to exclude character 137 from the allowed list of delimiters

Leverage the Magic

A magical feature, no action by Data Magicians required.

ADI

Nailed it!, the filedrop capability really can eat most files our Data Magicaians find in the wild.

Customer

Voilà!, the filedrop feature just seems load any file we upload like magic.

Magician Partner

Census files use _ as a padding character in the header row, there are so many that the file parser assumes they are the delimiter for the file. Add underscore to the exclusion list to prevent this. Character code 137 (decimal) or 89 (hexadecimal) is the underscore character (_) in the Windows-1252 and ISO 8859-1 encoding standards. In other character encodings, such as UTF-8 or Unicode, character code 137 does not represent a specific character.

Last Refreshed

Doc Refreshed: 2024-03-02