Table Of Contents
Table Of Contents

2023-07-18 - Update File Header Parser

Release

Status: Available

Type: DataOps

Date: 2023-07-18

Problem

Update the file header parser for two more scenarios (both generated from csv exports from excel)

  1. trailing commas in header because of empty columns

  2. invalid column names eg code.description

Solution

Strip columns that don’t have a name from the schema generator Check for invalid column names - eg code.description, and replace characters with an underscore > code_description

Leverage the Magic

Update the existing file header parser and build in more checks

Last Refreshed

Doc Refreshed: 2024-05-20