Table Of Contents
Table Of Contents

2023-08-10 - Error Message Parsing - Take 2

Release

Status: Available

Type: Bug-Squash

Date: 2023-08-10

Problem

Jobs with less than 10 errors were showing up as failed, instead of completed with warnings. This meant users were being incorrectly notified the job had failed and the load pipelines were stopping.

Solution

Error parsing logic was updated to let the pipeline continue and return a data contract ‘warning’ notification to the user, and show a job status of ‘warning’ in the event history (although its technically a success)

Leverage the Magic

Update the existing check_job_statsus() pattern which checks as jobs finish, notifies the user and starts the next job in the pipeline

ADI

Smashed ! I love it when annoying bugs get squashed.

Magician Partner

We have a set a limit of 10 errors before a BQ job fails. But, when we had 1 or 2 errors thrown by a job, the job was being marked as failed and the pipeline was stopped and the user notified that the load had failed.

Error parsing logic was updated to let the pipeline continue and return a data contract ‘warning’ notification to the user, and show a job status of ‘warning’ in the event history (although its technically a success)

Last Refreshed

Doc Refreshed: 2024-05-20