Releases - DataOps¶
The AgileData DataOps contains features that are automate the technical work required to operate the Data Platform.
The AgileData team are constantly adding more features to the AgileData Data Platform which results in less effort and cost for the AgileData team and in better performance for Data Magicians.
As we release each new DataOps feature we document what it does, using the Amazon Press Release pattern.
- 2024-11-22 - Websockets - Join Validation
- 2024-11-22 - History layer - joins for late arriving records
- 2024-11-22 - Autogenerated Activity Consumes
- 2024-11-22 - Ask ADI - Update from Bison to Gemini
- 2024-11-22 - Activity Manifest
- 2024-10-23 - Handling deleted records from source
- 2024-10-23 - Filedrop queueing for large volumes of files - take 2
- 2024-10-23 - Event ensembles - late arriving details
- 2024-09-30 - Spanner Database Optimisation
- 2024-09-30 - File Header Parser - invalid delimiters
- 2024-08-15 - Processing Enhancements
- 2024-08-15 - Housekeeping of achived tiles
- 2024-08-15 - File Name Tidy
- 2024-08-15 - Bucket Sync
- 2024-07-13 - s3 integration
- 2024-05-13 - sla misses - take 2
- 2024-05-13 - Rebuild all downstream tiles (admin feature)
- 2024-05-13 - llm response validation and feedback
- 2024-02-28 - Partner Metrics
- 2024-02-28 - New template for shared google sheets
- 2024-02-28 - Landing to history for intraday change data
- 2024-02-28 - Landing to History - allow users to override column list
- 2024-02-28 - Initial source data guess for rule screen
- 2024-02-28 - History to Event - Take 2 (new template for change data)
- 2024-02-28 - History to Detail - Take 2 (new template for change data)
- 2024-01-16 - Spanner workload - ensemble config checking
- 2024-01-16 - Data Quality Jobs Optimisation - Take 2
- 2023-12-06 - NodeJS and Uvicorn Async Workers
- 2023-11-10 - Refine identification of Change vs Event data sources
- 2023-11-10 - Cache schedule endpoint for performance
- 2023-10-27 - User Policies and Table Policies
- 2023-10-27 - Spanner Housekeeping
- 2023-10-27 - Row Level Security
- 2023-10-27 - Archiving View Tiles
- 2023-09-18 - Row Level Security for Views
- 2023-09-18 - New system trust rule - future effective date
- 2023-09-18 - Converting Spanner payloads to BQ format payloads
- 2023-08-10 - Upgrade Python for websockets, apis and cloud functions
- 2023-08-10 - Svelte to SvelteKit Upgrade
- 2023-08-10 - LogRocket for SvelteKit
- 2023-07-18 - Update File Header Parser
- 2023-07-18 - New Partner config table in Spanner
- 2023-07-18 - Coalesce user defined event keys
- 2023-05-22 - Version Dropped Files Automatically
- 2023-05-22 - Rule Templates for Given Tiles Iteration
- 2023-05-22 - Filedrop Header Parser Iteration
- 2023-05-22 - Centralised Storage of Commoin Change Rule Export Files
- 2023-05-22 - Blast Radius
- 2023-05-22 - Autogenerated History Rules Iteration
- 2023-05-22 - Add AgileData User Agent to BQ and GCS calls
- 2023-04-11 - Split Tenancy Features into a Separate Config Table
- 2023-04-11 - Share bigquery-public datasets in other Tenancies
- 2023-04-11 - Persist Error and Load graphs on notification messages
- 2023-04-11 - Notifications Endpoint Iteration
- 2023-04-11 - Multi File Drop File Queuing
- 2023-04-11 - Move Notification Logging
- 2023-04-11 - Cross Tenancy AutoSync for Shared Consume Tiles
- 2023-03-27 - View Data Profiles on Shared Tiles
- 2023-03-27 - Tag Notifications
- 2023-03-27 - Orchestration Iteration
- 2023-03-27 - Multiple Tile Sources Iteration
- 2023-03-27 - Iterate UNION Rule Step Template to Support Shared Tiles
- 2023-03-27 - Impact Assessment (Blast Radius) for Upstream Config
- 2023-03-27 - Geolocation as a Tenancy Deployment Feature
- 2023-03-27 - Enable Secure Communication Across Tenancies
- 2023-02-21 - Move Data Trust Rules Execution into Core
- 2023-02-21 - Consumer Last Access Date
- 2022-12-14 - JSON New Line Auto Converter for Uploaded Files