.. _2026-01-10-duplicate-file-detection-improvement_release:

=================================================================================
2026-01-10 - Duplicate File Detection Improvement
=================================================================================

Release
==================
AgileData.io - Bug Fix: Smarter Duplicate File Detection

We improved our ability to detect duplicate files in your file drops, preventing accidental reprocessing of the same data.

**What was happening:**

* Files with the same content but different names could be processed multiple times
* Checksum-based detection wasn't catching all duplicate scenarios

**What we did:**

* Enhanced duplicate detection to compare actual file bytes and size
* Added checks for files with same size and target but different names
* Improved logic to identify truly identical files regardless of naming

**What this means for you:**

* Prevents accidentally processing the same data multiple times even if filenames differ
* More reliable data loading with intelligent duplicate detection
* Reduced risk of data duplication in your pipelines

Last Refreshed
===========================
*Doc Refreshed: 2026-01-10*