Skip to content

Add a sanity check for duplicate file actions in commitImpl - #6892

Merged
murali-db merged 1 commit into
delta-io:masterfrom
dhruvarya-db:oss-duplicate-addfile-check
May 27, 2026
Merged

Add a sanity check for duplicate file actions in commitImpl#6892
murali-db merged 1 commit into
delta-io:masterfrom
dhruvarya-db:oss-duplicate-addfile-check

Conversation

@dhruvarya-db

@dhruvarya-db dhruvarya-db commented May 27, 2026

Copy link
Copy Markdown
Collaborator

What changes were proposed in this pull request?

This PR adds a sanity check to the commit path that rejects a commit containing duplicate file actions for the same (path, deletion vector id).commitImpl previously did not validate that the prepared actions were free of duplicates. This change adds that validation:

  • A new checkNoDuplicateActions helper on the ConflictChecker companion object that streams the actions in a single pass and throws if it finds the same file added twice, the same file removed twice, or the same path + deletion vector both added and removed within one commit.
  • A new error class DELTA_DUPLICATE_ACTIONS_FOUND and the corresponding DeltaErrors.duplicateActionCheckFailed helper.
  • A call site in commitImpl, guarded by the DELTA_DUPLICATE_ACTION_CHECK_ENABLED configuration, that runs the check on the prepared actions before a commit is written.
  • Re-validation after conflict resolution: ConflictChecker.checkConflictsAndValidateActions re-runs the duplicate check whenever conflict resolution changes the current transaction's actions.

How was this patch tested?

Added 10 tests to OptimisticTransactionSuiteBase covering duplicate add, duplicate remove, and add/remove overlap on the same path, including deletion-vector variants (same DV, DV vs. no DV, and different DVs). They run in every concrete suite that mixes in the base trait.

Does this PR introduce any user-facing changes?

No.

@murali-db
murali-db merged commit 8849639 into delta-io:master May 27, 2026
40 checks passed
@felipepessoto

Copy link
Copy Markdown
Contributor

@dhruvarya-db @prakharjain09 @murali-db I think it would be possible to be even more strict and disallow duplicated path regardless of DV:

Moreover, for simplicity it is required that there is at most one file action of the same type for any path (regardless of dvId)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants