fix(common): use ISO week/year in detect_datetime_format so week-date cursors round-trip - #4061
Merged
rudolfix merged 4 commits intoJun 19, 2026
Conversation
detect_datetime_format mapped "YYYY-Www" to "%Y-W%W". pendulum parses week strings as ISO 8601 week dates, but %W is the locale week-of-year keyed on the calendar year %Y, which disagrees with the ISO week number (%V) and ISO week-numbering year (%G) around year boundaries. This breaks incremental cursors with a lag: a string cursor is parsed, shifted, then re-serialized with the detected format. e.g. "2026-W01" parses to 2025-12-29 and was rendered back as "2025-W52", silently corrupting the saved cursor. "2020-W53" rendered as "2020-W52". Use the ISO directives %G-W%V and %G-W%V-%u so the detected format round-trips for every valid ISO week. Add a round-trip test in test_time.py and boundary cases to test_apply_lag_to_str_value.
yashs33244
force-pushed
the
fix/iso-week-detect-datetime-format
branch
from
June 15, 2026 11:48
a4af372 to
1ae1e69
Compare
Contributor
Author
|
The failing checks look unrelated to this change, which only touches
I've rebased onto the latest |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
detect_datetime_format(dlt/common/time.py) mapped ISO 8601 week dates (YYYY-Www) to"%Y-W%W"/"%Y-W%W-%u".%W(locale week-of-year, tied to the calendar year%Y) disagrees with the ISO week%V+ ISO week-numbering year%Gat year boundaries.The detected format is used by
dlt/extract/incremental/lag.pyto re-serialize a parsed-and-lagged string cursor, so it must round-trip — and it did not:"2026-W01"parsed to 2025-12-29 and rendered back as"2025-W52", silently corrupting the saved incremental cursor at year boundaries.Fix: emit
"%G-W%V"/"%G-W%V-%u".Red-green proven: 9 tests fail without the fix. An exhaustive 2015–2030 round-trip scan shows 0 regressions and 303 previously-broken week values fixed; out-of-range weeks (e.g. W53 in a 52-week year) are still rejected at parse time.
tests/common/test_time.py361 passed,tests/extract/test_lag.py426 passed.black --check,ruff check,flake8, andmypyare clean on the touched files.