[Sriram-PR]

Fixes #208

2025年10月12日 (Chinese/Japanese) and 2025년 10월 12일 (Korean) weren't matched by any pattern in custom_parse, since YMD_PATTERN and YM_PATTERN only accept -, / and . between the parts. Fast mode returned nothing and extensive mode usually fell back to the year.

This adds CJK_YMD_PATTERN to extractors.py and tries it in _date_candidates right after the Y-M-D patterns. examine_text() cuts trailing non-digits (NON_DIGITS_REGEX), so by the time element text like 公開日: 2025年10月12日 reaches custom_parse the final 日 is gone. The day suffix is therefore optional when no further digit follows.

Year-month alone (2025年10月) is left out on purpose: it mostly shows up in calendar widgets and placeholders (2019年12月XX日), and the day-1 dates it gave beat the real date on the page.

find_date() with <p class="date">...</p>, before and after:

text	fast before	extensive before	after (both)
公開日: 2025年10月12日	None	2025-01-01	2025-10-12
2025年1月5日	None	2025-01-01	2025-01-05
发布时间：2025年10月12日	None	2025-01-01	2025-10-12
發佈日期：2025年10月12日	None	2025-01-01	2025-10-12
게시일: 2025년 10월 12일	None	2025-01-01	2025-10-12
2025年10月12日 10:30	None	2025-10-12	2025-10-12

Tests: new test_cjk_dates in tests/unit_tests.py (valid dates through custom_parse, invalid ones like month 13 and Feb 30, year-month only and placeholders giving no date, find_date in both modes with <p class="date"> and <time>, and a placeholder or archive list next to the real date). It fails on master.

• pytest: 42 passed
• TZ=Europe/Berlin python eval_gate.py: extensive F1 0.9490, fast 0.9265, both at the floor, so no change on the benchmark
• ruff check ., ruff format --check htmldate tests and mypy -p htmldate with the dev extras: clean

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (2c4be93) to head (0e947e5).

Additional details and impacted files

@@            Coverage Diff            @@
##            master      #209   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files            8         8           
  Lines          882       897   +15     
=========================================
+ Hits           882       897   +15     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Sriram-PR]

hi @adbar, just checking in on this one. CI is green and it still merges cleanly on top of #198. happy to change anything if you'd like it done differently

[adbar]

@Sriram-PR Thanks, it would be better for the extraction.

Could you remove CJK_YM_PATTERN and its tests? It picks up calendar widgets (2026年10月) and placeholders (2019年12月XX日), and the resulting day-1 dates beat the real dates on the same page.

[Sriram-PR]

done, CJK_YM_PATTERN and its tests are gone. i added your two examples as cases that must give no date, plus a find_date check where a 2019年12月XX日 placeholder or a 2026年10月 archive list sits next to the real date. on the old code those returned 2019-12-01 and 2026-10-01, now both give the article's 2025-10-12. eval gate unchanged (0.9490 / 0.9265).