Repository navigation
feat: support casting strings to durations - #11572
miguelcorderopamphile wants to merge 3 commits into
Conversation
Add casts from Utf8/LargeUtf8/Utf8View to Duration for all four units, reusing the existing Parser implementations for the Duration*Types (same integer parsing as the JSON reader).
|
|
||
| #[test] | ||
| fn test_cast_string_to_duration() { | ||
| let array = StringArray::from(vec![Some("1"), Some("-2"), None, Some("invalid")]); |
There was a problem hiding this comment.
do we support only casting from numeric values? no units such as 1 hour 10 minutes?
There was a problem hiding this comment.
Currently numeric only: the cast reuses the existing Duration*Type parser impls (the same integer parsing the JSON reader uses).
I can extend it to unit strings like 1 hour 10 minutes by going through the existing string→interval parser and the Interval(MonthDayNano) → Duration cast, which keeps the current semantics (months/days are not convertible — cast_month_day_nano_to_duration nulls/errors them).
I could add it in this PR or keep this one numeric-only — your call.
There was a problem hiding this comment.
it would be more useful if we could parse units
There was a problem hiding this comment.
Done in ffa5e92 — non-integer strings now fall back to the interval parser, so "1 hour 10 minutes" casts to any Duration unit. Integer strings behave as before.
Months/days stay non-convertible to match the interval → duration cast (null when safe, error otherwise). I can change that if you'd rather treat "1 day" as 24h.
There was a problem hiding this comment.
a good rule of thumb is to be able to roundtrip through the display representation:
arrow-rs/arrow-cast/src/display.rs
Lines 1591 to 1613 in 6db2d4f
can see how we display days here, so it would make sense to support parsing days
There was a problem hiding this comment.
Done in d047bf0 — days now convert at 24h/day (weeks fold into days in the parser), so pretty display strings roundtrip: "45 days 14 hours 2 mins 34 secs" → 3938554s for Duration(s). Months stay non-convertible since the duration display never emits them.
Reuse the interval parser for non-integer strings (e.g. '1 hour 10 minutes'). Intervals with non-zero months or days remain non-convertible (null/error), consistent with the interval to duration cast.
Convert days at 24h so pretty display strings roundtrip, e.g. '45 days 14 hours 2 mins 34 secs'. Weeks fold into days in the interval parser. Months stay non-convertible (never displayed).
Fixes #4555.
The cast kernels supported strings to intervals, but not to durations. This adds casts from
Utf8,LargeUtf8andUtf8ViewtoDurationin all four units, reusing the existingParserimplementations for theDuration*Types (the same integer parsing the JSON reader uses, #6683).can_cast_typesaccepts the string → duration combinationparse_string(Utf8/LargeUtf8) andparse_string_view(Utf8View)cargo test -p arrow-cast --lib: 368 passed.