-
Notifications
You must be signed in to change notification settings - Fork 92
Labels
enhancementNew feature or requestNew feature or request
Description
Fortunately, there's a relatively small surface area of pandas we need to be concerned with. In looking at the backwards incompatible API changes, most of the changes that'll concern us revolve around datetime64
& timedelta64
dtype resolutions.
Things we'll need to consider
- Default use should be consistent with pandas 1.x behavior for backward compatibility and unsurprising differences between using 1.x vs 2.x:
- Pandas 2.x supports resolutions “s”, “ms”, “us”, and “ns”. 1.x did not. In 1.x when constructing a Series or DataFrame and passing a
datetime64
ortimedelta64
dtype with unsupported resolution (i.e. anything other than “ns”), pandas would silently replace the given dtype with its nanosecond analogue. Captured in Using pandas 2.0 cause query_df to return columns with different/unexpected dtypes #165
- Pandas 2.x supports resolutions “s”, “ms”, “us”, and “ns”. 1.x did not. In 1.x when constructing a Series or DataFrame and passing a
- Will need to incorporate running the relevant tests in CI against pandas 1.x and 2.x
Probably 2 additional tests: (Python 3.13, ClickHouse latest, pandas 1.x) and then (Python 3.13, ClickHouse latest, pandas 2.x)- See this comment for more context, but we'll need to add one new test: (Python 3.11, ClickHouse latest, pandas 1.x). Captured in Add pandas 1.x and 2.x testing to CI #528
Useful enhancements once supported
- Leverage pandas 2.x support of
"pyarrow"
dtype_backend
: Support PyArrow backed Pandas DataFrames. #326 - We should expose a setting in common that'll allow users of 2.x to opt in to the more appropriate resolutions (e.g.
[s]
forTime
,[us]
forTime64(6)
) or[D]
forDate
, etc.): Expose setting allowing pandas 2.x datetime64/timedelta64 variable resolutions #531
Sub-issues
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or request