GH-48593: [C++] C++20: use standard calendar / timezone APIs

[rok]

Rationale for this change

Switch to std::chrono for MSVC to be able to use the system-provided timezone automatically on Windows.

What changes are included in this PR?

This adds chrono_internal.h that uses C++20 std::chrono timezone/calendar APIs on compilers with support (MSVC only for now) and falls back to vendored date.h otherwise.

Are these changes tested?

Partially tested locally and partially to be tested on CI.

Are there any user-facing changes?

Yes, Windows users will no longer need to install the IANA tzdb (see instructions here and here). We possibly have tzdb download set up in CI too and should update it appropriately.

• GitHub Issue: [C++] C++20: use standard calendar / timezone APIs #48593

[github-actions]

⚠️ GitHub issue #48593 has been automatically assigned in GitHub to PR creator.

[rok]

It seems that std::chrono on GCC (14.3.0, 15.2.0) potentially has a bug that triggers some of our tests. Meanwhile std::chrono on MSVC 19.44 ( 14.44) appears to be pass them and is correct or at least consistent with vendored date.h. I would therefore advise we only switch to std::chrono on MSVC for now as that gives us the most benefit anyway (users no longer have to deal with the tz db).

Below is the explanation and reproduction of the bug.

// GCC libstdc++ DST bug reproduction
//
// The AN to AS Transition Bug
// ---------------------------
// Source https://github.com/eggert/tz/blob/c37fbc3249c1a1334948b38f3bca47dee5c11dd1/australasia#L165-L192
// Australia/Broken_Hill used the AN (New South Wales) rules until 2000, then
// switched to AS (South Australia) rules. Under AN rules, DST started the last
// Sunday of October and ended the last Sunday of March. For the 1999-2000
// summer, DST started October 31, 1999 and would end March 26, 2000. February
// 29, 2000 falls squarely within this DST period, so the correct offset should
// be 9:30 base + 1:00 DST = 10:30 (630 minutes).
//
// Why GCC's Data is Wrong
// -----------------------
// When libstdc++ processes the zone transition from AN rules to AS rules (which
// happens in year 2000), it appears to lose or reset the DST state inherited
// from the AN rules. Instead of recognizing that DST is still active from the
// October 1999 transition, it reports offset=570 (just the 9:30 base) with
// save=0. The inconsistency is evident: it returns abbrev="ACDT" (daylight
// time) but the offset and save values indicate standard time. The AN rules
// clearly show DST should be active until the last Sunday of March 2000.
//
// Compile: g++ -std=c++20 -o gcc_dst_bug gcc_libstdcxx_dst_bug.cpp
// Expected: 630 (10:30 = 9:30 base + 1:00 DST)
// Actual:   570 (9:30 = base only, DST missing)

#include <chrono>
#include <iostream>

int main() {
  using namespace std::chrono;
  auto* tz = locate_zone("Australia/Broken_Hill");
  auto info = tz->get_info(sys_days{2000y / February / 29d} + 23h + 23min + 23s);
  std::cout << duration_cast<minutes>(info.offset).count() << "\n";
}

[rok]

@pitrou

[pitrou]

It seems that std::chrono on GCC (14.3.0, 15.2.0) potentially has a bug that triggers some of our tests.

Is the bug reported somewhere? If not, can you do that?

[pitrou]

We possibly have tzdb download set up in CI too and should update it appropriately.

Yes, I think we should do so. There are also a bunch of code snippets in the C++ and Python codebase that could be removed, IIRC.

[rok]

Is the bug reported somewhere? If not, can you do that?

It seems to be related to a known issue, I added comment explaining our case.

[raulcd]

@github-actions crossbow submit -g python

[github-actions]

Revision: 4c5113f

Submitted crossbow builds: ursacomputing/crossbow @ actions-e83bbadd74

Task	Status
example-python-minimal-build-fedora-conda
example-python-minimal-build-ubuntu-venv
test-conda-python-3.10
test-conda-python-3.10-hdfs-2.9.2
test-conda-python-3.10-hdfs-3.2.1
test-conda-python-3.10-pandas-1.3.4-numpy-1.21.2
test-conda-python-3.11
test-conda-python-3.11-dask-latest
test-conda-python-3.11-dask-upstream_devel
test-conda-python-3.11-hypothesis
test-conda-python-3.11-pandas-latest-numpy-latest
test-conda-python-3.11-spark-master
test-conda-python-3.12
test-conda-python-3.12-cpython-debug
test-conda-python-3.12-pandas-latest-numpy-1.26
test-conda-python-3.12-pandas-latest-numpy-latest
test-conda-python-3.13
test-conda-python-3.13-pandas-nightly-numpy-nightly
test-conda-python-3.13-pandas-upstream_devel-numpy-nightly
test-conda-python-3.14
test-conda-python-emscripten
test-debian-13-python-3-amd64
test-debian-13-python-3-i386
test-fedora-42-python-3
test-ubuntu-22.04-python-3
test-ubuntu-22.04-python-313-freethreading
test-ubuntu-24.04-python-3

[raulcd]

@github-actions crossbow submit wheel-windows-*

[github-actions]

Revision: cde409e

Submitted crossbow builds: ursacomputing/crossbow @ actions-be2cb464a0

Task	Status
wheel-windows-cp310-cp310-amd64
wheel-windows-cp311-cp311-amd64
wheel-windows-cp312-cp312-amd64
wheel-windows-cp313-cp313-amd64
wheel-windows-cp313-cp313t-amd64
wheel-windows-cp314-cp314-amd64
wheel-windows-cp314-cp314t-amd64

[raulcd]

@github-actions verify-rc-source-windows

[raulcd]

Sorry for being the bearer of bad news but the Windows wheels failures are related:

 >       check_example_file(path, table, need_fix=True)

Python310\lib\site-packages\pyarrow\tests\test_orc.py:145: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
Python310\lib\site-packages\pyarrow\tests\test_orc.py:101: in check_example_file
    table = orc_file.read()
Python310\lib\site-packages\pyarrow\orc.py:187: in read
    return self.reader.read(columns=columns)
pyarrow/_orc.pyx:374: in pyarrow._orc.ORCReader.read
    ???
pyarrow/error.pxi:155: in pyarrow.lib.pyarrow_internal_check_status
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   ???
E   pyarrow.lib.ArrowException: Unknown error: Time zone file /usr/share/zoneinfo/America/Los_Angeles does not exist. Please install IANA time zone database and set TZDIR env.

pyarrow/error.pxi:92: ArrowException

[rok]

@github-actions crossbow submit wheel-windows-cp314-cp314t-amd64

[github-actions]

Revision: 7e49c63

Submitted crossbow builds: ursacomputing/crossbow @ actions-e400d8f35f

Task	Status
wheel-windows-cp314-cp314t-amd64

[rok]

@github-actions crossbow submit wheel-windows-cp314-cp314t-amd64

[github-actions]

Revision: 54017ee

Submitted crossbow builds: ursacomputing/crossbow @ actions-3a266783b2

Task	Status
wheel-windows-cp314-cp314t-amd64

[rok]

@github-actions crossbow submit wheel-windows-*

[github-actions]

Revision: 54017ee

Submitted crossbow builds: ursacomputing/crossbow @ actions-b3fc0a238b

Task	Status
wheel-windows-cp310-cp310-amd64
wheel-windows-cp311-cp311-amd64
wheel-windows-cp312-cp312-amd64
wheel-windows-cp313-cp313-amd64
wheel-windows-cp313-cp313t-amd64
wheel-windows-cp314-cp314-amd64
wheel-windows-cp314-cp314t-amd64

[rok]

@jonkeane in light of CRAN build issues are we ok to merge here?

[jonkeane]

@jonkeane in light of CRAN build issues are we ok to merge here?

I wish we had a way of testing more reliably on the older macOS framework. What's in #49221 (comment) I'm not really comfortable merging to main, but does exercise the code path.

There's at least one other place we need to clean up on main already #49223 so maybe it's fine to merge this and when we get a better way to test this manualy or in CI we'll need to revert this or add a work around. I don't want worries about CRAN to totally slow down everything C++(20) related, but there is a bit of an annoying place we are in!