fix test failures on 1.12, avoid race condition in multithreaded partitioned writes

[palday]

There is race condition fundamental to the current architecture for creating and writing dictionary encodings. The relevant lock is created on a worker thread and thus there is a race to create the lock and initialize the relevant data structure. This race condition has existed for a long time and consistently occurs when testing on 1.12, but I have occasionally been able to see it occur on Julia 1.10.

Reworking this goes well beyond what I currently have time for, so I have simply disabled multithreaded writing as a stopgap. This may seem extreme, but:

1. This is a correctness bug and correctness is far more important than speed.
2. The test failures that this race condition causes on 1.12 are blocking the release of 2.8.1, which includes Avoid extending Type from Base on Julia 1.12 #543 and addresses another source of potential correctness issues on Julia 1.12+.

[codecov-commenter]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.98%. Comparing base (3712291) to head (db35d62).
⚠️ Report is 42 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #582      +/-   ##
==========================================
- Coverage   87.43%   86.98%   -0.46%     
==========================================
  Files          26       27       +1     
  Lines        3288     3396     +108     
==========================================
+ Hits         2875     2954      +79     
- Misses        413      442      +29     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[palday]

@quinnj I think there is a race condition baked into the current architecture that can't be addressed without a very large refactoring. The current architecture creates the locks on a worker thread if they don't already exist, which means that threads are competing for the creation of the initial lock. The locks should be created before any tasks are spawned.

[palday]

given how long the Arrow tests take, it's useful to have some indication of progress so that we can tell if tests have hung. ExtendedTestSet shows a . for each completed test.

(We also get colored diffs of arrays when tests fail, which is nice.)

[palday]

@kou @ericphanson Could I get review on this please? It's not a pretty solution, but it works.

[ericphanson]

I think correct writing of multiply partitioned files is better than fast-but-wrong so this seems like an improvement. In general it looks like there's a lot to cleanup here with the threading, e.g. we shouldn't really be spawning tasks without holding onto them so we can fetch them and recover any errors they threw. We should prefer structured concurrency primitives like threaded map / asyncmap etc. But that's unrelated to this PR.

[kou]

I'll merge this in a few days if nobody has more comments.

We can try 2.8.4 RC2 release with this, right?

See also: The 2.8.4 RC1 vote: https://lists.apache.org/thread/7g3wr39wlbs8dj08hpb87mf9z2mlqrft

[palday]

We can try 2.8.4 RC2 release with this, right?

@kou Yes, exactly. 😄