Evaluate replacing MinIO in the test fixture: the vendor has withdrawn public images #1560

tests/conftest.py starts MinIO to exercise the object-storage paths. MinIO has withdrawn from public distribution, so that dependency now rests on a third party continuing to publish builds of an archived codebase. #1559 restored CI by switching to Chainguard's build; this issue is about whether to keep depending on MinIO at all.

Filing it separately on purpose: #1559 was the release-eve fix and needed to change no test code. This one needs measurement, and picking wrong here produces the worst kind of green — a suite that passes while the behavior it claims to cover is untested.

Why the current position is not stable

Chainguard's build is a good mitigation and the right call for now — it is public, credential-free, drop-in for MinioContainer, and rebuilt continuously, where the last community image is a year stale.

But continuous rebuilds address the base image and dependencies. A defect in MinIO's own Go code has no upstream fix, because upstream is archived — worth confirming against Chainguard's advisory data rather than taking from me. And the arrangement depends on a third party's free tier continuing, for software whose vendor has already shown its direction. That is a reasonable place to stand for a release, and a poor place to stand indefinitely.

What a replacement has to satisfy

This is the part that decides it, and it rules out the obvious shortcut.

DataJoint reaches object storage through fsspec/s3fs, not boto3. The storage layer calls:

isdir · walk · size · exists · rm · open · pipe_file · get_file · put · put_file · cat_file

S3 has no directories. isdir and walk are emulated by s3fs over list_objects_v2 using common prefixes and zero-byte directory markers, and the emulation is sensitive to how a server reports prefixes, delimiters, and empty keys.

Garbage collection is built directly on that emulation — gc.py:205 and gc.py:255 both walk a store recursively to decide what is orphaned. A server that diverges on listing will not throw; it will return a different set of keys, and GC will classify live files as orphans or miss real ones. test_gc is 44 tests and exists precisely to catch that.

So a lightweight S3 mock is the tempting choice and the risky one. Fidelity on listing matters more here than startup time.

Candidates

The container images pull publicly today; moto installs from PyPI.

Staying on cgr.dev/chainguard/minio is also a legitimate outcome if the alternatives cost more fidelity than the dependency risk is worth. The point of the exercise is to answer that with evidence rather than assume it.

Recommendation

1. Evaluate SeaweedFS first as the replacement for the default fixture, against the acceptance criteria below. It is a real object store, so it is the candidate most likely to match MinIO on prefix and delimiter listing.
2. Garage is the fallback if SeaweedFS diverges on listing or is awkward to start under testcontainers.
3. If both diverge, stay on Chainguard MinIO for 2.4 and revisit.
4. Do not make moto or s3mock the default. GC's correctness rests on listing semantics, and a lenient emulator would turn test_gc green without testing it. moto is still worth a separate look later as a fast path for tests that never list — that is a speed question, not a replacement, and is out of scope here.

Acceptance

• The five object-storage suites pass against the candidate from a cold pull, with any local image cache deleted — 118 tests: test_gc (44), test_object (46), test_npy_codec (22), test_codecs (4), test_attach (2).
• test_gc passes specifically, including the prefix-scoping and orphan-detection cases, since that is where listing semantics bite.
• Directory-shaped objects round-trip: isdir and walk agree with what MinIO reported for the same tree.
• The image is public, needs no credentials, and is actively maintained.
• Nothing changes outside tests/conftest.py and the fixture; DJ_TEST_MINIO_IMAGE already makes the source swappable.

Out of scope

What DataJoint recommends to customers for production object storage. That is a product question, it touches an existing partnership conversation, and it should not be decided as a side effect of a test-fixture choice.