test_os.ExtendedAttributeTests fail on filesystems with low xattr limits (e.g. ext4 with small blocks)

[mgorny]

Bug report

Bug description:

This is roughly the same as #66210, except that bug was closed due to "not enough info", and I'm here to provide "enough info", and I can't reopen that one.

I'm seeing the following tests fail on Gentoo ARM devboxen:

Details

$ ./python -m test test_os -v -m ExtendedAttributeTests
== CPython 3.14.0a1+ (heads/main:2313f84, Nov 16 2024, 16:54:48) [GCC 13.3.1 20241024]
== Linux-5.15.169-gentoo-dist-aarch64-with-glibc2.40 little-endian
== Python build: release
== cwd: /var/tmp/cpython/build/test_python_worker_802177æ
== CPU count: 96
== encodings: locale=UTF-8 FS=utf-8
== resources: all test resources are disabled, use -u option to unskip tests

Using random seed: 892697182
0:00:00 load avg: 1.53 Run 1 test sequentially in a single process
0:00:00 load avg: 1.53 [1/1] test_os
test_fds (test.test_os.ExtendedAttributeTests.test_fds) ... ERROR
test_lpath (test.test_os.ExtendedAttributeTests.test_lpath) ... ERROR
test_simple (test.test_os.ExtendedAttributeTests.test_simple) ... ERROR

======================================================================
ERROR: test_fds (test.test_os.ExtendedAttributeTests.test_fds)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/tmp/cpython/Lib/test/test_os.py", line 4006, in test_fds
    self._check_xattrs(getxattr, setxattr, removexattr, listxattr)
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/tmp/cpython/Lib/test/test_os.py", line 3979, in _check_xattrs
    self._check_xattrs_str(str, *args, **kwargs)
    ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/var/tmp/cpython/Lib/test/test_os.py", line 3970, in _check_xattrs_str
    setxattr(fn, s("user.test"), b"a"*1024, **kwargs)
    ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/tmp/cpython/Lib/test/test_os.py", line 3999, in setxattr
    os.setxattr(fp.fileno(), *args)
    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 28] No space left on device: 3

======================================================================
ERROR: test_lpath (test.test_os.ExtendedAttributeTests.test_lpath)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/tmp/cpython/Lib/test/test_os.py", line 3990, in test_lpath
    self._check_xattrs(os.getxattr, os.setxattr, os.removexattr,
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                       os.listxattr, follow_symlinks=False)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/var/tmp/cpython/Lib/test/test_os.py", line 3979, in _check_xattrs
    self._check_xattrs_str(str, *args, **kwargs)
    ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/var/tmp/cpython/Lib/test/test_os.py", line 3970, in _check_xattrs_str
    setxattr(fn, s("user.test"), b"a"*1024, **kwargs)
    ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 28] No space left on device: '@test_802177_tmpæ'

======================================================================
ERROR: test_simple (test.test_os.ExtendedAttributeTests.test_simple)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/var/tmp/cpython/Lib/test/test_os.py", line 3986, in test_simple
    self._check_xattrs(os.getxattr, os.setxattr, os.removexattr,
    ~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                       os.listxattr)
                       ^^^^^^^^^^^^^
  File "/var/tmp/cpython/Lib/test/test_os.py", line 3979, in _check_xattrs
    self._check_xattrs_str(str, *args, **kwargs)
    ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
  File "/var/tmp/cpython/Lib/test/test_os.py", line 3970, in _check_xattrs_str
    setxattr(fn, s("user.test"), b"a"*1024, **kwargs)
    ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 28] No space left on device: '@test_802177_tmpæ'

----------------------------------------------------------------------
Ran 3 tests in 0.008s

FAILED (errors=3)
test test_os failed
test_os failed (3 errors)

== Tests result: FAILURE ==

1 test failed:
    test_os

Total duration: 154 ms
Total tests: run=3 (filtered)
Total test files: run=1/1 (filtered) failed=1
Result: FAILURE

These tests attempt to set quite a fair number of extended attributes, notably including one attribute with 1024-byte value and 100 short attributes (that should take another 1 KiB). However, according to xattr(7):

In the current ext2, ext3, and ext4 filesystem implementations, the total bytes used by the names and values of all of a file’s extended attributes must fit in a single filesystem block (1024, 2048 or 4096 bytes, depending on the block size specified when the filesystem was created).

Well, I don't know why exactly, but the filesystems here (on both ARM machines we have) are 1024 byte long. Hence, attempting to write over 2 KiB of xattrs to them triggers ENOSPC.

I can get the test to pass if I lower the numbers significantly, e.g.:

diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py
index 9a4be78..919ed92 100644
--- a/Lib/test/test_os.py
+++ b/Lib/test/test_os.py
@@ -3967,10 +3967,10 @@ def _check_xattrs_str(self, s, getxattr, setxattr, removexattr, listxattr, **kwa
         xattr.remove("user.test")
         self.assertEqual(set(listxattr(fn)), xattr)
         self.assertEqual(getxattr(fn, s("user.test2"), **kwargs), b"foo")
-        setxattr(fn, s("user.test"), b"a"*1024, **kwargs)
-        self.assertEqual(getxattr(fn, s("user.test"), **kwargs), b"a"*1024)
+        setxattr(fn, s("user.test"), b"a"*256, **kwargs)
+        self.assertEqual(getxattr(fn, s("user.test"), **kwargs), b"a"*256)
         removexattr(fn, s("user.test"), **kwargs)
-        many = sorted("user.test{}".format(i) for i in range(100))
+        many = sorted("user.test{}".format(i) for i in range(32))
         for thing in many:
             setxattr(fn, thing, b"x", **kwargs)
         self.assertEqual(set(listxattr(fn)), set(init_xattr) | set(many))

However, I don't know if that's desirable. The alternatives might be to catch ENOSPC and use lower numbers then, or use a larger value in supports_extended_attributes() to have the tests skipped when the filesystem has an xattr limit lower than 4096 bytes.

CPython versions tested on:

3.9, 3.10, CPython main branch

Operating systems tested on:

Linux

Linked PRs

• gh-126909: Fix running xattr tests on systems with lower limits #126930
• [3.12] gh-126909: Fix running xattr tests on systems with lower limits (GH-126930) #126964
• [3.13] gh-126909: Fix running xattr tests on systems with lower limits (GH-126930) #126965

[mgorny]

I've filed #126930 with the proposed solution. I'm open to implementing a different approach, should another one be preferred.

[vstinner]

Fixed by 2c0a21c.