Skip to content

Reading a small valid workbook can abort the Python process (allocation failure in calamine's Range::from_sparse) #263

Open

Description

@ZhijieQ

Problem

CalamineWorkbook.get_sheet_by_name() / get_sheet_by_index() (and from_filelike() for .xls) can abort the whole Python process on a small, valid workbook. The cause is upstream: calamine densifies a sheet into rows × cols cells in one allocation (tafia/calamine#693), and an allocation failure in Rust aborts — it is not a panic, so PyO3 cannot turn it into a Python exception and try/except BaseException around the call does not help.

From Python there is currently no way to check a sheet's size before that allocation happens, so a caller that reads untrusted files cannot defend itself except by parsing in a separate process.

Reproduce (python-calamine 0.9.1, openpyxl 3.1.5, Linux x86-64)

import io, subprocess, sys
from openpyxl import Workbook

CHILD = """
import io, sys
from python_calamine import CalamineWorkbook
try:
    wb = CalamineWorkbook.from_filelike(io.BytesIO(sys.stdin.buffer.read()))
    ws = wb.get_sheet_by_index(0)
    print("read", ws.total_height, ws.total_width)
except BaseException as exc:
    print("caught", type(exc).__name__, exc)
"""
for ref in ("XFD10", "A1048576", "XFD1048576"):
    wb = Workbook(); ws = wb.active
    ws["A1"], ws[ref] = "x", 1
    buf = io.BytesIO(); wb.save(buf)
    out = subprocess.run([sys.executable, "-c", CHILD], input=buf.getvalue(), capture_output=True)
    print(ref, out.returncode, out.stdout.decode().strip(), out.stderr.decode().strip()[:60])

Output:

XFD10 0 read 9 16383
A1048576 0 read 1048575 0
XFD1048576 -6  memory allocation of 549755813888 bytes failed

The last file is 4.8 KB and opens normally in Excel and LibreOffice. A damaged 12.8 KB .xls (a few bytes changed in a generated workbook) does the same at from_filelike() time, asking for 33,698,529,984 bytes; LibreOffice opens that one too. I can share the .xls if useful.

Suggestion

Something a Python caller can use before the grid is built, for example:

  • a sheet dimension that does not densify (e.g. from calamine's cell reader, or the sheet's declared/used range), so callers can refuse oversized sheets; or
  • an option such as max_cells= on get_sheet_by_* (and on workbook loading for .xls) that raises a normal Python exception when rows × cols would exceed it.

Upstream, tafia/calamine#697 would make Range::from_sparse return an error (CellLimitExceeded, over 100,000,000 cells) instead of aborting; it is still under review. When it lands, mapping that error to a Python exception (e.g. CalamineError) would fix most of this. An option like the above would still help: the upstream limit is fixed at 100 M cells (about 3.2 GB at 32 bytes per cell), which a web service may want to set lower.

Related: tafia/calamine#693, tafia/calamine#726, tafia/calamine#697.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions