[DE-7720] Bump minimum Python to 3.10 and patch all dependency CVEs

[edwinpav]

Summary

Bumps the minimum supported Python version from 3.7 to 3.10 (now supports 3.10–3.14) so we can adopt patched versions of our locked dependencies. The patched releases of aiohttp, urllib3, requests, Pillow, numpy, setuptools, and others now require Python ≥ 3.10, so we couldn't pull them in while still supporting 3.7–3.9.

This is the headline reason for the bump: it unblocks the dependency upgrades that resolve all current Trivy findings for this repo.

Why

Local Trivy scan before this PR: 48 vulnerabilities (1 CRITICAL, 11 HIGH, 22 MEDIUM, 14 LOW), including:

(See https://linear.app/scale-epd/issue/DE-7720 for file of all vulns)

• Pillow CVE-2023-50447 (CRITICAL — RCE)
• aiohttp CVE-2024-23334 (HIGH — path traversal) and a long tail of HTTP-smuggling/DoS CVEs
• urllib3 CVE-2025-66418 (HIGH — decompression DoS)
• setuptools CVE-2024-6345 (HIGH — RCE)
• plus issues in requests, numpy, idna, certifi, marshmallow, pygments, tqdm, zipp, scikit-learn

Local Trivy scan after this PR: 0 vulnerabilities. See .trivy/fs.txt for the clean run (.trivy/fs-strict.txt shows the original findings).

Python 3.7, 3.8, and 3.9 are all past upstream end-of-life (2023-06-27, 2024-10-07, and 2025-10-31 respectively) and no longer receive security patches even from CPython itself.

What's in this PR

• Drop Python 3.7/3.8/3.9 support. pyproject.toml now requires python = ">=3.10,<4.0".
• Refresh locked dependencies to versions that resolve the Trivy findings (regenerated poetry.lock).
• Expand CI installation matrix from [3.10, 3.11] to [3.10, 3.11, 3.12, 3.13, 3.14] so every supported Python version is exercised on every PR. The matrix builds the sdist, installs it (no extras, [metrics], [launch], and [metrics, launch]), and smoke-tests import nucleus on each Python version.
• Fix pytest 9 collection errors. pytest 9 turns @pytest.mark.* decorators on fixtures into hard errors. Removed/converted no-op fixture marks across tests/cli/conftest.py, tests/validate/conftest.py, tests/test_scene.py, and tests/test_video_scene.py.
• NumPy 2.x compatibility. Replaced np.float_ and np.float (both removed in modern NumPy) with np.float64 in nucleus/metrics/segmentation_utils.py and nucleus/metrics/segmentation_metrics.py.
• Lint cleanup to keep CI green on the new dependency versions: pylint findings (E0606, W3101, R1737, R1728, C3001, C3002, W0719), pylint disables updated (+R0913, -R0201), black re-applied. One exception was made more specific.
• Mypy clean. poetry run mypy --ignore-missing-imports nucleus now reports Success: no issues found in 76 source files. Notable user-visible adjustments:
  • DatasetItem.reference_id is now typed Optional[str] (still required at runtime via __post_init__ assertion); the internal "DUMMY_VALUE" sentinel is gone.
  • nucleus/async_utils.py wraps timeouts in aiohttp.ClientTimeout(total=...).
  • NucleusClient.list_autotags always returns a list regardless of response shape.
• Release 0.18.0 with a CHANGELOG entry.

Impact / risk

• Anyone still on Python 3.7/3.8/3.9 will see pip install scale-nucleus==0.18.0 fail with a clear version-requirement error (no silent breakage).
• All other changes are either internal lint/typing cleanup or backwards-compatible. The one observable Python-side change is DatasetItem.reference_id's type — the previous docstring already declared it Optional[str], so this aligns the type with the documented contract.

Verification

• make scan — 0 Trivy findings (.trivy/fs.txt).
• poetry run mypy --ignore-missing-imports nucleus — Success: no issues found in 76 source files.
• poetry run ruff . / poetry run pylint nucleus / poetry run isort --check-only . — clean.
• poetry run pytest --collect-only tests/ — collects cleanly under pytest 9 with no fixture-mark errors.
• CircleCI installation matrix runs against Python 3.10, 3.11, 3.12, 3.13, and 3.14.

Linear

DE-7720

Greptile Summary

This PR bumps the minimum Python version from 3.7 to 3.10 and refreshes all locked dependencies to resolve 48 Trivy CVEs (including a critical Pillow RCE and high-severity aiohttp/urllib3/setuptools findings). Alongside the version bump, it fixes NumPy 2.x compatibility (np.float_ → np.float64), wraps aiohttp timeouts in ClientTimeout, resolves pytest 9 fixture-mark collection errors, and applies mypy/pylint/black cleanup throughout.

Confidence Score: 5/5

Safe to merge — all changes are correct compatibility and security fixes with no functional regressions.

No P0 or P1 issues found. All changes are either formatting-only, legitimate security/compatibility upgrades, or well-understood type-annotation corrections. The aiohttp timeout fix, NumPy dtype replacements, pytest 9 fixture adjustments, and cuboid_utils NameError pre-init are all correct. CI matrix expansion exercises 3.10–3.14. Trivy reports 0 findings post-merge.

No files require special attention.

Important Files Changed

Filename	Overview
pyproject.toml	Python constraint bumped to >=3.10, version to 0.18.0, and all security-relevant deps updated (aiohttp ^3.10, requests ^2.32, Pillow >=12.2.0,<13, boto3 ^1.42, numpy >=1.22.0).
nucleus/async_utils.py	Fixes aiohttp timeout by wrapping scalar with aiohttp.ClientTimeout(total=…), and de-duplicates getattr call for api_key before each request — correct and clean.
nucleus/dataset_item.py	reference_id sentinel DUMMY_VALUE replaced with Optional[str] = None; post_init assertion updated accordingly; Exception → ValueError for privacy-mode check.
nucleus/metrics/segmentation_utils.py	np.float / np.float_ replaced with np.float64 for NumPy 2.x compatibility; shape[1] → shape[-1] (equivalent for 2-D arrays); setup_iou_thresholds updated similarly.
nucleus/metrics/cuboid_utils.py	Pre-initialises distance_mask to an all-False array so the variable is always defined even when both input corner arrays are empty, preventing a NameError.
nucleus/init.py	list_autotags now always returns a list; Exception → RuntimeError in two places; minor formatting and API-key logic simplification.
nucleus/annotation.py	Segment.index made Optional[int] = None to match real JSON payloads (mypy fix); Exception → ValueError/FileNotFoundError; type_key default changed from None to empty string (both fall through to SegmentationAnnotation).
nucleus/dataset.py	Multiple Exception → ValueError promotions, items default changed from None to [] to avoid None iteration, yield from refactors, and formatting cleanup.
.circleci/config.yml	CI matrix updated from [3.7–3.11] to [3.10–3.14]; Docker images bumped to python:3.10-bullseye and cimg/python:3.10.
tests/validate/conftest.py	@pytest.mark.usefixtures on fixture replaced with explicit annotations parameter to fix pytest 9 collection errors; intent preserved via comment.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Python >=3.10 requirement] --> B[Unlocks patched deps]
    B --> C1[aiohttp ^3.10 - path-traversal CVEs fixed]
    B --> C2[Pillow >=12.2.0 - RCE CVE-2023-50447 fixed]
    B --> C3[urllib3/requests/numpy - security patches]
    B --> C4[setuptools/boto3 - RCE / other CVEs fixed]
    A --> D[Code compatibility fixes]
    D --> D1[NumPy 2.x - np.float_ to np.float64]
    D --> D2[aiohttp ClientTimeout - wrap scalar timeout]
    D --> D3[pytest 9 - remove marks on fixtures]
    D --> D4[mypy/pylint/black - type and style cleanup]

Loading

_{Reviews (2): Last reviewed commit: "Update CHANGELOG.md to include recent co..." | Re-trigger Greptile}

[socket-security]

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff	Package	Supply Chain Security	Vulnerability	Quality	Maintenance	License
	lark@​1.3.1
	astroid@​2.11.7 ⏵ 4.0.4	+1
	botocore@​1.33.13 ⏵ 1.42.97
	certifi@​2023.11.17 ⏵ 2026.4.22	+1	+1
	debugpy@​1.7.0 ⏵ 1.8.20					-20
	ipython@​7.34.0 ⏵ 8.39.0	+1	+1
	ipython@​7.34.0 ⏵ 9.13.0		+1
	numpy@​1.26.3 ⏵ 2.2.6	+1
	pathspec@​0.11.2 ⏵ 1.1.1	+1
	pyzmq@​24.0.1 ⏵ 27.1.0	+2
	scipy@​1.6.1 ⏵ 1.15.3	+1
	unidecode@​1.3.7 ⏵ 1.4.0
	nbclassic@​1.0.0 ⏵ 1.3.3					-19
	tornado@​6.2 ⏵ 6.5.5	+1	+40
	rasterio@​1.2.10 ⏵ 1.4.4	-9
	jupyterlab@​3.6.6 ⏵ 3.6.8		+22
	scikit-learn@​1.0.2 ⏵ 1.7.2	+1	+2
	dill@​0.3.7 ⏵ 0.4.1	-8
	jupyterlab-pygments@​0.2.2 ⏵ 0.3.0	+1
	notebook@​6.5.6 ⏵ 6.5.7
	black@​23.3.0 ⏵ 23.12.1
	prompt-toolkit@​3.0.43 ⏵ 3.0.52	+1
	joblib@​1.3.2 ⏵ 1.5.3
	distlib@​0.3.8 ⏵ 0.4.0
	psutil@​5.9.7 ⏵ 7.2.2	+23
	jupyter-server@​1.24.0 ⏵ 2.17.0		+20			+20
	nbconvert@​7.6.0 ⏵ 7.17.1		+19
	execnet@​2.0.2 ⏵ 2.1.2
	virtualenv@​20.25.0 ⏵ 21.3.0	+1	+18
	pygments@​2.17.2 ⏵ 2.20.0	+1	+1
	anyio@​3.7.1 ⏵ 4.13.0
	ipykernel@​6.16.2 ⏵ 6.29.5	+4				+20
	urllib3@​2.0.7 ⏵ 2.6.3	+1	+31
See 101 more rows in the dashboard

View full report