gh-149353: Preserve JIT shim object during PGO clean

[diegorusso]

• Issue: JIT shim build changes in #148872 can cause build steps to be unnecessarily repeated and with possible build breakage #149353

[diegorusso]

@ned-deily can you try this please?

[read-the-docs-community]

Documentation build overview

📚 cpython-previews | 🛠️ Build #32534730 | 📁 Comparing 3883cef against main (8c79678)

  🔍 Preview build  

3 files changed

± library/argparse.html
± whatsnew/3.15.html
± whatsnew/changelog.html

[ned-deily]

@ned-deily can you try this please?

@diegorusso Sorry, at least on macOS I don't see any difference: the make install step still starts by rebuilding jit.o.

[ned-deily]

Ah, I should have tried this before.

gmake --debug=w install
Makefile:979: update target 'check-clean-src' due to: target is .PHONY
Makefile:3395: update target 'Python/jit.o' due to: jit_stencils-aarch64-apple-darwin.h
gcc -c -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer  -fno-strict-overflow -Wsign-compare -Wunreachable-code -DNDEBUG -g -O3 -Wall    -std=c11 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden -Werror=unguarded-availability -D_Py_TIER2=3 -D_Py_JIT  -I./Include/internal -I./Include/internal/mimalloc  -I. -I./Include    -DPy_BUILD_CORE -o Python/jit.o Python/jit.c
Makefile:2115: update target 'Modules/getbuildinfo.o' due to: Python/jit.o
gcc -c -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer  -fno-strict-overflow -Wsign-compare -Wunreachable-code -DNDEBUG -g -O3 -Wall    -std=c11 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden -Werror=unguarded-availability -D_Py_TIER2=3 -D_Py_JIT  -I./Include/internal -I./Include/internal/mimalloc  -I. -I./Include    -DPy_BUILD_CORE \
      -DGITVERSION="\"`LC_ALL=C git --git-dir ./.git rev-parse --short HEAD`\"" \
      -DGITTAG="\"`LC_ALL=C git --git-dir ./.git describe --all --always --dirty`\"" \
      -DGITBRANCH="\"`LC_ALL=C git --git-dir ./.git name-rev --name-only HEAD`\"" \
      -o Modules/getbuildinfo.o ./Modules/getbuildinfo.c
Makefile:1953: update target 'Programs/_freeze_module' due to: Modules/getbuildinfo.o Python/jit.o
[...]

[ned-deily]

And, following the first make but before the make install:

-rw-r-----   1 nad staff      904 May  4 20:01 jit_shim-aarch64-apple-darwin.o
-rw-r-----   1 nad staff      253 May  4 20:01 jit_unwind_info-aarch64-apple-darwin.h
-rw-r-----   1 nad staff  2046781 May  4 20:02 jit_stencils-aarch64-apple-darwin.h
-rw-r-----  1 nad  staff  1356520 May  4 20:01 Python/jit.o

So it does think that jit.o is out-of-date.

[diegorusso]

That's odd, it works on my side:

To find the necessary bits, look in configure.ac and config.log.

Checked 115 modules (37 built-in, 77 shared, 0 n/a on macosx-26.4-arm64, 0 disabled, 1 missing, 0 failed on import)

❯ make install
Creating directory /tmp/bin
Creating directory /tmp/lib
if test "no-framework" = "no-framework" ; then \
		/usr/bin/install -c python.exe /tmp/bin/python3.15; \
	else \
		/usr/bin/install -c -s Mac/pythonw /tmp/bin/python3.15; \
	fi
if test "3.15" != "3.15"; then \
		if test -f /tmp/bin/python3.15 -o -h /tmp/bin/python3.15; \
		then rm -f /tmp/bin/python3.15; \
		fi; \
		(cd /tmp/bin; ln python3.15 python3.15); \
	fi

What version of make do you have?

❯ make --version
GNU Make 3.81
Copyright (C) 2006  Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

This program built for i386-apple-darwin11.3.0

[diegorusso]

OK, I was able to replicate it. If you build in-tree it does happen. I tend to build out-of-tree hence I didn't see it.
This happens on both Linux and macOS.

[diegorusso]

I've pushed the fix. clean-retain-profile left an inconsistent JIT group:

kept:    .jit-stamp
kept:    jit_stencils*.h
kept:    jit_unwind_info*.h
deleted: jit_shim*.o

Now we are excluding jit_shim*.o when cleaning so all the jit files are kept and we have a consistent JIT group.

kept:    .jit-stamp
kept:    jit_stencils*.h
kept:    jit_unwind_info*.h
kept:    jit_shim*.o

I've tested this on macOS/Linux, in tree and out of tree.

[ned-deily]

LGTM, thanks for the quick fix!

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Debian root 3.x (tier-1) has failed when building commit 4fa5c04.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/345/builds/14151) and take a look at the build logs.
4. Check if the failure is related to this commit (4fa5c04) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/345/builds/14151

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/root/buildarea/3.x.angelico-debian-amd64/build/Lib/test/support/__init__.py", line 948, in gc_collect
    gc.collect()
    ~~~~~~~~~~^^
ResourceWarning: unclosed file <_io.FileIO name=11 mode='wb' closefd=True>

[bedevere-bot]

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot wasm32-wasi 3.x (tier-2) has failed when building commit 4fa5c04.

What do you need to do:

1. Don't panic.
2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
3. Go to the page of the buildbot that failed (https://buildbot.python.org/#/builders/1046/builds/11720) and take a look at the build logs.
4. Check if the failure is related to this commit (4fa5c04) or if it is a false positive.
5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/#/builders/1046/builds/11720

Failed tests:

• test_zipfile

Failed subtests:

• test_write_without_source_date_epoch - test.test_zipfile.test_core.OtherTests.test_write_without_source_date_epoch

Summary of the results of the build (if available):

==

Click to see traceback logs

Traceback (most recent call last):
  File "/Lib/test/test_zipfile/test_core.py", line 1906, in test_write_without_source_date_epoch
    self.assertAlmostEqual(z_time, c_time, delta=1)
    ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 52 != 54 within 1 delta (2 difference)