Skip to content

Commit f4eff71

Browse files
pks-tgitster
authored andcommitted
builtin/pack-objects: don't fetch objects when merging packs
The "--stdin-packs" option can be used to merge objects from multiple packfiles given via stdin into a new packfile. One big upside of this option is that we don't have to perform a complete rev walk to enumerate objects. Instead, we can simply enumerate all objects that are part of the specified packfiles, which can be significantly faster in very large repositories. There is one downside though: when we don't perform a rev walk we also don't have a good way to learn about the respective object's names. As a consequence, we cannot use the name hashes as a heuristic to get better delta selection. We try to offset this downside though by performing a localized rev walk: we queue all objects that we're about to repack as interesting, and all objects from excluded packfiles as uninteresting. We then perform a best-effort rev walk that allows us to fill in object names. There is one gotcha here though: when "--exclude-promisor-objects" has not been given we will perform backfill fetches for any promised objects that are missing. This used to not be an issue though as this option was mutually exclusive with "--stdin-packs". But that has changed recently, and starting with dcc9c7e (builtin/repack: handle promisor packs with geometric repacking, 2026-01-05) we will now repack promisor packs during geometric compaction. The consequence is that a geometric repack may now perform a bunch of backfill fetches. We of course cannot pass "--exclude-promisor-objects" to fix this issue -- after all, the whole intent is to repack objects part of a promisor pack. But arguably we don't have to: the rev walk is intended as best effort, and we already configure it to ignore missing links to other objects. So we can adapt the walk to unconditionally disable fetching any missing objects. Do so and add a test that verifies we don't backfill any objects. Reported-by: Lukas Wanko <lwanko@gitlab.com> Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
1 parent 67ad421 commit f4eff71

2 files changed

Lines changed: 28 additions & 0 deletions

File tree

builtin/pack-objects.c

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -3927,8 +3927,16 @@ static void add_unreachable_loose_objects(struct rev_info *revs);
39273927

39283928
static void read_stdin_packs(enum stdin_packs_mode mode, int rev_list_unpacked)
39293929
{
3930+
int prev_fetch_if_missing = fetch_if_missing;
39303931
struct rev_info revs;
39313932

3933+
/*
3934+
* The revision walk may hit objects that are promised, only. As the
3935+
* walk is best-effort though we don't want to perform backfill fetches
3936+
* for them.
3937+
*/
3938+
fetch_if_missing = 0;
3939+
39323940
repo_init_revisions(the_repository, &revs, NULL);
39333941
/*
39343942
* Use a revision walk to fill in the namehash of objects in the include
@@ -3964,6 +3972,8 @@ static void read_stdin_packs(enum stdin_packs_mode mode, int rev_list_unpacked)
39643972
stdin_packs_found_nr);
39653973
trace2_data_intmax("pack-objects", the_repository, "stdin_packs_hints",
39663974
stdin_packs_hints_nr);
3975+
3976+
fetch_if_missing = prev_fetch_if_missing;
39673977
}
39683978

39693979
static void add_cruft_object_entry(const struct object_id *oid, enum object_type type,

t/t5331-pack-objects-stdin.sh

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -358,6 +358,24 @@ test_expect_success '--stdin-packs with promisors' '
358358
)
359359
'
360360

361+
test_expect_success '--stdin-packs does not perform backfill fetch' '
362+
test_when_finished "rm -rf remote client" &&
363+
364+
git init remote &&
365+
test_commit_bulk -C remote 10 &&
366+
git -C remote config set --local uploadpack.allowfilter 1 &&
367+
git -C remote config set --local uploadpack.allowanysha1inwant 1 &&
368+
369+
git clone --filter=tree:0 "file://$(pwd)/remote" client &&
370+
(
371+
cd client &&
372+
ls .git/objects/pack/*.promisor | sed "s|.*/||; s/\.promisor$/.pack/" >packs &&
373+
test_line_count -gt 1 packs &&
374+
GIT_TRACE2_EVENT="$(pwd)/event.log" git pack-objects --stdin-packs pack <packs &&
375+
test_grep ! "\"event\":\"child_start\"" event.log
376+
)
377+
'
378+
361379
stdin_packs__follow_with_only () {
362380
rm -fr stdin_packs__follow_with_only &&
363381
git init stdin_packs__follow_with_only &&

0 commit comments

Comments
 (0)