Re: Size of the security-tracker repository

From	Sylvain Beucler <beuc@beuc.net>
Newsgroups	linux.debian.security
Subject	Re: Size of the security-tracker repository
Date	2026-06-12 07:40 +0200
Message-ID	<N52Mx-cciO-1@gated-at.bofh.it> (permalink)
References	<N2kcV-ajpd-1@gated-at.bofh.it> <N2Tkl-aHHx-1@gated-at.bofh.it> <N3rm1-b5kq-9@gated-at.bofh.it> <N4G9j-bWTZ-7@gated-at.bofh.it>
Organization	linux.* mail to news gateway

Show all headers | View raw

Hi,

(moving the discussion to debian-security-tracker, last post to 
debian-security :))

On 11/06/2026 07:21, Sylvain Beucler wrote:
> On 07/06/2026 21:24, Christoph Biedl wrote:
>> This I can confirm, the auto-gc brought this down to 5.3G last night.
> [...]
>> Still, this is a lot worse than ealier: On April 7th the total size of
>> .git/objects/ was just 1571M here - a day later things started to go
>> downhill.
> 
> Interesting. Do you graph the .git usage somehow, and do you have more 
> data?
> 
> 
> I believe that Salsa is still sending very suboptimal packs over the 
> network, which are then reused by the local Git.
> 
> 'git gc --aggressive --keep-largest-pack' repacked the most recents 
> packs from last week MUCH more efficiently (1.6GB -> 12MB), which tends 
> to confirm this hypothesis.
> 
> 'git gc --aggressive' is supposed to repack everything, but I'm not able 
> to run it on my computer anymore at this currently OOMs at >20GB RAM, 
> which means previous packs may stay fat and non-optimized forever.

FTR I ran:
   $ git -c pack.windowMemory=2G gc --aggressive
which took 15h and peeked at 30GB RAM (over the nproc*2GB limit).

Result:
   $ du -sh .git
   1,8G	.git
(from 7GB)

Not very practical of course, but mainly this fits my above hypothesis.

>> It's a bit saddening nothing has happened about this while everybody
>> should have been aware this will not resolve by itself - but rather get
>> worse over time :(
> 
> I also see that files >50MB now get a warning at GitHub (before getting 
> rejected at >100MB), so with our data/CVE/list at 53MB we're starting to 
> hit all kinds of hard limits, confirming we have to do something sooner 
> than later.
> 
> Maybe this is something where the LTS Team can help with, let's wait for 
> the bookworm LTS handover to take place this week, and I'll try and 
> reach out to involved teams.

Cheers!
Sylvain Beucler
Debian LTS Team

Back to linux.debian.security | Previous | Next — Previous in thread | Find similar | Unroll thread

Thread

Size of the security-tracker repository Christoph Biedl <debian.axhn@manchmal.in-ulm.de> - 2026-06-04 19:40 +0200
  Re: Size of the security-tracker repository Sylvain Beucler <beuc@beuc.net> - 2026-06-06 09:10 +0200
    Re: Size of the security-tracker repository Christoph Biedl <debian.axhn@manchmal.in-ulm.de> - 2026-06-07 21:30 +0200
      Re: Size of the security-tracker repository Sylvain Beucler <beuc@beuc.net> - 2026-06-11 07:30 +0200
        Re: Size of the security-tracker repository Sylvain Beucler <beuc@beuc.net> - 2026-06-12 07:40 +0200

csiph-web