Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.bugs.dist > #1250201 > unrolled thread

Bug#1103033: emacs: generate versioned Provides for all :core packages

Back to article view | Back to linux.debian.bugs.dist

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Bug#1103033: emacs: generate versioned Provides for all :core packages Xiyue Deng <manphiz@gmail.com> - 2025-06-17 03:10 +0200
    Bug#1103033: emacs: generate versioned Provides for all :core packages Sean Whitton <spwhitton@spwhitton.name> - 2025-06-18 16:20 +0200
      Bug#1103033: emacs: generate versioned Provides for all :core packages Xiyue Deng <manphiz@gmail.com> - 2025-06-19 04:10 +0200
        Bug#1103033: emacs: generate versioned Provides for all :core packages Sean Whitton <spwhitton@spwhitton.name> - 2025-06-19 14:50 +0200
          Bug#1103033: emacs: generate versioned Provides for all :core packages Sean Whitton <spwhitton@spwhitton.name> - 2025-06-20 13:40 +0200
            Bug#1103033: emacs: generate versioned Provides for all :core packages Sean Whitton <spwhitton@spwhitton.name> - 2025-07-04 14:40 +0200
              Bug#1103033: emacs: generate versioned Provides for all :core packages Sean Whitton <spwhitton@spwhitton.name> - 2025-07-04 22:50 +0200
                Bug#1103033: emacs: generate versioned Provides for all :core packages Sean Whitton <spwhitton@spwhitton.name> - 2025-07-15 11:20 +0200
                  Bug#1103033: emacs: generate versioned Provides for all :core packages Sean Whitton <spwhitton@spwhitton.name> - 2025-07-15 23:00 +0200
                    Bug#1103033: emacs: generate versioned Provides for all :core packages Sean Whitton <spwhitton@spwhitton.name> - 2025-07-16 11:40 +0200
                      Bug#1103033: emacs: generate versioned Provides for all :core packages Sean Whitton <spwhitton@spwhitton.name> - 2025-07-16 20:40 +0200
                        Bug#1103033: emacs: generate versioned Provides for all :core packages Sean Whitton <spwhitton@spwhitton.name> - 2025-07-18 11:20 +0200
                          Bug#1103033: emacs: generate versioned Provides for all :core packages Sean Whitton <spwhitton@spwhitton.name> - 2025-07-20 16:40 +0200
                            Bug#1103033: emacs: generate versioned Provides for all :core packages Sean Whitton <spwhitton@spwhitton.name> - 2025-07-21 10:30 +0200
                  Bug#1103033: emacs: generate versioned Provides for all :core packages Sean Whitton <spwhitton@spwhitton.name> - 2025-07-15 23:00 +0200
                    Bug#1103033: emacs: generate versioned Provides for all :core packages Sean Whitton <spwhitton@spwhitton.name> - 2025-07-16 12:10 +0200

#1250201 — Bug#1103033: emacs: generate versioned Provides for all :core packages

[Multipart message — attachments visible in raw view] — view raw

Sean Whitton <spwhitton@spwhitton.name> writes:

> control: reassign -1 src:emacs
> control: retitle -1 emacs: generate versioned Provides for all :core packages
>
> Hello,
>
> On Sun 13 Apr 2025 at 08:55pm -07, Xiyue Deng wrote:
>
>> Option 1 also sounds good to me.
>
> Cool.  I'm not planning to work on it but I can review patches.
>

I have made proof-of-concept patches as a pseudo-MR at [1] (this MR is
against my own fork just for illustration).  Basically, it uses an Emacs
script to generate Debian compatible versions from the builtin package
list, which is integrated with `debian-sync' in debian/rules (similar to
the handling of debian/copyright.in).  I'm currently making the provide
list in emacs-common package which ships the compiled elisp files.

Please review and comment.

[1] https://salsa.debian.org/manphiz/deb-emacs/-/merge_requests/1

-- 
Regards,
Xiyue Deng

[toc] | [next] | [standalone]

#1250352

[Multipart message — attachments visible in raw view] — view raw

Hello,

On Mon 16 Jun 2025 at 05:59pm -07, Xiyue Deng wrote:

> Sean Whitton <spwhitton@spwhitton.name> writes:
>
>> control: reassign -1 src:emacs
>> control: retitle -1 emacs: generate versioned Provides for all :core packages
>>
>> Hello,
>>
>> On Sun 13 Apr 2025 at 08:55pm -07, Xiyue Deng wrote:
>>
>>> Option 1 also sounds good to me.
>>
>> Cool.  I'm not planning to work on it but I can review patches.
>>
>
> I have made proof-of-concept patches as a pseudo-MR at [1] (this MR is
> against my own fork just for illustration).  Basically, it uses an Emacs
> script to generate Debian compatible versions from the builtin package
> list, which is integrated with `debian-sync' in debian/rules (similar to
> the handling of debian/copyright.in).  I'm currently making the provide
> list in emacs-common package which ships the compiled elisp files.
>
> Please review and comment.

Very interesting.

Have you considered using debhelper substvars for this?  Like how we
have ${elpa:Depends}.  It avoids needing a control.in.  A control.in
isn't the end of the world but it might be cleaner not to have one.

Don't sharpquote lambdas; unlike in CL, doing so will mean they don't
get compiled (I think).

You can rewrite the cond using cl-case or pcase.

Use cl-incf (just incf in Emacs 31) instead of setq and 1+.

I see you're using package--builtin-versions.  I'm not happy to be using
an internal variable -- it got us in trouble before!

-- 
Sean Whitton

[toc] | [prev] | [next] | [standalone]

#1250412

[Multipart message — attachments visible in raw view] — view raw

Sean Whitton <spwhitton@spwhitton.name> writes:

> Hello,
>
> On Mon 16 Jun 2025 at 05:59pm -07, Xiyue Deng wrote:
>
>> Sean Whitton <spwhitton@spwhitton.name> writes:
>>
>>> control: reassign -1 src:emacs
>>> control: retitle -1 emacs: generate versioned Provides for all :core packages
>>>
>>> Hello,
>>>
>>> On Sun 13 Apr 2025 at 08:55pm -07, Xiyue Deng wrote:
>>>
>>>> Option 1 also sounds good to me.
>>>
>>> Cool.  I'm not planning to work on it but I can review patches.
>>>
>>
>> I have made proof-of-concept patches as a pseudo-MR at [1] (this MR is
>> against my own fork just for illustration).  Basically, it uses an Emacs
>> script to generate Debian compatible versions from the builtin package
>> list, which is integrated with `debian-sync' in debian/rules (similar to
>> the handling of debian/copyright.in).  I'm currently making the provide
>> list in emacs-common package which ships the compiled elisp files.
>>
>> Please review and comment.
>
> Very interesting.
>
> Have you considered using debhelper substvars for this?  Like how we
> have ${elpa:Depends}.  It avoids needing a control.in.  A control.in
> isn't the end of the world but it might be cleaner not to have one.
>

Ah didn't think about that.  I think it's better than using control.in
in general, though there are the "unused" warning on other packages
(using "?=" doesn't seem to help).

One advantage of generating d/control that I like is that we can see the
provide list directly and can inspect the diff on regeneration.  To get
something similar, I have each package version pair as one line of
comment in the substvars file before setting emacs:Provides (the
substitute variable I'm using for now).  This way we get something
similar.

Somehow I cannot use debian/substvars or debian/<package>.substvars
(which probably got removed by "clean") and need to use
override_dh_gencontrol to pass the generated substvars file (I'm using
debian/emacs-substvars).  Let me know if there is a better way.

> Don't sharpquote lambdas; unlike in CL, doing so will mean they don't
> get compiled (I think).
>

Ack.  TIL.

> You can rewrite the cond using cl-case or pcase.
>

Done

> Use cl-incf (just incf in Emacs 31) instead of setq and 1+.
>

Done

> I see you're using package--builtin-versions.  I'm not happy to be using
> an internal variable -- it got us in trouble before!
>

Current there seems to be no way to get the builtin package info
otherwise.  Fortunately we can easily verify whether it still works by
running "debian/rules debian-sync" and inspect the diff in
debian/emacs-substvars.

Meanwhile I'll try to file a wishlist bug upstream and migrate in
future.

> -- 
> Sean Whitton

I have updated my branch accordingly.  Also attached the formatted patch
in case it's preferred.

PTAL.

-- 
Regards,
Xiyue Deng

[toc] | [prev] | [next] | [standalone]

#1250456

Hello,

On Wed 18 Jun 2025 at 07:07pm -07, Xiyue Deng wrote:

> Ah didn't think about that.  I think it's better than using control.in
> in general, though there are the "unused" warning on other packages
> (using "?=" doesn't seem to help).

I don't follow.

> One advantage of generating d/control that I like is that we can see the
> provide list directly and can inspect the diff on regeneration.  To get
> something similar, I have each package version pair as one line of
> comment in the substvars file before setting emacs:Provides (the
> substitute variable I'm using for now).  This way we get something
> similar.

True.

> Somehow I cannot use debian/substvars or debian/<package>.substvars
> (which probably got removed by "clean") and need to use
> override_dh_gencontrol to pass the generated substvars file (I'm using
> debian/emacs-substvars).  Let me know if there is a better way.

I don't know how to do it off the top of my head, either.  But I'm sure
it can be done.  Look at how dh_elpa does it, I guess?

>> I see you're using package--builtin-versions.  I'm not happy to be using
>> an internal variable -- it got us in trouble before!
>>
>
> Current there seems to be no way to get the builtin package info
> otherwise.  Fortunately we can easily verify whether it still works by
> running "debian/rules debian-sync" and inspect the diff in
> debian/emacs-substvars.
>
> Meanwhile I'll try to file a wishlist bug upstream and migrate in
> future.

I'm not up for merging that, I'm afraid.  I think we need a public
function first.

-- 
Sean Whitton

[toc] | [prev] | [next] | [standalone]

#1250544

[Multipart message — attachments visible in raw view] — view raw

Hello,

On Thu 19 Jun 2025 at 05:16pm -07, Xiyue Deng wrote:

> Sean Whitton <spwhitton@spwhitton.name> writes:
>
>> Hello,
>>
>> On Wed 18 Jun 2025 at 07:07pm -07, Xiyue Deng wrote:
>>
>>> Ah didn't think about that.  I think it's better than using control.in
>>> in general, though there are the "unused" warning on other packages
>>> (using "?=" doesn't seem to help).
>>
>> I don't follow.
>>
>
> Excerpt from build log (basically it shows a warning of substitution
> variable unused for all binary packages except emacs-common where it's
> used.)

Ah, I see.  Could you define it as the empty string for those packages
and then add a dummy entry to Provides ?

> I have filed the bug upstream[1], but that would be for 31.x.  I was
> hoping this would be useful for Trixie to handle security updates,
> though it may have already been a bit too late for that.

I intended to say, once the interface is settled upstream, I would be
happy to backport it for 30.1 in forky, so we can deploy your change
before the release of 31.1.

-- 
Sean Whitton

[toc] | [prev] | [next] | [standalone]

#1251821

[Multipart message — attachments visible in raw view] — view raw

Hello,

On Thu 03 Jul 2025 at 09:51pm -07, Xiyue Deng wrote:

> It looks like the upstream discussion[1] may still go on for a while:
> the direction has consensus, while the details of the functions are
> still under discussion.
>
> In the meantime, I wonder whether we can move forward on the Debian side
> and have this in Trixie (with an unblock request), which will help users
> who upgrade from Bookworm avoid surprises when older versions of addons
> are installed.  My branch[2] is updated to the latest version of
> upstream bug.  Patches are also attached.

I don't think the full change is appropriate for trixie, and I would
like to see the upstream changes committed to Emacs 31 before
backporting them here.

I think though that we could add the necessary dependency relations
manually to deal with potentially broken upgrades.  There is no need to
do every built-in package, right?  We just need to add values for those
built-in packages that were RM'd for trixie?

-- 
Sean Whitton

[toc] | [prev] | [next] | [standalone]

#1251846

[Multipart message — attachments visible in raw view] — view raw

Hello,

On Fri 04 Jul 2025 at 11:47am -07, Xiyue Deng wrote:

> I think there are values for including other packages which helps with
> security updates.  For example, we have Org 7.9.11 shipped with Emacs
> 30.1; later if a security bug is found in 7.9.11, and Emacs 30.2 ships a
> newer version of Org to fix that, this gets automatically taken care of
> by this.

We would have to upload security updates for both packages anyway,
though.

-- 
Sean Whitton

[toc] | [prev] | [next] | [standalone]

#1252837

[Multipart message — attachments visible in raw view] — view raw

Hello,

On Mon 14 Jul 2025 at 03:33am -07, Xiyue Deng wrote:

> FYI, upstream merged my proposed patch in bug#78844[1].  I'd still
> advocate for its inclusion in Trixie to make the upgrade experience
> smoother for dropped packages (e.g. eglot, project, etc.)  Though I do
> acknowledge that we are already in the late stage of the Trixie release
> cycle, and it's probably safe to maintain the status quo.  If the latter
> is chosen, I wonder whether this may still be a good candidate for a
> future stable update, or only be suitable for a backport?

Unfortunately I don't think we can include it in trixie.  The Release
Team's freeze policy says we should upload only targeted fixes.  And the
Stable Release Managers only allow fixes for bugs of Severity: important
or higher for stable updates, but this bug is wishlist.

But I think we can do the following:

- Let's start preparing the versioned Provides generation in
  experimental.  Assume I'l backport your patch in #78844 to Emacs 30.
  Can you prepare a patch implementing the Provides generation?

- If you want to work on it, we can propose adding manually generated
  Provides to trixie.

-- 
Sean Whitton

[toc] | [prev] | [next] | [standalone]

#1252907

[Multipart message — attachments visible in raw view] — view raw

Hello,

On Tue 15 Jul 2025 at 11:46am -07, Xiyue Deng wrote:

> Xiyue Deng <manphiz@gmail.com> writes:
>
>> [...]
>>> But I think we can do the following:
>>>
>>> - Let's start preparing the versioned Provides generation in
>>>   experimental.  Assume I'l backport your patch in #78844 to Emacs 30.
>>>   Can you prepare a patch implementing the Provides generation?
>>>
>>
>> The patches I attached to [3] implemented this (as well as in my
>> branch[4] which could be newer).  It didn't backport bug#78844 in the
>> Emacs source, but host identify copies of the new functions in the
>> provides/breaks/replaces generation code (which are guarded by fboundp
>> so that the Emacs implementations will be used when available in Emacs
>> 31).  I think this has the advantage that we won't hit any conflicts
>> when upgrading.
>>
>
> I just realized that you have already done the backporting of #78844.
> The patches still work (as it's guarded by fboundp), but do you want me
> to remove the vendored functions?

Yes, please prepare a patch against current debian/d/sid/master.

-- 
Sean Whitton

[toc] | [prev] | [next] | [standalone]

#1252962

Hello Xiyue,

This is great work.  Thank you for the respin.  I'd just like to ask a
few questions about it.

- What is the emacs-provided-package-versions variable for?  I don't
  think it gets set anywhere?

- Why pregenerate emacs-common-substvars?  Why not generate it during
  the build using the version of Emacs we just built?

-- 
Sean Whitton

[toc] | [prev] | [next] | [standalone]

#1253018

[Multipart message — attachments visible in raw view] — view raw

Hello,

On Wed 16 Jul 2025 at 10:30am -07, Xiyue Deng wrote:

> Sean Whitton <spwhitton@spwhitton.name> writes:
>
>> Hello Xiyue,
>>
>> This is great work.  Thank you for the respin.  I'd just like to ask a
>> few questions about it.
>>
>> - What is the emacs-provided-package-versions variable for?  I don't
>>   think it gets set anywhere?
>>
>
> It's used later in the `emacs-provided-package-versions` function, which
> will populate it with a mapping of package name to version.  This will
> then be used later to actually generate the substvars values.

I don't think it needs to be a defvar.  It can be a lexical variable
inside that function.

> It also gets regenerated with `debian/rules debian-sync': notice that
> the build rule is marked `.PHONY'.  I think this should be run on each
> update according to d/README.source.  Maybe this could be added to
> dh_auto_clean in case someone (like me) forgets to run it by hand?

Why not generate it during the build?  That's how other substvars work.

-- 
Sean Whitton

[toc] | [prev] | [next] | [standalone]

#1253130

[Multipart message — attachments visible in raw view] — view raw

Hello,

On Wed 16 Jul 2025 at 11:59am -07, Xiyue Deng wrote:

> This makes the development easier so that I don't have to sbuild it
> every time to test (which takes ~20min for me).  Also, running the
> script needs a working emacs executable (I'm using `/usr/bin/emacs')
> which is not available yet during build (or it can some built emacs
> executable in some build-path, but I think it's a bit too fragile).

That wouldn't be fragile, and it's what we should be doing.  We don't
want to use /usr/bin/emacs.  We want to be sure we are using the latest
information, from the thing we are building.

> And I realize that because I removed the vendored lisp functions to get
> the built-in package information, the script currently cannot work until
> Emacs with the backported package.el is released.  So probably we should
> upload -6 first (in experimental)?

This would also be solved by using the Emacs that's part of the build.

-- 
Sean Whitton

[toc] | [prev] | [next] | [standalone]

#1253375

[Multipart message — attachments visible in raw view] — view raw

Hello Xiyue,

Thanks.  I think you sent the wrong patches but I had a look at your
branch on salsa.  This still isn't quite what I mean.  I think that

- you should be using the Emacs built binary (as you are now doing)
- we should not commit the file emacs-common-substvars to git.
  Instead it should be generated during the build.

So the main thing missing is that the generating of the file should be
incorporated into the build.  Probably it should go into the existing
override_dh_auto_install.

-- 
Sean Whitton

[toc] | [prev] | [next] | [standalone]

#1253459

[Multipart message — attachments visible in raw view] — view raw

Hello,

On Sun 20 Jul 2025 at 11:00am -07, Xiyue Deng wrote:

> Hi Sean,
>
> Sean Whitton <spwhitton@spwhitton.name> writes:
>
>> Hello Xiyue,
>>
>> Thanks.  I think you sent the wrong patches
>
> Oops, sorry.
>
>> but I had a look at your
>> branch on salsa.  This still isn't quite what I mean.  I think that
>>
>> - you should be using the Emacs built binary (as you are now doing)
>> - we should not commit the file emacs-common-substvars to git.
>>   Instead it should be generated during the build.
>>
>> So the main thing missing is that the generating of the file should be
>> incorporated into the build.  Probably it should go into the existing
>> override_dh_auto_install.
>>
>
> I understand that this file is mostly useful during package building.
> Still, I'd like to give another try to propose to have
> debian/emacs-common-substvars committed in git: this will help with
> debugging in the following scenarios:
>
> * When we want to understand why some packages are (or not) being
>   replaces by Emacs, and
>
> * When we want to see what packages are updated between Emacs releases
>   (which is also why I put a package+version per line in the comments).
>
> Without this file, we need to get this information from `apt show' or
> from https://packages.debian.org/ which would be very cumbersome and
> hard to get it right.

I don't mind also committing the information in a text file somewhere
(with a version annotation), but the substvar generation should be
wholly dynamic.

-- 
Sean Whitton

[toc] | [prev] | [next] | [standalone]

#1252908

[Multipart message — attachments visible in raw view] — view raw

Hello,

On Tue 15 Jul 2025 at 10:45am -07, Xiyue Deng wrote:

> Acknowledged.  I thought there was a bug report from Stefano about the
> upgrading issue regarding Emacs backports, which turns out to be just an
> email to debian-backports[1].  That said, Cyril seemed to support this
> idea for a better upgrade experience, though not officially as a RT
> member.  As the issue is real, maybe we can file this as an important
> bug? (It would sound a bit like tricking the system though but not
> really :P)
>
> Still, I'm also OK with postponing this to Forky to avoid causing
> additional issues.

No -- my suggestion is to add manually generated
Provides/Conflicts/etc. to trixie to fix this problem.

-- 
Sean Whitton

[toc] | [prev] | [next] | [standalone]

#1252966

[Multipart message — attachments visible in raw view] — view raw

Hello,

On Tue 15 Jul 2025 at 04:15pm -07, Xiyue Deng wrote:

> Sean Whitton <spwhitton@spwhitton.name> writes:
>
>> Hello,
>>
>> On Tue 15 Jul 2025 at 10:45am -07, Xiyue Deng wrote:
>>
>>> Acknowledged.  I thought there was a bug report from Stefano about the
>>> upgrading issue regarding Emacs backports, which turns out to be just an
>>> email to debian-backports[1].  That said, Cyril seemed to support this
>>> idea for a better upgrade experience, though not officially as a RT
>>> member.  As the issue is real, maybe we can file this as an important
>>> bug? (It would sound a bit like tricking the system though but not
>>> really :P)
>>>
>>> Still, I'm also OK with postponing this to Forky to avoid causing
>>> additional issues.
>>
>> No -- my suggestion is to add manually generated
>> Provides/Conflicts/etc. to trixie to fix this problem.
>>
>
> Just to confirm: is the substvars based solution OK for Trixie (as done
> in my updated patchs in my other email), or were you expecting the
> provides/replaces info directly put in d/control (like my earlier
> approach based on d/control.in but only keep the generated d/control)?

I mean the latter.

-- 
Sean Whitton

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.bugs.dist

csiph-web