Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.bugs.dist > #1113757 > unrolled thread

Bug#1017739: emacs-lucid cannot start after upgrade

Back to article view | Back to linux.debian.bugs.dist

Contents

  Bug#1017739: emacs-lucid cannot start after upgrade Russ Allbery <rra@debian.org> - 2022-08-19 20:50 +0200
    Bug#1017739: emacs-lucid cannot start after upgrade Stefan Monnier <monnier@iro.umontreal.ca> - 2022-08-22 01:40 +0200
      Bug#1017739: emacs-lucid cannot start after upgrade Andrea Corallo <akrl@sdf.org> - 2022-08-22 11:30 +0200
    Bug#1017739: emacs-lucid cannot start after upgrade Adam Lackorzynski <adam@os.inf.tu-dresden.de> - 2022-08-22 21:30 +0200
    Bug#1017739: emacs-lucid cannot start after upgrade Rob Browning <rlb@defaultvalue.org> - 2022-08-23 06:30 +0200
      Bug#1017739: emacs-lucid cannot start after upgrade Russ Allbery <rra@debian.org> - 2022-08-23 06:50 +0200
        Bug#1017739: emacs-lucid cannot start after upgrade Rob Browning <rlb@defaultvalue.org> - 2022-08-23 07:10 +0200
    Bug#1017739: emacs-lucid cannot start after upgrade Adam Lackorzynski <adam@os.inf.tu-dresden.de> - 2022-08-23 11:10 +0200
    Bug#1017739: emacs-lucid cannot start after upgrade Stefan Monnier <monnier@iro.umontreal.ca> - 2022-08-23 16:40 +0200
      Bug#1017739: emacs-lucid cannot start after upgrade Rob Browning <rlb@defaultvalue.org> - 2022-08-24 04:00 +0200
        Bug#1017739: emacs-lucid cannot start after upgrade Stefan Monnier <monnier@iro.umontreal.ca> - 2022-08-24 04:10 +0200
          Bug#1017739: emacs-lucid cannot start after upgrade Russ Allbery <rra@debian.org> - 2022-08-24 05:10 +0200
            Bug#1017739: emacs-lucid cannot start after upgrade Stefan Monnier <monnier@iro.umontreal.ca> - 2022-08-24 14:00 +0200
            Bug#1017739: emacs-lucid cannot start after upgrade David Bremner <bremner@debian.org> - 2022-09-12 00:20 +0200
              Bug#1017739: emacs-lucid cannot start after upgrade Stefan Monnier <monnier@iro.umontreal.ca> - 2022-09-12 03:40 +0200
                Bug#1017739: emacs-lucid cannot start after upgrade David Bremner <bremner@debian.org> - 2022-09-13 02:10 +0200

#1113757 — Bug#1017739: emacs-lucid cannot start after upgrade

Package: emacs-lucid
Version: 1:28.1+1-1
Severity: grave
X-Debbugs-Cc: rra@debian.org

The 28.1 version of emacs-lucid fails on startup with a cryptic error
message:

% emacs
Cannot find suitable directory for output in ‘comp-native-load-path’.

Running emacs -q allows it to start, but it still reports the same error
immediately during startup, and attempting to do someting as basic as
M-x set-variable (to try to enable debug-on-error) fails with a similar
error message:

defalias: Cannot find suitable directory for output in ‘comp-native-load-path’.

emacs -q --no-site-file avoids the error on startup, but M-x set-variable
still immediately fails with the same error.

-- System Information:
Debian Release: bookworm/sid
  APT prefers unstable
  APT policy: (990, 'unstable'), (500, 'unstable-debug'), (1, 'experimental')
Architecture: amd64 (x86_64)

Kernel: Linux 5.18.0-4-amd64 (SMP w/12 CPU threads; PREEMPT)
Kernel taint flags: TAINT_WARN
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled

Versions of packages emacs-lucid depends on:
ii  emacs-bin-common     1:28.1+1-1
ii  emacs-common         1:28.1+1-1
ii  libacl1              2.3.1-1
ii  libasound2           1.2.7.2-1
ii  libc6                2.34-4
ii  libcairo2            1.16.0-6
ii  libdbus-1-3          1.14.0-2
ii  libfontconfig1       2.13.1-4.4
ii  libfreetype6         2.12.1+dfsg-3
ii  libgccjit0           12.1.0-8
ii  libgdk-pixbuf-2.0-0  2.42.9+dfsg-1
ii  libgif7              5.2.1-2.5
ii  libglib2.0-0         2.72.3-1+b1
ii  libgmp10             2:6.2.1+dfsg1-1
ii  libgnutls30          3.7.7-2
ii  libgpm2              1.20.7-10
ii  libharfbuzz0b        2.7.4-1+b1
ii  libice6              2:1.0.10-1
ii  libjansson4          2.14-2
ii  libjpeg62-turbo      1:2.1.2-1
ii  liblcms2-2           2.13.1-1
ii  libm17n-0            1.8.0-4
ii  libotf1              0.9.16-3
ii  libpng16-16          1.6.37-5
ii  librsvg2-2           2.54.4+dfsg-1
ii  libselinux1          3.4-1+b1
ii  libsm6               2:1.2.3-1
ii  libsystemd0          251.4-1
ii  libtiff5             4.4.0-4
ii  libtinfo6            6.3+20220423-2
ii  libx11-6             2:1.8.1-2
ii  libxext6             2:1.3.4-1
ii  libxfixes3           1:6.0.0-1
ii  libxinerama1         2:1.1.4-3
ii  libxml2              2.9.14+dfsg-1+b1
ii  libxmu6              2:1.1.3-3
ii  libxpm4              1:3.5.12-1
ii  libxrandr2           2:1.5.2-2+b1
ii  libxrender1          1:0.9.10-1.1
ii  libxt6               1:1.2.1-1
ii  xaw3dg               1.5+F-1
ii  zlib1g               1:1.2.11.dfsg-4.1

Versions of packages emacs-lucid recommends:
pn  fonts-noto-color-emoji  <none>

Versions of packages emacs-lucid suggests:
pn  emacs-common-non-dfsg  <none>

-- no debconf information

[toc] | [next] | [standalone]

#1114028

Russ Allbery [2022-08-19 11:42:30] wrote:
> The 28.1 version of emacs-lucid fails on startup with a cryptic error
> message:
>
> % emacs
> Cannot find suitable directory for output in ‘comp-native-load-path’.
>
> Running emacs -q allows it to start, but it still reports the same error
> immediately during startup, and attempting to do someting as basic as
> M-x set-variable (to try to enable debug-on-error) fails with a similar
> error message:
>
> defalias: Cannot find suitable directory for output in ‘comp-native-load-path’.
>
> emacs -q --no-site-file avoids the error on startup, but M-x set-variable
> still immediately fails with the same error.

This error is signaled by `comp-el-to-eln-filename` and it usually
indicates that that function was unable to find a writable directory in
which to put the auto-generated `.eln` native-compiled files.

I don't know why it fails to find a writable directory.  Maybe

    emacs --debug-init` would help
or
    emacs -e '(message "%S" comp-native-load-path)'

might help track down the origin of the problem.

This said, Emacs shouldn't become unusable in such a circumstance, so we
should maybe use a patch along the lines of the one below (not sure if
it wraps the relevant invocation of `comp-el-to-eln-filename`, tho).


        Stefan


diff --git a/lisp/emacs-lisp/comp.el b/lisp/emacs-lisp/comp.el
index 304ea8cc6c1..5a599f8a0e8 100644
--- a/lisp/emacs-lisp/comp.el
+++ b/lisp/emacs-lisp/comp.el
@@ -3923,10 +3926,13 @@ comp-run-async-workers
                        "`comp-files-queue' should be \".el\" files: %s"
                        source-file)
          when (or native-comp-always-compile
-                  load ; Always compile when the compilation is
-                       ; commanded for late load.
-                  (file-newer-than-file-p
-                   source-file (comp-el-to-eln-filename source-file)))
+                  load        ; Always compile when the compilation is
+                              ; commanded for late load.
+                  ;; Skip compilation if `comp-el-to-eln-filename' fails
+                  ;; to find a writable directory.
+                  (with-demoted-errors "Async compilation :%S"
+                    (file-newer-than-file-p
+                     source-file (comp-el-to-eln-filename source-file))))
          do (let* ((expr `((require 'comp)
                            (setq comp-async-compilation t)
                            (setq warning-fill-column most-positive-fixnum)

[toc] | [prev] | [next] | [standalone]

#1114072

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Russ Allbery [2022-08-19 11:42:30] wrote:
>> The 28.1 version of emacs-lucid fails on startup with a cryptic error
>> message:
>>
>> % emacs
>> Cannot find suitable directory for output in ‘comp-native-load-path’.
>>
>> Running emacs -q allows it to start, but it still reports the same error
>> immediately during startup, and attempting to do someting as basic as
>> M-x set-variable (to try to enable debug-on-error) fails with a similar
>> error message:
>>
>> defalias: Cannot find suitable directory for output in ‘comp-native-load-path’.
>>
>> emacs -q --no-site-file avoids the error on startup, but M-x set-variable
>> still immediately fails with the same error.
>
> This error is signaled by `comp-el-to-eln-filename` and it usually
> indicates that that function was unable to find a writable directory in
> which to put the auto-generated `.eln` native-compiled files.
>
> I don't know why it fails to find a writable directory.  Maybe
>
>     emacs --debug-init` would help
> or
>     emacs -e '(message "%S" comp-native-load-path)'
>
> might help track down the origin of the problem.
>
> This said, Emacs shouldn't become unusable in such a circumstance, so we
> should maybe use a patch along the lines of the one below (not sure if
> it wraps the relevant invocation of `comp-el-to-eln-filename`, tho).

Agree Emacs should stay usable and I think the one you've identified
should be the right invocation to be wrapped.

Best Regards

  Andrea

[toc] | [prev] | [next] | [standalone]

#1114153

On Fri Aug 19, 2022 at 11:42:30 -0700, Russ Allbery wrote:
> Package: emacs-lucid
> Version: 1:28.1+1-1
> Severity: grave
> X-Debbugs-Cc: rra@debian.org
> 
> The 28.1 version of emacs-lucid fails on startup with a cryptic error
> message:
> 
> % emacs
> Cannot find suitable directory for output in ‘comp-native-load-path’.

I had the same issue. Turned out my personal .emacs.d/eln-cache
directory and its folders belonged to root:

$ ls -la $HOME/.emacs.d/eln-cache
total 16
drwxr-xr-x 4 root root  4096 Aug 21 19:32 .
drwx------ 4 adam users 4096 Aug 19 19:43 ..
drwxr-xr-x 2 root root  4096 Aug 21 19:32 28.1-20961986
drwxr-xr-x 2 root root  4096 Aug 19 19:43 28.1-aa5da5cc

chown'ing it to me fixed it.


Thanks,
Adam

[toc] | [prev] | [next] | [standalone]

#1114186

Russ Allbery <rra@debian.org> writes:

> Oh, good catch!  The same thing was true of mine.
>
> I think the dates were when I upgraded Emacs.  Let me take a guess: are
> you also old-school and use su from a regular user account when installing
> new packages?  HOME gets overridden by su, but LOGNAME and USER do not,
> and I suspect something in Emacs is deciding where to write files based on
> USER, and the installation process of emacs-lucid creates the eln-cache
> directory for some reason.

Agreed, nice catch, very glad y'all found this.  And not sure it'd be
appropriate in your case, and you may well know, but adding the "-"
argument to su may avoid the issue for you.

> I'm downgrading the severity of this bug because I suspect the average
> user using sudo may not run into it (although the maintainer should feel
> free to raise the severity again if they disagree).  If I'm right, the
> best place to solve the problem may be in the Debian maintainer scripts,
> overwriting USER (and possibly LOGNAME, not sure if it matters) to some
> safe value like root while performing the installation.  This may also be
> necessary to do when installing Emacs add-on packges; I'm not sure.

Possibly, but I'd assumed the installation processs should expect a
"normal" root environment, and if so, "su" without the dash wouldn't
qualify.  Otherwise, random user .bashrc changes (that su doesn't reset)
could affect the install.

> I've removed the directory with the wrong ownership and upgraded again
> with USER and LOGNAME set to root, and now everything works fine.  (Well,
> my laptop gets extremely hot the first time I start the new Emacs, but I
> assume that's expected for the new compilation system.)

Yes, I believe that should only happen once per .el file, per user[1],
but it's not cheap.

[1] ...until/unless we decide to ship NATIVE_FULL_AOT packages (all
    upstream files precompiled).  But if nothing else, that's not ready
    for broad use yet.

-- 
Rob Browning
rlb @defaultvalue.org and @debian.org
GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A
GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4

[toc] | [prev] | [next] | [standalone]

#1114187

Rob Browning <rlb@defaultvalue.org> writes:

> Possibly, but I'd assumed the installation processs should expect a
> "normal" root environment, and if so, "su" without the dash wouldn't
> qualify.  Otherwise, random user .bashrc changes (that su doesn't reset)
> could affect the install.

For what it's worth, this is the first package that I recall having
trouble with during installation, and I've been using Debian for quite a
while.  :)

That doesn't mean you're wrong -- you've got a good point and I should get
in the habit of using -.  But I do wonder if there's a bug or at least
some surprising behavior here in Emacs.  Blindly using USER when HOME is
set correctly is pretty unusual (and, for whatever it's worth, contrary to
the BaseDir specification, not that Emacs is currently following that).
Generally HOME should override USER when used to locate the current user's
home directory.

-- 
Russ Allbery (rra@debian.org)              <https://www.eyrie.org/~eagle/>

[toc] | [prev] | [next] | [standalone]

#1114188

Russ Allbery <rra@debian.org> writes:

> For what it's worth, this is the first package that I recall having
> trouble with during installation, and I've been using Debian for quite a
> while.  :)
>
> That doesn't mean you're wrong -- you've got a good point and I should get
> in the habit of using -.

Oh, I don't know whether I'm right or wrong there either, just
describing my defensive behavior :)

> But I do wonder if there's a bug or at least some surprising behavior
> here in Emacs.  Blindly using USER when HOME is set correctly is
> pretty unusual (and, for whatever it's worth, contrary to the BaseDir
> specification, not that Emacs is currently following that).  Generally
> HOME should override USER when used to locate the current user's home
> directory.

Ahh, yeah, speaking more generally, I fully expect we may hit and have
to (help) fix a relatively large number of edge cases as a result of the
addition of native compilation.  Lots of new moving parts, and the
arrangement in Debian is likely different from what upstream typically
tests in important ways.

(cf. the other problem right now where installs segfault in some cases
 if you don't already have emacs-el installed.  That's likely an
 upstream bug of some kind.)

-- 
Rob Browning
rlb @defaultvalue.org and @debian.org
GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A
GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4

[toc] | [prev] | [next] | [standalone]

#1114207

On Mon Aug 22, 2022 at 14:53:57 -0700, Russ Allbery wrote:
> Control: severity -1 important
> 
> Adam Lackorzynski <adam@os.inf.tu-dresden.de> writes:
> 
> > I had the same issue. Turned out my personal .emacs.d/eln-cache
> > directory and its folders belonged to root:
> 
> > $ ls -la $HOME/.emacs.d/eln-cache
> > total 16
> > drwxr-xr-x 4 root root  4096 Aug 21 19:32 .
> > drwx------ 4 adam users 4096 Aug 19 19:43 ..
> > drwxr-xr-x 2 root root  4096 Aug 21 19:32 28.1-20961986
> > drwxr-xr-x 2 root root  4096 Aug 19 19:43 28.1-aa5da5cc
> 
> > chown'ing it to me fixed it.
> 
> Oh, good catch!  The same thing was true of mine.
> 
> I think the dates were when I upgraded Emacs.  Let me take a guess: are
> you also old-school and use su from a regular user account when installing
> new packages?  HOME gets overridden by su, but LOGNAME and USER do not,
> and I suspect something in Emacs is deciding where to write files based on
> USER, and the installation process of emacs-lucid creates the eln-cache
> directory for some reason.

Indeed, yes. I'd follow the lines here that a Debian package should
ensure that content generated for global use (which I assume the package
positinst is generating) shall be in a location accessible for everyone.

> I'm downgrading the severity of this bug because I suspect the average
> user using sudo may not run into it (although the maintainer should feel
> free to raise the severity again if they disagree).  If I'm right, the
> best place to solve the problem may be in the Debian maintainer scripts,
> overwriting USER (and possibly LOGNAME, not sure if it matters) to some
> safe value like root while performing the installation.  This may also be
> necessary to do when installing Emacs add-on packges; I'm not sure.
> 
> I've removed the directory with the wrong ownership and upgraded again
> with USER and LOGNAME set to root, and now everything works fine.  (Well,
> my laptop gets extremely hot the first time I start the new Emacs, but I
> assume that's expected for the new compilation system.)
> 
> -- 
> Russ Allbery (rra@debian.org)              <https://www.eyrie.org/~eagle/>

Adam

[toc] | [prev] | [next] | [standalone]

#1114244

>>     emacs -e '(message "%S" comp-native-load-path)'

Sorry, that should have been:

    emacs --eval '(message "%S" comp-native-load-path)'

(which you can then try with `-q` and friends if needed).

> I'm a bit mystified as to why everyone else on Debian isn't seeing this.
> I would have assumed it must be something in my startup files that is
> incompatible with the latest release of Emacs, except I thought -q
> --no-site-file should completely disable loading anything from my local
> configuration.

My crystal ball suggests maybe your ~/.emacs.d is marked as read-only?


        Stefan

[toc] | [prev] | [next] | [standalone]

#1114295

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>>>     emacs -e '(message "%S" comp-native-load-path)'
>
> Sorry, that should have been:
>
>     emacs --eval '(message "%S" comp-native-load-path)'
>
> (which you can then try with `-q` and friends if needed).
>
>> I'm a bit mystified as to why everyone else on Debian isn't seeing this.
>> I would have assumed it must be something in my startup files that is
>> incompatible with the latest release of Emacs, except I thought -q
>> --no-site-file should completely disable loading anything from my local
>> configuration.
>
> My crystal ball suggests maybe your ~/.emacs.d is marked as read-only?

I don't know if this helps, but in addition to the recent issue we've
identified where not having emacs-el installed can cause emacs from the
current 28 packages to segfault on startup, people just figured out that
if you try to install emacs via "su apt install emacs" (note the lack of
a "-" argument to su) from a non-root account, emacs can end up failing
for that account because the install ends up creating root-only
directories (for I think the eln files, etc.) in ~.

I'm not sure whether that's something that we'd expect to support
(i.e. apt installs via sudo without -i or su without -), but I wanted to
mention it since it's been reported (I think) more than once, and in
case it indicates something that emacs might want to change (use of USER
vs HOME, etc.).

No personal position there right now either way.

Thanks
-- 
Rob Browning
rlb @defaultvalue.org and @debian.org
GPG as of 2011-07-10 E6A9 DA3C C9FD 1FF8 C676 D2C4 C0F0 39E9 ED1B 597A
GPG as of 2002-11-03 14DD 432F AE39 534D B592 F9A0 25C8 D377 8C7E 73A4

[toc] | [prev] | [next] | [standalone]

#1114297

> I'm not sure whether that's something that we'd expect to support
> (i.e. apt installs via sudo without -i or su without -), but I wanted to
> mention it since it's been reported (I think) more than once, and in
> case it indicates something that emacs might want to change (use of USER
> vs HOME, etc.).

You might want to report it, at least so Emacs maintainers are aware of
the problem.  And also maybe the startup code can then report a more
specific error when it encounters a `~/.emacs.d/eln-cache` that belongs
to some other user.

Personally I think running something like Emacs as root with $HOME
pointing to some other user's home directory is a "pilot error", but
then again I'm one of those who didn't notice the infamous "su" change
because I never use `su` without `-` (because I simply don't
understand what is its intended semantics).


        Stefan

[toc] | [prev] | [next] | [standalone]

#1114299

Stefan Monnier <monnier@iro.umontreal.ca> writes:

> Personally I think running something like Emacs as root with $HOME
> pointing to some other user's home directory is a "pilot error", but
> then again I'm one of those who didn't notice the infamous "su" change
> because I never use `su` without `-` (because I simply don't understand
> what is its intended semantics).

To be clear, $HOME was pointing correctly at /root.  Emacs ignored $HOME
and instead created $USER/.emacs.d/eln-cache, which was surprising.  (su
without - still changes $HOME but not $USER.)

Or at least I think that's happened, and the correct thing happened when I
overrode $USER (and $LOGNAME) and then did the same thing again with
otherwise the same environment variables.

-- 
Russ Allbery (rra@debian.org)              <https://www.eyrie.org/~eagle/>

[toc] | [prev] | [next] | [standalone]

#1114338

> To be clear, $HOME was pointing correctly at /root.  Emacs ignored $HOME
> and instead created $USER/.emacs.d/eln-cache, which was surprising.  (su
> without - still changes $HOME but not $USER.)

Oh, yes, that's a behavior we have in Emacs which I also find horrible
(and to add insult to injury it takes extra work to implement it; it's
only done when running as root).

I encourage you to file this as a bug.


        Stefan

[toc] | [prev] | [next] | [standalone]

#1116891

[Multipart message — attachments visible in raw view] — view raw

I think what we want in Debian is no native compilation when installing
Emacs. There is really no benefit for most people to caching the eln
files in ~root, and there are possible bad effects as demonstrated by
this bug.

Attached is a proposed patch to emacsen-common. I'll reassign this bug
there for now.

[toc] | [prev] | [next] | [standalone]

#1116900

> -"$flavor" --no-init-file --no-site-file -batch -f batch-byte-compile \
> +"$flavor" --no-init-file --no-site-file -batch -f batch-byte-compile --eval='(setq native-comp-speed -1)' \

I don't think `(setq native-comp-speed -1)` does what we need here:

     [...]
     options of the compiler.  The value −1 means disable
     native-compilation: functions and files will be only byte-compiled;
     however, the ‘*.eln’ files will still be produced, they will just
     contain the compiled code in bytecode form.  (This can be achieved
     [...]

I'm not sure what's the best way to go about disabling automatic native
compilation, but I think `native-comp-deferred-compilation-deny-list`
might work.

Of course, another way to fix this is to actually fix the original
problem where the wrong directory (i.e. `~$USER/.emacs.d` instead of
`$HOME/.emacs.d`) is chosen.  This original problem affects other
circumstances where Emacs will write to the `.emacs.d` directory so it
would be good to fix it more generally.

A "quick fix" is to start Emacs with an explicit
`--init-directory` argument.

It's too late to get this fixed for Emacs-28,2, but this really should
be reported as a bug.


        Stefan

[toc] | [prev] | [next] | [standalone]

#1117004

Stefan Monnier <monnier@iro.umontreal.ca> writes:

>
> Of course, another way to fix this is to actually fix the original
> problem where the wrong directory (i.e. `~$USER/.emacs.d` instead of
> `$HOME/.emacs.d`) is chosen.  This original problem affects other
> circumstances where Emacs will write to the `.emacs.d` directory so it
> would be good to fix it more generally.
>

I think we probably want both, at least in Debian. Even if that is
fixed, emacs would then write into some user's (possibly root's) home
directory during package installation. That seems like bug to me.

d

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.bugs.dist

csiph-web