Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.bugs.dist > #1200633 > unrolled thread

Bug#1073038: po4a: Fails due to undefined subroutine Locale::Po4a::Pod::dgettext

Back to article view | Back to linux.debian.bugs.dist

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  Bug#1073038: po4a: Fails due to undefined subroutine Locale::Po4a::Pod::dgettext Martin Quinson <Martin.Quinson@ens-rennes.fr> - 2024-06-12 10:50 +0200
    Bug#1073038: po4a: Fails due to undefined subroutine Locale::Po4a::Pod::dgettext Guillem Jover <guillem@debian.org> - 2024-06-12 15:50 +0200
      Bug#1073038: po4a: Fails due to undefined subroutine Locale::Po4a::Pod::dgettext Guillem Jover <guillem@debian.org> - 2024-06-12 17:20 +0200
        Bug#1073038: po4a: Fails due to undefined subroutine Locale::Po4a::Pod::dgettext Guillem Jover <guillem@debian.org> - 2024-06-13 00:40 +0200
          Bug#1073038: po4a: Fails due to undefined subroutine Locale::Po4a::Pod::dgettext Martin Quinson <Martin.Quinson@ens-rennes.fr> - 2024-06-13 00:50 +0200
            Bug#1073038: po4a: Fails due to undefined subroutine Locale::Po4a::Pod::dgettext Guillem Jover <guillem@debian.org> - 2024-06-13 01:20 +0200
              Bug#1073038: po4a: Fails due to undefined subroutine Locale::Po4a::Pod::dgettext Martin Quinson <Martin.Quinson@ens-rennes.fr> - 2024-06-13 12:40 +0200

#1200633 — Bug#1073038: po4a: Fails due to undefined subroutine Locale::Po4a::Pod::dgettext

[Multipart message — attachments visible in raw view] — view raw

Le mercredi 12 juin 2024 à 04:47 +0200, Guillem Jover a écrit :
> Package: po4a
> Version: 0.70
> Severity: serious
> 
> Hi!
> 
> When building dpkg, it now fails with something like this:
> 
>   ,---
>   Making check in man
>   make[1]: Entering directory '<UPDIR>/dpkg/man'
>   /usr/bin/po4a --previous --srcdir . --destdir . --no-backups --porefs file
> --msgmerge-opt=--add-location=file --package-name dpkg-man --package-version
> 1.22.6-80-gec07 --copyright-holder "Dpkg Developers" --msgid-bugs-address
> debian-dpkg@lists.debian.org  ./po/po4a.cfg
>   Undefined subroutine &Locale::Po4a::Pod::dgettext called at
> /usr/share/perl5/Locale/Po4a/Pod.pm line 94, <$fh> line 21.
>   make[1]: *** [Makefile:902: man.stamp] Error 255
>   make[1]: Leaving directory '<UPDIR>/dpkg/man'
>   `---
> 
> It seems the module is missing some import.

Hello,

the attached patch could help, but I fail to reproduce the failure in the first
place. I built dpkg v1.22.6 against po4a v0.72 and it builds with no failure.
The "Making check in man" section LGTM but I'm not sure of what it should look
like.

I tried the git version of po4a with git-pbuilder, but it complains about not
being able to determine whether it's a git or a release build. I must be
missing some config on my side.

Could you please do the following steps, please?

 * Try whether the failure still happens with the latest po4a
 * Try whether the attached patch applied to the latest po4a helps
 * Tell me more details about how to reproduce the failure

Thanks in advance, and sorry for the inconvenience. To me, this part of po4a
code was not changed in years, so I need help to understand the issue.

Mt

[toc] | [next] | [standalone]

#1200652

Hi!

On Wed, 2024-06-12 at 10:36:52 +0200, Martin Quinson wrote:
> Le mercredi 12 juin 2024 à 04:47 +0200, Guillem Jover a écrit :
> > Package: po4a
> > Version: 0.70
> > Severity: serious

> > When building dpkg, it now fails with something like this:
> > 
> >   ,---
> >   Making check in man
> >   make[1]: Entering directory '<UPDIR>/dpkg/man'
> >   /usr/bin/po4a --previous --srcdir . --destdir . --no-backups --porefs file
> > --msgmerge-opt=--add-location=file --package-name dpkg-man --package-version
> > 1.22.6-80-gec07 --copyright-holder "Dpkg Developers" --msgid-bugs-address
> > debian-dpkg@lists.debian.org  ./po/po4a.cfg
> >   Undefined subroutine &Locale::Po4a::Pod::dgettext called at
> > /usr/share/perl5/Locale/Po4a/Pod.pm line 94, <$fh> line 21.
> >   make[1]: *** [Makefile:902: man.stamp] Error 255
> >   make[1]: Leaving directory '<UPDIR>/dpkg/man'
> >   `---
> > 
> > It seems the module is missing some import.

> the attached patch could help, but I fail to reproduce the failure in the first
> place. I built dpkg v1.22.6 against po4a v0.72 and it builds with no failure.
> The "Making check in man" section LGTM but I'm not sure of what it should look
> like.

I guess that's because the tarballl has already been bootstrapped. Building
from git should reproduce it. But in any case if that's too cumbersome I'm
happy to keep testing stuff.

> I tried the git version of po4a with git-pbuilder, but it complains about not
> being able to determine whether it's a git or a release build. I must be
> missing some config on my side.

Perhaps you meant you tried to build the git version of dpkg here?
(If so you'd need to run ./autogen first.)

> Could you please do the following steps, please?

>  * Try whether the failure still happens with the latest po4a
>  * Try whether the attached patch applied to the latest po4a helps
>  * Tell me more details about how to reproduce the failure
> 
> Thanks in advance, and sorry for the inconvenience. To me, this part of po4a
> code was not changed in years, so I need help to understand the issue.

It looked like at least the initial regression was introduced in po4a
commit b2333d54845976f4804f024b5cea61db16fb4f36.

> diff --git a/lib/Locale/Po4a/Pod.pm b/lib/Locale/Po4a/Pod.pm
> index 4b3dc042..bd3d5c1a 100644
> --- a/lib/Locale/Po4a/Pod.pm
> +++ b/lib/Locale/Po4a/Pod.pm
> @@ -27,6 +27,8 @@ use vars qw(@ISA);
>  
>  use Carp qw(croak confess);
>  
> +use Locale::Po4a::Common qw(dgettext);
> +
>  sub initialize { }
>  
>  sub translate {

I hot-edited this in my installed system and then got:

  ,---
    PO4A   man.stamp
  Undefined subroutine &Locale::Po4a::Pod::wrap_mod called at /usr/share/perl5/Locale/Po4a/Pod.pm line 96, <$fh> line 21.
  make[2]: *** [Makefile:902: man.stamp] Error 255
  `---

I then instead removed the import restrictions and simply added:

  ,---
  use Locale::Po4a::Common;
  `---

That fixed that issue but then it fails with:

  ,---
    PO4A   man.stamp
  po4a::pod: The file deb-buildinfo.pod declares utf8 as encoding, but you 
             provided UTF-8 as master charset. Please change either setting.
   at /usr/bin/po4a line 1663.
  make[2]: *** [Makefile:902: man.stamp] Error 255
  `---

Thanks,
Guillem

[toc] | [prev] | [next] | [standalone]

#1200666

Hi!

On Wed, 2024-06-12 at 15:42:57 +0200, Guillem Jover wrote:
> On Wed, 2024-06-12 at 10:36:52 +0200, Martin Quinson wrote:
> > Le mercredi 12 juin 2024 à 04:47 +0200, Guillem Jover a écrit :
> > > Package: po4a
> > > Version: 0.70
> > > Severity: serious

> > Could you please do the following steps, please?
> 
> >  * Try whether the failure still happens with the latest po4a
> >  * Try whether the attached patch applied to the latest po4a helps
> >  * Tell me more details about how to reproduce the failure
> > 
> > Thanks in advance, and sorry for the inconvenience. To me, this part of po4a
> > code was not changed in years, so I need help to understand the issue.
> 
> It looked like at least the initial regression was introduced in po4a
> commit b2333d54845976f4804f024b5cea61db16fb4f36.

Ah, and missed that the later error I got was precisely the one being
emitted by that commit, which if you do not trigger then you'd not see
the earlier undefined symbol errors.

> > diff --git a/lib/Locale/Po4a/Pod.pm b/lib/Locale/Po4a/Pod.pm
> > index 4b3dc042..bd3d5c1a 100644
> > --- a/lib/Locale/Po4a/Pod.pm
> > +++ b/lib/Locale/Po4a/Pod.pm
> > @@ -27,6 +27,8 @@ use vars qw(@ISA);
> >  
> >  use Carp qw(croak confess);
> >  
> > +use Locale::Po4a::Common qw(dgettext);
> > +
> >  sub initialize { }
> >  
> >  sub translate {
> 
> I hot-edited this in my installed system and then got:
> 
>   ,---
>     PO4A   man.stamp
>   Undefined subroutine &Locale::Po4a::Pod::wrap_mod called at /usr/share/perl5/Locale/Po4a/Pod.pm line 96, <$fh> line 21.
>   make[2]: *** [Makefile:902: man.stamp] Error 255
>   `---
> 
> I then instead removed the import restrictions and simply added:
> 
>   ,---
>   use Locale::Po4a::Common;
>   `---
> 
> That fixed that issue but then it fails with:
> 
>   ,---
>     PO4A   man.stamp
>   po4a::pod: The file deb-buildinfo.pod declares utf8 as encoding, but you 
>              provided UTF-8 as master charset. Please change either setting.
>    at /usr/bin/po4a line 1663.
>   make[2]: *** [Makefile:902: man.stamp] Error 255
>   `---

I think the charset comparison is too naive though, and does not cover
for example any aliases listed in «man Encode::Supported». I think a
better comparison logic could look like this:

  ,---
  use Encode;

  my $enc_charset = Encode::find_encoding($charset);
  my $enc_master_charset = Encode::find_encoding($master_charset);

  say 'match' if $enc_charset->mime_name eq $enc_master_charset->mime_name;
  `---

Unfortunately neither Encode::find_encoding()->name nor
Encode::resolve_alias() seem helpful here because they return
"utf-8-strict" for "UTF-8" which will not match against "utf-8" for
the canonical "utf8".

I'm not sure whether lib/Locale/Po4a/Xml.pod might need a similar
treatment, because that might depend on what encodings are supported
in there, against what is supported in po4a I guess.

Thanks,
Guillem

[toc] | [prev] | [next] | [standalone]

#1200730

Hi!

On Thu, 2024-06-13 at 00:02:43 +0200, Martin Quinson wrote:
> Le mercredi 12 juin 2024 à 17:14 +0200, Guillem Jover a écrit :
> > I think the charset comparison is too naive though, and does not cover
> > for example any aliases listed in «man Encode::Supported». I think a
> > better comparison logic could look like this:
> > 
> >   ,---
> >   use Encode;
> > 
> >   my $enc_charset = Encode::find_encoding($charset);
> >   my $enc_master_charset = Encode::find_encoding($master_charset);
> > 
> >   say 'match' if $enc_charset->mime_name eq $enc_master_charset->mime_name;
> >   `---
> > 
> > Unfortunately neither Encode::find_encoding()->name nor
> > Encode::resolve_alias() seem helpful here because they return
> > "utf-8-strict" for "UTF-8" which will not match against "utf-8" for
> > the canonical "utf8".
> 
> The problem is that Perl has a rather unexpected behavior wrt utf8, UTF-8 and
> UTF8. These names are not aliases of others in Perl. See
> https://perldoc.perl.org/Encode#UTF-8-vs.-utf8-vs.-UTF8

Sure, but here I think this does not matter (in theory), because it
depends on how the pod parser interprets the encoding name, and from
checking the perl code it seems it maps /utf-?8/i to ":encoding(UTF-8)".
So they are really treated the same, at least when it comes to POD, that
does not mean perl has that distinction for the encoding in other
contexts. The following two places I found that are in charge of parsing
POD lowercase and remove or ignore - (and _) when parsing the encoding
name:

  perl/Pod-Perldoc/lib/Pod/Perldoc.pm:set_encoding
  https://sources.debian.org/src/perl/5.38.2-5/cpan/Pod-Perldoc/lib/Pod/Perldoc.pm/#L1054

  perl/Pod-Simple/lib/Pod/Simple/BlackBox.pm:_handle_encoding_line
  https://sources.debian.org/src/perl/5.38.2-5/cpan/Pod-Simple/lib/Pod/Simple/BlackBox.pm/#L604

> So I don't think that you will find a piece of Perl code that will allow to
> merge these names to declare that they are equivalent. In some sense, that's
> the essence of all issues I introduced in po4a v0.70 when introducing perlIO to
> control the encoding issues. I went from a lax system being utf8 by default but
> happily taking latin-1 encoding files to a much stricter UTF-8 system by
> default and forcing the user to specify if they want another encoding.

I think the problem is that this might be conflating independent contexts,
such as IO, perl source code, POD, etc.

> Instead, I just pushed a commit changing the error message when charsets are
> utf8 and UTF-8 to insist on the difference between these encodings in Perl.
> https://github.com/mquinson/po4a/commit/afe6e1344ffad9d87dd807a81ed6467d6101b15f

I think this might break more than dpkg though. And from the above Pod
handling it might be unnecessary?

Thanks,
Guillem

[toc] | [prev] | [next] | [standalone]

#1200731

[Multipart message — attachments visible in raw view] — view raw

Le jeudi 13 juin 2024 à 00:29 +0200, Guillem Jover a écrit :
> On Thu, 2024-06-13 at 00:02:43 +0200, Martin Quinson wrote:
> > Le mercredi 12 juin 2024 à 17:14 +0200, Guillem Jover a écrit :
> > > 
> > > Unfortunately neither Encode::find_encoding()->name nor
> > > Encode::resolve_alias() seem helpful here because they return
> > > "utf-8-strict" for "UTF-8" which will not match against "utf-8" for
> > > the canonical "utf8".
> > 
> > The problem is that Perl has a rather unexpected behavior wrt utf8, UTF-8
> > and UTF8. These names are not aliases of others in Perl. See
> > https://perldoc.perl.org/Encode#UTF-8-vs.-utf8-vs.-UTF8
> 
> Sure, but here I think this does not matter (in theory), because it
> depends on how the pod parser interprets the encoding name, and from
> checking the perl code it seems it maps /utf-?8/i to ":encoding(UTF-8)".
> So they are really treated the same, at least when it comes to POD, that
> does not mean perl has that distinction for the encoding in other
> contexts. 

Ok, ok. You are perfectly right. This distinction between utf8 and UTF-8 is
perfectly useless in the context of POD files. I just pushed yet another commit
to not do that in POD.

> 
Please tell me whether it looks good to you now.

Thanks,
Mt

[toc] | [prev] | [next] | [standalone]

#1200733

Hi!

On Thu, 2024-06-13 at 00:45:14 +0200, Martin Quinson wrote:
> Le jeudi 13 juin 2024 à 00:29 +0200, Guillem Jover a écrit :
> > On Thu, 2024-06-13 at 00:02:43 +0200, Martin Quinson wrote:
> > > The problem is that Perl has a rather unexpected behavior wrt utf8, UTF-8
> > > and UTF8. These names are not aliases of others in Perl. See
> > > https://perldoc.perl.org/Encode#UTF-8-vs.-utf8-vs.-UTF8
> > 
> > Sure, but here I think this does not matter (in theory), because it
> > depends on how the pod parser interprets the encoding name, and from
> > checking the perl code it seems it maps /utf-?8/i to ":encoding(UTF-8)".
> > So they are really treated the same, at least when it comes to POD, that
> > does not mean perl has that distinction for the encoding in other
> > contexts. 
> 
> Ok, ok. You are perfectly right. This distinction between utf8 and UTF-8 is
> perfectly useless in the context of POD files. I just pushed yet another commit
> to not do that in POD.

Perfect, thanks! :)

> Please tell me whether it looks good to you now.

I just hot-fixed my installed po4a and it seems to be working fine
now. Thank you!

You might also perhaps want to update the NEWS file after that change
though? :)

Regards,
Guillem

[toc] | [prev] | [next] | [standalone]

#1200769

[Multipart message — attachments visible in raw view] — view raw

Hello,

I just updated the NEWS file. I hope it's OK now, but any potential improvement
is welcome.

Mt

Le jeudi 13 juin 2024 à 01:09 +0200, Guillem Jover a écrit :
> On Thu, 2024-06-13 at 00:45:14 +0200, Martin Quinson wrote:
> > Le jeudi 13 juin 2024 à 00:29 +0200, Guillem Jover a écrit :
> > > On Thu, 2024-06-13 at 00:02:43 +0200, Martin Quinson wrote:
> > > > The problem is that Perl has a rather unexpected behavior wrt utf8,
> > > > UTF-8
> > > > and UTF8. These names are not aliases of others in Perl. See
> > > > https://perldoc.perl.org/Encode#UTF-8-vs.-utf8-vs.-UTF8
> > > 
> > > Sure, but here I think this does not matter (in theory), because it
> > > depends on how the pod parser interprets the encoding name, and from
> > > checking the perl code it seems it maps /utf-?8/i to ":encoding(UTF-8)".
> > > So they are really treated the same, at least when it comes to POD, that
> > > does not mean perl has that distinction for the encoding in other
> > > contexts. 
> > 
> > Ok, ok. You are perfectly right. This distinction between utf8 and UTF-8 is
> > perfectly useless in the context of POD files. I just pushed yet another
> > commit
> > to not do that in POD.
> 
> Perfect, thanks! :)
> 
> > Please tell me whether it looks good to you now.
> 
> I just hot-fixed my installed po4a and it seems to be working fine
> now. Thank you!
> 
> You might also perhaps want to update the NEWS file after that change
> though? :)
> 
> Regards,
> Guillem

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.bugs.dist

csiph-web