Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.devel > #101492 > unrolled thread

Ideas for a dh-privacy-helper

Back to article view | Back to linux.debian.devel

Contents

  Ideas for a dh-privacy-helper "Bastien Roucariès" <roucaries.bastien@gmail.com> - 2021-09-02 18:00 +0200
    Re: Ideas for a dh-privacy-helper Jonas Smedegaard <jonas@jones.dk> - 2021-09-02 18:20 +0200
      Re: Ideas for a dh-privacy-helper "Bastien Roucariès" <roucaries.bastien@gmail.com> - 2021-09-02 23:50 +0200
        Re: Ideas for a dh-privacy-helper Jonas Smedegaard <jonas@jones.dk> - 2021-09-03 03:10 +0200
          Re: Ideas for a dh-privacy-helper Jonas Smedegaard <jonas@jones.dk> - 2021-09-04 20:10 +0200
            Re: Ideas for a dh-privacy-helper Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-04 20:50 +0200
              Re: Ideas for a dh-privacy-helper Jonas Smedegaard <jonas@jones.dk> - 2021-09-04 23:00 +0200
                Re: Ideas for a dh-privacy-helper Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-05 00:00 +0200
                Re: Ideas for a dh-privacy-helper Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-05 00:40 +0200
              Re: Ideas for a dh-privacy-helper Jonas Smedegaard <jonas@jones.dk> - 2021-09-04 23:00 +0200
            Re: Ideas for a dh-privacy-helper Jonas Smedegaard <jonas@jones.dk> - 2021-09-09 00:20 +0200
          Re: Ideas for a dh-privacy-helper Bastien ROUCARIES <roucaries.bastien@gmail.com> - 2021-09-04 20:10 +0200
    Re: Ideas for a dh-privacy-helper Paul Wise <pabs@debian.org> - 2021-09-03 03:30 +0200

#101492 — Ideas for a dh-privacy-helper

[Multipart message — attachments visible in raw view] — view raw

Hi,

A few year ago I have created the privacy-breach lintian checks in order to 
detect trackers in our doc

I think we are losing the battle here.

I believe that we need better tools than sed in order to fix this kind of 
problem.

I have some idea like:
- read the html tree
- convert the html tree dom representation to xml serialization (so called 
XHTML5 or polyglot)
- apply to this xhtml5 xslt2 rules for fixing the privacy breach

The problem are the tools to use...

I will like to use javascript for this kind of transformation but nodejs does 
not compile on armel, and for saxon-ce I need gwt that is not in debian...

I could use saxon2,but it will need java.

Any ideas is welcomed

Bastien

[toc] | [next] | [standalone]

#101493

[Multipart message — attachments visible in raw view] — view raw

Quoting Bastien Roucariès (2021-09-02 17:53:18)
> A few year ago I have created the privacy-breach lintian checks in 
> order to detect trackers in our doc
> 
> I think we are losing the battle here.
> 
> I believe that we need better tools than sed in order to fix this kind 
> of problem.
> 
> I have some idea like:
> - read the html tree
> - convert the html tree dom representation to xml serialization (so called 
>   XHTML5 or polyglot)
> - apply to this xhtml5 xslt2 rules for fixing the privacy breach
> 
> The problem are the tools to use...
> 
> I will like to use javascript for this kind of transformation but 
> nodejs does not compile on armel, and for saxon-ce I need gwt that is 
> not in debian...
> 
> I could use saxon2,but it will need java.

Perl is famous for its text juggling features, and sloppy parsing of 
html can be done e.g. with HTML::HTML5::Parser (i.e. Debian package 
libhtml-html5-parser-perl).

Also, debhelper itself is written in perl, so is likely easier to 
integrate plugins written in perl as well.  If perl is an option at all, 
obviously...

I am sure Python/Ruby/PHP/Haskell/Scheme/Rust/etc. folks will argue that 
their pet language is the right for the task as well: I think it will 
help the conversation if you clarify what you are open to and what are 
constraints for you.

E.g. do you mean that it *must* be JavaScript when you mention that?  Or 
are you perhaps asking if someone else wants to take over the challenge 
from you, so it does not matter how it is done?


 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

[toc] | [prev] | [next] | [standalone]

#101499

[Multipart message — attachments visible in raw view] — view raw

Le jeudi 2 septembre 2021, 16:11:48 UTC Jonas Smedegaard a écrit :
> Quoting Bastien Roucariès (2021-09-02 17:53:18)
> 
> > A few year ago I have created the privacy-breach lintian checks in
> > order to detect trackers in our doc
> > 
> > I think we are losing the battle here.
> > 
> > I believe that we need better tools than sed in order to fix this kind
> > of problem.
> > 
> > I have some idea like:
> > - read the html tree
> > - convert the html tree dom representation to xml serialization (so called
> > 
> >   XHTML5 or polyglot)
> > 
> > - apply to this xhtml5 xslt2 rules for fixing the privacy breach
> > 
> > The problem are the tools to use...
> > 
> > I will like to use javascript for this kind of transformation but
> > nodejs does not compile on armel, and for saxon-ce I need gwt that is
> > not in debian...
> > 
> > I could use saxon2,but it will need java.
> 
> Perl is famous for its text juggling features, and sloppy parsing of
> html can be done e.g. with HTML::HTML5::Parser (i.e. Debian package
> libhtml-html5-parser-perl).
> 
> Also, debhelper itself is written in perl, so is likely easier to
> integrate plugins written in perl as well.  If perl is an option at all,
> obviously...

Perl is an option I implemented the privacy breach test in perl. The problem 
is I prefer to drop a debian/package.privacy.xslt file in the package instead 
of asking maintainer to code the removal of privacy problems...

Generic one could be coded in perl, but for the end side I need something like 
xslt2

> I am sure Python/Ruby/PHP/Haskell/Scheme/Rust/etc. folks will argue that
> their pet language is the right for the task as well: I think it will
> help the conversation if you clarify what you are open to and what are
> constraints for you.
> 
> E.g. do you mean that it *must* be JavaScript when you mention that?  Or
> are you perhaps asking if someone else wants to take over the challenge
> from you, so it does not matter how it is done?

No it must no be javascript, but using V8 or something like browser internal 
in order to fail to get a dom tree in case of broken html file, like a browser 
do. But may be I am overconcious

Bastien
> 
> 
>  - Jonas

[toc] | [prev] | [next] | [standalone]

#101506

[Multipart message — attachments visible in raw view] — view raw

Quoting Bastien Roucariès (2021-09-02 23:45:30)
> Perl is an option I implemented the privacy breach test in perl. The 
> problem is I prefer to drop a debian/package.privacy.xslt file in the 
> package instead of asking maintainer to code the removal of privacy 
> problems...
> 
> Generic one could be coded in perl, but for the end side I need 
> something like xslt2

If you are asking how to sloppily parse HTML5 files from upstream source 
and XSLT2 files provided by package maintainers, then with perl you 
could use HTML::HTML5::Parser for the first and XML::Saxon::XSLT2 for 
the second.


> > I am sure Python/Ruby/PHP/Haskell/Scheme/Rust/etc. folks will argue 
> > that their pet language is the right for the task as well: I think 
> > it will help the conversation if you clarify what you are open to 
> > and what are constraints for you.
> > 
> > E.g. do you mean that it *must* be JavaScript when you mention that?  
> > Or are you perhaps asking if someone else wants to take over the 
> > challenge from you, so it does not matter how it is done?
> 
> No it must no be javascript, but using V8 or something like browser 
> internal in order to fail to get a dom tree in case of broken html 
> file, like a browser do. But may be I am overconcious

If you are asking how to parse HTML5 files like a web browser, then with 
perl you could use Gtk3::WebKit2 for that.


 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

[toc] | [prev] | [next] | [standalone]

#101537

[Multipart message — attachments visible in raw view] — view raw

Quoting Bastien ROUCARIES (2021-09-04 19:52:50)
> Le ven. 3 sept. 2021 à 01:03, Jonas Smedegaard <jonas@jones.dk> a écrit :
> >
> > Quoting Bastien Roucariès (2021-09-02 23:45:30)
> > > Perl is an option I implemented the privacy breach test in perl. The
> > > problem is I prefer to drop a debian/package.privacy.xslt file in the
> > > package instead of asking maintainer to code the removal of privacy
> > > problems...
> > >
> > > Generic one could be coded in perl, but for the end side I need
> > > something like xslt2
> >
> > If you are asking how to sloppily parse HTML5 files from upstream source
> > and XSLT2 files provided by package maintainers, then with perl you
> > could use HTML::HTML5::Parser for the first and XML::Saxon::XSLT2 for
> > the second.
> 
> Unfortunatly HTML::HTML5::Parser is RC buggy since 4 years due to a
> bug for handling UTF-8 (#750946)
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=750946

Ouch!

I keep forgetting which packages are affected by that annoying bug :-/


> Your suggestion will work fine but we need to get some solution for 
> this utf-8 problem...

I have recently grown somewhat more familiar with UTF-8 and perl (in my 
work towards fixing bug#867305 in licensecheck), and will try take a 
fresh look at bug#750946...


 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

[toc] | [prev] | [next] | [standalone]

#101539

Le sam. 4 sept. 2021 à 18:03, Jonas Smedegaard <jonas@jones.dk> a écrit :
>
> Quoting Bastien ROUCARIES (2021-09-04 19:52:50)
> > Le ven. 3 sept. 2021 à 01:03, Jonas Smedegaard <jonas@jones.dk> a écrit :
> > >
> > > Quoting Bastien Roucariès (2021-09-02 23:45:30)
> > > > Perl is an option I implemented the privacy breach test in perl. The
> > > > problem is I prefer to drop a debian/package.privacy.xslt file in the
> > > > package instead of asking maintainer to code the removal of privacy
> > > > problems...
> > > >
> > > > Generic one could be coded in perl, but for the end side I need
> > > > something like xslt2
> > >
> > > If you are asking how to sloppily parse HTML5 files from upstream source
> > > and XSLT2 files provided by package maintainers, then with perl you
> > > could use HTML::HTML5::Parser for the first and XML::Saxon::XSLT2 for
> > > the second.
> >
> > Unfortunatly HTML::HTML5::Parser is RC buggy since 4 years due to a
> > bug for handling UTF-8 (#750946)
> > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=750946
>
> Ouch!
>
> I keep forgetting which packages are affected by that annoying bug :-/
>
>
> > Your suggestion will work fine but we need to get some solution for
> > this utf-8 problem...
>
> I have recently grown somewhat more familiar with UTF-8 and perl (in my
> work towards fixing bug#867305 in licensecheck), and will try take a
> fresh look at bug#750946...

The solution is straightforward just send you a mail. Use html5
sniffing and add an optional parameter to method to specify encoding.

Bastien

>
>  - Jonas
>
> --
>  * Jonas Smedegaard - idealist & Internet-arkitekt
>  * Tlf.: +45 40843136  Website: http://dr.jones.dk/
>
>  [x] quote me freely  [ ] ask before reusing  [ ] keep private

[toc] | [prev] | [next] | [standalone]

#101541

[Multipart message — attachments visible in raw view] — view raw

Quoting Jonas Smedegaard (2021-09-04 22:51:40)
> It is my understanding that upstream would considers the API being 
> tied to the API - i.e. if you want a different API then look for 
> different module.

Seems the upstream author of HTML::HTML5::Parser even himself switched 
to HTML5::DOM for his newer work: https://metacpan.org/pod/Types::HTML5

 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

[toc] | [prev] | [next] | [standalone]

#101543

Le sam. 4 sept. 2021 à 20:54, Jonas Smedegaard <jonas@jones.dk> a écrit :
>
> Quoting Jonas Smedegaard (2021-09-04 22:51:40)
> > It is my understanding that upstream would considers the API being
> > tied to the API - i.e. if you want a different API then look for
> > different module.
>
> Seems the upstream author of HTML::HTML5::Parser even himself switched
> to HTML5::DOM for his newer work: https://metacpan.org/pod/Types::HTML5
No I really need to ouptut xml and after pass to saxon...

>  - Jonas
>
> --
>  * Jonas Smedegaard - idealist & Internet-arkitekt
>  * Tlf.: +45 40843136  Website: http://dr.jones.dk/
>
>  [x] quote me freely  [ ] ask before reusing  [ ] keep private

[toc] | [prev] | [next] | [standalone]

#101544

Le sam. 4 sept. 2021 à 20:54, Jonas Smedegaard <jonas@jones.dk> a écrit :
>
> Quoting Jonas Smedegaard (2021-09-04 22:51:40)
> > It is my understanding that upstream would considers the API being
> > tied to the API - i.e. if you want a different API then look for
> > different module.
>
> Seems the upstream author of HTML::HTML5::Parser even himself switched
> to HTML5::DOM for his newer work: https://metacpan.org/pod/Types::HTML5

Ok reading the source i need HTML5::DOM and Types::HTML5

Types::HTML5 seems to offer a html_to_xml method that I need.

Bastien
>  - Jonas
>
> --
>  * Jonas Smedegaard - idealist & Internet-arkitekt
>  * Tlf.: +45 40843136  Website: http://dr.jones.dk/
>
>  [x] quote me freely  [ ] ask before reusing  [ ] keep private

[toc] | [prev] | [next] | [standalone]

#101542

[Multipart message — attachments visible in raw view] — view raw

Quoting Bastien ROUCARIES (2021-09-04 20:28:49)
> Le sam. 4 sept. 2021 à 18:03, Jonas Smedegaard <jonas@jones.dk> a écrit :
> >
> > Quoting Bastien ROUCARIES (2021-09-04 19:52:50)
> > > Le ven. 3 sept. 2021 à 01:03, Jonas Smedegaard <jonas@jones.dk> a écrit :
> > > >
> > > > Quoting Bastien Roucariès (2021-09-02 23:45:30)
> > > > > Perl is an option I implemented the privacy breach test in perl. The
> > > > > problem is I prefer to drop a debian/package.privacy.xslt file in the
> > > > > package instead of asking maintainer to code the removal of privacy
> > > > > problems...
> > > > >
> > > > > Generic one could be coded in perl, but for the end side I need
> > > > > something like xslt2
> > > >
> > > > If you are asking how to sloppily parse HTML5 files from upstream source
> > > > and XSLT2 files provided by package maintainers, then with perl you
> > > > could use HTML::HTML5::Parser for the first and XML::Saxon::XSLT2 for
> > > > the second.
> > >
> > > Unfortunatly HTML::HTML5::Parser is RC buggy since 4 years due to a
> > > bug for handling UTF-8 (#750946)
> > > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=750946
> >
> > Ouch!
> >
> > I keep forgetting which packages are affected by that annoying bug :-/
> >
> >
> > > Your suggestion will work fine but we need to get some solution for
> > > this utf-8 problem...
> >
> > I have recently grown somewhat more familiar with UTF-8 and perl (in my
> > work towards fixing bug#867305 in licensecheck), and will try take a
> > fresh look at bug#750946...
> 
> The solution is straightforward just send you a mail. Use html5
> sniffing and add an optional parameter to method to specify encoding.

Seems to me - and seems from your posts to upstream bugreport that you 
agree - that a "straightforward" solution breaks the API, whereas a 
solution which preserves the API is hard.

It is my understanding that upstream would considers the API being tied 
to the API - i.e. if you want a different API then look for different 
module.

Therefore: How do you think about instead using HTML5::DOM?  It is not 
yet in Debian so will need a "sudo apt install cpanminus; cpanm 
HTML5::DOM".  If useful then I can offer to package it for Debian.


 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

[toc] | [prev] | [next] | [standalone]

#101604

[Multipart message — attachments visible in raw view] — view raw

Quoting Jonas Smedegaard (2021-09-04 20:02:57)
> Quoting Bastien ROUCARIES (2021-09-04 19:52:50)
> > Le ven. 3 sept. 2021 à 01:03, Jonas Smedegaard <jonas@jones.dk> a écrit :
> > >
> > > Quoting Bastien Roucariès (2021-09-02 23:45:30)
> > > > Perl is an option I implemented the privacy breach test in perl. 
> > > > The problem is I prefer to drop a debian/package.privacy.xslt 
> > > > file in the package instead of asking maintainer to code the 
> > > > removal of privacy problems...
> > > >
> > > > Generic one could be coded in perl, but for the end side I need 
> > > > something like xslt2
> > >
> > > If you are asking how to sloppily parse HTML5 files from upstream 
> > > source and XSLT2 files provided by package maintainers, then with 
> > > perl you could use HTML::HTML5::Parser for the first and 
> > > XML::Saxon::XSLT2 for the second.
> > 
> > Unfortunatly HTML::HTML5::Parser is RC buggy since 4 years due to a 
> > bug for handling UTF-8 (#750946) 
> > https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=750946
> 
> Ouch!
> 
> I keep forgetting which packages are affected by that annoying bug :-/
> 
> 
> > Your suggestion will work fine but we need to get some solution for 
> > this utf-8 problem...
> 
> I have recently grown somewhat more familiar with UTF-8 and perl (in 
> my work towards fixing bug#867305 in licensecheck), and will try take 
> a fresh look at bug#750946...

HTML::HTML5::Parser should now be in better shape.

Please try version 0.992 now in unstable, if still relevant for your 
work.


 - Jonas

-- 
 * Jonas Smedegaard - idealist & Internet-arkitekt
 * Tlf.: +45 40843136  Website: http://dr.jones.dk/

 [x] quote me freely  [ ] ask before reusing  [ ] keep private

[toc] | [prev] | [next] | [standalone]

#101538

Le ven. 3 sept. 2021 à 01:03, Jonas Smedegaard <jonas@jones.dk> a écrit :
>
> Quoting Bastien Roucariès (2021-09-02 23:45:30)
> > Perl is an option I implemented the privacy breach test in perl. The
> > problem is I prefer to drop a debian/package.privacy.xslt file in the
> > package instead of asking maintainer to code the removal of privacy
> > problems...
> >
> > Generic one could be coded in perl, but for the end side I need
> > something like xslt2
>
> If you are asking how to sloppily parse HTML5 files from upstream source
> and XSLT2 files provided by package maintainers, then with perl you
> could use HTML::HTML5::Parser for the first and XML::Saxon::XSLT2 for
> the second.

Unfortunatly HTML::HTML5::Parser is RC buggy since 4 years due to a
bug for handling UTF-8 (#750946)
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=750946

Your suggestion will work fine but we need to get some solution for
this utf-8 problem...

Bastien






>
> > > I am sure Python/Ruby/PHP/Haskell/Scheme/Rust/etc. folks will argue
> > > that their pet language is the right for the task as well: I think
> > > it will help the conversation if you clarify what you are open to
> > > and what are constraints for you.
> > >
> > > E.g. do you mean that it *must* be JavaScript when you mention that?
> > > Or are you perhaps asking if someone else wants to take over the
> > > challenge from you, so it does not matter how it is done?
> >
> > No it must no be javascript, but using V8 or something like browser
> > internal in order to fail to get a dom tree in case of broken html
> > file, like a browser do. But may be I am overconcious
>
> If you are asking how to parse HTML5 files like a web browser, then with
> perl you could use Gtk3::WebKit2 for that.
>
>
>  - Jonas
>
> --
>  * Jonas Smedegaard - idealist & Internet-arkitekt
>  * Tlf.: +45 40843136  Website: http://dr.jones.dk/
>
>  [x] quote me freely  [ ] ask before reusing  [ ] keep private

[toc] | [prev] | [next] | [standalone]

#101507

[Multipart message — attachments visible in raw view] — view raw

On Thu, 2021-09-02 at 15:53 +0000, Bastien Roucariès wrote:

> A few year ago I have created the privacy-breach lintian checks in
> order to detect trackers in our doc
> 
> I think we are losing the battle here.

These lintian checks are a good start, but they are just heuristics
that cannot detect how documentation will behave when loaded into a
browser or the other appropriate documentation viewer. Especially for
documentation with scripting languages and or interactive features.

Another thing to do would be to load the documentation in the most
appropriate viewer, interact with it in expected ways and monitor for
network activity or other data leaking mechanisms (eg WebBluetooth).

There are also many other types of privacy issues with using Debian:

https://wiki.debian.org/PrivacyIssues

> I believe that we need better tools than sed in order to fix this kind
> of problem.

Could you detail the kinds of issues you are seeing with sed that make
you want to replace it with something else?

-- 
bye,
pabs

https://wiki.debian.org/PaulWise

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.devel

csiph-web