Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python.announce > #2732 > unrolled thread

ANN: htmldammit 0.1

Back to article view | Back to comp.lang.python.announce

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.

Contents

  ANN: htmldammit 0.1 Tal Einat <taleinat@gmail.com> - 2018-01-07 17:26 +0200

#2732 — ANN: htmldammit 0.1

I'm happy to announce htmldammit <https://pypi.python.org/pypi/htmldammit>,
a library for decoding binary HTML data into Unicode in the best possible
way.

Suggestions and comments are most welcome!


Installation: `pip install htmldammit`

Source: GitHub <https://github.com/taleinat/htmldammit>

Rationale:

When it comes to decoding HTML,Common libraries such as requests and
BeautifulSoup do not make the best possible use of information from both
the raw data as well as the HTTP Content-Type header. Combining them to get
the best possible result is surprisingly far from straightforward.

BeauftulSoup comes closest with it UnicodeDammit utility, but gives no
direct support for taking advantage of Content-Type header information.
Furthermore, in order to utilize the Content-Type header while preferring
embedded charset declarations in a document when those exist, quite some
hacking is required.

htmldammit takes care of all of this. It uses UnicodeDammit in a
non-trivial manner under the hood. It supplies simple functions to extract
all of the needed data from a urlopen() and requests.get() response objects
and return Unicode HTML.

Additionally, a request hook for requests and an opener for urllib are
supplied to make using this consistently simpler.

Regards,
- Tal Einat

[toc] | [standalone]

Back to top | Article view | comp.lang.python.announce

csiph-web