Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.compression > #2310 > unrolled thread

Standardisation of header compression in HTTP 2

Back to article view | Back to comp.compression

Contents

  Standardisation of header compression in HTTP 2 pgut001@cs.auckland.ac.nz (Peter Gutmann) - 2014-05-13 04:50 +0000
    Re: Standardisation of header compression in HTTP 2 Thomas Richter <thor@math.tu-berlin.de> - 2014-05-13 08:37 +0200
      Re: Standardisation of header compression in HTTP 2 Robert Wessel <robertwessel2@yahoo.com> - 2014-05-13 02:26 -0500
        Re: Standardisation of header compression in HTTP 2 Thomas Richter <thor@math.tu-berlin.de> - 2014-05-13 13:39 +0200
          Re: Standardisation of header compression in HTTP 2 Eli the Bearded <*@eli.users.panix.com> - 2014-05-13 19:19 +0000
            Re: Standardisation of header compression in HTTP 2 Thomas Richter <thor@math.tu-berlin.de> - 2014-05-14 11:44 +0200
              Re: Standardisation of header compression in HTTP 2 Robert Wessel <robertwessel2@yahoo.com> - 2014-05-14 12:08 -0500
            Re: Standardisation of header compression in HTTP 2 Phil Carmody <thefatphil_demunged@yahoo.co.uk> - 2014-06-01 12:11 +0300
              Re: Standardisation of header compression in HTTP 2 Eli the Bearded <*@eli.users.panix.com> - 2014-06-03 20:00 +0000
                Re: Standardisation of header compression in HTTP 2 matrix29bear@gmail.com - 2014-06-11 00:53 -0700
      Re: Standardisation of header compression in HTTP 2 glen herrmannsfeldt <gah@ugcs.caltech.edu> - 2014-05-13 08:35 +0000

#2310 — Standardisation of header compression in HTTP 2

Just a heads-up for folks in this group, the Hypertext Transfer Protocol Bis
(httpbis) working group is currently in the process of standardising HTTP
header compression using, unfortunately, techniques that were state-of-the-
art in about 1975.  It seems like they're in desperate need of people with
knowledge of data compression technology to help them out.  The spec for this
is draft-ietf-httpbis-header-compression-07,
http://tools.ietf.org/html/draft-ietf-httpbis-header-compression-07, the
discussion list is at http://lists.w3.org/Archives/Public/ietf-http-wg/.

Peter.

[toc] | [next] | [standalone]

#2311

Am 13.05.2014 06:50, schrieb Peter Gutmann:
> Just a heads-up for folks in this group, the Hypertext Transfer Protocol Bis
> (httpbis) working group is currently in the process of standardising HTTP
> header compression using, unfortunately, techniques that were state-of-the-
> art in about 1975.  It seems like they're in desperate need of people with
> knowledge of data compression technology to help them out.  The spec for this
> is draft-ietf-httpbis-header-compression-07,
> http://tools.ietf.org/html/draft-ietf-httpbis-header-compression-07, the
> discussion list is at http://lists.w3.org/Archives/Public/ietf-http-wg/.

To be honest, I don't get the point. The points. In multiple ways.

Surely, I'm not an expert in internet protocols.

First, the http header is, if I understand it correctly, only a minor 
part in the transmission of a message. It is something we (as in ISO 
SC29WG1) would call metadata. And we don't even bother compressing such 
data since it's not worth it, it's a small percentage of the traffic, so 
just leave it clear text.

Second: As I read the draft, this is already in revision 7 and is 
supposed to be ready in October this year. If I understand the preamble 
correctly. I believe this makes it pretty unrealistic to come up with 
any new or novel scheme and replace the text by something completely 
different. Bad timing, unlikely to happen.

Third: The text says that the header compression can allow an attacker 
to get access to secured data. Sorry, I don't quite get this. If you 
want to secure the data, that's what HTTPS or SSL is good for. If you 
send your data compressed, I wonder why that adds *additional* 
information - compared to sending it in clear text that is - that allows 
an attack. I also wonder why the simple compression algorithm presented 
in the draft is not vulnerable to the same type of problem. Compression 
and encryption are (almost always) orthogonal concepts. Compress, then 
encrypt. Leave the design of the encryption to experts in encryption, 
not to compression experts.

Obviously, I know nothing about the IETF goals and use cases for this, 
thus the stupid questions, but given the limited time you have, it seems 
unrealistic that anything other will come out of this. If its good 
enough, its good enough. If DEFLATE (as in SPDY) works, I wonder what 
the problem with that is either. It's a free algorithm after all, 
probably not ideal for the data, and probably not ideal for the use 
cases. But for that one first has to understand the latter.

Greetings,
Thomas

[toc] | [prev] | [next] | [standalone]

#2312

On Tue, 13 May 2014 08:37:36 +0200, Thomas Richter
<thor@math.tu-berlin.de> wrote:

>Am 13.05.2014 06:50, schrieb Peter Gutmann:
>> Just a heads-up for folks in this group, the Hypertext Transfer Protocol Bis
>> (httpbis) working group is currently in the process of standardising HTTP
>> header compression using, unfortunately, techniques that were state-of-the-
>> art in about 1975.  It seems like they're in desperate need of people with
>> knowledge of data compression technology to help them out.  The spec for this
>> is draft-ietf-httpbis-header-compression-07,
>> http://tools.ietf.org/html/draft-ietf-httpbis-header-compression-07, the
>> discussion list is at http://lists.w3.org/Archives/Public/ietf-http-wg/.
>
>To be honest, I don't get the point. The points. In multiple ways.
>
>Surely, I'm not an expert in internet protocols.
>
>First, the http header is, if I understand it correctly, only a minor 
>part in the transmission of a message. It is something we (as in ISO 
>SC29WG1) would call metadata. And we don't even bother compressing such 
>data since it's not worth it, it's a small percentage of the traffic, so 
>just leave it clear text.
>
>Second: As I read the draft, this is already in revision 7 and is 
>supposed to be ready in October this year. If I understand the preamble 
>correctly. I believe this makes it pretty unrealistic to come up with 
>any new or novel scheme and replace the text by something completely 
>different. Bad timing, unlikely to happen.
>
>Third: The text says that the header compression can allow an attacker 
>to get access to secured data. Sorry, I don't quite get this. If you 
>want to secure the data, that's what HTTPS or SSL is good for. If you 
>send your data compressed, I wonder why that adds *additional* 
>information - compared to sending it in clear text that is - that allows 
>an attack. I also wonder why the simple compression algorithm presented 
>in the draft is not vulnerable to the same type of problem. Compression 
>and encryption are (almost always) orthogonal concepts. Compress, then 
>encrypt. Leave the design of the encryption to experts in encryption, 
>not to compression experts.


By reducing the length of the message to (approximately) the number of
bits implied by its entropy, a compressed and encrypted messages ends
up leaking some number of bits representing the entropy of the input
message.  Looked at another way, compression encodes the entropy of
the message in its length, which is something that's usually visible
even when encrypted.

For long and highly variable messages, this is not much of a practical
issue.  But for short messages of a few types, the effective leak can
be significant.

Consider a hypothetical system with two messages types: one is an
effectively random string of 100 bytes, the other is a string of 100
A's.  After compressing, the first message type would still be
approximately 100 bytes long, while the second would always be reduced
to only a few bytes.  Recognizing the second type of message, even if
encrypted, is then trivial.  OTOH, if not compressed, encryption would
hide all the message types.

As to the purpose of HTTP header compression, many of the headers have
gotten fairly bulky, what with contents of cookies, the web site name,
modifications dates, encoding types, agent/server IDs, and whatnot,
plus you often see many headers when you have HTTP 1.1 persistent
connections, often for many smallish chunks of web page, especially
when you see dozens of GETs for unchanged pages in a single session,
where there are thousands of header bytes exchanged, with no data at
all.  Still, it's not likely to be a huge improvement, but it is a
reasonable one.

[toc] | [prev] | [next] | [standalone]

#2314

Am 13.05.2014 09:26, schrieb Robert Wessel:
> On Tue, 13 May 2014 08:37:36 +0200, Thomas Richter
> <thor@math.tu-berlin.de>  wrote:
>
>
> By reducing the length of the message to (approximately) the number of
> bits implied by its entropy, a compressed and encrypted messages ends
> up leaking some number of bits representing the entropy of the input
> message.  Looked at another way, compression encodes the entropy of
> the message in its length, which is something that's usually visible
> even when encrypted.
>
> For long and highly variable messages, this is not much of a practical
> issue.  But for short messages of a few types, the effective leak can
> be significant.
>
> Consider a hypothetical system with two messages types: one is an
> effectively random string of 100 bytes, the other is a string of 100
> A's.  After compressing, the first message type would still be
> approximately 100 bytes long, while the second would always be reduced
> to only a few bytes.  Recognizing the second type of message, even if
> encrypted, is then trivial.  OTOH, if not compressed, encryption would
> hide all the message types.

Well. Ok. Maybe. Thanks for the explanation. However, then I don't 
understand in how far the simple encoding proposed by IETF is of any 
help preventing this type of attack. It doesn't seem to be that much 
different, i.e. replacing substrings by references, though more explicit 
and less generic than LZ (as in DEFLATE) does.

IOW, I don't quite get the substantial difference in the appraoches.

> As to the purpose of HTTP header compression, many of the headers have
> gotten fairly bulky, what with contents of cookies, the web site name,
> modifications dates, encoding types, agent/server IDs, and whatnot,
> plus you often see many headers when you have HTTP 1.1 persistent
> connections, often for many smallish chunks of web page, especially
> when you see dozens of GETs for unchanged pages in a single session,
> where there are thousands of header bytes exchanged, with no data at
> all.  Still, it's not likely to be a huge improvement, but it is a
> reasonable one.

Well... ok. That's all I can say, looks weird to me as I would have 
guessed otherwise, but if you say.

Greetings,
	Thomas

[toc] | [prev] | [next] | [standalone]

#2317

In comp.compression, Thomas Richter  <thor@math.tu-berlin.de> wrote:
> Am 13.05.2014 09:26, schrieb Robert Wessel:
>> As to the purpose of HTTP header compression, many of the headers have
>> gotten fairly bulky, what with contents of cookies, the web site name,
>> modifications dates, encoding types, agent/server IDs, and whatnot,
>> plus you often see many headers when you have HTTP 1.1 persistent
>> connections, often for many smallish chunks of web page, especially
>> when you see dozens of GETs for unchanged pages in a single session,
>> where there are thousands of header bytes exchanged, with no data at
> Well... ok. That's all I can say, looks weird to me as I would have 
> guessed otherwise, but if you say.

There are a few very obvious large items in a typical web page, and
many, many more small and less obvious ones. Look at a web site
load analysis for examples. With Pingdom Website Speed Test:

http://tools.pingdom.com/fpt/

And entering "www.latimes.com", it finds 146 individual requests making
up the page. Many of these have a content body size of zero bytes, and
probably half of them are 1kB or less. 

Compressed headers will make web bugs and other tracking tools much less
bandwidth intensive, and thus cheaper to do. Don't you want site
operators to be able to track you better?

Elijah
------
think of the poor data miners

[toc] | [prev] | [next] | [standalone]

#2318

Am 13.05.2014 21:19, schrieb Eli the Bearded:

> There are a few very obvious large items in a typical web page, and
> many, many more small and less obvious ones. Look at a web site
> load analysis for examples. With Pingdom Website Speed Test:
>
> http://tools.pingdom.com/fpt/
>
> And entering "www.latimes.com", it finds 146 individual requests making
> up the page. Many of these have a content body size of zero bytes, and
> probably half of them are 1kB or less.
>
> Compressed headers will make web bugs and other tracking tools much less
> bandwidth intensive, and thus cheaper to do. Don't you want site
> operators to be able to track you better?

Sorry, but I believe that there are a couple of things that are mixed up 
here, at least IMHO. Yes, when making many requests, header sizes add 
up. But the problem for loading such pages is, IMHO, not caused by the 
additional bandwidth required to send the headers, but by the additional 
latency every request causes. It's one TCP round-trip from my location 
to the location of the http server. Whether you compress headers or not 
makes no difference for that. A "bulk request" that collects several 
resources at once, or that can transmit resources that the receiver will 
likely need anyhow even though they have not been requested yet would be 
something that would likely help more than just compressing the headers. 
Again, I'm not an expert in this, but that's my best guess.

Second, I don't quite see how that's related to user tracking. If its 
compressed or not does not matter for this. If its encrypted, well, at 
least not *everybody* can look inside.

Greetings,
	Thomas

[toc] | [prev] | [next] | [standalone]

#2319

On Wed, 14 May 2014 11:44:49 +0200, Thomas Richter
<thor@math.tu-berlin.de> wrote:

>Am 13.05.2014 21:19, schrieb Eli the Bearded:
>
>> There are a few very obvious large items in a typical web page, and
>> many, many more small and less obvious ones. Look at a web site
>> load analysis for examples. With Pingdom Website Speed Test:
>>
>> http://tools.pingdom.com/fpt/
>>
>> And entering "www.latimes.com", it finds 146 individual requests making
>> up the page. Many of these have a content body size of zero bytes, and
>> probably half of them are 1kB or less.
>>
>> Compressed headers will make web bugs and other tracking tools much less
>> bandwidth intensive, and thus cheaper to do. Don't you want site
>> operators to be able to track you better?
>
>Sorry, but I believe that there are a couple of things that are mixed up 
>here, at least IMHO. Yes, when making many requests, header sizes add 
>up. But the problem for loading such pages is, IMHO, not caused by the 
>additional bandwidth required to send the headers, but by the additional 
>latency every request causes. It's one TCP round-trip from my location 
>to the location of the http server. Whether you compress headers or not 
>makes no difference for that. A "bulk request" that collects several 
>resources at once, or that can transmit resources that the receiver will 
>likely need anyhow even though they have not been requested yet would be 
>something that would likely help more than just compressing the headers. 
>Again, I'm not an expert in this, but that's my best guess.


Persistent connections with HTTP 1.1 do just that.  While the initial
base page may wait for the whole round trip, all the request for the
sub-bits (at least those heading to the same server), can just be
blasted out the same TCP connection, and all the responses can come in
on that connection in bulk as well.

In any event, most request and response headers are probably averaging
200-300 bytes these days.  And those tend to happen even if you've
already got the content.  Eli's LA Times example is illustrative - a
number of the items it downloads would actually generate "unchanged"
responses for most requests (much the Javascript, for example, would
likely change only rarely).


>Second, I don't quite see how that's related to user tracking. If its 
>compressed or not does not matter for this. If its encrypted, well, at 
>least not *everybody* can look inside.


He was making a bit of a joke.  If the stuff that goes into tracking
(page bugs, cookies, etc.), is transmitted more efficiently, you'll be
less bothered (FSVO "bothered") by it, so there can be more of it.

[toc] | [prev] | [next] | [standalone]

#2342

Eli the Bearded <*@eli.users.panix.com> writes:
> In comp.compression, Thomas Richter  <thor@math.tu-berlin.de> wrote:
> > Am 13.05.2014 09:26, schrieb Robert Wessel:
> >> As to the purpose of HTTP header compression, many of the headers have
> >> gotten fairly bulky, what with contents of cookies, the web site name,
> >> modifications dates, encoding types, agent/server IDs, and whatnot,
> >> plus you often see many headers when you have HTTP 1.1 persistent
> >> connections, often for many smallish chunks of web page, especially
> >> when you see dozens of GETs for unchanged pages in a single session,
> >> where there are thousands of header bytes exchanged, with no data at
> > Well... ok. That's all I can say, looks weird to me as I would have 
> > guessed otherwise, but if you say.
> 
> There are a few very obvious large items in a typical web page, and
> many, many more small and less obvious ones. Look at a web site
> load analysis for examples. With Pingdom Website Speed Test:
> 
> http://tools.pingdom.com/fpt/
> 
> And entering "www.latimes.com", it finds 146 individual requests making
> up the page. Many of these have a content body size of zero bytes, and
> probably half of them are 1kB or less. 
> 
> Compressed headers will make web bugs and other tracking tools much less
> bandwidth intensive, and thus cheaper to do. Don't you want site
> operators to be able to track you better?

The obvious optimisation is to simply not send all that shit.

HTTP 1.1 was clearly a prototype of a much larger system...
http://www.youtube.com/watch?v=9L_73vDgBjU

Phil
-- 
Religion is too important a matter to its devotees to be a subject of 
ridicule. If they indulge in absurdities, they are to be pitied rather
than ridiculed. -- Immanuel Kant (1724-1804), lecture at Konigsberg, 1775

[toc] | [prev] | [next] | [standalone]

#2344

In comp.compression, Phil Carmody  <thefatphil_demunged@yahoo.co.uk> wrote:
> Eli the Bearded <*@eli.users.panix.com> writes:
>> Compressed headers will make web bugs and other tracking tools much less
>> bandwidth intensive, and thus cheaper to do. Don't you want site
>> operators to be able to track you better?
> The obvious optimisation is to simply not send all that shit.
> HTTP 1.1 was clearly a prototype of a much larger system...
> http://www.youtube.com/watch?v=9L_73vDgBjU

Some sort of video game cutscreen about surveillance? Three minutes of
video for perhaps four hundred words of content? I though this was
comp.compression. Second link here has a text version:

https://duckduckgo.com/?q=deus+ex+morpheus+conversation+transcript

How about some further reading?

http://www.rifters.com/real/shorts/TheScorchedEarthSociety-transcript.pdf

Summary: A biologist argues that mammals by instinct become paranoid
when closely watched. Turning around and watching the watchers can be
dangerous for your continued well-being. Don't assure people you can
keep data private, instead assure people you can burn it to the ground
before they make you hand it over.

Elijah
------
not sure that "not sending all that shit" stops the big data problem

[toc] | [prev] | [next] | [standalone]

#2372

On Tuesday, June 3, 2014 4:00:37 PM UTC-4, Eli the Bearded wrote:
> In comp.compression, Phil Carmody  <thefatphil_demunged@yahoo.co.uk> wrote:
> 
> > Eli the Bearded <*@eli.users.panix.com> writes:
> 
> >> Compressed headers will make web bugs and other tracking tools much less
> 
> >> bandwidth intensive, and thus cheaper to do. Don't you want site
> 
> >> operators to be able to track you better?
> 
> > The obvious optimisation is to simply not send all that shit.
> 
> > HTTP 1.1 was clearly a prototype of a much larger system...
> 
> > http://www.youtube.com/watch?v=9L_73vDgBjU
> 
> 
> 
> Some sort of video game cutscreen about surveillance? Three minutes of
> 
> video for perhaps four hundred words of content? I though this was
> 
> comp.compression. Second link here has a text version:
> 
> 
> 
> https://duckduckgo.com/?q=deus+ex+morpheus+conversation+transcript
> 
> 
> 
> How about some further reading?
> 
> 
> 
> http://www.rifters.com/real/shorts/TheScorchedEarthSociety-transcript.pdf
> 
> 
> 
> Summary: A biologist argues that mammals by instinct become paranoid
> 
> when closely watched. Turning around and watching the watchers can be
> 
> dangerous for your continued well-being. Don't assure people you can
> 
> keep data private, instead assure people you can burn it to the ground
> 
> before they make you hand it over.
> 
> 
> 
> Elijah
> 
> ------
> 
> not sure that "not sending all that shit" stops the big data problem


"Don't assure people you can keep data private, instead assure people you can burn it to the ground before they make you hand it over."


    MEGA UPLOAD anyone?
    "So why do we keep the file servers on a steel treadmill leading to a wall of these again? Oh yeah, that thing that happened LAST TIME..."

SSI's Shred of the Month: Engine Blocks Shredding (D) 
https://www.youtube.com/watch?v=9JUBBEhrvfc

[toc] | [prev] | [next] | [standalone]

#2313

Thomas Richter <thor@math.tu-berlin.de> wrote:

(snip)
> Surely, I'm not an expert in internet protocols.
 
> First, the http header is, if I understand it correctly, only a minor 
> part in the transmission of a message. It is something we (as in ISO 
> SC29WG1) would call metadata. And we don't even bother compressing such 
> data since it's not worth it, it's a small percentage of the traffic, so 
> just leave it clear text.

If I remember right, SLIP does compress the header. Protocols like
telnet tend to send one packet for each character typed, and on
slow lines that can be significant.

Some years ago (before the usual ISPs existed) I had my home
connected through a 9600 baud modem. The IP header overhead
is fairly significant at that speed.

-- glen

[toc] | [prev] | [standalone]

Back to top | Article view | comp.compression

csiph-web