Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #178566 > unrolled thread

why does latest jessie apache2 reject _ in http request path?

Back to article view | Back to linux.debian.user

Contents

  why does latest jessie apache2 reject _ in http request path? Juha Heinanen <jh@tutpro.com> - 2017-03-08 08:20 +0100
    Re: why does latest jessie apache2 reject _ in http request path? <tomas@tuxteam.de> - 2017-03-08 10:20 +0100
      Re: why does latest jessie apache2 reject _ in http request path? Juha Heinanen <jh@tutpro.com> - 2017-03-08 10:40 +0100
        Re: why does latest jessie apache2 reject _ in http request path? Juha Heinanen <jh@tutpro.com> - 2017-03-08 10:50 +0100
        Re: why does latest jessie apache2 reject _ in http request path? tomas@tuxteam.de - 2017-03-08 11:00 +0100
    Re: why does latest jessie apache2 reject _ in http request path sf@debian.org - 2017-03-08 12:00 +0100
      Re: why does latest jessie apache2 reject _ in http request path Juha Heinanen <jh@tutpro.com> - 2017-03-08 21:10 +0100
        Re: why does latest jessie apache2 reject _ in http request path Greg Wooledge <wooledg@eeg.ccf.org> - 2017-03-08 21:10 +0100

#178566 — why does latest jessie apache2 reject _ in http request path?

My web app stopped working in apache2 2.4.10-10+deb8u8 and looks like
the reason is this:

  * CVE-2016-8743: Enforce more HTTP conformance for request lines and
    request headers, to prevent response splitting and cache pollution
    by malicious clients or downstream proxies.
    If this causes problems with non-conforming clients, some checks can
    be relaxed by adding the new directive 'HttpProtocolOptions unsafe'
    to the configuration.
    Differently than the upstream 2.4.25 release which will also be in the
    Debian 9 (stretch) release, this update for Debian 8 (jessie) accepts
    underscores in host and domain names even while 'HttpProtocolOptions
    strict' is in effect.
    More information is available at
    http://httpd.apache.org/docs/2.4/mod/core.html#httpprotocoloptions

I checked at the referenced RFCs and underscore IS a valid character in
a segment (rfc3986):

absolute-path = 1*( "/" segment )
segment       = *pchar
pchar         = unreserved / pct-encoded / sub-delims / ":" / "@"
unreserved    = ALPHA / DIGIT / "-" / "." / "_" / "~"

Why is it that if I have _ in my segment, apache2 rejects the request
without 'HttpProtocolOptions strict'?

-- Juha

[toc] | [next] | [standalone]

#178568

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, Mar 08, 2017 at 09:07:53AM +0200, Juha Heinanen wrote:
> My web app stopped working in apache2 2.4.10-10+deb8u8 and looks like
> the reason is this:
> 
>   * CVE-2016-8743: Enforce more HTTP conformance for request lines and
>     request headers, to prevent response splitting and cache pollution
>     by malicious clients or downstream proxies.
>     If this causes problems with non-conforming clients, some checks can
>     be relaxed by adding the new directive 'HttpProtocolOptions unsafe'
>     to the configuration.
>     Differently than the upstream 2.4.25 release which will also be in the
>     Debian 9 (stretch) release, this update for Debian 8 (jessie) accepts
>     underscores in host and domain names even while 'HttpProtocolOptions
                     ^^^^     ^^^^^^
>     strict' is in effect.
>     More information is available at
>     http://httpd.apache.org/docs/2.4/mod/core.html#httpprotocoloptions
> 
> I checked at the referenced RFCs and underscore IS a valid character in
> a segment (rfc3986):
    ^^^^^^^

Note the underscored parts. You are talking about (path) segments.
Underscore is fine there. Problem is host and domain names, and 3986 is
pretty deliberately handwavy there (3.2.2 host). Apart from IP addresses
it refers to good ol' DNS (1123, 952. Ah, Those folks knew how to write
RFCs ;-), which *doesn't* include underscore (but dash). But then it
goes on to say that you can locally do what you want with the host part
anyway, and that it hasn't to be tied to the DNS (even percent-encode
it, yikes).

So the restriction up there is pure prudence (but actually makes sense
to me).

Regards
- -- tomás
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAli/y5QACgkQBcgs9XrR2kaB3gCdFPiUnELQippWf8rR1S03MFK+
fhUAn37WSCnBj3/52UQ2bcuBzc/+l92p
=tS21
-----END PGP SIGNATURE-----

[toc] | [prev] | [next] | [standalone]

#178569

tomas@tuxteam.de writes:

> Note the underscored parts. You are talking about (path) segments.
> Underscore is fine there. Problem is host and domain names, and 3986 is
> pretty deliberately handwavy there (3.2.2 host). Apart from IP addresses
> it refers to good ol' DNS (1123, 952. Ah, Those folks knew how to write
> RFCs ;-), which *doesn't* include underscore (but dash). But then it
> goes on to say that you can locally do what you want with the host part
> anyway, and that it hasn't to be tied to the DNS (even percent-encode
> it, yikes).

Thanks for your answer.  The request below works over TLS in apache2
2.4.10-10+deb8u7, but fails in 2.4.10-10+deb8u8 unless I turn on

#HttpProtocolOptions unsafe

There is crlf after each line and there are no tabs.

I can't figure out what is wrong with it.

-- Juha

########
T 2017/03/08 11:28:05.427711 127.0.0.1:49612 -> 127.0.0.1:80 [AP]
POST /manager/xml-rpc-server.php HTTP/1.1.
Host: 127.0.0.1.
Connection: close.
Content-Type: text/xml.
Content-Length: 841.

[toc] | [prev] | [next] | [standalone]

#178570

Juha Heinanen writes:

> Thanks for your answer.  The request below works over TLS in apache2
> 2.4.10-10+deb8u7, but fails in 2.4.10-10+deb8u8 unless I turn on
> 
> #HttpProtocolOptions unsafe
> 
> There is crlf after each line and there are no tabs.
> 
> I can't figure out what is wrong with it.

Now when I compare HTTP version of the client to HTTPS version, I see
difference.  The client is written in PHP and if HTTP is used, the
request is set like this:

        $headers =
	"POST $location HTTP/1.1\r\n" .
	"Host: $site\r\n" .
	"Connection: close\r\n" .
	"Content-Type: text/xml\r\n" .
	"Content-Length: " . strlen($data) . "\r\n\r\n";

But if HTTPS is used, the request is set like this:

	curl_setopt($curl, CURLOPT_HTTPHEADER,
	  array('Content-Type: text/xml',
	    'POST $location HTTP/1.1',
 	    'Host: $site',
 	    'Connection: close',
	    'Content-Type: text/xml',
	    'Content-Length: ' . strlen($data)));

Perhaps curl_setops() does not properly add crlf after each header line?
I'll investigate.

-- Juha

[toc] | [prev] | [next] | [standalone]

#178571

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, Mar 08, 2017 at 11:34:54AM +0200, Juha Heinanen wrote:
> tomas@tuxteam.de writes:
> 
> > Note the underscored parts. You are talking about (path) segments.

[...]

> Thanks for your answer.  The request below works over TLS in apache2
> 2.4.10-10+deb8u7, but fails in 2.4.10-10+deb8u8 unless I turn on
> 
> #HttpProtocolOptions unsafe
> 
> There is crlf after each line and there are no tabs.
> 
> I can't figure out what is wrong with it.
> 
> -- Juha
> 
> ########
> T 2017/03/08 11:28:05.427711 127.0.0.1:49612 -> 127.0.0.1:80 [AP]
> POST /manager/xml-rpc-server.php HTTP/1.1.
> Host: 127.0.0.1.
> Connection: close.
> Content-Type: text/xml.
> Content-Length: 841.

Hm. I haven't the resources to track down everything, but I'd try first
changing the host to localhost (although rfc3986 explicitly allows a
"naked" ipv4 address as host part). A quick and brainless search points
to Debian bug 849082 [1], which mentions rfc7230 as reference (this seems
at first glance to refer back to 3986 and to *still* allow naked ipv4
addresses as host part, though).

Keep us updated :-)

regards

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=849082
- -- t
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iEYEARECAAYFAli/1FAACgkQBcgs9XrR2kYWQgCfQc0trfHuqfXj4MnBaYNeq7Xm
8D0AnRGSMxuTJr5GBLKVfk367/Cj3asV
=5Pml
-----END PGP SIGNATURE-----

[toc] | [prev] | [next] | [standalone]

#178572 — Re: why does latest jessie apache2 reject _ in http request path

On Wed, 8 Mar 2017, Juha Heinanen wrote:
> Why is it that if I have _ in my segment, apache2 rejects the request
> without 'HttpProtocolOptions strict'?

Try setting

Loglevel core:debug http:debug

and look if the error log provides more information.

[toc] | [prev] | [next] | [standalone]

#178584 — Re: why does latest jessie apache2 reject _ in http request path

sf@debian.org writes:

> > Why is it that if I have _ in my segment, apache2 rejects the request
> > without 'HttpProtocolOptions strict'?
> 
> Try setting
> 
> Loglevel core:debug http:debug
> 
> and look if the error log provides more information.

Thanks for your help.  The above didn't show all headers, but I found
the bug in the curl function call of the client:

	curl_setopt($curl, CURLOPT_HTTPHEADER,
	  array('Content-Type: text/xml',
	    'POST $location HTTP/1.0',
 	    'Host: $site',
         ...

The last line should of course be:

        'Host: ' . $site,

Otherwise the header looks like this:

Host: $site

which then gets rejected due to the $ char in hostname.

-- Juha

[toc] | [prev] | [next] | [standalone]

#178585 — Re: why does latest jessie apache2 reject _ in http request path

On Wed, Mar 08, 2017 at 10:04:05PM +0200, Juha Heinanen wrote:
> Thanks for your help.  The above didn't show all headers, but I found
> the bug in the curl function call of the client:
> 
> 	curl_setopt($curl, CURLOPT_HTTPHEADER,
> 	  array('Content-Type: text/xml',
> 	    'POST $location HTTP/1.0',
>  	    'Host: $site',
>          ...
> 
> The last line should of course be:
> 
>         'Host: ' . $site,

In that case, I would expect the $location variable to have the same
issue.  Not that I know much about PHP.  That is PHP, I think...?

Does this language have different kinds of quoting, like '...' vs.
"..." in perl or sh?  If so, you might try using the other quotes.

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.user

csiph-web