Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #85980 > unrolled thread

Python Requests logging 401 immediately before 200

Back to article view | Back to comp.lang.python

Contents

  Python Requests logging 401 immediately before 200 Zach Dunlap <zrdunlap@gmail.com> - 2015-02-20 09:16 -0800
    Re: Python Requests logging 401 immediately before 200 Chris Angelico <rosuav@gmail.com> - 2015-02-21 04:30 +1100
      Re: Python Requests logging 401 immediately before 200 Zach Dunlap <zrdunlap@gmail.com> - 2015-02-20 09:39 -0800
    Re: Python Requests logging 401 immediately before 200 Mark Lawrence <breamoreboy@yahoo.co.uk> - 2015-02-20 18:34 +0000

#85980 — Python Requests logging 401 immediately before 200

Hi everyone,

I'm using MarkLogic and have a Python script set up to check for documents using Requests (http://docs.python-requests.org/en/latest/) and I have logging included in the script.

I have logging set to the DEBUG level.

When I set the script to simple search and return the HTTP status code for items it works, but when I look at what is generated by logging I see the following for everything:

INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): localhost
DEBUG:requests.packages.urllib3.connectionpool:"GET /v1/documents?uri=000248e4331d4db5856df8fd427b3cdb.xml HTTP/1.1" 401 211
DEBUG:requests.packages.urllib3.connectionpool:"GET /v1/documents?uri=000248e4331d4db5856df8fd427b3cdb.xml HTTP/1.1" 200 18327

As you can see the 400 and the 200 are the same document (only 1 can exist in ML with the same URI) and in MarkLogic's logs both are executed at the same time.

When I search for a document using something like CURL for instance, I only get the 200 status code which seems correct to me.

Is this something I should be concerned about and should fix or is it just the log level I have set or something else I shouldn't worry about?

Thank you for your help!

Here's a snipped of my code in case it helps:

import requests
import logging

# initiate logging
log_path = os.path.join('my_log.log')
logging.basicConfig(filename=log_path,level=logging.DEBUG)

# make request
r = requests.get('http://localhost:8004/v1/documents?uri=000248e4331d4db5856df8fd427b3cdb.xml',auth=HTTPDigestAuth('USER', 'PASSWORD'))

print r.status_code

[toc] | [next] | [standalone]

#85981

On Sat, Feb 21, 2015 at 4:16 AM, Zach Dunlap <zrdunlap@gmail.com> wrote:
> INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): localhost
> DEBUG:requests.packages.urllib3.connectionpool:"GET /v1/documents?uri=000248e4331d4db5856df8fd427b3cdb.xml HTTP/1.1" 401 211
> DEBUG:requests.packages.urllib3.connectionpool:"GET /v1/documents?uri=000248e4331d4db5856df8fd427b3cdb.xml HTTP/1.1" 200 18327
>
> As you can see the 400 and the 200 are the same document (only 1 can exist in ML with the same URI) and in MarkLogic's logs both are executed at the same time.
>
> Is this something I should be concerned about and should fix or is it just the log level I have set or something else I shouldn't worry about?
>
> r = requests.get('http://localhost:8004/v1/documents?uri=000248e4331d4db5856df8fd427b3cdb.xml',auth=HTTPDigestAuth('USER', 'PASSWORD'))
>

Short explanation: It's part of the protocol.

You're using Digest Auth, so what's happening here is that a request
is sent without the authentication, and the inevitable 401 response is
the signal that the request should be re-sent with credentials.

There's a massive debate as to whether or not it's correct to send
unsolicited credentials. On the one hand, this "401 then 200" pattern
takes extra internet round-trips (not that that's a big deal with
localhost, but it's the same issue everywhere) and puts extra load on
both server and client; but on the other hand, nobody wants their
credentials sent to the wrong server, and it's not always easy to tell
when they'll be needed. Without the 401, you can't know whether or not
you need to authenticate, so if once you start sending unsolicited
credentials, you basically have to keep on doing so.

If you don't have a problem with performance or latency, then consider
this to be nothing more than a bit of log file spam. I'm not sure if
the Python requests module can be easily configured to send
credentials on the first query, but my advice is: don't even bother
looking into that unless you have need to.

All the best!

ChrisA

[toc] | [prev] | [next] | [standalone]

#85983

On Friday, February 20, 2015 at 12:31:00 PM UTC-5, Chris Angelico wrote:
> On Sat, Feb 21, 2015 at 4:16 AM, Zach Dunlap <zrdunlap@gmail.com> wrote:
> > INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): localhost
> > DEBUG:requests.packages.urllib3.connectionpool:"GET /v1/documents?uri=000248e4331d4db5856df8fd427b3cdb.xml HTTP/1.1" 401 211
> > DEBUG:requests.packages.urllib3.connectionpool:"GET /v1/documents?uri=000248e4331d4db5856df8fd427b3cdb.xml HTTP/1.1" 200 18327
> >
> > As you can see the 400 and the 200 are the same document (only 1 can exist in ML with the same URI) and in MarkLogic's logs both are executed at the same time.
> >
> > Is this something I should be concerned about and should fix or is it just the log level I have set or something else I shouldn't worry about?
> >
> > r = requests.get('http://localhost:8004/v1/documents?uri=000248e4331d4db5856df8fd427b3cdb.xml',auth=HTTPDigestAuth('USER', 'PASSWORD'))
> >
> 
> Short explanation: It's part of the protocol.
> 
> You're using Digest Auth, so what's happening here is that a request
> is sent without the authentication, and the inevitable 401 response is
> the signal that the request should be re-sent with credentials.
> 
> There's a massive debate as to whether or not it's correct to send
> unsolicited credentials. On the one hand, this "401 then 200" pattern
> takes extra internet round-trips (not that that's a big deal with
> localhost, but it's the same issue everywhere) and puts extra load on
> both server and client; but on the other hand, nobody wants their
> credentials sent to the wrong server, and it's not always easy to tell
> when they'll be needed. Without the 401, you can't know whether or not
> you need to authenticate, so if once you start sending unsolicited
> credentials, you basically have to keep on doing so.
> 
> If you don't have a problem with performance or latency, then consider
> this to be nothing more than a bit of log file spam. I'm not sure if
> the Python requests module can be easily configured to send
> credentials on the first query, but my advice is: don't even bother
> looking into that unless you have need to.
> 
> All the best!
> 
> ChrisA

Thanks very much for this it's extremely helpful! This clears it up and I think I will ignore it as log spam as you suggest.

[toc] | [prev] | [next] | [standalone]

#85988

On 20/02/2015 17:16, Zach Dunlap wrote:
> Hi everyone,
>
> I'm using MarkLogic

A name that frightens me to death, what idiot thought of that? :)

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web