Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #53323 > unrolled thread

UnicodeDecodeError issue

Back to article view | Back to comp.lang.python

Contents

  UnicodeDecodeError issue Ferrous Cranus <nikos@superhost.gr> - 2013-08-31 09:41 +0300
    Re: UnicodeDecodeError issue Chris Angelico <rosuav@gmail.com> - 2013-08-31 16:53 +1000
      Re: UnicodeDecodeError issue Ferrous Cranus <nikos@superhost.gr> - 2013-08-31 10:02 +0300
        Re: UnicodeDecodeError issue Ferrous Cranus <nikos@superhost.gr> - 2013-08-31 10:18 +0300
    Re: UnicodeDecodeError issue Peter Otten <__peter__@web.de> - 2013-08-31 09:25 +0200
      Re: UnicodeDecodeError issue Ferrous Cranus <nikos@superhost.gr> - 2013-08-31 10:58 +0300
        Re: UnicodeDecodeError issue Ferrous Cranus <nikos@superhost.gr> - 2013-08-31 11:31 +0300
          Re: UnicodeDecodeError issue Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-08-31 11:28 +0000
            Re: UnicodeDecodeError issue Ferrous Cranus <nikos@superhost.gr> - 2013-08-31 15:58 +0300
              Re: UnicodeDecodeError issue Ferrous Cranus <nikos@superhost.gr> - 2013-08-31 16:07 +0300
              Re: UnicodeDecodeError issue Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-08-31 15:44 +0000
    Re: UnicodeDecodeError issue Ferrous Cranus <nikos.gr33k@gmail.com> - 2013-08-31 23:50 -0700
      Re: UnicodeDecodeError issue Chris Angelico <rosuav@gmail.com> - 2013-09-01 17:12 +1000
        Re: UnicodeDecodeError issue Ferrous Cranus <nikos@superhost.gr> - 2013-09-01 10:23 +0300
          Re: UnicodeDecodeError issue Chris Angelico <rosuav@gmail.com> - 2013-09-01 17:28 +1000
          Re: UnicodeDecodeError issue Dave Angel <davea@davea.name> - 2013-09-01 10:35 +0000
            Re: UnicodeDecodeError issue Ferrous Cranus <nikos@superhost.gr> - 2013-09-01 16:59 +0300
              Re: UnicodeDecodeError issue Dave Angel <davea@davea.name> - 2013-09-01 15:40 +0000
          Re: UnicodeDecodeError issue Chris Angelico <rosuav@gmail.com> - 2013-09-01 20:51 +1000
      Re: UnicodeDecodeError issue Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-09-01 08:35 +0000
        Re: UnicodeDecodeError issue Ferrous Cranus <nikos@superhost.gr> - 2013-09-01 17:08 +0300
          Re: UnicodeDecodeError issue Ferrous Cranus <nikos@superhost.gr> - 2013-09-01 17:25 +0300
          Re: UnicodeDecodeError issue Dave Angel <davea@davea.name> - 2013-09-01 15:36 +0000
            Re: UnicodeDecodeError issue Ferrous Cranus <nikos@superhost.gr> - 2013-09-01 19:10 +0300
              Re: UnicodeDecodeError issue Ferrous Cranus <nikos@superhost.gr> - 2013-09-02 01:23 +0300
                Re: UnicodeDecodeError issue Dave Angel <davea@davea.name> - 2013-09-01 23:14 +0000
                  Re: UnicodeDecodeError issue Ferrous Cranus <nikos@superhost.gr> - 2013-09-02 07:16 +0300
                    Re: UnicodeDecodeError issue Dave Angel <davea@davea.name> - 2013-09-02 11:38 +0000
                      Re: UnicodeDecodeError issue Ferrous Cranus <nikos@superhost.gr> - 2013-09-02 14:49 +0300
                        Re: UnicodeDecodeError issue Dave Angel <davea@davea.name> - 2013-09-02 12:21 +0000
                          Re: UnicodeDecodeError issue Ferrous Cranus <nikos@superhost.gr> - 2013-09-02 18:05 +0300
                            Re: UnicodeDecodeError issue Dave Angel <davea@davea.name> - 2013-09-02 18:28 +0000
                              Re: UnicodeDecodeError issue Ferrous Cranus <nikos.gr33k@gmail.com> - 2013-09-04 01:35 -0700
                                Re: UnicodeDecodeError issue Dave Angel <davea@davea.name> - 2013-09-04 11:26 +0000
                                  Re: UnicodeDecodeError issue Ferrous Cranus <nikos@superhost.gr> - 2013-09-04 14:38 +0300
                                    Re: UnicodeDecodeError issue Dave Angel <davea@davea.name> - 2013-09-04 12:38 +0000
                                      Re: UnicodeDecodeError issue Ferrous Cranus <nikos@superhost.gr> - 2013-09-04 17:29 +0300
                                        Re: UnicodeDecodeError issue Dave Angel <davea@davea.name> - 2013-09-05 00:17 +0000
                                          Re: UnicodeDecodeError issue Steven D'Aprano <steve@pearwood.info> - 2013-09-05 03:07 +0000
                                            Re: UnicodeDecodeError issue Chris Angelico <rosuav@gmail.com> - 2013-09-05 13:59 +1000
                                              Re: UnicodeDecodeError issue Steven D'Aprano <steve@pearwood.info> - 2013-09-05 05:28 +0000
                    Re: UnicodeDecodeError issue MRAB <python@mrabarnett.plus.com> - 2013-09-02 12:56 +0100
                    Re: UnicodeDecodeError issue Dave Angel <davea@davea.name> - 2013-09-02 12:24 +0000
                    Re: UnicodeDecodeError issue MRAB <python@mrabarnett.plus.com> - 2013-09-02 15:44 +0100
                      Re: UnicodeDecodeError issue wxjmfauth@gmail.com - 2013-09-03 08:23 -0700
                        Re: UnicodeDecodeError issue Antoon Pardon <antoon.pardon@rece.vub.ac.be> - 2013-09-04 10:01 +0200
                          Re: UnicodeDecodeError issue wxjmfauth@gmail.com - 2013-09-04 07:08 -0700
                    Re: UnicodeDecodeError issue Chris Angelico <rosuav@gmail.com> - 2013-09-03 08:45 +1000
                      Re: UnicodeDecodeError issue Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-09-03 14:56 +0000
                    Re: UnicodeDecodeError issue Joel Goldstick <joel.goldstick@gmail.com> - 2013-09-02 20:49 -0400

Page 1 of 3  [1] 2 3  Next page →

#53323 — UnicodeDecodeError issue

Suddenly my webiste superhost.gr running my main python script presents 
me with this error:

Code:
UnicodeDecodeError('utf-8', b'\xb6\xe3\xed\xf9\xf3\xf4\xef 
\xfc\xed\xef\xec\xe1 \xf3\xf5\xf3\xf4\xde\xec\xe1\xf4\xef\xf2', 0, 1, 
'invalid start byte')


Does anyone know what this means?


-- 
Webhost <http://superhost.gr>

[toc] | [next] | [standalone]

#53325

On Sat, Aug 31, 2013 at 4:41 PM, Ferrous Cranus <nikos@superhost.gr> wrote:
> Suddenly my webiste superhost.gr running my main python script presents me
> with this error:
>
> Code:
> UnicodeDecodeError('utf-8', b'\xb6\xe3\xed\xf9\xf3\xf4\xef
> \xfc\xed\xef\xec\xe1 \xf3\xf5\xf3\xf4\xde\xec\xe1\xf4\xef\xf2', 0, 1,
> 'invalid start byte')
>
>
> Does anyone know what this means?

Yes. It means that 0xB6 is an invalid start byte in UTF-8. If that
seems unhelpful, it's because your question was; there is not enough
information for us to be able to give any further information.

ChrisA

[toc] | [prev] | [next] | [standalone]

#53326

Στις 31/8/2013 9:53 πμ, ο/η Chris Angelico έγραψε:
> On Sat, Aug 31, 2013 at 4:41 PM, Ferrous Cranus <nikos@superhost.gr> wrote:
>> Suddenly my webiste superhost.gr running my main python script presents me
>> with this error:
>>
>> Code:
>> UnicodeDecodeError('utf-8', b'\xb6\xe3\xed\xf9\xf3\xf4\xef
>> \xfc\xed\xef\xec\xe1 \xf3\xf5\xf3\xf4\xde\xec\xe1\xf4\xef\xf2', 0, 1,
>> 'invalid start byte')
>>
>>
>> Does anyone know what this means?
>
> Yes. It means that 0xB6 is an invalid start byte in UTF-8. If that
> seems unhelpful, it's because your question was; there is not enough
> information for us to be able to give any further information.


Hello Chris,

I provided you with only that info because that only info is displayed 
at http://superhost.gr at the top left.

If you please tell me what to try to provide you guys with more info 
about this problem.


-- 
Webhost <http://superhost.gr>

[toc] | [prev] | [next] | [standalone]

#53328

Στις 31/8/2013 10:02 πμ, ο/η Ferrous Cranus έγραψε:
> Στις 31/8/2013 9:53 πμ, ο/η Chris Angelico έγραψε:
>> On Sat, Aug 31, 2013 at 4:41 PM, Ferrous Cranus <nikos@superhost.gr>
>> wrote:
>>> Suddenly my webiste superhost.gr running my main python script
>>> presents me
>>> with this error:
>>>
>>> Code:
>>> UnicodeDecodeError('utf-8', b'\xb6\xe3\xed\xf9\xf3\xf4\xef
>>> \xfc\xed\xef\xec\xe1 \xf3\xf5\xf3\xf4\xde\xec\xe1\xf4\xef\xf2', 0, 1,
>>> 'invalid start byte')
>>>
>>>
>>> Does anyone know what this means?
>>
>> Yes. It means that 0xB6 is an invalid start byte in UTF-8. If that
>> seems unhelpful, it's because your question was; there is not enough
>> information for us to be able to give any further information.
>
>
> Hello Chris,
>
> I provided you with only that info because that only info is displayed
> at http://superhost.gr at the top left.
>
> If you please tell me what to try to provide you guys with more info
> about this problem.
>
>
And the wird thign is that few the last days this error was not there, 
just only today it started dissaperaing.

-- 
Webhost <http://superhost.gr>

[toc] | [prev] | [next] | [standalone]

#53329

Ferrous Cranus wrote:

> Suddenly my webiste superhost.gr running my main python script presents
> me with this error:
 
> Code:
> UnicodeDecodeError('utf-8', b'\xb6\xe3\xed\xf9\xf3\xf4\xef
> \xfc\xed\xef\xec\xe1 \xf3\xf5\xf3\xf4\xde\xec\xe1\xf4\xef\xf2', 0, 1,
> 'invalid start byte')
 
> Does anyone know what this means?

>>> b'\xb6\xe3\xed\xf9\xf3\xf4\xef\xfc\xed\xef\xec\xe1\xf3\xf5\xf3\xf4\xde\xec\xe1\xf4\xef\xf2'.decode("iso-8859-7")
'Unknown hostname'

[toc] | [prev] | [next] | [standalone]

#53332

Στις 31/8/2013 10:25 πμ, ο/η Peter Otten έγραψε:
> Ferrous Cranus wrote:
>
>> Suddenly my webiste superhost.gr running my main python script presents
>> me with this error:
>
>> Code:
>> UnicodeDecodeError('utf-8', b'\xb6\xe3\xed\xf9\xf3\xf4\xef
>> \xfc\xed\xef\xec\xe1 \xf3\xf5\xf3\xf4\xde\xec\xe1\xf4\xef\xf2', 0, 1,
>> 'invalid start byte')
>
>> Does anyone know what this means?
>
>>>> b'\xb6\xe3\xed\xf9\xf3\xf4\xef\xfc\xed\xef\xec\xe1\xf3\xf5\xf3\xf4\xde\xec\xe1\xf4\xef\xf2'.decode("iso-8859-7")
> 'Unknown hostname'
>
>
Thanks you decoded in greek-iso, coll thinking!

But llok:

Why can the host varibale be unset since iam using this:


try:
	gi = pygeoip.GeoIP('/usr/local/share/GeoIPCity.dat')
	city = gi.time_zone_by_addr( os.environ['REMOTE_ADDR'] ) or 
gi.time_zone_by_addr( os.environ['HTTP_CF_CONNECTING_IP'] )
	host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] ) or 
socket.gethostbyaddr( os.environ['HTTP_CF_CONNECTING_IP'] ) or 'Αγνωστη 
Προέλευση'
except Exception as e:
	err = repr(e)

==================
it gets set to string 'Αγνωστη Προέλευση' if in case visitor's remote ip 
address cannot be resolved.

So that error should arise at all.

-- 
Webhost <http://superhost.gr>

[toc] | [prev] | [next] | [standalone]

#53336

Here is the code inside files.py:

try:
	gi = pygeoip.GeoIP('/usr/local/share/GeoIPCity.dat')
	city = gi.time_zone_by_addr( os.environ['REMOTE_ADDR'] ) or 
gi.time_zone_by_addr( os.environ['HTTP_CF_CONNECTING_IP'] )
	host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] ) or 
socket.gethostbyaddr( os.environ['HTTP_CF_CONNECTING_IP'] ) or 
os.environ['REMOTE_ADDR']
except Exception as e:
	print( repr(e), file=open( '/tmp/err.out', 'w' ) )


that produces this:


[Sat Aug 31 08:29:33 2013] [error] [client 108.162.229.116] ValueError: 
underlying buffer has been detached, referer: http://superhost.gr/
[Sat Aug 31 08:29:33 2013] [error] [client 108.162.229.116] , referer: 
http://superhost.gr/
[Sat Aug 31 08:29:33 2013] [error] [client 108.162.229.116] Original 
exception was:, referer: http://superhost.gr/
[Sat Aug 31 08:29:33 2013] [error] [client 108.162.229.116] Traceback 
(most recent call last):, referer: http://superhost.gr/
[Sat Aug 31 08:29:33 2013] [error] [client 108.162.229.116]   File 
"/home/nikos/public_html/cgi-bin/files.py", line 135, in <module>, 
referer: http://superhost.gr/
[Sat Aug 31 08:29:33 2013] [error] [client 108.162.229.116] 
cur.execute('''INSERT INTO files (url, host, city, lastvisit) VALUES 
(%s, %s, %s, %s)''', (filename, host, city, lastvisit) ), referer: 
http://superhost.gr/
[Sat Aug 31 08:29:33 2013] [error] [client 108.162.229.116] NameError: 
name 'host' is not defined, referer: http://superhost.gr/


But 'host' defaults to an ip address if it cannot resolve the hostname.
Why the errro says its undefined?

[toc] | [prev] | [next] | [standalone]

#53344

On Sat, 31 Aug 2013 11:31:13 +0300, Ferrous Cranus wrote:

> Here is the code inside files.py:

The code you show is not the ENTIRE code inside of files.py, is it? You 
are only showing us a small piece, correct?

The code you show:

> try:
> 	gi = pygeoip.GeoIP('/usr/local/share/GeoIPCity.dat') 
> 	city = gi.time_zone_by_addr( os.environ['REMOTE_ADDR'] ) or
> 	  gi.time_zone_by_addr( os.environ['HTTP_CF_CONNECTING_IP'] )
> 	host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] ) or
> 	  socket.gethostbyaddr( os.environ['HTTP_CF_CONNECTING_IP'] ) or
> 	  os.environ['REMOTE_ADDR']
> except Exception as e:
> 	print( repr(e), file=open( '/tmp/err.out', 'w' ) )


does not contain a call to cur.execute. And here is your error:


> [error] [client 108.162.229.116] Traceback (most recent call last):,
> referer: http://superhost.gr/ [Sat Aug 31 08:29:33 2013] [error] [client
> 108.162.229.116]   File "/home/nikos/public_html/cgi-bin/files.py", line
> 135, in <module>, referer: http://superhost.gr/
> [Sat Aug 31 08:29:33 2013] [error] [client 108.162.229.116]
> cur.execute('''INSERT INTO files (url, host, city, lastvisit) VALUES
> (%s, %s, %s, %s)''', (filename, host, city, lastvisit) ), referer:
> http://superhost.gr/
> [Sat Aug 31 08:29:33 2013] [error] [client 108.162.229.116] NameError:
> name 'host' is not defined, referer: http://superhost.gr/


Extracting out the critical part:

line 135
cur.execute('''INSERT INTO files (url, host, city, lastvisit) VALUES
(%s, %s, %s, %s)''', (filename, host, city, lastvisit) ),

Where is the call to cur.execute in the code snippet you show above?



> But 'host' defaults to an ip address if it cannot resolve the hostname.
> Why the errro says its undefined?

Because it is is undefined. Python is not lying to you. If Python tells 
you there is an error, BELIEVE IT. Resolving the hostname is irrelevant.

print(host)  ### ERROR OCCURS HERE BECAUSE HOST IS UNDEFINED ###
host = socket.gethostbyaddr(addr)

The second line is too late, the error has already occurred.


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#53351

Στις 31/8/2013 2:28 μμ, ο/η Steven D'Aprano έγραψε:
> On Sat, 31 Aug 2013 11:31:13 +0300, Ferrous Cranus wrote:
>
>> Here is the code inside files.py:
>
> The code you show is not the ENTIRE code inside of files.py, is it? You
> are only showing us a small piece, correct?
>
> The code you show:
>
>> try:
>> 	gi = pygeoip.GeoIP('/usr/local/share/GeoIPCity.dat')
>> 	city = gi.time_zone_by_addr( os.environ['REMOTE_ADDR'] ) or
>> 	  gi.time_zone_by_addr( os.environ['HTTP_CF_CONNECTING_IP'] )
>> 	host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] ) or
>> 	  socket.gethostbyaddr( os.environ['HTTP_CF_CONNECTING_IP'] ) or
>> 	  os.environ['REMOTE_ADDR']
>> except Exception as e:
>> 	print( repr(e), file=open( '/tmp/err.out', 'w' ) )
>
>
> does not contain a call to cur.execute. And here is your error:
>
>
>> [error] [client 108.162.229.116] Traceback (most recent call last):,
>> referer: http://superhost.gr/ [Sat Aug 31 08:29:33 2013] [error] [client
>> 108.162.229.116]   File "/home/nikos/public_html/cgi-bin/files.py", line
>> 135, in <module>, referer: http://superhost.gr/
>> [Sat Aug 31 08:29:33 2013] [error] [client 108.162.229.116]
>> cur.execute('''INSERT INTO files (url, host, city, lastvisit) VALUES
>> (%s, %s, %s, %s)''', (filename, host, city, lastvisit) ), referer:
>> http://superhost.gr/
>> [Sat Aug 31 08:29:33 2013] [error] [client 108.162.229.116] NameError:
>> name 'host' is not defined, referer: http://superhost.gr/
>
>
> Extracting out the critical part:
>
> line 135
> cur.execute('''INSERT INTO files (url, host, city, lastvisit) VALUES
> (%s, %s, %s, %s)''', (filename, host, city, lastvisit) ),
>
> Where is the call to cur.execute in the code snippet you show above?
>
>
>
>> But 'host' defaults to an ip address if it cannot resolve the hostname.
>> Why the errro says its undefined?
>
> Because it is is undefined. Python is not lying to you. If Python tells
> you there is an error, BELIEVE IT. Resolving the hostname is irrelevant.
>
> print(host)  ### ERROR OCCURS HERE BECAUSE HOST IS UNDEFINED ###
> host = socket.gethostbyaddr(addr)
I'm not saying Python is lying of course it does not.

But how is this possible since:


host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] ) or 
socket.gethostbyaddr( os.environ['HTTP_CF_CONNECTING_IP'] ) or 
os.environ['REMOTE_ADDR']


it must have a value by defaulting to something.

The cur.execute fails because it make use of 'host' which is undefined.

		# Try to insert the file into the database
		cur.execute('''INSERT INTO files (url, host, city, lastvisit) VALUES 
(%s, %s, %s, %s)''', (filename, host, city, lastvisit) )


And the question remain as to why 'host' is undefined.
-- 
Webhost <http://superhost.gr>

[toc] | [prev] | [next] | [standalone]

#53353

Στις 31/8/2013 3:58 μμ, ο/η Ferrous Cranus έγραψε:
> Στις 31/8/2013 2:28 μμ, ο/η Steven D'Aprano έγραψε:
>> On Sat, 31 Aug 2013 11:31:13 +0300, Ferrous Cranus wrote:
>>
>>> Here is the code inside files.py:
>>
>> The code you show is not the ENTIRE code inside of files.py, is it? You
>> are only showing us a small piece, correct?
>>
>> The code you show:
>>
>>> try:
>>>     gi = pygeoip.GeoIP('/usr/local/share/GeoIPCity.dat')
>>>     city = gi.time_zone_by_addr( os.environ['REMOTE_ADDR'] ) or
>>>       gi.time_zone_by_addr( os.environ['HTTP_CF_CONNECTING_IP'] )
>>>     host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] ) or
>>>       socket.gethostbyaddr( os.environ['HTTP_CF_CONNECTING_IP'] ) or
>>>       os.environ['REMOTE_ADDR']
>>> except Exception as e:
>>>     print( repr(e), file=open( '/tmp/err.out', 'w' ) )
>>
>>
>> does not contain a call to cur.execute. And here is your error:
>>
>>
>>> [error] [client 108.162.229.116] Traceback (most recent call last):,
>>> referer: http://superhost.gr/ [Sat Aug 31 08:29:33 2013] [error] [client
>>> 108.162.229.116]   File "/home/nikos/public_html/cgi-bin/files.py", line
>>> 135, in <module>, referer: http://superhost.gr/
>>> [Sat Aug 31 08:29:33 2013] [error] [client 108.162.229.116]
>>> cur.execute('''INSERT INTO files (url, host, city, lastvisit) VALUES
>>> (%s, %s, %s, %s)''', (filename, host, city, lastvisit) ), referer:
>>> http://superhost.gr/
>>> [Sat Aug 31 08:29:33 2013] [error] [client 108.162.229.116] NameError:
>>> name 'host' is not defined, referer: http://superhost.gr/
>>
>>
>> Extracting out the critical part:
>>
>> line 135
>> cur.execute('''INSERT INTO files (url, host, city, lastvisit) VALUES
>> (%s, %s, %s, %s)''', (filename, host, city, lastvisit) ),
>>
>> Where is the call to cur.execute in the code snippet you show above?
>>
>>
>>
>>> But 'host' defaults to an ip address if it cannot resolve the hostname.
>>> Why the errro says its undefined?
>>
>> Because it is is undefined. Python is not lying to you. If Python tells
>> you there is an error, BELIEVE IT. Resolving the hostname is irrelevant.
>>
>> print(host)  ### ERROR OCCURS HERE BECAUSE HOST IS UNDEFINED ###
>> host = socket.gethostbyaddr(addr)
> I'm not saying Python is lying of course it does not.
>
> But how is this possible since:
>
>
> host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] ) or
> socket.gethostbyaddr( os.environ['HTTP_CF_CONNECTING_IP'] ) or
> os.environ['REMOTE_ADDR']
>
>
> it must have a value by defaulting to something.
>
> The cur.execute fails because it make use of 'host' which is undefined.
>
>          # Try to insert the file into the database
>          cur.execute('''INSERT INTO files (url, host, city, lastvisit)
> VALUES (%s, %s, %s, %s)''', (filename, host, city, lastvisit) )
>
>
> And the question remain as to why 'host' is undefined.
Thsi fails too:

host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0] or 
socket.gethostbyaddr( os.environ['HTTP_CF_CONNECTING_IP'] )[0] or 
"Unkknown host"

-- 
Webhost <http://superhost.gr>

[toc] | [prev] | [next] | [standalone]

#53367

On Sat, 31 Aug 2013 15:58:11 +0300, Ferrous Cranus wrote:

Failure is here, line 135:

>> cur.execute('''INSERT INTO files (url, host, city, lastvisit) VALUES
>> (%s, %s, %s, %s)''', (filename, host, city, lastvisit) ),

[...]
> But how is this possible since:
> 
> 
> host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] ) or
> socket.gethostbyaddr( os.environ['HTTP_CF_CONNECTING_IP'] ) or
> os.environ['REMOTE_ADDR']

What's the line number of that line of code? My guess is that it is AFTER 
line 135, which is where the error occurs. Or possibly it is inside a 
function that hasn't been called. Here is the same failure:

print(host)
host = "this is too late"


Here is the same failure again:

if 0:
    host = "this never gets called"
print(host)


Here is the same failure again:

data = []
for x in data:
    host = "This never happens"
print(host)


And again, a trickier one this time:

host = "something"
del host  # and now it is gone
print(host)


And again:

def func():
    global host
    print(host)
func()
host = "Too late, the error has already occurred"


One last one:

def set_host():
    host = "This is a local variable"
set_host()
print(host)


Study all these examples. Now read your own code. host has not been 
defined at the time the cur.execute line is reached. You have to work out 
which of my examples best matches your code.


> it must have a value by defaulting to something.

No, there is no default value for variables. How long have you been 
programming in Python? Six months? A year?


> The cur.execute fails because it make use of 'host' which is undefined.

Correct.


> 		# Try to insert the file into the database cur.execute
('''INSERT INTO
> 		files (url, host, city, lastvisit) VALUES
> (%s, %s, %s, %s)''', (filename, host, city, lastvisit) )
> 
> 
> And the question remain as to why 'host' is undefined.

Because you haven't defined it BEFORE you try to use it. There is no 
point defining it AFTER you use it, the error has already occurred.

You have to make the coffee before you drink it:

# This fails too
drink(coffee)
coffee = make_coffee()


# This doesn't help at all
drink(coffee)
coffee = make_coffee() or prepare_coffee()


# This is also useless
drink(coffee)
coffee = make_coffee() or prepare_coffee() or "coffee"


# But this works
coffee = make_coffee()
drink(coffee)



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#53399

Τη Σάββατο, 31 Αυγούστου 2013 9:41:27 π.μ. UTC+3, ο χρήστης Ferrous Cranus έγραψε:
> Suddenly my webiste superhost.gr running my main python script presents 
> 
> me with this error:
> 
> 
> 
> Code:
> 
> UnicodeDecodeError('utf-8', b'\xb6\xe3\xed\xf9\xf3\xf4\xef 
> 
> \xfc\xed\xef\xec\xe1 \xf3\xf5\xf3\xf4\xde\xec\xe1\xf4\xef\xf2', 0, 1, 
> 
> 'invalid start byte')
> 
> 
> 
> 
> 
> Does anyone know what this means?
> 
> 
> 
> 
> 
> -- 
> 
> Webhost <http://superhost.gr>

Good morning Steven,

Ye i'm aware that i need to define variables before i try to make use of them.
I have study all of your examples and then re-view my code and i can *assure* you that the line statement that tied to set the 'host' variable is very early at the top of the script(of course after imports), and the cur.execute comes after.

The problem here is not what you say, that i try to drink k a coffee before actually making one first but rather than i cannot drink the coffee although i know *i have tried* to make one first.


i will upload the code for you to prove my sayings at pastebin.

http://pastebin.com/J97guApQ

[toc] | [prev] | [next] | [standalone]

#53401

On Sun, Sep 1, 2013 at 4:50 PM, Ferrous Cranus <nikos.gr33k@gmail.com> wrote:
> Ye i'm aware that i need to define variables before i try to make use of them.
> I have study all of your examples and then re-view my code and i can *assure* you that the line statement that tied to set the 'host' variable is very early at the top of the script(of course after imports), and the cur.execute comes after.
>
> The problem here is not what you say, that i try to drink k a coffee before actually making one first but rather than i cannot drink the coffee although i know *i have tried* to make one first.
>
>
> i will upload the code for you to prove my sayings at pastebin.
>
> http://pastebin.com/J97guApQ

You're setting host inside a try/except block. Are you getting
exceptions that prevent it from being set, perhaps?

Also... do you seriously (a) block access if no Referer: header, and
(b) permit that access anyway if the user has a non-blank cookie named
'admin'? Seriously??

ChrisA

[toc] | [prev] | [next] | [standalone]

#53402

Στις 1/9/2013 10:12 πμ, ο/η Chris Angelico έγραψε:
> On Sun, Sep 1, 2013 at 4:50 PM, Ferrous Cranus <nikos.gr33k@gmail.com> wrote:
>> Ye i'm aware that i need to define variables before i try to make use of them.
>> I have study all of your examples and then re-view my code and i can *assure* you that the line statement that tied to set the 'host' variable is very early at the top of the script(of course after imports), and the cur.execute comes after.
>>
>> The problem here is not what you say, that i try to drink k a coffee before actually making one first but rather than i cannot drink the coffee although i know *i have tried* to make one first.
>>
>>
>> i will upload the code for you to prove my sayings at pastebin.
>>
>> http://pastebin.com/J97guApQ
>
> You're setting host inside a try/except block. Are you getting
> exceptions that prevent it from being set, perhaps?
>
> Also... do you seriously (a) block access if no Referer: header, and
> (b) permit that access anyway if the user has a non-blank cookie named
> 'admin'? Seriously??
>
> ChrisA
>

I have checked th output of the erro log a tthe very moment that 
files.py tris to run and i receive this:


[Sun Sep 01 07:17:52 2013] [error] [client 108.162.231.113] ValueError: 
underlying buffer has been detached, referer: http://superhost.gr/
[Sun Sep 01 07:17:52 2013] [error] [client 108.162.231.113] , referer: 
http://superhost.gr/
[Sun Sep 01 07:17:52 2013] [error] [client 108.162.231.113] Original 
exception was:, referer: http://superhost.gr/
[Sun Sep 01 07:17:52 2013] [error] [client 108.162.231.113] Traceback 
(most recent call last):, referer: http://superhost.gr/
[Sun Sep 01 07:17:52 2013] [error] [client 108.162.231.113]   File 
"/home/nikos/public_html/cgi-bin/files.py", line 135, in <module>, 
referer: http://superhost.gr/
[Sun Sep 01 07:17:52 2013] [error] [client 108.162.231.113] 
cur.execute('''INSERT INTO files (url, host, city, lastvisit) VALUES 
(%s, %s, %s, %s)''', (filename, host, city, lastvisit) ), referer: 
http://superhost.gr/
[Sun Sep 01 07:17:52 2013] [error] [client 108.162.231.113] NameError: 
name 'host' is not defined, referer: http://superhost.gr/

Let alone that i when i try to set the 'host' variable i get this line 
at my '/tmp/err.out'


nikos@superhost.gr [~]# cat /tmp/err.out
UnicodeDecodeError('utf-8', b'\xb6\xe3\xed\xf9\xf3\xf4\xef 
\xfc\xed\xef\xec\xe1 \xf3\xf5\xf3\xf4\xde\xec\xe1\xf4\xef\xf2', 0, 1, 
'invalid start byte')

-- 
Webhost <http://superhost.gr>

[toc] | [prev] | [next] | [standalone]

#53403

On Sun, Sep 1, 2013 at 5:23 PM, Ferrous Cranus <nikos@superhost.gr> wrote:
> Let alone that i when i try to set the 'host' variable i get this line at my
> '/tmp/err.out'
>
>
> nikos@superhost.gr [~]# cat /tmp/err.out
>
> UnicodeDecodeError('utf-8', b'\xb6\xe3\xed\xf9\xf3\xf4\xef
> \xfc\xed\xef\xec\xe1 \xf3\xf5\xf3\xf4\xde\xec\xe1\xf4\xef\xf2', 0, 1,
> 'invalid start byte')

So host isn't being set because of this error, which you're blithely
carrying on after and assuming that things have happened. Solve this
problem, only then move on. Otherwise you WILL confuse yourself
further.

ChrisA

[toc] | [prev] | [next] | [standalone]

#53408

On 1/9/2013 03:23, Ferrous Cranus wrote:

> Στις 1/9/2013 10:12 πμ, ο/η Chris Angelico έγραψε:
>> On Sun, Sep 1, 2013 at 4:50 PM, Ferrous Cranus <nikos.gr33k@gmail.com> wrote:
>>> Ye i'm aware that i need to define variables before i try to make use of them.
>>> I have study all of your examples and then re-view my code and i can *assure* you that the line statement that tied to set the 'host' variable is very early at the top of the script(of course after imports), and the cur.execute comes after.
>>>
>>> The problem here is not what you say, that i try to drink k a coffee before actually making one first but rather than i cannot drink the coffee although i know *i have tried* to make one first.
>>>
>>>
>>> i will upload the code for you to prove my sayings at pastebin.
>>>
>>> http://pastebin.com/J97guApQ
>>
>> You're setting host inside a try/except block. Are you getting
>> exceptions that prevent it from being set, perhaps?
>>

   <SNIP>
>
> Let alone that i when i try to set the 'host' variable i get this line 
> at my '/tmp/err.out'
>
>
> nikos@superhost.gr [~]# cat /tmp/err.out
> UnicodeDecodeError('utf-8', b'\xb6\xe3\xed\xf9\xf3\xf4\xef 
> \xfc\xed\xef\xec\xe1 \xf3\xf5\xf3\xf4\xde\xec\xe1\xf4\xef\xf2', 0, 1, 
> 'invalid start byte')
>

Let's examine the relevant code from your pastebin above:


try:
        gi = pygeoip.GeoIP('/usr/local/share/GeoIPCity.dat')
        city = gi.time_zone_by_addr( os.environ['REMOTE_ADDR'] ) or gi.time_zone_by_addr( os.environ['HTTP_CF_CONNECTING_IP'] )
        host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0] or socket.gethostbyaddr( os.environ['HTTP_CF_CONNECTING_IP'] )[0] or "Proxy Detected"
except Exception as e:
        print( repr(e), file=open( '/tmp/err.out', 'w' ) )
 
Presumably you do realize that any exception on any of the three lines
will skip the remainder, and print that line to the /tmp/err.out file? 
And that if any exception occurs, it'll be before 'host' is initialized
at all?  Possibly before 'city' is initialized, or even 'gi' ?   So
there's no reason yet to assume that somehow setting the 'host'
variable triggers anything. When in doubt, consider including only one
line in any given try block. But let's try simpler changes.

Your file mode is 'w' which will wipe out anything written earlier.  Yet
if the exception does not happen, the file will still contain error(s)
from some earlier run.  You should open (and truncate) the file before
the first use, then use the same handle in each exception at which you
want to record any information.  Further, you ought to put more into the
file than just the repr(e) value.  For starters, the Traceback would be
nice.

If I were doing it, I'd be printing  repr(sys.exc_info()) to that file. 
I'd also have some kind of label, so that once I had more than one of
these in the code, I'd have an easyt way to tell which was which.

This is my first crack at it (untested):

errout = open("/tmp/err.out", "w")   #opens and truncates the error
output file
try:
    gi = pygeoip.GeoIP('/usr/local/share/GeoIPCity.dat')
    city = gi.time_zone_by_addr( os.environ['REMOTE_ADDR'] ) or
gi.time_zone_by_addr( os.environ['HTTP_CF_CONNECTING_IP'] )
    host =socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0] or
socket.gethostbyaddr( os.environ['HTTP_CF_CONNECTING_IP'] )[0] or
"Proxy Detected"
except Exception as e:
    print( "Xyzzy exception-", repr(sys.exc_info()), file=errout)
    errout.flush()


Note that I haven't had to use exc_info() in my own code, so I'm sure it
could be formatted prettier.  But right now, you need to stop throwing
away useful information.


-- 
DaveA

[toc] | [prev] | [next] | [standalone]

#53414

Στις 1/9/2013 1:35 μμ, ο/η Dave Angel έγραψε:
> This is my first crack at it (untested):
>
> errout = open("/tmp/err.out", "w")   #opens and truncates the error
> output file
> try:
>      gi = pygeoip.GeoIP('/usr/local/share/GeoIPCity.dat')
>      city = gi.time_zone_by_addr( os.environ['REMOTE_ADDR'] ) or
> gi.time_zone_by_addr( os.environ['HTTP_CF_CONNECTING_IP'] )
>      host =socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0] or
> socket.gethostbyaddr( os.environ['HTTP_CF_CONNECTING_IP'] )[0] or
> "Proxy Detected"
> except Exception as e:
>      print( "Xyzzy exception-", repr(sys.exc_info()), file=errout)
>      errout.flush()
>
>
> Note that I haven't had to use exc_info() in my own code, so I'm sure it
> could be formatted prettier.  But right now, you need to stop throwing
> away useful information.

First of all thank you for your detailed information Dave.
I have tried all you said, the above example you provided me, but i'm 
afraid even with your approach which should have given more error 
specific information the output of the err file remains.


nikos@superhost.gr [~]# cat /tmp/err.out
UnicodeDecodeError('utf-8', b'\xb6\xe3\xed\xf9\xf3\xf4\xef 
\xfc\xed\xef\xec\xe1 \xf3\xf5\xf3\xf4\xde\xec\xe1\xf4\xef\xf2', 0, 1, 
'invalid start byte')



-- 
Webhost <http://superhost.gr>

[toc] | [prev] | [next] | [standalone]

#53418

On 1/9/2013 09:59, Ferrous Cranus wrote:

> Στις 1/9/2013 1:35 μμ, ο/η Dave Angel έγραψε:
>> This is my first crack at it (untested):
>>
>> errout = open("/tmp/err.out", "w")   #opens and truncates the error
>> output file
>> try:
>>      gi = pygeoip.GeoIP('/usr/local/share/GeoIPCity.dat')
>>      city = gi.time_zone_by_addr( os.environ['REMOTE_ADDR'] ) or
>> gi.time_zone_by_addr( os.environ['HTTP_CF_CONNECTING_IP'] )
>>      host =socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0] or
>> socket.gethostbyaddr( os.environ['HTTP_CF_CONNECTING_IP'] )[0] or
>> "Proxy Detected"
>> except Exception as e:
>>      print( "Xyzzy exception-", repr(sys.exc_info()), file=errout)
>>      errout.flush()
>>
>>
>> Note that I haven't had to use exc_info() in my own code, so I'm sure it
>> could be formatted prettier.  But right now, you need to stop throwing
>> away useful information.
>
> First of all thank you for your detailed information Dave.
> I have tried all you said, the above example you provided me, but i'm 
> afraid even with your approach which should have given more error 
> specific information the output of the err file remains.
>
>
> nikos@superhost.gr [~]# cat /tmp/err.out
> UnicodeDecodeError('utf-8', b'\xb6\xe3\xed\xf9\xf3\xf4\xef 
> \xfc\xed\xef\xec\xe1 \xf3\xf5\xf3\xf4\xde\xec\xe1\xf4\xef\xf2', 0, 1, 
> 'invalid start byte')
>
>
>

See my other response.  The above file did NOT result from running the
code above.  It is missing the "Xyzzy" label.

-- 
Signature file not found

[toc] | [prev] | [next] | [standalone]

#53409

On Sun, Sep 1, 2013 at 8:35 PM, Dave Angel <davea@davea.name> wrote:
> Your file mode is 'w' which will wipe out anything written earlier.  Yet
> if the exception does not happen, the file will still contain error(s)
> from some earlier run.  You should open (and truncate) the file before
> the first use, then use the same handle in each exception at which you
> want to record any information.

Hmm, I'd go the other way, and append to the file every time there's
an error - otherwise, one successful page load will wipe out the error
from a previous one. But I agree that the inconsistency is not helping
him.

ChrisA

[toc] | [prev] | [next] | [standalone]

#53406

On Sat, 31 Aug 2013 23:50:23 -0700, Ferrous Cranus wrote:

> Τη Σάββατο, 31 Αυγούστου 2013 9:41:27 π.μ. UTC+3, ο χρήστης Ferrous
> Cranus έγραψε:
>> Suddenly my webiste superhost.gr running my main python script presents
>> 
>> me with this error:
>> 
>> 
>> 
>> Code:
>> 
>> UnicodeDecodeError('utf-8', b'\xb6\xe3\xed\xf9\xf3\xf4\xef
>> 
>> \xfc\xed\xef\xec\xe1 \xf3\xf5\xf3\xf4\xde\xec\xe1\xf4\xef\xf2', 0, 1,
>> 
>> 'invalid start byte')
>> 
>> 
>> 
>> 
>> 
>> Does anyone know what this means?
>> 
>> 
>> 
>> 
>> 
>> --
>> 
>> Webhost <http://superhost.gr>
> 
> Good morning Steven,
> 
> Ye i'm aware that i need to define variables before i try to make use of
> them. I have study all of your examples and then re-view my code and i
> can *assure* you that the line statement that tied to set the 'host'
> variable is very early at the top of the script(of course after
> imports), and the cur.execute comes after.
> 
> The problem here is not what you say, that i try to drink k a coffee
> before actually making one first but rather than i cannot drink the
> coffee although i know *i have tried* to make one first.
> 
> 
> i will upload the code for you to prove my sayings at pastebin.
> 
> http://pastebin.com/J97guApQ


You are mistaken. In line 20-25, you have this:

try:
    gi = pygeoip.GeoIP('/usr/local/share/GeoIPCity.dat')
    city = gi.time_zone_by_addr( os.environ['REMOTE_ADDR'] ) or
        gi.time_zone_by_addr( os.environ['HTTP_CF_CONNECTING_IP'] )
    host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0] or
        socket.gethostbyaddr( os.environ['HTTP_CF_CONNECTING_IP'] )[0] 
        or "Proxy Detected"
except Exception as e:
        print( repr(e), file=open( '/tmp/err.out', 'w' ) )


An error occurs inside that block, *before* host gets set. Who knows what 
the error is? You have access to the err.out file, but apparently you 
aren't reading it to find out.

Then, 110 lines later, at line 135, you try to access the value of "host" 
that never got set.

Your job is to read the error in /tmp/err.out, see what is failing, and 
fix it.


-- 
Steven

[toc] | [prev] | [next] | [standalone]

Page 1 of 3  [1] 2 3  Next page →

Back to top | Article view | comp.lang.python

csiph-web