Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #48354 > unrolled thread

Why 'files.py' does not print the filenames into a table format?

Back to article view | Back to comp.lang.python

Contents

  Why 'files.py' does not print the filenames into a table format? Nick the Gr33k <support@superhost.gr> - 2013-06-15 22:38 +0300
    Re: Why 'files.py' does not print the filenames into a table format? Jarrod Henry <jarrodhenry@gmail.com> - 2013-06-15 14:46 -0500
      Re: Why 'files.py' does not print the filenames into a table format? Nick the Gr33k <support@superhost.gr> - 2013-06-15 22:51 +0300
        Re: Why 'files.py' does not print the filenames into a table format? Benjamin Schollnick <benjamin@schollnick.net> - 2013-06-15 16:29 -0400
        Re: Why 'files.py' does not print the filenames into a table format? Joshua Landau <joshua.landau.ws@gmail.com> - 2013-06-15 21:37 +0100
          Re: Why 'files.py' does not print the filenames into a table format? Νίκος <support@superhost.gr> - 2013-06-17 08:58 +0300
            Re: Why 'files.py' does not print the filenames into a table format? Νίκος <support@superhost.gr> - 2013-06-17 09:11 +0300
              Re: Why 'files.py' does not print the filenames into a table format? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-06-17 07:00 +0000
                Re: Why 'files.py' does not print the filenames into a table format? Simpleton <support@superhost.gr> - 2013-06-17 12:07 +0300
                  Re: Why 'files.py' does not print the filenames into a table format? Simpleton <support@superhost.gr> - 2013-06-17 12:18 +0300
                  Re: Why 'files.py' does not print the filenames into a table format? Simpleton <support@superhost.gr> - 2013-06-17 13:26 +0300
        Re: Why 'files.py' does not print the filenames into a table format? Chris Angelico <rosuav@gmail.com> - 2013-06-16 08:51 +1000
          Re: Why 'files.py' does not print the filenames into a table format? Nick the Gr33k <support@superhost.gr> - 2013-06-16 04:07 +0300
            Python Greek mailing list [was Re: Why 'files.py' does not print the filenames into a table format?] Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-06-16 05:06 +0000
              Re: Python Greek mailing list [was Re: Why 'files.py' does not print the filenames into a table format?] Nick the Gr33k <support@superhost.gr> - 2013-06-16 11:28 +0300
                Re: Python Greek mailing list [was Re: Why 'files.py' does not print the filenames into a table format?] Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-06-16 10:33 +0000
                  Re: Python Greek mailing list [was Re: Why 'files.py' does not print the filenames into a table format?] Ferrous Cranus <support@superhost.gr> - 2013-06-16 13:57 +0300
                    Re: Python Greek mailing list [was Re: Why 'files.py' does not print the filenames into a table format?] Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-06-16 12:09 +0100
                      Re: Python Greek mailing list [was Re: Why 'files.py' does not print the filenames into a table format?] Ferrous Cranus <support@superhost.gr> - 2013-06-16 14:12 +0300
                    Re: Python Greek mailing list [was Re: Why 'files.py' does not print the filenames into a table format?] Chris “Kwpolska” Warrick <kwpolska@gmail.com> - 2013-06-16 13:19 +0200
                  Re: Python Greek mailing list [was Re: Why 'files.py' does not print the filenames into a table format?] Chris “Kwpolska” Warrick <kwpolska@gmail.com> - 2013-06-16 13:03 +0200
            Re: Why 'files.py' does not print the filenames into a table format? Ferrous Cranus <support@superhost.gr> - 2013-06-16 16:57 +0300
          Re: Why 'files.py' does not print the filenames into a table format? Nick the Gr33k <support@superhost.gr> - 2013-06-16 04:10 +0300
          Re: Why 'files.py' does not print the filenames into a table format? Nick the Gr33k <support@superhost.gr> - 2013-06-16 04:24 +0300
        Re: Why 'files.py' does not print the filenames into a table format? Grant Edwards <invalid@invalid.invalid> - 2013-06-17 14:50 +0000
    Re: Why 'files.py' does not print the filenames into a table format? Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-06-15 22:29 +0100
      Re: Why 'files.py' does not print the filenames into a table format? Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2013-06-16 00:10 +0000
        Re: Why 'files.py' does not print the filenames into a table format? alex23 <wuwei23@gmail.com> - 2013-06-16 17:04 -0700
          Re: Why 'files.py' does not print the filenames into a table format? Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-06-17 01:31 +0100
      Re: Why 'files.py' does not print the filenames into a table format? Nick the Gr33k <support@superhost.gr> - 2013-06-16 04:03 +0300
        Re: Why 'files.py' does not print the filenames into a table format? Mark Lawrence <breamoreboy@yahoo.co.uk> - 2013-06-16 02:10 +0100
          Re: Why 'files.py' does not print the filenames into a table format? Nick the Gr33k <support@superhost.gr> - 2013-06-16 04:28 +0300
    Re: Why 'files.py' does not print the filenames into a table format? Denis McMahon <denismfmcmahon@gmail.com> - 2013-06-16 07:23 +0000
      Re: Why 'files.py' does not print the filenames into a table format? Nick the Gr33k <support@superhost.gr> - 2013-06-16 11:35 +0300
        Re: Why 'files.py' does not print the filenames into a table format? Denis McMahon <denismfmcmahon@gmail.com> - 2013-06-16 08:55 +0000
        Re: Why 'files.py' does not print the filenames into a table format? Ferrous Cranus <support@superhost.gr> - 2013-06-16 16:59 +0300
    Re: Why 'files.py' does not print the filenames into a table format? jmfauth <wxjmfauth@gmail.com> - 2013-06-16 05:57 -0700

Page 1 of 2  [1] 2  Next page →

#48354 — Why 'files.py' does not print the filenames into a table format?

Hello,

Trying to browse http://superhost.gr/?page=files.py with tailing -F of 
the error_log i noticed that error log outputs no error!

So that means that the script is correct.

here are the directory app's files.

nikos@superhost.gr [~/www/data/apps]# ls -l
total 412788
drwxr-xr-x 2 nikos nikos     4096 Jun 12 12:03 ./
drwxr-xr-x 6 nikos nikos     4096 May 26 21:13 ../
-rwxr-xr-x 1 nikos nikos 13157283 Mar 17 12:57 100\ Mythoi\ tou\ 
Aiswpou.pdf*
-rwxr-xr-x 1 nikos nikos 29524686 Mar 11 18:17 Anekdotologio.exe*
-rw-r--r-- 1 nikos nikos 42413964 Jun  2 20:29 Battleship.exe
-rw-r--r-- 1 nikos nikos 51819750 Jun  2 20:04 Luxor\ Evolved.exe
-rw-r--r-- 1 nikos nikos 60571648 Jun  2 14:59 Monopoly.exe
-rwxr-xr-x 1 nikos nikos  1788164 Mar 14 11:31 Online\ Movie\ Player.zip*
-rw-r--r-- 1 nikos nikos  5277287 Jun  1 18:35 O\ Nomos\ tou\ Merfy\ 
v1-2-3.zip
-rwxr-xr-x 1 nikos nikos 16383001 Jun 22  2010 Orthodoxo\ Imerologio.exe*
-rw-r--r-- 1 nikos nikos  6084806 Jun  1 18:22 Pac-Man.exe
-rw-r--r-- 1 nikos nikos 45297713 Jun 10 12:38 Raptor\ Chess.exe
-rw-r--r-- 1 nikos nikos 25476584 Jun  2 19:50 Scrabble.exe
-rwxr-xr-x 1 nikos nikos 49141166 Mar 17 12:48 To\ 1o\ mou\ vivlio\ gia\ 
to\ skaki.pdf*
-rwxr-xr-x 1 nikos nikos  3298310 Mar 17 12:45 Vivlos\ gia\ Atheofovous.pdf*
-rw-r--r-- 1 nikos nikos  1764864 May 29 21:50 V-Radio\ v2.4.msi
-rw-r--r-- 1 nikos nikos  3511233 Jun  4 14:11 Ευχή\ του\ Ιησού.mp3
-rwxr-xr-x 1 nikos nikos 66896732 Mar 17 13:13 Κοσμάς\ Αιτωλός\ -\ 
Προφητείες.pdf*
-rw-r--r-- 1 nikos nikos   236032 Jun  4 14:10 Σκέψου\ έναν\ αριθμό.exe


The code is as follows:

# 
=================================================================================================================
# Convert wrongly encoded filenames to utf-8
# 
=================================================================================================================
path = b'/home/nikos/public_html/data/apps/'
filenames = os.listdir( path )

utf8_filenames = []

for filename in filenames:
	# Compute 'path/to/filename'
	filename_bytes = path + filename
	encoding = guess_encoding( filename_bytes )
	
	if encoding == 'utf-8':
		# File name is valid UTF-8, so we can skip to the next file.
		utf8_filenames.append( filename_bytes )
		continue
	elif encoding is None:
		# No idea what the encoding is. Hit it with a hammer until it stops 
moving.
		filename = filename_bytes.decode( 'utf-8', 'xmlcharrefreplace' )
	else:
		filename = filename_bytes.decode( encoding )

	# Rename the file to something which ought to be UTF-8 clean.
	newname_bytes = filename.encode('utf-8')
	os.rename( filename_bytes, newname_bytes )
	utf8_filenames.append( newname_bytes )
	
	# Once we get here, the file ought to be UTF-8 clean and the Unicode 
name ought to exist:
	assert os.path.exists( newname_bytes.decode('utf-8') )


# Switch filenames from utf8 bytestrings => unicode strings
filenames = []

for utf8_filename in utf8_filenames:
	filenames.append( utf8_filename.decode('utf-8') )

# Check the presence of a database file against the dir files and delete 
record if it doesn't exist
cur.execute('''SELECT url FROM files''')
data = cur.fetchall()

for url in data:
	if url not in filenames:
		# Delete spurious
		cur.execute('''DELETE FROM files WHERE url = %s''', url )


# 
=================================================================================================================
# Display ALL files, each with its own download button
# 
=================================================================================================================
print('''<body background='/data/images/star.jpg'>
		 <center><img src='/data/images/download.gif'><br><br>
		 <table border=5 cellpadding=5 bgcolor=green>
''')

try:
	cur.execute( '''SELECT * FROM files ORDER BY lastvisit DESC''' )
	data = cur.fetchall()
	
	for row in data:
		(filename, hits, host, lastvisit) = row
		lastvisit = lastvisit.strftime('%A %e %b, %H:%M')
		
		print('''
		<form method="get" action="/cgi-bin/files.py">
			<tr>
				<td> <center> <input type="submit" name="filename" value="%s"> </td>
				<td> <center> <font color=yellow size=5> %s </td>
				<td> <center> <font color=orange size=4> %s </td>
				<td> <center> <font color=silver size=4> %s </td>
			</tr>
		</form>
		''' % (filename, hits, host, lastvisit) )
	print( '''</table><br><br>''' )
except pymysql.ProgrammingError as e:
	print( repr(e) )

===========================================
PLEASE take a look, its not a huge code, the encoding was of Steven 
idea's, so from another thread is a bit more or less already known to 
the most of you.

I just want to know why it doesn't print anything.

Thank you and please whoever does not feel like helping, please at least 
not spam the thread.

-- 
What is now proved was at first only imagined!

[toc] | [next] | [standalone]

#48356

[Multipart message — attachments visible in raw view] — view raw

Nick, at this point, you need to hire someone to do your work for you.

We are not here to do your job.  I would suggest finding a coder for hire
and letting them do this job correctly.

Thanks.


On Sat, Jun 15, 2013 at 2:38 PM, Nick the Gr33k <support@superhost.gr>wrote:

> Hello,
>
> Trying to browse http://superhost.gr/?page=**files.py<http://superhost.gr/?page=files.py>with tailing -F of the error_log i noticed that error log outputs no error!
>
> So that means that the script is correct.
>
> here are the directory app's files.
>
> nikos@superhost.gr [~/www/data/apps]# ls -l
> total 412788
> drwxr-xr-x 2 nikos nikos     4096 Jun 12 12:03 ./
> drwxr-xr-x 6 nikos nikos     4096 May 26 21:13 ../
> -rwxr-xr-x 1 nikos nikos 13157283 Mar 17 12:57 100\ Mythoi\ tou\
> Aiswpou.pdf*
> -rwxr-xr-x 1 nikos nikos 29524686 Mar 11 18:17 Anekdotologio.exe*
> -rw-r--r-- 1 nikos nikos 42413964 Jun  2 20:29 Battleship.exe
> -rw-r--r-- 1 nikos nikos 51819750 Jun  2 20:04 Luxor\ Evolved.exe
> -rw-r--r-- 1 nikos nikos 60571648 Jun  2 14:59 Monopoly.exe
> -rwxr-xr-x 1 nikos nikos  1788164 Mar 14 11:31 Online\ Movie\ Player.zip*
> -rw-r--r-- 1 nikos nikos  5277287 Jun  1 18:35 O\ Nomos\ tou\ Merfy\
> v1-2-3.zip
> -rwxr-xr-x 1 nikos nikos 16383001 Jun 22  2010 Orthodoxo\ Imerologio.exe*
> -rw-r--r-- 1 nikos nikos  6084806 Jun  1 18:22 Pac-Man.exe
> -rw-r--r-- 1 nikos nikos 45297713 Jun 10 12:38 Raptor\ Chess.exe
> -rw-r--r-- 1 nikos nikos 25476584 Jun  2 19:50 Scrabble.exe
> -rwxr-xr-x 1 nikos nikos 49141166 Mar 17 12:48 To\ 1o\ mou\ vivlio\ gia\
> to\ skaki.pdf*
> -rwxr-xr-x 1 nikos nikos  3298310 Mar 17 12:45 Vivlos\ gia\
> Atheofovous.pdf*
> -rw-r--r-- 1 nikos nikos  1764864 May 29 21:50 V-Radio\ v2.4.msi
> -rw-r--r-- 1 nikos nikos  3511233 Jun  4 14:11 Ευχή\ του\ Ιησού.mp3
> -rwxr-xr-x 1 nikos nikos 66896732 Mar 17 13:13 Κοσμάς\ Αιτωλός\ -\
> Προφητείες.pdf*
> -rw-r--r-- 1 nikos nikos   236032 Jun  4 14:10 Σκέψου\ έναν\ αριθμό.exe
>
>
> The code is as follows:
>
> # ==============================**==============================**
> ==============================**=======================
> # Convert wrongly encoded filenames to utf-8
> # ==============================**==============================**
> ==============================**=======================
> path = b'/home/nikos/public_html/**data/apps/'
> filenames = os.listdir( path )
>
> utf8_filenames = []
>
> for filename in filenames:
>         # Compute 'path/to/filename'
>         filename_bytes = path + filename
>         encoding = guess_encoding( filename_bytes )
>
>         if encoding == 'utf-8':
>                 # File name is valid UTF-8, so we can skip to the next
> file.
>                 utf8_filenames.append( filename_bytes )
>                 continue
>         elif encoding is None:
>                 # No idea what the encoding is. Hit it with a hammer until
> it stops moving.
>                 filename = filename_bytes.decode( 'utf-8',
> 'xmlcharrefreplace' )
>         else:
>                 filename = filename_bytes.decode( encoding )
>
>         # Rename the file to something which ought to be UTF-8 clean.
>         newname_bytes = filename.encode('utf-8')
>         os.rename( filename_bytes, newname_bytes )
>         utf8_filenames.append( newname_bytes )
>
>         # Once we get here, the file ought to be UTF-8 clean and the
> Unicode name ought to exist:
>         assert os.path.exists( newname_bytes.decode('utf-8') )
>
>
> # Switch filenames from utf8 bytestrings => unicode strings
> filenames = []
>
> for utf8_filename in utf8_filenames:
>         filenames.append( utf8_filename.decode('utf-8') )
>
> # Check the presence of a database file against the dir files and delete
> record if it doesn't exist
> cur.execute('''SELECT url FROM files''')
> data = cur.fetchall()
>
> for url in data:
>         if url not in filenames:
>                 # Delete spurious
>                 cur.execute('''DELETE FROM files WHERE url = %s''', url )
>
>
> # ==============================**==============================**
> ==============================**=======================
> # Display ALL files, each with its own download button
> # ==============================**==============================**
> ==============================**=======================
> print('''<body background='/data/images/star.**jpg'>
>                  <center><img src='/data/images/download.**gif'><br><br>
>                  <table border=5 cellpadding=5 bgcolor=green>
> ''')
>
> try:
>         cur.execute( '''SELECT * FROM files ORDER BY lastvisit DESC''' )
>         data = cur.fetchall()
>
>         for row in data:
>                 (filename, hits, host, lastvisit) = row
>                 lastvisit = lastvisit.strftime('%A %e %b, %H:%M')
>
>                 print('''
>                 <form method="get" action="/cgi-bin/files.py">
>                         <tr>
>                                 <td> <center> <input type="submit"
> name="filename" value="%s"> </td>
>                                 <td> <center> <font color=yellow size=5>
> %s </td>
>                                 <td> <center> <font color=orange size=4>
> %s </td>
>                                 <td> <center> <font color=silver size=4>
> %s </td>
>                         </tr>
>                 </form>
>                 ''' % (filename, hits, host, lastvisit) )
>         print( '''</table><br><br>''' )
> except pymysql.ProgrammingError as e:
>         print( repr(e) )
>
> ==============================**=============
> PLEASE take a look, its not a huge code, the encoding was of Steven
> idea's, so from another thread is a bit more or less already known to the
> most of you.
>
> I just want to know why it doesn't print anything.
>
> Thank you and please whoever does not feel like helping, please at least
> not spam the thread.
>
> --
> What is now proved was at first only imagined!
> --
> http://mail.python.org/**mailman/listinfo/python-list<http://mail.python.org/mailman/listinfo/python-list>
>

[toc] | [prev] | [next] | [standalone]

#48357

On 15/6/2013 10:46 μμ, Jarrod Henry wrote:
> Nick, at this point, you need to hire someone to do your work for you.

The code is completely ready.
Some detail is missing and its not printing the files as expected.

Irrelevant to my question i just noticed weird behavior about my 
pelatologio.py script which can be seen here:

http://superhost.gr/?show=stats

The first 3 files are of my doing.
All the rest are of someone else's that managed to append entries into 
my counters database utilizing this code:

================

	try:
		#find the needed counter for the page URL
		cur.execute('''SELECT ID FROM counters WHERE url = %s''', page )
		data = cur.fetchone()        #URL is unique, so should only be one
			
		if not data:
			#first time for page; primary key is automatic, hit is defaulted
			cur.execute('''INSERT INTO counters (url) VALUES (%s)''', page )
			cID = cur.lastrowid        #get the primary key value of the new record
======================

Does someone want to state something?




-- 
What is now proved was at first only imagined!

[toc] | [prev] | [next] | [standalone]

#48365

[Multipart message — attachments visible in raw view] — view raw

>> Nick, at this point, you need to hire someone to do your work for you.
> 
> The code is completely ready.
> Some detail is missing and its not printing the files as expected.

Then the code is not completely ready, it has bugs, and your trying to have the list debug and fix it for you for free.

> Irrelevant to my question i just noticed weird behavior about my pelatologio.py script which can be seen here:
> 
> http://superhost.gr/?show=stats
> 
> The first 3 files are of my doing.
> All the rest are of someone else's that managed to append entries into my counters database utilizing this code:
> 
> ================
> 
> 	try:
> 		#find the needed counter for the page URL
> 		cur.execute('''SELECT ID FROM counters WHERE url = %s''', page )
> 		data = cur.fetchone()        #URL is unique, so should only be one
> 			
> 		if not data:
> 			#first time for page; primary key is automatic, hit is defaulted
> 			cur.execute('''INSERT INTO counters (url) VALUES (%s)''', page )
> 			cID = cur.lastrowid        #get the primary key value of the new record
> ======================
> 
> Does someone want to state something?

Sure, whoever wrote that code is a fool.

http://xkcd.com/327/

<img src="http://imgs.xkcd.com/comics/exploits_of_a_mom.png">

They didn't sanitize your database inputs.

My suggestion would be for you to stop trying to re-invent the wheel, and use COTS software.  

		- Benjamin

[toc] | [prev] | [next] | [standalone]

#48367

On 15 June 2013 20:51, Nick the Gr33k <support@superhost.gr> wrote:
> On 15/6/2013 10:46 μμ, Jarrod Henry wrote:
>>
>> Nick, at this point, you need to hire someone to do your work for you.
>
>
> The code is completely ready.
> Some detail is missing and its not printing the files as expected.

Look, Nick,

A lot of people are frustrated by you. You should understand that. If
you cannot, you need to step back and consider, or you really are a
troll.

Now, obviously it's not going to get you any help to have half of the
forum angry at you. People have stopped helping, at least in large.
This is fine; people here are volunteers. But you want help.

So, Nick, listen. You need to learn how to ask *smart* questions. If
you do, I *guarantee* that people will respect you a lot more. I'll be
willing to give a bit of time to explain what I mean.

1) What is your problem. Not "I want to know why it doesn't print
anything." Here's an example, for some random idea:

> I've written some code to find the first file in a directory which
> is not UTF-8. Lines 40-42 are meant to print out the file found
> to a log ("/home/joshua/.logs/log"). Unfortunately, although
> there is no error, no file is printed to the log.

2) What have you tried? What debugging have you done? For someone of
your skill level, it's also important to tell us what you think your
code is doing. Example:

> I've tried checking for a failure - when there is no non-UTF-8 file
> in the directory the appropriate error is raised. I think this should
> mean that the "else" after the "for" loop would be run, and this
> should run the lines 40-42 above when there *is* a non-UTF-8
> file.

3) If possible, give us an example we can run.

> To make helping easier, I've removed the code that searches the
> directory as I know that works, and instead there's a list of BytesIO
> and StringIO objects that pretend to be them. The bug is still
> there.

Do you see the difference?

> Irrelevant to my question i just noticed weird behavior about my
> pelatologio.py script which can be seen here:
>
> http://superhost.gr/?show=stats
>
> The first 3 files are of my doing.
> All the rest are of someone else's that managed to append entries into my
> counters database utilizing this code:
>
> ================
>
>         try:
>                 #find the needed counter for the page URL
>                 cur.execute('''SELECT ID FROM counters WHERE url = %s''',
> page )
>                 data = cur.fetchone()        #URL is unique, so should only
> be one
>
>                 if not data:
>                         #first time for page; primary key is automatic, hit
> is defaulted
>                         cur.execute('''INSERT INTO counters (url) VALUES
> (%s)''', page )
>                         cID = cur.lastrowid        #get the primary key
> value of the new record
> ======================
>
> Does someone want to state something?

Sure. Here I go:

What's the question?

[toc] | [prev] | [next] | [standalone]

#48495

On 15/6/2013 11:37 μμ, Joshua Landau wrote:
> On 15 June 2013 20:51, Nick the Gr33k <support@superhost.gr> wrote:
>> On 15/6/2013 10:46 μμ, Jarrod Henry wrote:
>>>
>>> Nick, at this point, you need to hire someone to do your work for you.
>>
>>
>> The code is completely ready.
>> Some detail is missing and its not printing the files as expected.
>
> Look, Nick,
>
> A lot of people are frustrated by you. You should understand that. If
> you cannot, you need to step back and consider, or you really are a
> troll.
>
> Now, obviously it's not going to get you any help to have half of the
> forum angry at you. People have stopped helping, at least in large.
> This is fine; people here are volunteers. But you want help.
>
> So, Nick, listen. You need to learn how to ask *smart* questions. If
> you do, I *guarantee* that people will respect you a lot more. I'll be
> willing to give a bit of time to explain what I mean.
>
> 1) What is your problem. Not "I want to know why it doesn't print
> anything." Here's an example, for some random idea:
>
>> I've written some code to find the first file in a directory which
>> is not UTF-8. Lines 40-42 are meant to print out the file found
>> to a log ("/home/joshua/.logs/log"). Unfortunately, although
>> there is no error, no file is printed to the log.
>
> 2) What have you tried? What debugging have you done? For someone of
> your skill level, it's also important to tell us what you think your
> code is doing. Example:
>
>> I've tried checking for a failure - when there is no non-UTF-8 file
>> in the directory the appropriate error is raised. I think this should
>> mean that the "else" after the "for" loop would be run, and this
>> should run the lines 40-42 above when there *is* a non-UTF-8
>> file.
>
> 3) If possible, give us an example we can run.
>
>> To make helping easier, I've removed the code that searches the
>> directory as I know that works, and instead there's a list of BytesIO
>> and StringIO objects that pretend to be them. The bug is still
>> there.
>
> Do you see the difference?
>
>> Irrelevant to my question i just noticed weird behavior about my
>> pelatologio.py script which can be seen here:
>>
>> http://superhost.gr/?show=stats
>>
>> The first 3 files are of my doing.
>> All the rest are of someone else's that managed to append entries into my
>> counters database utilizing this code:
>>
>> ================
>>
>>          try:
>>                  #find the needed counter for the page URL
>>                  cur.execute('''SELECT ID FROM counters WHERE url = %s''',
>> page )
>>                  data = cur.fetchone()        #URL is unique, so should only
>> be one
>>
>>                  if not data:
>>                          #first time for page; primary key is automatic, hit
>> is defaulted
>>                          cur.execute('''INSERT INTO counters (url) VALUES
>> (%s)''', page )
>>                          cID = cur.lastrowid        #get the primary key
>> value of the new record
>> ======================
>>
>> Does someone want to state something?
>
> Sure. Here I go:
>
> What's the question?


I DID, I FINALLY DID IT JUST NOW!!

HERE ARE THE MODIFICATIONS THAT MADE IT HAPPEN!


==========================================================================================================
# Convert wrongly encoded filenames to utf-8
==========================================================================================================

path = b'/home/nikos/public_html/data/apps/'
filenames = os.listdir( path )

utf8_filenames = []

for filename in filenames:
	# Compute 'path/to/filename'
	filename_bytes = path + filename
	encoding = guess_encoding( filename_bytes )
	
	if encoding == 'utf-8':
		# File name is valid UTF-8, so we can skip to the next file.
		utf8_filenames.append( filename_bytes )
		continue
	elif encoding is None:
		# No idea what the encoding is. Hit it with a hammer until it stops 
moving.
		filename = filename_bytes.decode( 'utf-8', 'xmlcharrefreplace' )
	else:
		filename = filename_bytes.decode( encoding )

	# Rename the file to something which ought to be UTF-8 clean.
	newname_bytes = filename.encode('utf-8')
	os.rename( filename_bytes, newname_bytes )
	utf8_filenames.append( newname_bytes )
	
	# Once we get here, the file ought to be UTF-8 clean and the Unicode 
name ought to exist:
	assert os.path.exists( newname_bytes.decode('utf-8') )
================================

i SMASHED MY HEAD INTO THE WALL, BUT I MADE IT!!!!
FINALLY AFTER > 15 DAYS!!

FEEL FREE TO CONGRATULATE ME!



-- 
What is now proved was at first only imagined!

[toc] | [prev] | [next] | [standalone]

#48496

On 17/6/2013 8:58 πμ, Νίκος wrote:
> On 15/6/2013 11:37 μμ, Joshua Landau wrote:
>> On 15 June 2013 20:51, Nick the Gr33k <support@superhost.gr> wrote:
>>> On 15/6/2013 10:46 μμ, Jarrod Henry wrote:
>>>>
>>>> Nick, at this point, you need to hire someone to do your work for you.
>>>
>>>
>>> The code is completely ready.
>>> Some detail is missing and its not printing the files as expected.
>>
>> Look, Nick,
>>
>> A lot of people are frustrated by you. You should understand that. If
>> you cannot, you need to step back and consider, or you really are a
>> troll.
>>
>> Now, obviously it's not going to get you any help to have half of the
>> forum angry at you. People have stopped helping, at least in large.
>> This is fine; people here are volunteers. But you want help.
>>
>> So, Nick, listen. You need to learn how to ask *smart* questions. If
>> you do, I *guarantee* that people will respect you a lot more. I'll be
>> willing to give a bit of time to explain what I mean.
>>
>> 1) What is your problem. Not "I want to know why it doesn't print
>> anything." Here's an example, for some random idea:
>>
>>> I've written some code to find the first file in a directory which
>>> is not UTF-8. Lines 40-42 are meant to print out the file found
>>> to a log ("/home/joshua/.logs/log"). Unfortunately, although
>>> there is no error, no file is printed to the log.
>>
>> 2) What have you tried? What debugging have you done? For someone of
>> your skill level, it's also important to tell us what you think your
>> code is doing. Example:
>>
>>> I've tried checking for a failure - when there is no non-UTF-8 file
>>> in the directory the appropriate error is raised. I think this should
>>> mean that the "else" after the "for" loop would be run, and this
>>> should run the lines 40-42 above when there *is* a non-UTF-8
>>> file.
>>
>> 3) If possible, give us an example we can run.
>>
>>> To make helping easier, I've removed the code that searches the
>>> directory as I know that works, and instead there's a list of BytesIO
>>> and StringIO objects that pretend to be them. The bug is still
>>> there.
>>
>> Do you see the difference?
>>
>>> Irrelevant to my question i just noticed weird behavior about my
>>> pelatologio.py script which can be seen here:
>>>
>>> http://superhost.gr/?show=stats
>>>
>>> The first 3 files are of my doing.
>>> All the rest are of someone else's that managed to append entries
>>> into my
>>> counters database utilizing this code:
>>>
>>> ================
>>>
>>>          try:
>>>                  #find the needed counter for the page URL
>>>                  cur.execute('''SELECT ID FROM counters WHERE url =
>>> %s''',
>>> page )
>>>                  data = cur.fetchone()        #URL is unique, so
>>> should only
>>> be one
>>>
>>>                  if not data:
>>>                          #first time for page; primary key is
>>> automatic, hit
>>> is defaulted
>>>                          cur.execute('''INSERT INTO counters (url)
>>> VALUES
>>> (%s)''', page )
>>>                          cID = cur.lastrowid        #get the primary key
>>> value of the new record
>>> ======================
>>>
>>> Does someone want to state something?
>>
>> Sure. Here I go:
>>
>> What's the question?
>
>
> I DID, I FINALLY DID IT JUST NOW!!
>
> HERE ARE THE MODIFICATIONS THAT MADE IT HAPPEN!
>
>
> ==========================================================================================================
>
> # Convert wrongly encoded filenames to utf-8
> ==========================================================================================================
>
>
> path = b'/home/nikos/public_html/data/apps/'
> filenames = os.listdir( path )
>
> utf8_filenames = []
>
> for filename in filenames:
>      # Compute 'path/to/filename'
>      filename_bytes = path + filename
>      encoding = guess_encoding( filename_bytes )
>
>      if encoding == 'utf-8':
>          # File name is valid UTF-8, so we can skip to the next file.
>          utf8_filenames.append( filename_bytes )
>          continue
>      elif encoding is None:
>          # No idea what the encoding is. Hit it with a hammer until it
> stops moving.
>          filename = filename_bytes.decode( 'utf-8', 'xmlcharrefreplace' )
>      else:
>          filename = filename_bytes.decode( encoding )
>
>      # Rename the file to something which ought to be UTF-8 clean.
>      newname_bytes = filename.encode('utf-8')
>      os.rename( filename_bytes, newname_bytes )
>      utf8_filenames.append( newname_bytes )
>
>      # Once we get here, the file ought to be UTF-8 clean and the
> Unicode name ought to exist:
>      assert os.path.exists( newname_bytes.decode('utf-8') )
> ================================
>
> i SMASHED MY HEAD INTO THE WALL, BUT I MADE IT!!!!
> FINALLY AFTER > 15 DAYS!!
>
> FEEL FREE TO CONGRATULATE ME!

oups!

everything work as expected but not the part when the counter of a 
filename gets increased when the file have been requested.

I don't see how since:

if filename:
	#update file counter
	cur.execute('''UPDATE files SET hits = hits + 1, host = %s, lastvisit = 
%s WHERE url = %s''', (host, lastvisit, filename) )

-- 
What is now proved was at first only imagined!

[toc] | [prev] | [next] | [standalone]

#48501

On Mon, 17 Jun 2013 09:11:05 +0300, Νίκος wrote:

> everything work as expected but not the part when the counter of a
> filename gets increased when the file have been requested.
> 
> I don't see how since:
> 
> if filename:
> 	#update file counter
> 	cur.execute('''UPDATE files SET hits = hits + 1, host = %s, 
>       lastvisit = %s WHERE url = %s''', (host, lastvisit, filename) )


Have you read these links yet?

http://sscce.org/‎

http://www.catb.org/esr/faqs/smart-questions.html‎


They will teach you how to successfully ask for help.




-- 
Steven

[toc] | [prev] | [next] | [standalone]

#48510

On 17/6/2013 10:00 πμ, Steven D'Aprano wrote:
> On Mon, 17 Jun 2013 09:11:05 +0300, Νίκος wrote:
>
>> everything work as expected but not the part when the counter of a
>> filename gets increased when the file have been requested.
>>
>> I don't see how since:
>>
>> if filename:
>> 	#update file counter
>> 	cur.execute('''UPDATE files SET hits = hits + 1, host = %s,
>>        lastvisit = %s WHERE url = %s''', (host, lastvisit, filename) )
>
>
There isn'tmuch to say ehre. You already know the code that im using 
inside files.pu and the question is that this execute never gets to execute.

# 
=================================================================================================================
# Make sure that ALL database records are filenames in existance
# 
=================================================================================================================
filenames = []

# Switch filenames from (utf8 bytestrings => unicode strings) and trim 
them from their paths
for utf8_filename in utf8_filenames:
	filenames.append( utf8_filename.decode('utf-8').replace( 
'/home/nikos/public_html/data/apps/', '' ) )

# Check the presence of a database file against the dir files and delete 
record if it doesn't exist
cur.execute('''SELECT url FROM files''')
data = cur.fetchall()

# Delete spurious database records
for rec in data:
	if rec not in filenames:
		cur.execute('''DELETE FROM files WHERE url = %s''', rec )

# Load'em
for filename in filenames:
	try:
		# Check the presence of current filename against it's database presence
		cur.execute('''SELECT url FROM files WHERE url = %s''', filename )
		data = cur.fetchone()

		if not data:
			# First time for file; primary key is automatic, hit is defaulted
			cur.execute('''INSERT INTO files (url, host, lastvisit) VALUES (%s, 
%s, %s)''', (filename, host, lastvisit) )
	except pymysql.ProgrammingError as e:
		print( repr(e) )


# 
=================================================================================================================
# Display ALL files, each with its own download button
# 
=================================================================================================================
print('''<body background='/data/images/star.jpg'>
		 <center><img src='/data/images/download.gif'><br><br>
		 <table border=5 cellpadding=5 bgcolor=green>
''')

try:
	cur.execute( '''SELECT * FROM files ORDER BY lastvisit DESC''' )
	data = cur.fetchall()
	
	for row in data:
		(filename, hits, host, lastvisit) = row
		lastvisit = lastvisit.strftime('%A %e %b, %H:%M')
		
		print('''
		<form method="get" action="/cgi-bin/files.py">
			<tr>
				<td> <center> <input type="submit" name="filename" value="%s"> </td>
				<td> <center> <font color=yellow size=5> %s </td>
				<td> <center> <font color=orange size=4> %s </td>
				<td> <center> <font color=silver size=4> %s </td>
			</tr>
		</form>
		''' % (filename, hits, host, lastvisit) )
	print( '''</table><br><br>''' )
except pymysql.ProgrammingError as e:
	print( repr(e) )
	
sys.exit(0)

After a spcific file gets selected then files.py is reloading grabbign 
the filename as a variable form and:

# 
=================================================================================================================
# If user downloaded a file, thank the user !!!
# 
=================================================================================================================
if filename:
	#update filename's counter if cookie does not exist
	cur.execute('''UPDATE files SET hits = hits + 1, host = %s, lastvisit = 
%s WHERE url = %s''', (host, lastvisit, filename) )

but the execute never happesn.
i ahve tested it

if data:
	print soemthing

but data is always empty.
-- 
What is now proved was at first only imagined!

[toc] | [prev] | [next] | [standalone]

#48511

On 17/6/2013 12:07 μμ, Simpleton wrote:
> # Load'em
> for filename in filenames:
>      try:
>          # Check the presence of current filename against it's database
> presence
>          cur.execute('''SELECT url FROM files WHERE url = %s''', filename )
>          data = cur.fetchone()
>
>          if not data:
>              # First time for file; primary key is automatic, hit is
> defaulted
>              cur.execute('''INSERT INTO files (url, host, lastvisit)
> VALUES (%s, %s, %s)''', (filename, host, lastvisit) )
>      except pymysql.ProgrammingError as e:
>          print( repr(e) )


Also i just alternated the above code to:

# Load'em
for filename in filenames:
	try:
		# try to insert the file into the database
		cur.execute('''INSERT INTO files (url, host, lastvisit) VALUES (%s, 
%s, %s)''', (filename, host, lastvisit) )
	except pymysql.ProgrammingError as e:
		# Insertion failed, file already into database, skip this, go to next 
filename
		pass

Isn't more compact and straightforward this way?
but i have to set the url's type into unique type for the abpve to work?

-- 
What is now proved was at first only imagined!

[toc] | [prev] | [next] | [standalone]

#48512

On 17/6/2013 12:07 μμ, Simpleton wrote:
> On 17/6/2013 10:00 πμ, Steven D'Aprano wrote:
>> On Mon, 17 Jun 2013 09:11:05 +0300, Νίκος wrote:
>>
>>> everything work as expected but not the part when the counter of a
>>> filename gets increased when the file have been requested.
>>>
>>> I don't see how since:
>>>
>>> if filename:
>>>     #update file counter
>>>     cur.execute('''UPDATE files SET hits = hits + 1, host = %s,
>>>        lastvisit = %s WHERE url = %s''', (host, lastvisit, filename) )
>>
>>
> There isn'tmuch to say ehre. You already know the code that im using
> inside files.pu and the question is that this execute never gets to
> execute.
>
> #
> =================================================================================================================
>
> # Make sure that ALL database records are filenames in existance
> #
> =================================================================================================================
>
> filenames = []
>
> # Switch filenames from (utf8 bytestrings => unicode strings) and trim
> them from their paths
> for utf8_filename in utf8_filenames:
>      filenames.append( utf8_filename.decode('utf-8').replace(
> '/home/nikos/public_html/data/apps/', '' ) )
>
> # Check the presence of a database file against the dir files and delete
> record if it doesn't exist
> cur.execute('''SELECT url FROM files''')
> data = cur.fetchall()
>
> # Delete spurious database records
> for rec in data:
>      if rec not in filenames:
>          cur.execute('''DELETE FROM files WHERE url = %s''', rec )
>
> # Load'em
> for filename in filenames:
>      try:
>          # Check the presence of current filename against it's database
> presence
>          cur.execute('''SELECT url FROM files WHERE url = %s''', filename )
>          data = cur.fetchone()
>
>          if not data:
>              # First time for file; primary key is automatic, hit is
> defaulted
>              cur.execute('''INSERT INTO files (url, host, lastvisit)
> VALUES (%s, %s, %s)''', (filename, host, lastvisit) )
>      except pymysql.ProgrammingError as e:
>          print( repr(e) )
>
>
> #
> =================================================================================================================
>
> # Display ALL files, each with its own download button
> #
> =================================================================================================================
>
> print('''<body background='/data/images/star.jpg'>
>           <center><img src='/data/images/download.gif'><br><br>
>           <table border=5 cellpadding=5 bgcolor=green>
> ''')
>
> try:
>      cur.execute( '''SELECT * FROM files ORDER BY lastvisit DESC''' )
>      data = cur.fetchall()
>
>      for row in data:
>          (filename, hits, host, lastvisit) = row
>          lastvisit = lastvisit.strftime('%A %e %b, %H:%M')
>
>          print('''
>          <form method="get" action="/cgi-bin/files.py">
>              <tr>
>                  <td> <center> <input type="submit" name="filename"
> value="%s"> </td>
>                  <td> <center> <font color=yellow size=5> %s </td>
>                  <td> <center> <font color=orange size=4> %s </td>
>                  <td> <center> <font color=silver size=4> %s </td>
>              </tr>
>          </form>
>          ''' % (filename, hits, host, lastvisit) )
>      print( '''</table><br><br>''' )
> except pymysql.ProgrammingError as e:
>      print( repr(e) )
>
> sys.exit(0)
>
> After a spcific file gets selected then files.py is reloading grabbign
> the filename as a variable form and:
>
> #
> =================================================================================================================
>
> # If user downloaded a file, thank the user !!!
> #
> =================================================================================================================
>
> if filename:
>      #update filename's counter if cookie does not exist
>      cur.execute('''UPDATE files SET hits = hits + 1, host = %s,
> lastvisit = %s WHERE url = %s''', (host, lastvisit, filename) )
>
> but the execute never happesn.
> i ahve tested it
>
> if data:
>      print soemthing
>
> but data is always empty.

So any ideas why the update statements never gets executed?

-- 
What is now proved was at first only imagined!

[toc] | [prev] | [next] | [standalone]

#48381

On Sun, Jun 16, 2013 at 6:29 AM, Benjamin Schollnick
<benjamin@schollnick.net> wrote:
> cur.execute('''SELECT ID FROM counters WHERE url = %s''', page )
> cur.execute('''INSERT INTO counters (url) VALUES (%s)''', page )
>
> Sure, whoever wrote that code is a fool.
>
> http://xkcd.com/327/
>
> They didn't sanitize your database inputs.

I assume you're talking about the above two lines of code? They're not
SQL injection targets. The clue is that the %s isn't in quotes. This
is an out-of-band argument passing method (actually, since he's using
MySQL (IIRC), it's probably just going to escape it and pass it on
through, but it comes to the same thing), so it's safe.

ChrisA

[toc] | [prev] | [next] | [standalone]

#48393

On 16/6/2013 1:51 πμ, Chris Angelico wrote:
> On Sun, Jun 16, 2013 at 6:29 AM, Benjamin Schollnick
> <benjamin@schollnick.net> wrote:
>> cur.execute('''SELECT ID FROM counters WHERE url = %s''', page )
>> cur.execute('''INSERT INTO counters (url) VALUES (%s)''', page )
>>
>> Sure, whoever wrote that code is a fool.
>>
>> http://xkcd.com/327/
>>
>> They didn't sanitize your database inputs.
>
> I assume you're talking about the above two lines of code? They're not
> SQL injection targets.

Then how those page entries found in the database Chris?

> The clue is that the %s isn't in quotes.

What happens if i write it like this?

cur.execute('''SELECT ID FROM counters WHERE url = "%s"''', page )

How quoting of %s helps here?

> This is an out-of-band argument passing method (actually, since he's using
> MySQL (IIRC), it's probably just going to escape it and pass it on
> through, but it comes to the same thing), so it's safe.

Yes iam using a comma and not a substitute operator, so input is mysql 
validates.

Please explain what is an "out-of-band argument passing method"

What your idea of those entries made it to the counters database table?


-- 
What is now proved was at first only imagined!

[toc] | [prev] | [next] | [standalone]

#48410 — Python Greek mailing list [was Re: Why 'files.py' does not print the filenames into a table format?]

Nikos,

Have you considered subscribing to this?

http://mail.python.org/mailman/listinfo/python-greece


Possibly some of these concepts will be easier for you to understand if 
explained to you in your native language. Or you might be able to join a 
local Users Group who can help you.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#48420 — Re: Python Greek mailing list [was Re: Why 'files.py' does not print the filenames into a table format?]

On 16/6/2013 8:06 πμ, Steven D'Aprano wrote:
> Nikos,
>
> Have you considered subscribing to this?
>
> http://mail.python.org/mailman/listinfo/python-greece
>
>
> Possibly some of these concepts will be easier for you to understand if
> explained to you in your native language. Or you might be able to join a
> local Users Group who can help you.

Thank you Steven i don't want to enter there as mail but wish to find it 
as a newsgroups, which i tried to subscribe but TB couldn't find it.

Also i have no trouble understand you guys in English or express myself 
here. I like English.

And i'm under the impression that foreigners are more helpful from 
Greeks. At least that's what experience have tought me in a local linux 
group for many years.

I prefer staying here but i can also subscribe there as well if you teel 
me what the groups name.

-- 
What is now proved was at first only imagined!

[toc] | [prev] | [next] | [standalone]

#48428 — Re: Python Greek mailing list [was Re: Why 'files.py' does not print the filenames into a table format?]

On Sun, 16 Jun 2013 11:28:00 +0300, Nick the Gr33k wrote:

> On 16/6/2013 8:06 πμ, Steven D'Aprano wrote:
>> Nikos,
>>
>> Have you considered subscribing to this?
>>
>> http://mail.python.org/mailman/listinfo/python-greece

[...]
> I prefer staying here but i can also subscribe there as well if you teel
> me what the groups name.

Nikos, this is exactly the sort of thing that makes it painful to try to 
help you. I've given you the URL. The name of the list is in the URL, and 
even if it isn't, you can just click on it and see for yourself.

Let me repeat the URL in case you cannot see it above:

http://mail.python.org/mailman/listinfo/python-greece

I will not answer any more questions about the python-greece list, 
because I do not know any more about it than what you can see by 
following that list.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#48433 — Re: Python Greek mailing list [was Re: Why 'files.py' does not print the filenames into a table format?]

On 16/6/2013 1:33 μμ, Steven D'Aprano wrote:
> On Sun, 16 Jun 2013 11:28:00 +0300, Nick the Gr33k wrote:
>
>> On 16/6/2013 8:06 πμ, Steven D'Aprano wrote:
>>> Nikos,
>>>
>>> Have you considered subscribing to this?
>>>
>>> http://mail.python.org/mailman/listinfo/python-greece
>
> [...]
>> I prefer staying here but i can also subscribe there as well if you teel
>> me what the groups name.
>
> Nikos, this is exactly the sort of thing that makes it painful to try to
> help you. I've given you the URL. The name of the list is in the URL, and
> even if it isn't, you can just click on it and see for yourself.
>
> Let me repeat the URL in case you cannot see it above:
>
> http://mail.python.org/mailman/listinfo/python-greece
>
> I will not answer any more questions about the python-greece list,
> because I do not know any more about it than what you can see by
> following that list.
>
>
>
i did Steven that why i asked in the 1st place

To post a message to all the list members, send email to 
python-greece@python.org.

this is not a valid nrewgroup name/

-- 
What is now proved was at first only imagined!

[toc] | [prev] | [next] | [standalone]

#48436 — Re: Python Greek mailing list [was Re: Why 'files.py' does not print the filenames into a table format?]

On 16/06/2013 11:57, Ferrous Cranus wrote:
> i did Steven that why i asked in the 1st place
>
> To post a message to all the list members, send email to
> python-greece@python.org.
>
> this is not a valid nrewgroup name/
>

Not valid in the same way that support@superhost.gr is not valid?

-- 
"Steve is going for the pink ball - and for those of you who are 
watching in black and white, the pink is next to the green." Snooker 
commentator 'Whispering' Ted Lowe.

Mark Lawrence

[toc] | [prev] | [next] | [standalone]

#48437 — Re: Python Greek mailing list [was Re: Why 'files.py' does not print the filenames into a table format?]

On 16/6/2013 2:09 μμ, Mark Lawrence wrote:
> On 16/06/2013 11:57, Ferrous Cranus wrote:
>> i did Steven that why i asked in the 1st place
>>
>> To post a message to all the list members, send email to
>> python-greece@python.org.
>>
>> this is not a valid nrewgroup name/
>>
>
> Not valid in the same way that support@superhost.gr is not valid?
>
There is not newsgroup called 'python-greece'. i searched.

-- 
What is now proved was at first only imagined!

[toc] | [prev] | [next] | [standalone]

#48439 — Re: Python Greek mailing list [was Re: Why 'files.py' does not print the filenames into a table format?]

On Sun, Jun 16, 2013 at 1:09 PM, Mark Lawrence <breamoreboy@yahoo.co.uk> wrote:
> On 16/06/2013 11:57, Ferrous Cranus wrote:
>>
>> i did Steven that why i asked in the 1st place
>>
>> To post a message to all the list members, send email to
>> python-greece@python.org.
>>
>> this is not a valid nrewgroup name/
>>
>
> Not valid in the same way that support@superhost.gr is not valid?

http://en.wikipedia.org/wiki/Usenet#Organization

--
Kwpolska <http://kwpolska.tk> | GPG KEY: 5EAAEA16
stop html mail                | always bottom-post
http://asciiribbon.org        | http://caliburn.nl/topposting.html

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.lang.python

csiph-web