Path: csiph.com!usenet.pasdenom.info!gegeweb.org!de-l.enfer-du-nord.net!feeder2.enfer-du-nord.net!cs.uu.nl!news.stack.nl!newsfeed.xs4all.nl!newsfeed4.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.000 X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'python,': 0.02; 'else:': 0.03; 'charset:iso-8859-7': 0.04; 'encoding': 0.05; 'mrab': 0.05; 'encoded': 0.07; 'paths': 0.07; 'rename': 0.07; 'skip:\\ 20': 0.07; 'utf-8': 0.07; 'bash': 0.09; 'encode': 0.09; 'filename': 0.09; 'filenames': 0.09; 'happen,': 0.09; 'root,': 0.09; 'skip:\\ 10': 0.09; 'try:': 0.09; 'compute': 0.16; 'files:': 0.16; 'from:addr:mrabarnett.plus.com': 0.16; 'from:addr:python': 0.16; 'from:name:mrab': 0.16; 'letters.': 0.16; 'message- id:@mrabarnett.plus.com': 0.16; 'name)': 0.16; 'new_path': 0.16; 'readable': 0.16; 'received:84.93': 0.16; 'received:84.93.230': 0.16; 'renaming': 0.16; 'set()': 0.16; 'somehow,': 0.16; 'sorts': 0.16; 'files.': 0.16; 'weird': 0.16; 'do,': 0.16; 'fix': 0.17; 'wrote:': 0.18; 'trying': 0.19; 'file,': 0.19; 'passing': 0.19; 'typing': 0.19; 'command': 0.22; 'appears': 0.22; 'import': 0.22; 'print': 0.22; 'header:User-Agent:1': 0.23; 'bytes': 0.24; 'skip:{ 20': 0.24; 'script': 0.25; 'extension': 0.26; 'first,': 0.26; 'skip:" 30': 0.26; 'suggested': 0.26; 'this:': 0.26; 'header:In- Reply-To:1': 0.27; 'idea': 0.28; 'appear': 0.29; 'testing': 0.29; 'correct': 0.29; "doesn't": 0.30; "skip:' 10": 0.31; 'away.': 0.31; "d'aprano": 0.31; 'steven': 0.31; 'file': 0.32; 'quite': 0.32; 'linux': 0.33; 'skip:b 30': 0.33; 'actual': 0.34; 'maybe': 0.34; 'subject:from': 0.34; 'could': 0.34; 'problem': 0.35; 'subject: (': 0.35; 'except': 0.35; 'tool': 0.35; 'something': 0.35; 'received:84': 0.35; 'test': 0.35; 'but': 0.35; "didn't": 0.36; 'turn': 0.37; 'skip:o 20': 0.38; 'to:addr:python-list': 0.38; 'files': 0.38; 'explain': 0.39; '(from': 0.39; 'sure': 0.39; 'to:addr:python.org': 0.39; 'how': 0.40; 'skip:u 10': 0.60; 'letters': 0.60; 'then,': 0.60; 'tell': 0.60; 'full': 0.61; 'new': 0.61; 'back': 0.62; 'name': 0.63; 'more': 0.64; '8bit%:95': 0.64; 'here': 0.66; 'header:Reply-To:1': 0.67; 'invalid': 0.68; 'reply- to:no real name:2**0': 0.71; '8bit%:92': 0.71; '8bit%:100': 0.72; 'walk': 0.74; 'as:': 0.81; 'skip:n 40': 0.81; 'greek': 0.84; 'reply-to:addr:python.org': 0.84; 'skip:\xef 10': 0.84; 'thoroughly': 0.91; '2013': 0.98 X-CM-Score: 0.00 X-CNFS-Analysis: v=2.1 cv=RZapVTdv c=1 sm=1 tr=0 a=0nF1XD0wxitMEM03M9B4ZQ==:117 a=0nF1XD0wxitMEM03M9B4ZQ==:17 a=0Bzu9jTXAAAA:8 a=oyR3mlnJdzkA:10 a=AdgiQdVXbpoA:10 a=ihvODaAuJD4A:10 a=OUOv7kDek9cA:10 a=E_FQspuQcigA:10 a=EBOSESyhAAAA:8 a=8AHkEIZyAAAA:8 a=uw23S90zXSUA:10 a=2QJReqtj--yO_bjdVQQA:9 a=o3X3QV6JOaEA:10 X-AUTH: mrabarnett:2500 Date: Thu, 06 Jun 2013 12:35:18 +0100 From: MRAB User-Agent: Mozilla/5.0 (Windows NT 5.1; rv:17.0) Gecko/20130509 Thunderbird/17.0.6 MIME-Version: 1.0 To: python-list@python.org Subject: Re: Changing filenames from Greeklish => Greek (subprocess complain) References: <2c425f2b-99de-4453-964e-c585f2043f71@googlegroups.com> <306a22ea-fbf7-4097-af31-121a999957d6@googlegroups.com> <9c482ba0-23ac-4e66-a0e1-a18be9fd82d8@googlegroup> <06a19483-65df-4fcd-9430-b45a01c9dbab@googlegroups.com> <51aed313$0$11118$c3e8da3@news.astraweb.com> <4c19b71d-4de5-41ad-b6ae-fb133a6c331e@googlegroups.com> <2be143c4-77c6-4c84-ba1c-46b02bd503ff@googlegroups.com> <1465c96b-c33e-4d5b-894e-b184c031a185@googlegroups.com> In-Reply-To: <1465c96b-c33e-4d5b-894e-b184c031a185@googlegroups.com> Content-Type: text/plain; charset=ISO-8859-7; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list Reply-To: python-list@python.org List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 176 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1370518526 news.xs4all.nl 15998 [2001:888:2000:d::a6]:35533 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:47218 On 06/06/2013 04:43, Íéêüëáïò Êïýñáò wrote: > Ôç ÔåôÜñôç, 5 Éïõíßïõ 2013 9:43:18 ì.ì. UTC+3, ï ÷ñÞóôçò Íéêüëáïò Êïýñáò Ýãñáøå: > > Ôç ÔåôÜñôç, 5 Éïõíßïõ 2013 9:32:15 ì.ì. UTC+3, ï ÷ñÞóôçò MRAB Ýãñáøå: > > > > > On 05/06/2013 18:43, �������� ������ wrote: > > > > > > > > > > > �� �������, 5 ������� 2013 8:56:36 �.�. UTC+3, � ������� Steven D'Aprano ������: > > > > > > > > > > > > > > > > > > > > > > Somehow, I don't know how because I didn't see it happen, you have one or > > > > > > > > > > > more files in that directory where the file name as bytes is invalid when > > > > > > > > > > > decoded as UTF-8, but your system is set to use UTF-8. So to fix this you > > > > > > > > > > > need to rename the file using some tool that doesn't care quite so much > > > > > > > > > > > about encodings. Use the bash command line to rename each file in turn > > > > > > > > > > > until the problem goes away. > > > > > > > > > > > > > > > > > > > > ' leade to that unknown encoding of this bytestream '\305\365\367\336\ \364\357\365\ \311\347\363\357\375.mp3' > > > > > > > > > > > > > > > > > > > > > > But please tell me Steven what linux tool you think it can encode the weird filename to proper '���� ��� �����.mp3' utf-8? > > > > > > > > > > > > > > > > > > > > > > or we cna write a script as i suggested to decode back the bytestream using all sorts of available decode charsets boiling down to the original greek letters. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Actually you were correct i was typing greek and is aw the fileneme here in gogole groups as: > > > > > > > > > > But renaming ia hsell access like 'mv 'Euxi tou Ihsou.mp3' '���� ��� �����.mp3 > > > > > > > > so maybe the filenames have to be decoded to greek-iso but then agian the contain both greek letters but their extension are in english chars like '.mp3' > > > > > > > > > > > > > Using Python, I think you could get the filenames using os.listdir, > > > > > passing the directory name as a bytestring so that it'll return the > > > > > names as bytestrings. > > > > > > > > > > > > > Then, for each name, you could decode from its current encoding and > > > > > encode to UTF-8 and rename the file, passing the old and new paths to > > > > > os.rename as bytestrings. > > > > > > > > Iam not sure i follow: > > > > > > > > Change this: > > > > > > > > # Compute a set of current fullpaths > > > > fullpaths = set() > > > > path = "/home/nikos/public_html/data/apps/" > > > > > > > > for root, dirs, files in os.walk(path): > > > > for fullpath in files: > > > > fullpaths.add( os.path.join(root, fullpath) ) > > > > > > > > > > > > to what to make the full url readable by files.py? > > MRAB can you please explain in more clarity your idea of solution? I was suggesting a way to rename the files so that their names are encoded in UTF-8 (they appear to be encoded in ISO-8859-7). You MUST TEST IT thoroughly first, of course, before trying it on the actual files. It could go something like this: import os # Give the path as a bytestring so that we'll get the names as bytestrings. root_folder = b"/home/nikos/public_html/data/apps/" # Setting TESTING to True will make it print out what renamings it will do, but # not actually do them. TESTING = True # Walk through the files. for root, dirs, files in os.walk(root_folder): for name in files: try: # Is this name encoded in UTF-8? name.decode("utf-8") except UnicodeDecodeError: # Decoding from UTF- failed, which means that the name is not valid # UTF-8. # It appears (from elsewhere) that the names are encoded in # ISO-8859-7, so decode from that and re-encode to UTF-8. new_name = name.decode("iso-8859-7").encode("utf-8") old_path = os.path.join(root, name) new_path = os.path.join(root, new_name) if TESTING: print("Will rename {!r} to {!r}".format(old_path, new_path)) else: print("Renaming {!r} to {!r}".format(old_path, new_path)) os.rename(old_path, new_path)