Path: csiph.com!usenet.pasdenom.info!news.albasani.net!newsfeed.freenet.ag!news2.euro.net!newsgate.cistron.nl!newsgate.news.xs4all.nl!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.000 X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'url:pypi': 0.03; 'win32': 0.03; 'binary': 0.05; 'test,': 0.05; 'assign': 0.07; 'automate': 0.07; 'conversions': 0.07; 'locale': 0.07; 'suppose': 0.07; 'utf-8': 0.07; '16-bit': 0.09; 'meaningful': 0.09; 'non-ascii': 0.09; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; 'windows,': 0.09; 'extension': 0.13; 'files.': 0.13; 'library': 0.15; 'encoding': 0.15; 'automation,': 0.16; 'csv': 0.16; 'encodings': 0.16; 'formats.': 0.16; 'received:80.91.229.3': 0.16; 'received:plane.gmane.org': 0.16; 'renamed': 0.16; 'stuff,': 0.16; 'subject:Different': 0.16; 'subject:Opening': 0.16; 'url:adobe': 0.16; 'word)': 0.16; 'wed,': 0.16; 'basically': 0.17; 'issue,': 0.17; '(or': 0.18; 'windows': 0.19; 'load': 0.19; '3.x': 0.22; 'ctypes': 0.22; 'doc': 0.22; 'parse': 0.22; 'txt': 0.22; 'proprietary': 0.23; 'skip:[ 10': 0.26; 'older': 0.27; 'plain': 0.27; 'tree': 0.27; 'header:X-Complaints-To:1': 0.28; 'rest': 0.28; 'embed': 0.29; 'image,': 0.29; 'implies': 0.29; 'url:developer': 0.29; 'usually': 0.30; 'compatible': 0.30; 'code': 0.31; '(and': 0.32; 'url:python': 0.32; 'file': 0.32; 'generally': 0.32; 'running': 0.32; 'could': 0.32; 'url:home': 0.33; 'to:addr:python-list': 0.33; 'excel': 0.33; 'likely': 0.33; 'that,': 0.34; 'version': 0.34; 'text': 0.34; 'whatever': 0.35; 'formats': 0.35; 'skip:. 20': 0.35; 'text.': 0.35; 'open': 0.35; 'something': 0.35; 'there': 0.35; 'received:org': 0.36; 'really': 0.36; 'ability': 0.36; 'but': 0.36; 'url:org': 0.36; 'should': 0.36; 'charset:us-ascii': 0.36; 'pdf': 0.37; 'xml': 0.37; 'option': 0.37; 'being': 0.37; '(for': 0.37; 'skip:z 10': 0.37; 'rather': 0.37; 'subject:: ': 0.38; 'files': 0.38; 'supports': 0.38; 'url:en': 0.38; 'page': 0.38; 'to:addr:python.org': 0.39; 'apply': 0.39; 'url:schemas': 0.39; 'application': 0.40; 'header:Received:5': 0.40; 'think': 0.40; 'your': 0.60; 'save': 0.61; 'first': 0.61; 'office': 0.61; 'url:public': 0.62; 'different': 0.63; 'email addr:gmail.com': 0.63; 'contents.': 0.65; 'jul': 0.65; 'believe': 0.69; '(once': 0.84; '(yes,': 0.84; 'acrobat': 0.91; 'dennis': 0.91; 'graphical': 0.91; 'received:108': 0.91 X-Injected-Via-Gmane: http://gmane.org/ To: python-list@python.org From: Dennis Lee Bieber Subject: Re: Opening multiple Files in Different Encoding Date: Wed, 11 Jul 2012 18:24:01 -0400 Organization: > Bestiaria Support Staff < References: <40633830-78ae-4cc6-8795-de5a352e0fb1@m2g2000pbv.googlegroups.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Gmane-NNTP-Posting-Host: adsl-108-73-117-76.dsl.klmzmi.sbcglobal.net X-Newsreader: Forte Agent 3.3/32.846 X-No-Archive: YES X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 94 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1342045454 news.xs4all.nl 6964 [2001:888:2000:d::a6]:41795 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:25181 On Wed, 11 Jul 2012 11:15:02 -0700 (PDT), subhabangalore@gmail.com declaimed the following in gmane.comp.python.general: > No generally I know the glob.glob or the encodings as I work lot on non-ASCII stuff, but I recently found an interesting issue, suppose there are .doc,.docx,.txt,.xls,.pdf files with different encodings. > 1) First I have to determine on the fly the file type. > 2) I can not assign encoding="..." whatever be the encoding I have to read it. > Many of those are (semi) proprietary formats (M$ Office ). DOCX (and XLSX) are, as I recall ZIP-compressed XML formats -- and I think that also implies UTF-8 (once you manage to decompress them)... Note that, for a test, I renamed a .docx to .zip and opened it in PowerArchiver... It generates 19 files in a multi-level tree -- one of which is named [content_types].xml and contains That should also apply to the rest of the new Office document formats. Plain DOC format could be a mishmash of three or four binary formats (Word6 being the last compatible with 16-bit Windows 3.x Word). I believe one Office version assigned DOC to what were really RTF format files rather than the binary (yes, binary -- there is no guarantee that you can find meaningful text without being able to parse a binary file format). PDF contents can by binary compressed; again there is no guarantee you can find meaningful text without being able to parse the contents. http://partners.adobe.com/public/developer/en/pdf/PDFReference.pdf (an older version than current standard, I suspect)... Heck, many of the cheaper PDF conversions basically embed each page as a graphical (bitmap) image, not as text. For the Office documents, if you are running on a Windows system (or can open them in something like OpenOffice), your best chances are likely to be programmatically open them in the application and then do a "save as..." TXT (for Word) and CSV (for Excel) -- then process the TXT/CSV files (or save as RTF if that is an option -- that's usually in whatever the locale specific Windows code page contains, if not plain ASCII). I believe there is a library to read Excel files directly: http://pypi.python.org/pypi/xlrd/ For PDF; I don't know if Acrobat Reader supports automation, to programmatically load and "save as text". http://p2p.wrox.com/vb-net-2002-2003-basics/39037-acrobat-reader-automation.html implies an ability to automate on Windows, so using the win32 extension library or ctypes may give you access to work with the files. -- Wulfraed Dennis Lee Bieber AF6VN wlfraed@ix.netcom.com HTTP://wlfraed.home.netcom.com/