Path: csiph.com!usenet.pasdenom.info!weretis.net!feeder4.news.weretis.net!newsreader4.netcologne.de!news.netcologne.de!xlned.com!feeder1.xlned.com!newsfeed.xs4all.nl!newsfeed2.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.007 X-Spam-Evidence: '*H*': 0.99; '*S*': 0.00; 'cpython': 0.05; 'encoding': 0.05; 'allowed.': 0.07; 'received:209.85.160.174': 0.07; 'subject:file': 0.07; 'utf-8': 0.07; 'bytes.': 0.09; 'filenames': 0.09; 'locale': 0.09; 'mostly': 0.14; 'windows': 0.15; 'encodings': 0.16; 'happily': 0.16; 'those,': 0.16; 'unicode,': 0.16; 'unix,': 0.16; 'wrote:': 0.18; 'settings': 0.22; 'header:User-Agent:1': 0.23; 'artist': 0.24; 'unicode': 0.24; 'decide': 0.24; 'fine': 0.24; "haven't": 0.24; 'certain': 0.27; 'header:In-Reply-To:1': 0.27; 'character': 0.29; 'generally': 0.29; 'characters': 0.30; 'ignored.': 0.30; '(which': 0.31; 'code': 0.31; 'adequate': 0.31; "user's": 0.31; 'file': 0.32; 'problem': 0.35; 'received:209.85': 0.35; 'but': 0.35; 'received:google.com': 0.35; 'changing': 0.37; 'reports': 0.37; 'received:209': 0.37; 'easily': 0.37; 'message-id:@gmail.com': 0.38; 'handle': 0.38; 'whatever': 0.38; 'to:addr:python-list': 0.38; 'issue': 0.38; 'moving': 0.39; 'to:addr:python.org': 0.39; 'system.': 0.39; 'how': 0.40; 'dave': 0.60; "you're": 0.61; 'first': 0.61; 'decided': 0.64; 'different': 0.65; 'between': 0.67; '3.3.1': 0.84; '9.1': 0.84; 'characters,': 0.84; 'freebsd': 0.84; 'subject:Making': 0.84; 'angel': 0.91 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=x-received:message-id:date:from:user-agent:mime-version:to:subject :references:in-reply-to:x-enigmail-version:content-type :content-transfer-encoding; bh=FdN6O4PS9wyLoz6O9X3+SLSPU2SXzmlxYtlraJrxWl0=; b=OnWsswe+aI5RwHlGeLA9I8/EtakNoR6cvPzahHAUpx55NoyRyvQiWEZEyExSL/BpmN /p94ZHxrHYtP1oxuCdcq60LSji+tqT3MPP59CXrEhPM/6yuwV8yDtvH5wyk5nJB1R7Dq d/A/dtJA7HaB2NbXgbV/vSDIpUycSUOH50Sr3AdDYxfIZxiNwVo34hJBfx4AaSHK1T2c 19Q3Oy4hEHP4CU8JFLbFx1sEFPplqQyGrPnrQIn1wF5TWfswj8Ht0AXyXe1aEK5OkPOY 2HNaMFC+Iv6CuDu16S8lyfvf/RFqFxvHoab43rK57FM1WeIhrw5D4ziH/oaM5GQSNUGr BOGw== X-Received: by 10.236.148.162 with SMTP id v22mr4265417yhj.131.1367974286361; Tue, 07 May 2013 17:51:26 -0700 (PDT) Date: Tue, 07 May 2013 19:51:24 -0500 From: Andrew Berg User-Agent: Mozilla/5.0 (Windows NT 6.2; WOW64; rv:17.0) Gecko/20130107 Thunderbird/17.0.2 MIME-Version: 1.0 To: "comp.lang.python" Subject: Re: Making safe file names References: <51895D03.4000300@gmail.com> <518998FE.6030805@davea.name> In-Reply-To: <518998FE.6030805@davea.name> X-Enigmail-Version: 1.5.1 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 22 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1367974288 news.xs4all.nl 15895 [2001:888:2000:d::a6]:52859 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:44923 On 2013.05.07 19:14, Dave Angel wrote: > You also need to decide how to handle Unicode characters, since they're > different for different OS. In Windows on NTFS, filenames are in > Unicode, while on Unix, filenames are bytes. So on one of those, you > will be encoding/decoding if your code is to be mostly portable. Characters outside whatever sys.getfilesystemencoding() returns won't be allowed. If the user's locale settings don't support Unicode, my program will be far from the only one to have issues with it. Any problem reports that arise from a user moving between legacy encodings will generally be ignored. I haven't yet decided how I will handle artist names with characters outside UTF-8, but inside UTF-16/32 (UTF-16 is just fine on Windows/NTFS, but on Unix(-ish) systems, many use UTF-8 in their locale settings). > Don't forget that ls and rm may not use the same encoding you're using. > So you may not consider it adequate to make the names legal, but you > may also want they easily typeable in the shell. I don't understand. I have no intention of changing Unicode characters. This is not a Unicode issue since (modern) file systems will happily accept it. The issue is that certain characters (which are ASCII) are not allowed on some file systems: \ / : * ? " < > | @ and the NUL character The first 9 are not allowed on NTFS, the @ is not allowed on ext3cow, and NUL and / are not allowed on pretty much any file system. Locale settings and encodings aside, these 11 characters will need to be escaped. -- CPython 3.3.1 | Windows NT 6.2.9200 / FreeBSD 9.1