Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]


Groups > comp.lang.python > #78183 > unrolled thread

Re: program to generate data helpful in finding duplicate large files

Started byChris Kaynor <ckaynor@zindagigames.com>
First post2014-09-22 12:47 -0700
Last post2014-09-22 12:47 -0700
Articles 1 — 1 participant

Back to article view | Back to comp.lang.python

This discussion starts older than the indexed window; earlier articles aren't shown. The article labeled Started by below is the oldest one visible, not the original post.


Contents

  Re: program to generate data helpful in finding duplicate large files Chris Kaynor <ckaynor@zindagigames.com> - 2014-09-22 12:47 -0700

#78183 — Re: program to generate data helpful in finding duplicate large files

FromChris Kaynor <ckaynor@zindagigames.com>
Date2014-09-22 12:47 -0700
SubjectRe: program to generate data helpful in finding duplicate large files
Message-ID<mailman.14235.1411415617.18130.python-list@python.org>

[Multipart message — attachments visible in raw view] — view raw

I went and looked up the PEPs regarding universal new-lines, and it seems
it would be platform-independent - all of "\r\n", "\r", and "\n" will
always be converted to "\n" in Python, unless explicitly modified on the
file object (or Universal newlines are disabled).

It still stands that for platform independence, "rb" should be used due to
some platforms treating binary files differently.

Additionally, and most importantly, for what the OP was doing (determining
if files are the same), you almost certainly want to read the raw bytes -
especially if some of the files are truly binary (such as *.exe on Windows).

Chris

On Mon, Sep 22, 2014 at 12:34 PM, <random832@fastmail.us> wrote:

> On Thu, Sep 18, 2014, at 14:45, Chris Kaynor wrote:
> > Additionally, you may want to specify binary mode by using
> > open(file_path,
> > 'rb') to ensure platform-independence ('r' uses Universal newlines, which
> > means on Windows, Python will convert "\r\n" to "\n" while reading the
> > file). Additionally, some platforms will treat binary files differently.
>
> Does 'r' not use universal newlines on unix? If not, why not? The only
> purpose seems to be to allow people to write bad programs and have them
> work on unix. It makes even less sense in python 3 where opening a file
> in text mode results in a TextIOWrapper with utf-8 encoding, and
> therefore can't be used on arbitrary binary files.
> --
> https://mail.python.org/mailman/listinfo/python-list
>

[toc] | [standalone]


Back to top | Article view | comp.lang.python


csiph-web