Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]
Groups > comp.lang.python > #78043
| Path | csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!eu.feeder.erje.net!newsfeed.xs4all.nl!newsfeed4.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail |
|---|---|
| Return-Path | <python-python-list@m.gmane.org> |
| X-Original-To | python-list@python.org |
| Delivered-To | python-list@mail.python.org |
| X-Spam-Status | OK 0.001 |
| X-Spam-Evidence | '*H*': 1.00; '*S*': 0.00; 'broken': 0.04; 'exercise': 0.04; 'method:': 0.09; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; 'subject:files': 0.09; 'received:80.91.229.3': 0.16; 'received:dip0.t-ipconnect.de': 0.16; 'received:plane.gmane.org': 0.16; 'received:t-ipconnect.de': 0.16; 'separator.': 0.16; 'splitting': 0.16; 'subject:program': 0.16; 'prevent': 0.16; 'size,': 0.16; 'wrote:': 0.18; 'memory': 0.22; 'separate': 0.22; 'print': 0.22; 'header:User-Agent:1': 0.23; 'skip:" 20': 0.27; 'header:X-Complaints-To:1': 0.27; 'record': 0.27; 'sep': 0.31; 'probably': 0.32; 'subject:data': 0.36; 'should': 0.36; 'being': 0.38; 'to:addr:python-list': 0.38; 'to:addr:python.org': 0.39; 'skip:p 20': 0.39; 'received:org': 0.40; 'providing': 0.61; 'field': 0.63; 'records.': 0.68; 'safe': 0.72; 'therefore': 0.72; 'records': 0.73 |
| X-Injected-Via-Gmane | http://gmane.org/ |
| To | python-list@python.org |
| From | Peter Otten <__peter__@web.de> |
| Subject | Re: program to generate data helpful in finding duplicate large files |
| Date | Thu, 18 Sep 2014 22:23:18 +0200 |
| Organization | None |
| References | <CALDD_==AYbQNPu29jRoLFp8WPZaZ9mMs79334-m_z3dgdxZRJw@mail.gmail.com> |
| Mime-Version | 1.0 |
| Content-Type | text/plain; charset="ISO-8859-1" |
| Content-Transfer-Encoding | 7Bit |
| X-Gmane-NNTP-Posting-Host | p57bdb9d2.dip0.t-ipconnect.de |
| User-Agent | KNode/4.13.3 |
| X-BeenThere | python-list@python.org |
| X-Mailman-Version | 2.1.15 |
| Precedence | list |
| List-Id | General discussion list for the Python programming language <python-list.python.org> |
| List-Unsubscribe | <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe> |
| List-Archive | <http://mail.python.org/pipermail/python-list/> |
| List-Post | <mailto:python-list@python.org> |
| List-Help | <mailto:python-list-request@python.org?subject=help> |
| List-Subscribe | <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe> |
| Newsgroups | comp.lang.python |
| Message-ID | <mailman.14123.1411071819.18130.python-list@python.org> (permalink) |
| Lines | 19 |
| NNTP-Posting-Host | 2001:888:2000:d::a6 |
| X-Trace | 1411071819 news.xs4all.nl 2899 [2001:888:2000:d::a6]:46776 |
| X-Complaints-To | abuse@xs4all.nl |
| Xref | csiph.com comp.lang.python:78043 |
Show key headers only | View raw
David Alban wrote:
> * sep = ascii_nul*
>
> * print "%s%c%s%c%d%c%d%c%d%c%d%c%s" % ( thishost, sep, md5sum, sep,
> dev, sep, ino, sep, nlink, sep, size, sep, file_path )*
file_path may contain newlines, therefore you should probably use "\0" to
separate the records. The other fields may not contain whitespace, so it's
safe to use " " as the field separator. When you deserialize the record you
can prevent the file_path from being broken by providing maxsplit to the
str.split() method:
for record in infile.read().split("\0"):
print(record.split(" ", 6))
Splitting into records without reading the whole data into memory left as an
exercise ;)
Back to comp.lang.python | Previous | Next | Find similar | Unroll thread
Re: program to generate data helpful in finding duplicate large files Peter Otten <__peter__@web.de> - 2014-09-18 22:23 +0200
csiph-web