Path: csiph.com!v102.xanadu-bbs.net!xanadu-bbs.net!feeder.erje.net!1.eu.feeder.erje.net!newsfeed.xs4all.nl!newsfeed1.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail
Return-Path: <torriem+gmail@torriefamily.org>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
X-Spam-Status: OK 0.060
X-Spam-Evidence: '*H*': 0.88; '*S*': 0.00; 'expressions': 0.07; 'file,': 0.15; 'from:addr:torriem': 0.16; 'from:name:michael torrie': 0.16; 'seconds.': 0.16; 'subject:Parser': 0.16; 'textfile': 0.16; 'wrote:': 0.16; 'input': 0.18; 'fairly': 0.22; 'parse': 0.22; 'parser': 0.22; 'module': 0.23; 'header:In-Reply- To:1': 0.24; 'somewhere': 0.24; 'install': 0.25; 'header:User- Agent:1': 0.26; 'figure': 0.27; 'regular': 0.29; 'along.': 0.29; 'flying': 0.29; 'environment': 0.29; 'lines': 0.30; 'certainly': 0.31; 'code': 0.31; 'usually': 0.33; 'quickly': 0.34; 'add': 0.34; 'google': 0.34; 'message-id:@gmail.com': 0.35; 'could': 0.35; 'to:addr:python-list': 0.35; 'along': 0.35; 'files,': 0.35; 'something': 0.35; 'but': 0.36; 'should': 0.37; 'subject:: ': 0.37; 'received:org': 0.38; 'pm,': 0.39; 'to:addr:python.org': 0.39; 'received:192': 0.39; 'data': 0.40; 'your': 0.60; 'between': 0.65; 'charset:windows-1252': 0.65; 'fantastic': 0.72; '20.000': 0.84; 'mock': 0.84; 'subject:needed.': 0.91; 'examine': 0.95
X-Virus-Scanned: amavisd-new at torriefamily.org
Date: Mon, 01 Jun 2015 19:12:12 -0600
From: Michael Torrie <torriem@gmail.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0
MIME-Version: 1.0
To: python-list@python.org
Subject: Re: Parser needed.
References: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
In-Reply-To: <949ff$556cf8e0$5419aafe$20874@news.ziggo.nl>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 7bit
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.20+
Precedence: list
List-Id: General discussion list for the Python programming language <python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>, <mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <http://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>, <mailto:python-list-request@python.org?subject=subscribe>
Newsgroups: comp.lang.python
Message-ID: <mailman.31.1433207544.13271.python-list@python.org>
Lines: 21
NNTP-Posting-Host: 2001:888:2000:d::a6
X-Trace: 1433207544 news.xs4all.nl 2917 [2001:888:2000:d::a6]:57090
X-Complaints-To: abuse@xs4all.nl
Xref: csiph.com comp.lang.python:91752

On 06/01/2015 06:29 PM, Skybuck Flying wrote:
> The parser should be able to parse a textfile of somewhere between 20.000 
> lines to 50.000 lines in about 1 to 2 seconds.
> 
> My environment is SikuliX 1.1

I don't have any inclination to examine your input files, but you could
certainly mock up a parser fairly quickly with the fantastic pyparsing
module (you'll have to install it from your distro's package manager, or
through pip.

Once you have figure out how to parse the data you can fill in your data
structures as you go along.  Pyparsing has pretty good docs; just google
for it.  Usually first I get pyparsing working on my input file, then I
add code to each of the steps along the way to actually do something
with the data that has been parsed.

You could parse it manually using regular expressions if the data is
fairly structured and regular.