Path: csiph.com!usenet.pasdenom.info!news.redatomik.org!newsfeed.xs4all.nl!newsfeed2a.news.xs4all.nl!xs4all!post.news.xs4all.nl!not-for-mail Return-Path: X-Original-To: python-list@python.org Delivered-To: python-list@mail.python.org X-Spam-Status: OK 0.016 X-Spam-Evidence: '*H*': 0.97; '*S*': 0.00; 'example:': 0.03; 'indicating': 0.07; 'defines': 0.09; 'linux/unix': 0.09; 'received:80.91': 0.09; 'received:80.91.229': 0.09; 'received:gmane.org': 0.09; 'received:list': 0.09; 'system?': 0.09; 'subject:How': 0.10; 'windows': 0.15; "function's": 0.16; 'lengths': 0.16; 'message-id:@4ax.com': 0.16; 'mode,': 0.16; 'monitors': 0.16; 'read?': 0.16; 'received:80.91.229.3': 0.16; 'received:plane.gmane.org': 0.16; 'subject:Problem': 0.16; 'torn': 0.16; 'folder': 0.16; 'module': 0.19; 'addition,': 0.20; 'later': 0.20; 'written': 0.21; 'refers': 0.24; 'url:home': 0.24; 'file.': 0.24; 'logging': 0.26; 'task': 0.26; 'pass': 0.26; '(for': 0.26; 'header:X-Complaints-To:1': 0.27; 'record': 0.27; 'ideal': 0.29; 'reporting': 0.29; 'moved': 0.30; 'lines': 0.31; 'dropped': 0.31; 'file': 0.32; 'regular': 0.32; 'text': 0.33; 'maybe': 0.34; 'could': 0.34; 'building': 0.35; 'module.': 0.36; 'processed': 0.36; 'scheduled': 0.36; 'next': 0.36; 'charset:us-ascii': 0.36; 'subject:?': 0.36; 'should': 0.36; 'throughout': 0.37; 'being': 0.38; 'to:addr:python-list': 0.38; 'files': 0.38; 'to:addr:python.org': 0.39; 'changed': 0.39; 'received:org': 0.40; 'how': 0.40; 'read': 0.60; 'length': 0.61; 'new': 0.61; "you're": 0.61; 'first': 0.61; 'such': 0.63; 'provide': 0.64; 'total': 0.65; 'taking': 0.65; 'between': 0.67; 'mar': 0.68; 'records': 0.73; 'day': 0.76; '100': 0.79; 'subject:this': 0.83; '2015': 0.84; 'capability': 0.84; 'short,': 0.84; 'stamp': 0.91; 'subject:Best': 0.91; 'dirty': 0.93; 'received:108': 0.93 X-Injected-Via-Gmane: http://gmane.org/ To: python-list@python.org From: Dennis Lee Bieber Subject: Re: Strategy/ Advice for How to Best Attack this Problem? Date: Sun, 29 Mar 2015 19:00:24 -0400 Organization: IISS Elusive Unicorn References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Gmane-NNTP-Posting-Host: adsl-108-73-116-210.dsl.klmzmi.sbcglobal.net X-Newsreader: Forte Agent 6.00/32.1186 X-No-Archive: YES X-BeenThere: python-list@python.org X-Mailman-Version: 2.1.19 Precedence: list List-Id: General discussion list for the Python programming language List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Newsgroups: comp.lang.python Message-ID: Lines: 41 NNTP-Posting-Host: 2001:888:2000:d::a6 X-Trace: 1427670045 news.xs4all.nl 2893 [2001:888:2000:d::a6]:55399 X-Complaints-To: abuse@xs4all.nl Xref: csiph.com comp.lang.python:88295 On Sun, 29 Mar 2015 04:32:48 -0700 (PDT), Saran Ahluwalia declaimed the following: >Below are the function's requirements. I am torn between using the OS module or some other quick and dirty module. In addition, my ideal assumption that this could be cross-platform. "Records" refers to contents in a file. What are some suggestions from the Pythonistas? > >* Monitors a folder for files that are dropped throughout the day > Other than spinning over a time.sleep() call, this is likely going to be OS specific: OS may provide a call-back/notification of when a directory is changed OS may provide a scheduled task capability that would let your task start at regular intervals -- you'd have to maintain some file identifying the last processed time stamp (maybe). Linux/UNIX cron jobs, Windows task scheduler >* When a file is dropped in the folder the program should scan the file > What happens if the file is still being written? Do you record the new file in one pass, then delay and process those files the next pass (recording new files for later processing)? >o IF all the records in the file have the same length > >o THEN the file should be moved to a "success" folder and a text file written indicating the total number of records processed > Text file? or maybe using the logging system? >o IF the file is empty OR the records are not all of the same length > >o THEN the file should be moved to a "failure" folder and a text file written indicating the cause for failure (for example: Empty file or line 100 was not the same length as the rest). How do you intend such a report to be determined? That is, what defines the "correct length" of the lines? The first record read? If that one is short, you're now reporting ALL OTHER lines as erroneous. Read all lines building a histogram of lengths and taking the mode, then reprocessing to flag lines that do not meet the mode? -- Wulfraed Dennis Lee Bieber AF6VN wlfraed@ix.netcom.com HTTP://wlfraed.home.netcom.com/