Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #82316 > unrolled thread

Python script isn't producing text in data file

Back to article view | Back to comp.lang.python

Contents

  Python script isn't producing text in data file Docfxit <docfxit@gmail.com> - 2014-12-10 15:04 -0800
    Re: Python script isn't producing text in data file Docfxit <docfxit@gmail.com> - 2014-12-10 15:07 -0800
    Re: Python script isn't producing text in data file Chris Angelico <rosuav@gmail.com> - 2014-12-11 10:11 +1100
      Re: Python script isn't producing text in data file sohcahtoa82@gmail.com - 2014-12-10 17:32 -0800
        Re: Python script isn't producing text in data file Chris Angelico <rosuav@gmail.com> - 2014-12-11 12:44 +1100
          Re: Python script isn't producing text in data file Steven D'Aprano <steve@pearwood.info> - 2014-12-11 02:41 +0000
            Re: Python script isn't producing text in data file Chris Angelico <rosuav@gmail.com> - 2014-12-11 13:50 +1100
          Re: Python script isn't producing text in data file Docfxit <docfxit@gmail.com> - 2014-12-10 19:30 -0800
            Re: Python script isn't producing text in data file Chris Angelico <rosuav@gmail.com> - 2014-12-11 14:40 +1100
            Re: Python script isn't producing text in data file Ben Finney <ben+python@benfinney.id.au> - 2014-12-11 14:52 +1100
              Re: Python script isn't producing text in data file Docfxit <docfxit@gmail.com> - 2014-12-10 20:23 -0800
                Re: Python script isn't producing text in data file Chris Angelico <rosuav@gmail.com> - 2014-12-11 15:35 +1100
                Re: Python script isn't producing text in data file Ben Finney <ben+python@benfinney.id.au> - 2014-12-11 15:38 +1100
                  Re: Python script isn't producing text in data file Grant Edwards <invalid@invalid.invalid> - 2014-12-11 05:22 +0000
                Re: Python script isn't producing text in data file Steven D'Aprano <steve@pearwood.info> - 2014-12-11 05:18 +0000
                  Re: Python script isn't producing text in data file Tim Golden <mail@timgolden.me.uk> - 2014-12-11 06:07 +0000
                Re: Python script isn't producing text in data file alister <alister.nospam.ware@ntlworld.com> - 2014-12-11 09:16 +0000
                Re: Python script isn't producing text in data file Peter Otten <__peter__@web.de> - 2014-12-11 10:30 +0100
                  Re: Python script isn't producing text in data file Docfxit <docfxit@gmail.com> - 2014-12-11 13:27 -0800
            Re: Python script isn't producing text in data file Grant Edwards <invalid@invalid.invalid> - 2014-12-11 05:19 +0000
              Re: Python script isn't producing text in data file alister <alister.nospam.ware@ntlworld.com> - 2014-12-11 09:06 +0000
    Re: Python script isn't producing text in data file Steven D'Aprano <steve@pearwood.info> - 2014-12-11 02:47 +0000
      Re: Python script isn't producing text in data file Docfxit <docfxit@gmail.com> - 2014-12-10 19:47 -0800

Page 1 of 2  [1] 2  Next page →

#82316 — Python script isn't producing text in data file

I have a Python script that runs with no errors but it doesn't produce the output it should in a text file.  I can't figure out why. Is this the correct forum to post this in or can someone suggest a more appropriate forum?

The script selects all files from the day before the script is run.  So if today is December 10th the script will select all log files from December 9th.

The script reads all files from the day before and summarizes the information in the log files.  It writes out a text file with the summary information.

What should show up in the text file is information that is in this zip file:
http://theoffice.la/m/SummaryExample.zip

In the script line 11 it states what directory to look for the input log files.
The input log file name will need to be changed to yesterdays date for the script to read it.
This is an example of an input log file:
http://theoffice.la/m/2014-12-05Test.zip

This is the Python Script that I'm having trouble with:
http://theoffice.la/m/CGPLogSummaryTest.py

If I haven't provided enough information please let me know.

Thank you,

Docfxit

[toc] | [next] | [standalone]

#82317

On Wednesday, December 10, 2014 3:05:07 PM UTC-8, Docfxit wrote:
> I have a Python script that runs with no errors but it doesn't produce the output it should in a text file.  I can't figure out why. Is this the correct forum to post this in or can someone suggest a more appropriate forum?
> 
> The script selects all files from the day before the script is run.  So if today is December 10th the script will select all log files from December 9th.
> 
> The script reads all files from the day before and summarizes the information in the log files.  It writes out a text file with the summary information.
> 
> What should show up in the text file is information that is in this zip file:
> http://theoffice.la/m/SummaryExample.zip
> 
> In the script line 11 it states what directory to look for the input log files.
> The input log file name will need to be changed to yesterdays date for the script to read it.
> This is an example of an input log file:
> http://theoffice.la/m/2014-12-05Test.zip
> 
> This is the Python Script that I'm having trouble with:
> http://theoffice.la/m/CGPLogSummaryTest.py
> 
> I'm running this with Python 3.2.2 In Windows 7 Pro 
> If I haven't provided enough information please let me know.
> 
> Thank you,
> 
> Docfxit

Edited for clarity.

[toc] | [prev] | [next] | [standalone]

#82318

On Thu, Dec 11, 2014 at 10:04 AM, Docfxit <docfxit@gmail.com> wrote:
> This is the Python Script that I'm having trouble with:
> http://theoffice.la/m/CGPLogSummaryTest.py
>
> If I haven't provided enough information please let me know.

It'd be better to include the code in the body of your email. You
never know when your hosting might go down and the code be
unavailable.... like right now. :)

ChrisA

[toc] | [prev] | [next] | [standalone]

#82319

On Wednesday, December 10, 2014 3:11:28 PM UTC-8, Chris Angelico wrote:
> On Thu, Dec 11, 2014 at 10:04 AM, Docfxit <docfxit@gmail.com> wrote:
> > This is the Python Script that I'm having trouble with:
> > http://theoffice.la/m/CGPLogSummaryTest.py
> >
> > If I haven't provided enough information please let me know.
> 
> It'd be better to include the code in the body of your email. You
> never know when your hosting might go down and the code be
> unavailable.... like right now. :)
> 
> ChrisA

Also, I'm weary about clicking links to random pages.  I like to think that I can trust my computer to not let me get pwned by simply downloading and opening up a ZIP, but with Windows's history of exploits, I don't trust it.

[toc] | [prev] | [next] | [standalone]

#82320

On Thu, Dec 11, 2014 at 12:32 PM,  <sohcahtoa82@gmail.com> wrote:
> On Wednesday, December 10, 2014 3:11:28 PM UTC-8, Chris Angelico wrote:
>> On Thu, Dec 11, 2014 at 10:04 AM, Docfxit <docfxit@gmail.com> wrote:
>> > This is the Python Script that I'm having trouble with:
>> > http://theoffice.la/m/CGPLogSummaryTest.py
>> >
>> > If I haven't provided enough information please let me know.
>>
>> It'd be better to include the code in the body of your email. You
>> never know when your hosting might go down and the code be
>> unavailable.... like right now. :)
>>
>> ChrisA
>
> Also, I'm weary about clicking links to random pages.  I like to think that I can trust my computer to not let me get pwned by simply downloading and opening up a ZIP, but with Windows's history of exploits, I don't trust it.
>

Agreed. There are ways around some of those problems (eg using wget to
fetch something, and then looking at it in a text editor - it's hard
to get pwned through a text editor... though I won't say impossible),
but there are other issues too, and all in all, it's just best to
include the text in-line.

ChrisA

[toc] | [prev] | [next] | [standalone]

#82323

On Thu, 11 Dec 2014 12:44:51 +1100, Chris Angelico wrote:

> Agreed. There are ways around some of those problems (eg using wget to
> fetch something, and then looking at it in a text editor - it's hard to
> get pwned through a text editor... though I won't say impossible), but
> there are other issues too, and all in all, it's just best to include
> the text in-line.

I believe that there was a recently discovered exploit on Linux where 
viewing a file with "less" could run arbitrary code.

But really, if the program is short enough, you should include the code 
in the body of your post, or at least as an attachment. And if it is not 
short enough, you should *make it shorter*.

http://sscce.org/


There are many reasons why people might not click through links to 
download your code:

- they may be reading your post somewhere where they have access
  to email, but not web (e.g. behind a corporate firewall)

- they may not appreciate the security risks

- they might just be lazy, and firing up a web browser too much
  a distraction

- they might already have 200+ tabs open in their browser, and 
  adding one more may be just enough to shift them from "yes, 
  I'd love to help" to "fuck 'em, they can solve their own 
  problems".



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#82325

On Thu, Dec 11, 2014 at 1:41 PM, Steven D'Aprano <steve@pearwood.info> wrote:
> On Thu, 11 Dec 2014 12:44:51 +1100, Chris Angelico wrote:
>
>> Agreed. There are ways around some of those problems (eg using wget to
>> fetch something, and then looking at it in a text editor - it's hard to
>> get pwned through a text editor... though I won't say impossible), but
>> there are other issues too, and all in all, it's just best to include
>> the text in-line.
>
> I believe that there was a recently discovered exploit on Linux where
> viewing a file with "less" could run arbitrary code.

Which is why I refused to say "impossible" :) Although you gain the
additional benefit of an unpredictable attack vector; one person might
use 'less', another might open it in SciTE, a third might just cat the
file and scroll through it some other way. It's hard to aim at a
target that exists in so many pieces, and probably isn't worth the
effort of attacking.

ChrisA

[toc] | [prev] | [next] | [standalone]

#82329

On Wednesday, December 10, 2014 5:45:14 PM UTC-8, Chris Angelico wrote:
> On Thu, Dec 11, 2014 at 12:32 PM,  <sohcahtoa82@gmail.com> wrote:
> > On Wednesday, December 10, 2014 3:11:28 PM UTC-8, Chris Angelico wrote:
> >> On Thu, Dec 11, 2014 at 10:04 AM, Docfxit <docfxit@gmail.com> wrote:
> >> > This is the Python Script that I'm having trouble with:
> >> > http://theoffice.la/m/CGPLogSummaryTest.py
> >> >
> >> > If I haven't provided enough information please let me know.
> >>
> >> It'd be better to include the code in the body of your email. You
> >> never know when your hosting might go down and the code be
> >> unavailable.... like right now. :)
> >>
> >> ChrisA
> >
> > Also, I'm weary about clicking links to random pages.  I like to think that I can trust my computer to not let me get pwned by simply downloading and opening up a ZIP, but with Windows's history of exploits, I don't trust it.
> >
> 
> Agreed. There are ways around some of those problems (eg using wget to
> fetch something, and then looking at it in a text editor - it's hard
> to get pwned through a text editor... though I won't say impossible),
> but there are other issues too, and all in all, it's just best to
> include the text in-line.
> 
> ChrisA

I am happy to paste it into a post.  The reason I didn't is because it's very large.  The Python script is 1239 lines long.
The example summary is 105 lines long.
The input log is 6810 lines long.

Are you sure you want me to post all of that here?  Or would you like me to post it on a pastebin?

Thanks,

Docfxit

[toc] | [prev] | [next] | [standalone]

#82331

On Thu, Dec 11, 2014 at 2:30 PM, Docfxit <docfxit@gmail.com> wrote:
> I am happy to paste it into a post.  The reason I didn't is because it's very large.  The Python script is 1239 lines long.
> The example summary is 105 lines long.
> The input log is 6810 lines long.
>
> Are you sure you want me to post all of that here?  Or would you like me to post it on a pastebin?

In that case, no matter where you post it, most of us are simply not
going to read it. So follow the advice that Steven linked to -
http://sscce.org/ - and make something that's actually manageable, and
then as an added bonus, it'll be small enough to post in-line.

ChrisA

[toc] | [prev] | [next] | [standalone]

#82334

Docfxit <docfxit@gmail.com> writes:

> I am happy to paste it into a post.  The reason I didn't is because
> it's very large.  The Python script is 1239 lines long.

That's too long to direct us toward, no matter where you put it.

Your task, then, is to construct a *much* smaller and simpler example
that still demonstrates the behaviour you want explained.

Once you have that, one of two things will be true:

* It will be small and clear enough that we can discuss it here
  usefully. Or

* You will have understood the problem well enough to solve it yourself.

Please, construct a Short, Self-Contained, Complete Example
<URL:http://sscce.org/> of code that exhibits the behaviour.

-- 
 \              “When cryptography is outlawed, bayl bhgynjf jvyy unir |
  `\                                              cevinpl.” —Anonymous |
_o__)                                                                  |
Ben Finney

[toc] | [prev] | [next] | [standalone]

#82337

On Wednesday, December 10, 2014 7:55:17 PM UTC-8, Ben Finney wrote:
> Docfxit <docfxit@gmail.com> writes:
> 
> > I am happy to paste it into a post.  The reason I didn't is because
> > it's very large.  The Python script is 1239 lines long.
> 
> That's too long to direct us toward, no matter where you put it.
> 
> Your task, then, is to construct a *much* smaller and simpler example
> that still demonstrates the behaviour you want explained.
> 
> Once you have that, one of two things will be true:
> 
> * It will be small and clear enough that we can discuss it here
>   usefully. Or
> 
> * You will have understood the problem well enough to solve it yourself.
> 
> Please, construct a Short, Self-Contained, Complete Example
> <URL:http://sscce.org/> of code that exhibits the behaviour.
> 
> -- 
>  \              "When cryptography is outlawed, bayl bhgynjf jvyy unir |
>   `\                                              cevinpl." --Anonymous |
> _o__)                                                                  |
> Ben Finney

Thank you all for the encouragement to make it smaller.
I don't know enough about Python to figure out how to isolate where the problem is happening.

Maybe it would be best If I could get some help in getting a debugger working so I can try to work through the script one line at a time.  I have started to install a debugger but I can't get it working.
I'm working in Win7 Pro.  I'm trying to get Winpdb working.
I have an error when I run the debugger.  It says winpdb is missing.

winpdb.py is located in:       -same version
C:\Python27\Lib\site-packages
C:\winpdb-winpdb

winpdb   is located in:
C:\winpdb-winpdb

Is winpdb supposed to be in c:\python27?

If it is, how is it supposed to get there?  (Some install?)

Thanks,

Docfxit

[toc] | [prev] | [next] | [standalone]

#82338

On Thu, Dec 11, 2014 at 3:23 PM, Docfxit <docfxit@gmail.com> wrote:
> Thank you all for the encouragement to make it smaller.
> I don't know enough about Python to figure out how to isolate where the problem is happening.
>
> Maybe it would be best If I could get some help in getting a debugger working so I can try to work through the script one line at a time.  I have started to install a debugger but I can't get it working.
> I'm working in Win7 Pro.  I'm trying to get Winpdb working.
> I have an error when I run the debugger.  It says winpdb is missing.

A debugger is a nice tool when you can get it working, but at the
moment, it sounds like you're heading down the path of yak shaving. I
would recommend taking a much MUCH simpler approach: divide and
conquer. Strip out a bunch of code, try the program again. If it still
has the problem, you know the problem wasn't in the code you just
removed. If it doesn't, you know the problem _was_ in that code. (If
you just created a brand new problem, back out the change and try
something different.)

By this method, you can usually cut a problem down to size. Sometimes
you'll find that the problem disappears when any of several pieces of
code is removed, which means it's some sort of interaction between
them; in that case, keep both, but see how much other code you can
remove. As long as the issue you're seeing is at least mostly
deterministic, this sort of technique will help you to pin it down,
even if you aren't too familiar with the language.

ChrisA

[toc] | [prev] | [next] | [standalone]

#82339

Docfxit <docfxit@gmail.com> writes:

> Thank you all for the encouragement to make it smaller.

Even if only for the purpose of demonstrating the behaviour that you'd
like to discuss.

This doesn't necessarily mean changing the actual program you're working
on (though it might lead to that as a benefit).

> I don't know enough about Python to figure out how to isolate where
> the problem is happening.

If so, then here's a simple brute-force method:

Begin with an empty program, and start constructing the behaviour from
scratch. Ignore anything else you want the program to do; focus only on
*this* behaviour which is confusing you.

Add only those parts which seem to be related; remove them to verify
whether they're *actually* part of the problematic behaviour.

Eventually you will have a small program which, if you take any part
away, does not exhibit the behaviour you want explained.

-- 
 \      “Properly read, the Bible is the most potent force for atheism |
  `\                                    ever conceived.” —Isaac Asimov |
_o__)                                                                  |
Ben Finney

[toc] | [prev] | [next] | [standalone]

#82343

On 2014-12-11, Ben Finney <ben+python@benfinney.id.au> wrote:
> Docfxit <docfxit@gmail.com> writes:
>
>> Thank you all for the encouragement to make it smaller.

> Begin with an empty program, and start constructing the behaviour
> from scratch. Ignore anything else you want the program to do; focus
> only on *this* behaviour which is confusing you.
>
> Add only those parts which seem to be related; remove them to verify
> whether they're *actually* part of the problematic behaviour.
>
> Eventually you will have a small program which, if you take any part
> away, does not exhibit the behaviour you want explained.

And more often than not, you will figure out the answer to your
problem while attempting to construct a suitably concise example of
your problem.

If you can clearly explain and demonstrate your problem, you're 90% of
the way to fixing it.

-- 
Grant

[toc] | [prev] | [next] | [standalone]

#82341

On Wed, 10 Dec 2014 20:23:56 -0800, Docfxit wrote:

> I don't know enough about Python to figure out how to isolate where the
> problem is happening.

Ouch! You have my sympathies. Nevertheless, I'm not going to run your 
code to see what it does. Even if I trusted it, and I don't, I can see 
that it assumes that it is running on Windows (it hard-codes '\\' as a 
directory separator). I'm not running Windows.

(I think it is funny that the script has a Unix "hash-bang" line at the 
top of the script, but is written such that it will only work on Windows.)


I have some questions. The answers may guide you towards a solution.

Has this script ever worked? If so, what changed before it stopped 
working?

When you run the script, does it print any output? If so, what?

Does it write any output to the output file?

What happens if you change the global variable "testing" to True?



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#82345

On 11/12/2014 05:18, Steven D'Aprano wrote:
> (I think it is funny that the script has a Unix "hash-bang" line at
> the top of the script, but is written such that it will only work on
> Windows.)

I didn't look at the code, but responding only to your comment...

Since the introduction of the PEP397 py.exe launcher in Python 3.4, it's 
made sense to put hash-bang lines into Python files on Windows 
especially if you have multiple versions installed.

TJG

[toc] | [prev] | [next] | [standalone]

#82354

On Wed, 10 Dec 2014 20:23:56 -0800, Docfxit wrote:

> On Wednesday, December 10, 2014 7:55:17 PM UTC-8, Ben Finney wrote:
>> Docfxit <docfxit@gmail.com> writes:
>> 
>> > I am happy to paste it into a post.  The reason I didn't is because
>> > it's very large.  The Python script is 1239 lines long.
>> 
>> That's too long to direct us toward, no matter where you put it.
>> 
>> Your task, then, is to construct a *much* smaller and simpler example
>> that still demonstrates the behaviour you want explained.
>> 
>> Once you have that, one of two things will be true:
>> 
>> * It will be small and clear enough that we can discuss it here
>>   usefully. Or
>> 
>> * You will have understood the problem well enough to solve it
>> yourself.
>> 
>> Please, construct a Short, Self-Contained, Complete Example
>> <URL:http://sscce.org/> of code that exhibits the behaviour.
>> 
>> --
>>  \              "When cryptography is outlawed, bayl bhgynjf jvyy unir
>>  |
>>   `\                                              cevinpl." --Anonymous
>>   |
>> _o__)                                                                 
>> |
>> Ben Finney
> 
> Thank you all for the encouragement to make it smaller.
> I don't know enough about Python to figure out how to isolate where the
> problem is happening.

That should then be your first port of call, check the Python 
documentation & step through the code manually.
> 
> Maybe it would be best If I could get some help in getting a debugger
> working so I can try to work through the script one line at a time. 

it is probably just as easy to add print statements into a copy of the 
code so you can where the code has got to & what your Values are.

> I have started to install a debugger but I can't get it working.
> I'm working in Win7 Pro.  I'm trying to get Winpdb working.
> I have an error when I run the debugger.  It says winpdb is missing.
> 
> winpdb.py is located in:       -same version
> C:\Python27\Lib\site-packages C:\winpdb-winpdb
> 
> winpdb   is located in:
> C:\winpdb-winpdb
> 
> Is winpdb supposed to be in c:\python27?
> 
> If it is, how is it supposed to get there?  (Some install?)
> 
> Thanks,
> 
> Docfxit





-- 
You canna change the laws of physics, Captain; I've got to have thirty 
minutes!

[toc] | [prev] | [next] | [standalone]

#82355

Docfxit wrote:

> I don't know enough about Python to figure out how to isolate where the
> problem is happening.

Can you get hold of the person who wrote the script? 

[toc] | [prev] | [next] | [standalone]

#82383

On Thursday, December 11, 2014 1:32:02 AM UTC-8, Peter Otten wrote:
> Docfxit wrote:
> 
> > I don't know enough about Python to figure out how to isolate where the
> > problem is happening.
> 
> Can you get hold of the person who wrote the script?

No.  I tried.  They are not answering their email.

Thanks for the suggestion,

Docfxit

[toc] | [prev] | [next] | [standalone]

#82342

On 2014-12-11, Docfxit <docfxit@gmail.com> wrote:

> I am happy to paste it into a post.  The reason I didn't is because
> it's very large.  The Python script is 1239 lines long.  The example
> summary is 105 lines long.  The input log is 6810 lines long.
>
> Are you sure you want me to post all of that here?  Or would you
> like me to post it on a pastebin?

You're going to need to create a _short_ example program that
demonstrates the problem you're asking about.  It needs to be included
in your post and fit in _one_ screen full.  A few people will look at
longer examples, but your chances of the right person seeing it and
figuring out your problem before their internal timer (or the one on
the microwave) goes "ding!"  drop precipitously as it gets longer.
Your chances approach 0 pretty fast if it requires following a URL to
some site full of malware[1].


[1] Yes, all sites are full of malware.

[toc] | [prev] | [next] | [standalone]

Page 1 of 2  [1] 2  Next page →

Back to top | Article view | comp.lang.python

csiph-web