Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #30705 > unrolled thread

final question: logging to stdout and updating files

Back to article view | Back to comp.lang.python

Contents

  final question: logging to stdout and updating files "Littlefield, Tyler" <tyler@tysdomain.com> - 2012-10-03 21:11 -0600
    Re: final question: logging to stdout and updating files Ramchandra Apte <maniandram01@gmail.com> - 2012-10-04 00:53 -0700
    Re: final question: logging to stdout and updating files Ramchandra Apte <maniandram01@gmail.com> - 2012-10-04 00:53 -0700
    Re: final question: logging to stdout and updating files Ramchandra Apte <maniandram01@gmail.com> - 2012-10-04 06:34 -0700
    Re: final question: logging to stdout and updating files Ramchandra Apte <maniandram01@gmail.com> - 2012-10-04 06:34 -0700
      Re: final question: logging to stdout and updating files Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2012-10-04 14:00 +0000
        Re: final question: logging to stdout and updating files Chris Angelico <rosuav@gmail.com> - 2012-10-05 00:27 +1000
        RE: final question: logging to stdout and updating files "Prasad, Ramit" <ramit.prasad@jpmorgan.com> - 2012-10-04 22:36 +0000
        Re: final question: logging to stdout and updating files Mark Lawrence <breamoreboy@yahoo.co.uk> - 2012-10-05 09:11 +0100
        Re: final question: logging to stdout and updating files Mark Lawrence <breamoreboy@yahoo.co.uk> - 2012-10-05 09:19 +0100
        Re: final question: logging to stdout and updating files Ramchandra Apte <maniandram01@gmail.com> - 2012-10-05 06:32 -0700
    Re: final question: logging to stdout and updating files Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2012-10-04 14:52 +0000

#30705 — final question: logging to stdout and updating files

pHello all:
I've seen frameworks like django reload files when it detects that 
they've been changed; how hard would it be to make my engine reload 
files that it detects were changed? I'm also curious how hard it would 
be to build in some error recovery. For example right now when an 
exception occurs, the player is sometimes just left hanging. It's a lot 
harder with Python for me, because I don't get the compile-time errors 
that I would with c++ for example to know that I did something wrong; 
while that's not always useful/and by far it doesn't catch everything, 
it does help. I'm familiar with things like pychecker, but it seems to 
be reporting a lot of issues that aren't issues. For example, I have a 
world module which is the core of the engine; it handles players, as 
well as keeps tracks of all rooms that are loaded in the game and that. 
Because player and world would have circular imports, I just pass the 
world object into player functions like logon/create. Pychecker tells me 
that the world parameter (which is a local var at that point) shadows 
the world variable in world; world is a singleton, so when you import 
world it just has a world = World() at the bottom of the module.

also: I have the following code:
     logging.basicConfig(filename=path.join("logs", "mud.log"), 
level=logging.DEBUG)
     logger = logging.getLogger(__name__)
     logger.addHandler(logging.StreamHandler())
I like it displaying to stderr since usually when I'm doing this I'm in 
screen bouncing back and forth between the output and the tt++ session, 
but right now I can't get a couple of things; I'm not sure how to set it 
to log and all other messages to stderr as I did for the file, and I'd 
like to use a rotating log handler so that it'll rotate when the files 
are say above 16 KB or something. Is it possible to do something like 
this; perhaps make it compress the file before it writes to disk, or 
call a command to do so, so that it wouldn't hang the entire mud while 
it compresses?
Thanks, and sorry again for all the questions.

-- 
Take care,
Ty
http://tds-solutions.net
The aspen project: a barebones light-weight mud engine:
http://code.google.com/p/aspenmud
He that will not reason is a bigot; he that cannot reason is a fool; he that dares not reason is a slave.

[toc] | [next] | [standalone]

#30713

On Thursday, 4 October 2012 08:41:35 UTC+5:30, Littlefield, Tyler  wrote:
> pHello all:
> 
> I've seen frameworks like django reload files when it detects that 
> 
> they've been changed; how hard would it be to make my engine reload 
> 
> files that it detects were changed? I'm also curious how hard it would 
> 
> be to build in some error recovery. For example right now when an 
> 
> exception occurs, the player is sometimes just left hanging. It's a lot 
> 
> harder with Python for me, because I don't get the compile-time errors 
> 
> that I would with c++ for example to know that I did something wrong; 
> 
> while that's not always useful/and by far it doesn't catch everything, 
> 
> it does help. I'm familiar with things like pychecker, but it seems to 
> 
> be reporting a lot of issues that aren't issues. For example, I have a 
> 
> world module which is the core of the engine; it handles players, as 
> 
> well as keeps tracks of all rooms that are loaded in the game and that. 
> 
> Because player and world would have circular imports, I just pass the 
> 
> world object into player functions like logon/create. Pychecker tells me 
> 
> that the world parameter (which is a local var at that point) shadows 
> 
> the world variable in world; world is a singleton, so when you import 
> 
> world it just has a world = World() at the bottom of the module.
> 
> 
> 
> also: I have the following code:
> 
>      logging.basicConfig(filename=path.join("logs", "mud.log"), 
> 
> level=logging.DEBUG)
> 
>      logger = logging.getLogger(__name__)
> 
>      logger.addHandler(logging.StreamHandler())
> 
> I like it displaying to stderr since usually when I'm doing this I'm in 
> 
> screen bouncing back and forth between the output and the tt++ session, 
> 
> but right now I can't get a couple of things; I'm not sure how to set it 
> 
> to log and all other messages to stderr as I did for the file, and I'd 
> 
> like to use a rotating log handler so that it'll rotate when the files 
> 
> are say above 16 KB or something. Is it possible to do something like 
> 
> this; perhaps make it compress the file before it writes to disk, or 
> 
> call a command to do so, so that it wouldn't hang the entire mud while 
> 
> it compresses?
> 
> Thanks, and sorry again for all the questions.
> 
> 
> 
> -- 
> 
> Take care,
> 
> Ty
> 
> http://tds-solutions.net
> 
> The aspen project: a barebones light-weight mud engine:
> 
> http://code.google.com/p/aspenmud
> 
> He that will not reason is a bigot; he that cannot reason is a fool; he that dares not reason is a slave.

I use pylint with NINJA IDE. NINJA IDE automatically shows common problems.

[toc] | [prev] | [next] | [standalone]

#30714

On Thursday, 4 October 2012 08:41:35 UTC+5:30, Littlefield, Tyler  wrote:
> pHello all:
> 
> I've seen frameworks like django reload files when it detects that 
> 
> they've been changed; how hard would it be to make my engine reload 
> 
> files that it detects were changed? I'm also curious how hard it would 
> 
> be to build in some error recovery. For example right now when an 
> 
> exception occurs, the player is sometimes just left hanging. It's a lot 
> 
> harder with Python for me, because I don't get the compile-time errors 
> 
> that I would with c++ for example to know that I did something wrong; 
> 
> while that's not always useful/and by far it doesn't catch everything, 
> 
> it does help. I'm familiar with things like pychecker, but it seems to 
> 
> be reporting a lot of issues that aren't issues. For example, I have a 
> 
> world module which is the core of the engine; it handles players, as 
> 
> well as keeps tracks of all rooms that are loaded in the game and that. 
> 
> Because player and world would have circular imports, I just pass the 
> 
> world object into player functions like logon/create. Pychecker tells me 
> 
> that the world parameter (which is a local var at that point) shadows 
> 
> the world variable in world; world is a singleton, so when you import 
> 
> world it just has a world = World() at the bottom of the module.
> 
> 
> 
> also: I have the following code:
> 
>      logging.basicConfig(filename=path.join("logs", "mud.log"), 
> 
> level=logging.DEBUG)
> 
>      logger = logging.getLogger(__name__)
> 
>      logger.addHandler(logging.StreamHandler())
> 
> I like it displaying to stderr since usually when I'm doing this I'm in 
> 
> screen bouncing back and forth between the output and the tt++ session, 
> 
> but right now I can't get a couple of things; I'm not sure how to set it 
> 
> to log and all other messages to stderr as I did for the file, and I'd 
> 
> like to use a rotating log handler so that it'll rotate when the files 
> 
> are say above 16 KB or something. Is it possible to do something like 
> 
> this; perhaps make it compress the file before it writes to disk, or 
> 
> call a command to do so, so that it wouldn't hang the entire mud while 
> 
> it compresses?
> 
> Thanks, and sorry again for all the questions.
> 
> 
> 
> -- 
> 
> Take care,
> 
> Ty
> 
> http://tds-solutions.net
> 
> The aspen project: a barebones light-weight mud engine:
> 
> http://code.google.com/p/aspenmud
> 
> He that will not reason is a bigot; he that cannot reason is a fool; he that dares not reason is a slave.

I use pylint with NINJA IDE. NINJA IDE automatically shows common problems.

[toc] | [prev] | [next] | [standalone]

#30725

On Thursday, 4 October 2012 08:41:35 UTC+5:30, Littlefield, Tyler  wrote:
> pHello all:
> 
> I've seen frameworks like django reload files when it detects that 
> 
> they've been changed; how hard would it be to make my engine reload 
> 
> files that it detects were changed? I'm also curious how hard it would 
> 
> be to build in some error recovery. For example right now when an 
> 
> exception occurs, the player is sometimes just left hanging. It's a lot 
> 
> harder with Python for me, because I don't get the compile-time errors 
> 
> that I would with c++ for example to know that I did something wrong; 
> 
> while that's not always useful/and by far it doesn't catch everything, 
> 
> it does help. I'm familiar with things like pychecker, but it seems to 
> 
> be reporting a lot of issues that aren't issues. For example, I have a 
> 
> world module which is the core of the engine; it handles players, as 
> 
> well as keeps tracks of all rooms that are loaded in the game and that. 
> 
> Because player and world would have circular imports, I just pass the 
> 
> world object into player functions like logon/create. Pychecker tells me 
> 
> that the world parameter (which is a local var at that point) shadows 
> 
> the world variable in world; world is a singleton, so when you import 
> 
> world it just has a world = World() at the bottom of the module.
> 
> 
> 
> also: I have the following code:
> 
>      logging.basicConfig(filename=path.join("logs", "mud.log"), 
> 
> level=logging.DEBUG)
> 
>      logger = logging.getLogger(__name__)
> 
>      logger.addHandler(logging.StreamHandler())
> 
> I like it displaying to stderr since usually when I'm doing this I'm in 
> 
> screen bouncing back and forth between the output and the tt++ session, 
> 
> but right now I can't get a couple of things; I'm not sure how to set it 
> 
> to log and all other messages to stderr as I did for the file, and I'd 
> 
> like to use a rotating log handler so that it'll rotate when the files 
> 
> are say above 16 KB or something. Is it possible to do something like 
> 
> this; perhaps make it compress the file before it writes to disk, or 
> 
> call a command to do so, so that it wouldn't hang the entire mud while 
> 
> it compresses?
> 
> Thanks, and sorry again for all the questions.
> 
> 
> 
> -- 
> 
> Take care,
> 
> Ty
> 
> http://tds-solutions.net
> 
> The aspen project: a barebones light-weight mud engine:
> 
> http://code.google.com/p/aspenmud
> 
> He that will not reason is a bigot; he that cannot reason is a fool; he that dares not reason is a slave.

Solution for the logging problem is to use to use logging.handlers.BaseRotatingHandler [0]
^0 http://docs.python.org/dev/library/logging.handlers.html#baserotatinghandler

"Optimize code always even if it causes bugs" - Ramchandra Apte, 2001-

[toc] | [prev] | [next] | [standalone]

#30726

On Thursday, 4 October 2012 08:41:35 UTC+5:30, Littlefield, Tyler  wrote:
> pHello all:
> 
> I've seen frameworks like django reload files when it detects that 
> 
> they've been changed; how hard would it be to make my engine reload 
> 
> files that it detects were changed? I'm also curious how hard it would 
> 
> be to build in some error recovery. For example right now when an 
> 
> exception occurs, the player is sometimes just left hanging. It's a lot 
> 
> harder with Python for me, because I don't get the compile-time errors 
> 
> that I would with c++ for example to know that I did something wrong; 
> 
> while that's not always useful/and by far it doesn't catch everything, 
> 
> it does help. I'm familiar with things like pychecker, but it seems to 
> 
> be reporting a lot of issues that aren't issues. For example, I have a 
> 
> world module which is the core of the engine; it handles players, as 
> 
> well as keeps tracks of all rooms that are loaded in the game and that. 
> 
> Because player and world would have circular imports, I just pass the 
> 
> world object into player functions like logon/create. Pychecker tells me 
> 
> that the world parameter (which is a local var at that point) shadows 
> 
> the world variable in world; world is a singleton, so when you import 
> 
> world it just has a world = World() at the bottom of the module.
> 
> 
> 
> also: I have the following code:
> 
>      logging.basicConfig(filename=path.join("logs", "mud.log"), 
> 
> level=logging.DEBUG)
> 
>      logger = logging.getLogger(__name__)
> 
>      logger.addHandler(logging.StreamHandler())
> 
> I like it displaying to stderr since usually when I'm doing this I'm in 
> 
> screen bouncing back and forth between the output and the tt++ session, 
> 
> but right now I can't get a couple of things; I'm not sure how to set it 
> 
> to log and all other messages to stderr as I did for the file, and I'd 
> 
> like to use a rotating log handler so that it'll rotate when the files 
> 
> are say above 16 KB or something. Is it possible to do something like 
> 
> this; perhaps make it compress the file before it writes to disk, or 
> 
> call a command to do so, so that it wouldn't hang the entire mud while 
> 
> it compresses?
> 
> Thanks, and sorry again for all the questions.
> 
> 
> 
> -- 
> 
> Take care,
> 
> Ty
> 
> http://tds-solutions.net
> 
> The aspen project: a barebones light-weight mud engine:
> 
> http://code.google.com/p/aspenmud
> 
> He that will not reason is a bigot; he that cannot reason is a fool; he that dares not reason is a slave.

Solution for the logging problem is to use to use logging.handlers.BaseRotatingHandler [0]
^0 http://docs.python.org/dev/library/logging.handlers.html#baserotatinghandler

"Optimize code always even if it causes bugs" - Ramchandra Apte, 2001-

[toc] | [prev] | [next] | [standalone]

#30728

On Thu, 04 Oct 2012 06:34:28 -0700, Ramchandra Apte wrote:

> "Optimize code always even if it causes bugs" - Ramchandra Apte, 2001-

Well, you've just added yourself into my list of people whose advice 
should always be ignored.

That is *terrible* advice. But if you insist on following it, you can 
optimize *any* Python program to this:

# === start code ===
pass  # this line is optional
# === end code ===


There you go. The most heavily optimized, fastest Python program in 
existence. Sure, it has a few bugs, but boy is it fast!!!


-- 
Steven

[toc] | [prev] | [next] | [standalone]

#30730

On Fri, Oct 5, 2012 at 12:00 AM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> That is *terrible* advice. But if you insist on following it, you can
> optimize *any* Python program to this:
>
> # === start code ===
> pass  # this line is optional
> # === end code ===
>
>
> There you go. The most heavily optimized, fastest Python program in
> existence. Sure, it has a few bugs, but boy is it fast!!!

Not many bugs though! I ran it in my Python 5.2.7 for GNU/Windows
256-bit (err, yeah, I borrowed Guido's time machine but had the silly
thing in reverse... oops) and it worked perfectly, except that
indentation has moved from "significant" to "mandatory". When I added
the necessary 5 space indent at the beginning, it correctly created
world peace, ensured that Australia won the next Test Match, and then
printed "Hello, world!\n" to stdout. Unfortunately, a bug in your "end
code" comment meant that the peace it created was by wiping out all
life, but that's pretty minor in the scheme of things.

Optimization really is that important, folks!

ChrisA
may need to schedule surgical detongueing of his cheek

[toc] | [prev] | [next] | [standalone]

#30754

Chris Angelico wrote:
> Sent: Thursday, October 04, 2012 9:28 AM
> To: python-list@python.org
> Subject: Re: final question: logging to stdout and updating files
> 
> On Fri, Oct 5, 2012 at 12:00 AM, Steven D'Aprano
> <steve+comp.lang.python@pearwood.info> wrote:
> > That is *terrible* advice. But if you insist on following it, you can
> > optimize *any* Python program to this:
> >
> > # === start code ===
> > pass  # this line is optional
> > # === end code ===
> >
> >
> > There you go. The most heavily optimized, fastest Python program in
> > existence. Sure, it has a few bugs, but boy is it fast!!!
> 
> Not many bugs though! I ran it in my Python 5.2.7 for GNU/Windows
> 256-bit (err, yeah, I borrowed Guido's time machine but had the silly
> thing in reverse... oops) and it worked perfectly, except that
> indentation has moved from "significant" to "mandatory". When I added
> the necessary 5 space indent at the beginning, it correctly created
> world peace, ensured that Australia won the next Test Match, and then
> printed "Hello, world!\n" to stdout. Unfortunately, a bug in your "end
> code" comment meant that the peace it created was by wiping out all
> life, but that's pretty minor in the scheme of things.

Python is a product for Americans! ;) It should ensure America
wins the Test Match....wait, do we even have a cricket team?

> 
> Optimization really is that important, folks!
> 
> ChrisA
> may need to schedule surgical detongueing of his cheek

Think we could get a group rate for c.l.p?

Ramit

This email is confidential and subject to important disclaimers and
conditions including on offers for the purchase or sale of
securities, accuracy and completeness of information, viruses,
confidentiality, legal privilege, and legal entity disclaimers,
available at http://www.jpmorgan.com/pages/disclosures/email.  

[toc] | [prev] | [next] | [standalone]

#30786

On 04/10/2012 15:27, Chris Angelico wrote:
> ensured that Australia won the next Test Match
>
> ChrisA
> may need to schedule surgical detongueing of his cheek
>

I'll arrange the cheek detonguing very cheaply after a comment like that :)

-- 
Cheers.

Mark Lawrence.

[toc] | [prev] | [next] | [standalone]

#30788

On 04/10/2012 23:36, Prasad, Ramit wrote:
>
> Python is a product for Americans! ;) It should ensure America
> wins the Test Match....wait, do we even have a cricket team?

ChrisA could have been talking rugby, your rugby union team isn't't too 
bad for a bunch of amateurs, some of whom had to take unpaid leave to 
play in the world cup.  I don't think there's any rugby league sides in 
the USA.  Then there's speedway and...?

>
>>
>> Optimization really is that important, folks!
>>
>> ChrisA
>> may need to schedule surgical detongueing of his cheek
>
> Think we could get a group rate for c.l.p?

I'm sure that with some appropriate grovelling the PSF could arrange 
this.  Perhaps fly everybody to the UK for PyCon and get the surgery 
done on the NHS at the same time.

>
> Ramit
>

-- 
Cheers.

Mark Lawrence.

[toc] | [prev] | [next] | [standalone]

#30813

On Thursday, 4 October 2012 19:30:26 UTC+5:30, Steven D'Aprano  wrote:
> On Thu, 04 Oct 2012 06:34:28 -0700, Ramchandra Apte wrote:
> 
> 
> 
> > "Optimize code always even if it causes bugs" - Ramchandra Apte, 2001-
> 
> 
> 
> Well, you've just added yourself into my list of people whose advice 
> 
> should always be ignored.
> 
> 
> 
> That is *terrible* advice. But if you insist on following it, you can 
> 
> optimize *any* Python program to this:
> 
> 
> 
> # === start code ===
> 
> pass  # this line is optional
> 
> # === end code ===
> 
> 
> 
> 
> 
> There you go. The most heavily optimized, fastest Python program in 
> 
> existence. Sure, it has a few bugs, but boy is it fast!!!
> 
> 
> 
> 
> 
> -- 
> 
> Steven

Please disclose the list of people.

This email is subject to important disclosures available at loobafoobajo.ke/legal.html
LOOBA FOOBA JOKE COMPANY MAKES NO WARRANTIES ABOUT THIS EMAIL
Read at your risk - you may die from reading this email. ;-P

[toc] | [prev] | [next] | [standalone]

#30732

On Wed, 03 Oct 2012 21:11:29 -0600, Littlefield, Tyler wrote:

> I've seen frameworks like django reload files when it detects that
> they've been changed; how hard would it be to make my engine reload
> files that it detects were changed? 

Oh, about as hard as writing a program.

What sort of files? What does your engine do? How does it do it? Without 
knowing the answers to these questions, how can we possibly tell you how 
hard it will be to reload them?

Detecting changed files is easy. If you google for "python monitor 
directory" and similar terms, you will find a metric tonne of solutions.

Having your engine reload files is entirely up to you: it's your engine, 
it will be as trivial or as difficult as you make it be.


> I'm also curious how hard it would be to build in some error recovery. 

How long is a piece of string? Again, that depends on you. If you design 
your application with error recovery in mind, it could be trivial. If you 
don't, it could be impossible.


> For example right now when an
> exception occurs, the player is sometimes just left hanging. It's a lot
> harder with Python for me, because I don't get the compile-time errors
> that I would with c++ for example to know that I did something wrong;
> while that's not always useful/and by far it doesn't catch everything,
> it does help. 

Sure, compiler-time checks can sometimes be useful. But in general, they 
tend to only detect the most trivial errors, syntax errors (Python does 
that too) and type errors.

In Python, the usual answer is to concentrate on writing good unit tests. 
Good unit tests will test far more than the compiler ever could, and will 
pay for themselves a hundred times over.


> I'm familiar with things like pychecker, but it seems to
> be reporting a lot of issues that aren't issues.

Pychecker, like other linters, don't just report fatal errors. They may 
also report *suspicious code* which may indicate an error, poor 
techniques, unused code, bad naming conventions, or other examples of 
poor style. You should be able to turn off such warnings.


> For example, I have a
> world module which is the core of the engine; it handles players, as
> well as keeps tracks of all rooms that are loaded in the game and that.
> Because player and world would have circular imports

Right there is a terrible *code smell*.

http://www.joelonsoftware.com/articles/Wrong.html

Maybe you have a good reason for a circular import, but alarm bells are 
ringing. Rather than having:

# world.py
import player

# player.py
import world

which leads to all sorts of complications, it is usually better to have a 
single module import both world and player and then combine them as 
needed.


-- 
Steven

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web