Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #90480 > unrolled thread

Python file structure

Back to article view | Back to comp.lang.python

Contents

  Python file structure zljubisicmob@gmail.com - 2015-05-12 12:13 -0700
    Re: Python file structure Chris Angelico <rosuav@gmail.com> - 2015-05-13 05:29 +1000
    Re: Python file structure Ned Batchelder <ned@nedbatchelder.com> - 2015-05-12 12:49 -0700
      Re: Python file structure zljubisicmob@gmail.com - 2015-05-12 12:58 -0700
        Re: Python file structure Dave Angel <davea@davea.name> - 2015-05-12 16:43 -0400
      Re: Python file structure Chris Angelico <rosuav@gmail.com> - 2015-05-13 06:02 +1000
      Re: Python file structure Terry Reedy <tjreedy@udel.edu> - 2015-05-12 17:34 -0400
    Re: Python file structure Ian Kelly <ian.g.kelly@gmail.com> - 2015-05-12 13:54 -0600
    Re: Python file structure Chris Angelico <rosuav@gmail.com> - 2015-05-13 06:07 +1000
    Re: Python file structure Tim Chase <python.list@tim.thechases.com> - 2015-05-13 13:34 -0500
    Re: Python file structure Chris Angelico <rosuav@gmail.com> - 2015-05-14 12:03 +1000

#90480 — Python file structure

Hi, I have python file with the following structure:

import...

A = configparser.get(...) 
B = configparser.get(...)

Command line parameters parsing [they can change variable A or B]

Def usage()
	Print how to use script parameters

def main():
	...

if __name__ == "__main__":
    main()

If I find an error in command line parameters section I cannot call function usage() because it is not defined yet. 

I have few options here:
1.	Put definition of usage function before command line parameters parsing section
2.	Make parameters global and put them in the main function
3.	...maybe some other options...

The basic idea is that variables A and B should be accessible globally, but after they are read by configparser and/or changed by command line parameters parsing section, the variables should preserve their values throughout the whole program.

If I set them in the way described above, even the variables are not global I can read their values everywhere with no option to change them.
If I set them with global keyword I am risking changing their values afterwards. 

What approach do you suggest?
The goal is to have global variables that should preserve their values and at the same time no meter if they are defined by configparser and/or command line parameter to be able to print usage if there is a problem with a parameter?

Regards.

[toc] | [next] | [standalone]

#90481

On Wed, May 13, 2015 at 5:13 AM,  <zljubisicmob@gmail.com> wrote:
> import...
>
> A = configparser.get(...)
> B = configparser.get(...)
>
> Command line parameters parsing [they can change variable A or B]
>
> Def usage()
>         Print how to use script parameters
>
> def main():
>         ...
>
> if __name__ == "__main__":
>     main()
>
> If I find an error in command line parameters section I cannot call function usage() because it is not defined yet.
>
> I have few options here:
> 1.      Put definition of usage function before command line parameters parsing section

I'd do this, unless there's a good reason not to. A simple usage
function probably doesn't have many dependencies, so it can logically
go high in the code. As a general rule, I like to organize code such
that things are defined higher up than they're used; it's not strictly
necessary (if they're used inside functions, the requirement is only
that they be defined before the function's called), but it helps with
clarity. That generally means that "def usage():" wants to go up above
any place where "usage()" occurs, but below the definitions of any
functions that usage() itself calls, and below the first assignments
to any global names it uses. It's not always possible, but when it is,
it tends to produce an easy-to-navigate source file.

ChrisA

[toc] | [prev] | [next] | [standalone]

#90482

On Tuesday, May 12, 2015 at 3:13:32 PM UTC-4, zljubi...@gmail.com wrote:
> Hi, I have python file with the following structure:
> 
> import...
> 
> A = configparser.get(...) 
> B = configparser.get(...)
> 
> Command line parameters parsing [they can change variable A or B]
> 
> Def usage()
> 	Print how to use script parameters
> 
> def main():
> 	...
> 
> if __name__ == "__main__":
>     main()
> 
> If I find an error in command line parameters section I cannot call function usage() because it is not defined yet. 
> 
> I have few options here:
> 1.	Put definition of usage function before command line parameters parsing section
> 2.	Make parameters global and put them in the main function
> 3.	...maybe some other options...
> 

I would put all of the code into a function some place.  Don't have
anything at the top level of the file except imports, function (and
class) definitions, and an "if __name__....." clause at the bottom.

If you need to use globals, assign them inside a parse_arguments
function that has a "global" statement in it.

This advice is consistent with Chris' "define things before they
are used."  It does it by defining everything before anything is
run.

As a side note, if you are going to have code at the top-level of
the file, then there's no point in the "if __name__..." clause.
That clause is designed to make a file both runnable and importable.
But your top-level code makes the file very difficult to import.

--Ned.

[toc] | [prev] | [next] | [standalone]

#90485

On Tuesday, May 12, 2015 at 9:49:20 PM UTC+2, Ned Batchelder wrote:
> On Tuesday, May 12, 2015 at 3:13:32 PM UTC-4, zljubi...@gmail.com wrote:
> > Hi, I have python file with the following structure:
> > 
> > import...
> > 
> > A = configparser.get(...) 
> > B = configparser.get(...)
> > 
> > Command line parameters parsing [they can change variable A or B]
> > 
> > Def usage()
> > 	Print how to use script parameters
> > 
> > def main():
> > 	...
> > 
> > if __name__ == "__main__":
> >     main()
> > 
> > If I find an error in command line parameters section I cannot call function usage() because it is not defined yet. 
> > 
> > I have few options here:
> > 1.	Put definition of usage function before command line parameters parsing section
> > 2.	Make parameters global and put them in the main function
> > 3.	...maybe some other options...
> > 
> 
> I would put all of the code into a function some place.  Don't have
> anything at the top level of the file except imports, function (and
> class) definitions, and an "if __name__....." clause at the bottom.
> 
> If you need to use globals, assign them inside a parse_arguments
> function that has a "global" statement in it.
> 
> This advice is consistent with Chris' "define things before they
> are used."  It does it by defining everything before anything is
> run.
> 
> As a side note, if you are going to have code at the top-level of
> the file, then there's no point in the "if __name__..." clause.
> That clause is designed to make a file both runnable and importable.
> But your top-level code makes the file very difficult to import.
> 
> --Ned.

It makes sense. The only drawback is that variables are global so they could be changed anywhere in the program.
I also agree that it is more python approach.

Thanks to both of you.

[toc] | [prev] | [next] | [standalone]

#90494

On 05/12/2015 03:58 PM, zljubisicmob@gmail.com wrote:
> On Tuesday, May 12, 2015 at 9:49:20 PM UTC+2, Ned Batchelder wrote:

>>
>> If you need to use globals, assign them inside a parse_arguments
>> function that has a "global" statement in it.
>>
>> This advice is consistent with Chris' "define things before they
>> are used."  It does it by defining everything before anything is
>> run.
>>
>> As a side note, if you are going to have code at the top-level of
>> the file, then there's no point in the "if __name__..." clause.
>> That clause is designed to make a file both runnable and importable.
>> But your top-level code makes the file very difficult to import.
>>
>> --Ned.
>
> It makes sense. The only drawback is that variables are global

only "if you need to use globals".  You can't have it both ways.  If 
they're needed, it's because you feel they must be changeable elsewhere 
in the program.  I try to avoid global variables, but make no such 
restraints on the use of global constants.  So for example, the argument 
parsing logic could very well export something as global, but it'd be 
all uppercase and anyone changing it subsequently would get their hand 
slapped by the linter.

> so they could be changed anywhere in the program.
> I also agree that it is more python approach.
>
> Thanks to both of you.
>


-- 
DaveA

[toc] | [prev] | [next] | [standalone]

#90486

On Wed, May 13, 2015 at 5:49 AM, Ned Batchelder <ned@nedbatchelder.com> wrote:
> I would put all of the code into a function some place.  Don't have
> anything at the top level of the file except imports, function (and
> class) definitions, and an "if __name__....." clause at the bottom.
>
> If you need to use globals, assign them inside a parse_arguments
> function that has a "global" statement in it.
>
> This advice is consistent with Chris' "define things before they
> are used."  It does it by defining everything before anything is
> run.

Consistent with, yes, but not for the reason you state. I'm talking
about code layout, not prevention of NameError. For instance, this
would fit your description, and wouldn't error out, but wouldn't fit
my ideal:

import sys

def main():
    if len(sys.argv) < 2: usage()
    # do stuff

def usage():
    print("USAGE: programname arguments")
    sys.exit(0)

if __name__ == '__main__': main()


I would shift the definition of usage() up above main(). Even though
both are executed before main() actually begins running, which
prevents NameError, it's harder to skim the file. Obviously this is an
ideal that can't always be attained (mutual references, for instance),
but where it can be done, it means that the first instance of any
token in a file is its definition - maybe in an import statement (so
"from X import *" violates the principle, but I think most people
avoid it anyway), or maybe in a def/class statement, or maybe a simple
assignment.

ChrisA

[toc] | [prev] | [next] | [standalone]

#90497

On 5/12/2015 3:49 PM, Ned Batchelder wrote:
> On Tuesday, May 12, 2015 at 3:13:32 PM UTC-4, zljubi...@gmail.com wrote:
>> Hi, I have python file with the following structure:
>>
>> import...
>>
>> A = configparser.get(...)
>> B = configparser.get(...)
>>
>> Command line parameters parsing [they can change variable A or B]
>>
>> Def usage()
>> 	Print how to use script parameters
>>
>> def main():
>> 	...
>>
>> if __name__ == "__main__":
>>      main()
>>
>> If I find an error in command line parameters section I cannot call function usage() because it is not defined yet.
>>
>> I have few options here:
>> 1.	Put definition of usage function before command line parameters parsing section
>> 2.	Make parameters global and put them in the main function
>> 3.	...maybe some other options...
>>
>
> I would put all of the code into a function some place.  Don't have
> anything at the top level of the file except imports, function (and
> class) definitions, and an "if __name__....." clause at the bottom.

I was about to suggest the same.  One advantage of 'write tests first' 
is that you force yourself to write testable code from the beginning, 
instead of refactoring later.

> If you need to use globals, assign them inside a parse_arguments
> function that has a "global" statement in it.

Better not to have mutable module globals if you can avoid it.  If you 
want application globals, put them in a separate module.

> As a side note, if you are going to have code at the top-level of
> the file, then there's no point in the "if __name__..." clause.
> That clause is designed to make a file both runnable and importable.
> But your top-level code makes the file very difficult to import.

And you need to import to test.

-- 
Terry Jan Reedy

[toc] | [prev] | [next] | [standalone]

#90484

On Tue, May 12, 2015 at 1:29 PM, Chris Angelico <rosuav@gmail.com> wrote:
> On Wed, May 13, 2015 at 5:13 AM,  <zljubisicmob@gmail.com> wrote:
>> If I find an error in command line parameters section I cannot call function usage() because it is not defined yet.
>>
>> I have few options here:
>> 1.      Put definition of usage function before command line parameters parsing section
>
> I'd do this, unless there's a good reason not to. A simple usage
> function probably doesn't have many dependencies, so it can logically
> go high in the code. As a general rule, I like to organize code such
> that things are defined higher up than they're used; it's not strictly
> necessary (if they're used inside functions, the requirement is only
> that they be defined before the function's called), but it helps with
> clarity. That generally means that "def usage():" wants to go up above
> any place where "usage()" occurs, but below the definitions of any
> functions that usage() itself calls, and below the first assignments
> to any global names it uses. It's not always possible, but when it is,
> it tends to produce an easy-to-navigate source file.

+1.

Also, I like to put command-line parsing inside the main function and
make that its *only* responsibility. The main function then calls the
real entry point of my script, which will be something more
specifically named. This also has the advantage that if some other
module needs to invoke my script, all it has to do is call the entry
point function which will be named something more suitable than
"main".

[toc] | [prev] | [next] | [standalone]

#90487

On Wed, May 13, 2015 at 5:54 AM, Ian Kelly <ian.g.kelly@gmail.com> wrote:
> Also, I like to put command-line parsing inside the main function and
> make that its *only* responsibility. The main function then calls the
> real entry point of my script, which will be something more
> specifically named. This also has the advantage that if some other
> module needs to invoke my script, all it has to do is call the entry
> point function which will be named something more suitable than
> "main".

That often makes sense, but sometimes doesn't. When it doesn't, you
can usually tell because your main function looks something like this:

def main():
    do_real_work(*sys.argv)
if __name__=="__main__": main()

A one-line function that's called from one place? In-line it.

if __name__ == "__main__":
    do_real_work(*sys.argv)

ChrisA

[toc] | [prev] | [next] | [standalone]

#90576

On 2015-05-13 06:07, Chris Angelico wrote:
> On Wed, May 13, 2015 at 5:54 AM, Ian Kelly <ian.g.kelly@gmail.com>
> wrote:
> > Also, I like to put command-line parsing inside the main function
> > and make that its *only* responsibility. The main function then
> > calls the real entry point of my script, which will be something
> > more specifically named. This also has the advantage that if some
> > other module needs to invoke my script, all it has to do is call
> > the entry point function which will be named something more
> > suitable than "main".
> 
> That often makes sense, but sometimes doesn't. When it doesn't, you
> can usually tell because your main function looks something like
> this:
> 
> def main():
>     do_real_work(*sys.argv)
> if __name__=="__main__": main()
> 
> A one-line function that's called from one place? In-line it.
> 
> if __name__ == "__main__":
>     do_real_work(*sys.argv)

Usually mine look something like

  def do_real_work(options, args):
    ...
  def main():
    parser = [optparse,argparse,docopt]....
    options, args = parser.parse_args()
    do_real_work(options, args)
  if __name__ == "__main__":
    main()

since my real-work function usually relies on configuration
(sometimes this also includes a config-file or environment variables
being munged into some "options" data structure).

-tkc



  

[toc] | [prev] | [next] | [standalone]

#90600

On Thu, May 14, 2015 at 4:34 AM, Tim Chase
<python.list@tim.thechases.com> wrote:
> Usually mine look something like
>
>   def do_real_work(options, args):
>     ...
>   def main():
>     parser = [optparse,argparse,docopt]....
>     options, args = parser.parse_args()
>     do_real_work(options, args)
>   if __name__ == "__main__":
>     main()
>
> since my real-work function usually relies on configuration
> (sometimes this also includes a config-file or environment variables
> being munged into some "options" data structure).
>

Sure. I rather dislike the whole duplication that that entails,
though, so I try to have the functions themselves do their own
argparse config. To that end, I put together a new project on PyPI,
but have now deprecated it in favour of Clize; the upshot is that my
main function becomes trivial again:

https://github.com/Rosuav/LetMeKnow/blob/master/letmeknow.py

@command
def await(...):
    """argparse config comes from here"""

if __name__ == "__main__":
    clize.run(commands)

So it comes and goes a bit. If there's real content in your main(),
then by all means, separate it out from do_real_work; but if the work
is all done elsewhere, not much point with main().

ChrisA

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web