Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #58001 > unrolled thread

Sharing common code between multiple scripts?

Back to article view | Back to comp.lang.python

Contents

  Sharing common code between multiple scripts? Victor Hooi <victorhooi@gmail.com> - 2013-10-29 20:58 -0700
    Re: Sharing common code between multiple scripts? Ben Finney <ben+python@benfinney.id.au> - 2013-10-30 15:31 +1100

#58001 — Sharing common code between multiple scripts?

Hi,

NB - I'm the original poster here - https://groups.google.com/d/topic/comp.lang.python/WUuRLEXJP4E/discussion - however, that post seems to have diverted, and I suspect my original question was poorly worded.

I have several Python scripts that use similar functions.

Currently, these functions are duplicated in each script.

These functions wrap things like connecting to databases, reading in config files, writing to CSV etc.

I'd like to pull them out, and move them to a common module for all the scripts to import.

Originally, I thought I'd create a package, and have it all work:

my_package
    __init__.py
    common/
        my_functions.py
    script1/
        __init__.py
        config.yaml
        script1.py
    script2/
        __init__.py
        config.yaml
        script2.py

However, there apparently isn't an easy way to have script1.py and script2.py import from common/my_functions.py.

So my new question is - what is the idiomatic way to structure this in Python, and easily share common functions between the scripts?

Ideally, I'd like to avoid having everything in a single directory - i.e. script1.py should be in it's own directory, as it has it's own config and other auxiliary files. However, if this is a bad idea, let me know.

Also, say I have a class in script1.py, and I want it pull in a common method as well. For example, I want multiples classes to have the following method:

    def gzip_csv_file(self):
        self.gzip_filename = '%s.gz' % self.output_csv
        with open(self.output_csv, 'rb') as uncompressed:
            with gzip.open(self.gzip_filename, 'wb') as compressed:
                compressed.writelines(uncompressed)

        self.logger.debug('Compressed to %s GZIP file.' % humansize(os.path.getsize(self.gzip_filename)))

How could I share this? Mixins? Or is there something better?

Cheers,
Victor

[toc] | [next] | [standalone]

#58003

Victor Hooi <victorhooi@gmail.com> writes:

> NB - I'm the original poster here - https://groups.google.com/d/topic/[…]

That is not the correct URL to a discussion on this forum. The official
archives are at <URL:https://mail.python.org/pipermail/python-list/>, so
that's the correct place to look for a canonical URL to your message.

> I'd like to pull them out, and move them to a common module for all
> the scripts to import.

Great! This is modular programming, and is good practice.

> Originally, I thought I'd create a package, and have it all work:
>
> my_package
>     __init__.py
>     common/
>         my_functions.py

You should make ‘common/’ a package directory, by creating
‘common/__init__.py’.

>     script1/
>         __init__.py
>         config.yaml
>         script1.py
>     script2/
>         __init__.py
>         config.yaml
>         script2.py
>
> However, there apparently isn't an easy way to have script1.py and
> script2.py import from common/my_functions.py.

Once ‘common/’ is a package directory, you can::

    from ..common import my_functions

> So my new question is - what is the idiomatic way to structure this in
> Python, and easily share common functions between the scripts?

Put your modules into one or more packages.

Make sure each subdirectory of modules is a package.

Use explicit relative imports within your application.

Use absolute imports for shared libraries (ones shared between different
applications).

-- 
 \          “Any sufficiently advanced bug is indistinguishable from a |
  `\                                          feature.” —Rich Kulawiec |
_o__)                                                                  |
Ben Finney

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web