Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > comp.lang.python > #80805 > unrolled thread

Question about installing python and modules on Red Hat Linux 6

Back to article view | Back to comp.lang.python

Contents

  Question about installing python and modules on Red Hat Linux 6 pythonista <software.by.python@gmail.com> - 2014-11-14 19:01 -0800
    Re: Question about installing python and modules on Red Hat Linux 6 Chris Angelico <rosuav@gmail.com> - 2014-11-15 16:09 +1100
    Re: Question about installing python and modules on Red Hat Linux 6 Cameron Simpson <cs@zip.com.au> - 2014-11-15 16:11 +1100
    Re: Question about installing python and modules on Red Hat Linux 6 Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-16 02:15 +1100
      Re: Question about installing python and modules on Red Hat Linux 6 Grant Edwards <invalid@invalid.invalid> - 2014-11-15 15:45 +0000
        Re: Question about installing python and modules on Red Hat Linux 6 Steven D'Aprano <steve+comp.lang.python@pearwood.info> - 2014-11-16 12:08 +1100
          Re: Question about installing python and modules on Red Hat Linux 6 Chris Angelico <rosuav@gmail.com> - 2014-11-16 12:32 +1100
          Re: Question about installing python and modules on Red Hat Linux 6 Michael Torrie <torriem@gmail.com> - 2014-11-15 18:57 -0700
          Re: Question about installing python and modules on Red Hat Linux 6 Chris Angelico <rosuav@gmail.com> - 2014-11-16 13:23 +1100
          Re: Question about installing python and modules on Red Hat Linux 6 Grant Edwards <invalid@invalid.invalid> - 2014-11-16 16:28 +0000
            Re: Question about installing python and modules on Red Hat Linux 6 Chris Angelico <rosuav@gmail.com> - 2014-11-17 09:16 +1100
      Re: Question about installing python and modules on Red Hat Linux 6 Michael Torrie <torriem@gmail.com> - 2014-11-15 10:05 -0700
    Re: Question about installing python and modules on Red Hat Linux 6 Michael Torrie <torriem@gmail.com> - 2014-11-15 10:02 -0700
    Re: Question about installing python and modules on Red Hat Linux 6 Michael Torrie <torriem@gmail.com> - 2014-11-15 10:13 -0700
    Re: Question about installing python and modules on Red Hat Linux 6 Michael Torrie <torriem@gmail.com> - 2014-11-15 10:23 -0700

#80805 — Question about installing python and modules on Red Hat Linux 6

I am developing a python application as a contractor.

I would like to know if someone can provide me with some insight into the problems that then infrastructure team has been having.

The scope of the project was to install python 2.7.8 and 4 modules/site packages on a fresh linux build.

The first team failed after almost 3 weeks of work.

Then  they put their star Linux administrator on the task and it took almost a week of considerable effort.

I was able to perform the same task on my windows desktop in less than a morning.

I cannot get a straight answer as to what the problems are.

They just seemed to be building and rebuilding all of the components from scratch.

Can anyone provide me with insight as to the scope what the problem could have been?

Thanks

[toc] | [next] | [standalone]

#80810

On Sat, Nov 15, 2014 at 2:01 PM, pythonista
<software.by.python@gmail.com> wrote:
> The scope of the project was to install python 2.7.8 and 4 modules/site packages on a fresh linux build.
>
> The first team failed after almost 3 weeks of work.
>
> Then  they put their star Linux administrator on the task and it took almost a week of considerable effort.
>
> I was able to perform the same task on my windows desktop in less than a morning.
>
> I cannot get a straight answer as to what the problems are.

Well, you're asking the wrong people. The crystal ball manufacturer
around here is horribly shoddy, shipping us faulty optics and devices
with frequent outages. But if you ask the people who did the work,
they should be able to tell you what was happening.

My suspicion here is that the admins were trying to use multiple
different installation methods, and they ran into issues. This
currently can't happen on Windows, as there simply isn't a
system-provided package manager (though that may be changing - soon
Windows people will have all the same advantages AND disadvantages
that everyone else has!), hence you perhaps found it easier. On my
Debian systems, I can use "apt-get install python python-bs4
python-django" and grab a consistent set of packages from the
OS-provided repositories; or alternatively, I can use pip to get
things from the Python-provided repository (PyPI). Trying to use both
can cause issues.

Want an easy solution? Get a recent Python (preferably 3.x, but
otherwise the latest builds of 2.7 should do I think), get pip, get
virtualenv, and then do everything inside a venv with "pip install"
(maybe using requirements.txt to save you the hassle of keying
everything in). You'll find that that "just works", pretty much all
the time. It's perhaps more work than you truly need (I personally
don't usually bother with a venv), but that way you can be sure you're
not trampling all over anything else.

ChrisA

[toc] | [prev] | [next] | [standalone]

#80811

On 14Nov2014 19:01, pythonista <software.by.python@gmail.com> wrote:
>I am developing a python application as a contractor.
>I would like to know if someone can provide me with some insight into the problems that then infrastructure team has been having.
>The scope of the project was to install python 2.7.8 and 4 modules/site packages on a fresh linux build.
>The first team failed after almost 3 weeks of work.
>Then  they put their star Linux administrator on the task and it took almost a week of considerable effort.
>I was able to perform the same task on my windows desktop in less than a morning.
>I cannot get a straight answer as to what the problems are.
>They just seemed to be building and rebuilding all of the components from scratch.
>Can anyone provide me with insight as to the scope what the problem could have been?

It depends. If they were trying to interoperate with the vendor supplied 
Python, they could be fighting with the RPM system.

When I do this these days I do two things:

  - fetch and build Python 2.7.8 from source, to install in /usr/local/python-2.7.8 (i.e.  use that as the --prefix option for configure).

At this point you could just proceed and use that python, installing inside its 
site-packages as usual. Or you could go a little further and make a virtualenv 
for your project:

  - fetch virtualenv and (blasphemy!) install it in the vendor space using: "python setup.py install"

  - create a virtualenv for your project: "/usr/local/python-2.7.8/bin/python2.7 virtualenv.py /usr/local/venv-2.7.8-your-project-name"

At that point you will have a "bin" directory inside 
/usr/local/venv-2.7.8-your-project-name with "python" and so forth.  Executing 
that "python" runs the python-2.7.8 you built earlier but with all the system 
paths set to install and use the lib tree inside the virtualenv directory.

The nice thing about that is that it is easy to make multiple virtualenvs, 
using distinct collections of additional packages, nicely isolated and easy to 
invoke (just run the "python" from within the appropriate virtualenv tree).

Cheers,
Cameron Simpson <cs@zip.com.au>

I just kept it wide-open thinking it would correct itself.
Then I ran out of talent.       - C. Fittipaldi

[toc] | [prev] | [next] | [standalone]

#80825

pythonista wrote:

> I am developing a python application as a contractor.
> 
> I would like to know if someone can provide me with some insight into the
> problems that then infrastructure team has been having.
> 
> The scope of the project was to install python 2.7.8 and 4 modules/site
> packages on a fresh linux build.

A "fresh linux build" of Red Hat Linux 6? RHL 6 was discontinued in 2000.
That's *at least* 14 years old. Why on earth are you using something so old
instead of a recent version of RHEL, Centos or Fedora?

The supported version of Python for RHL 6 was Python 1.5. (That's *one*
point five.) I'm not surprised that they had trouble installing Python 2.7
on something that old, especially if you needed support for optional but
important features like tkinter, ssh, readline, curses, etc.

I haven't tried it, so it is possible that installing 2.7 from source on
such an old Linux system will actually be trivially easy and need only half
an hour's work. But I wouldn't bet on it. I would expect all sorts of
difficulties, due to the age difference between RHL 6 and the Python 2.7. 

As for the four modules, that depends. If they are extension modules written
in C, who knows how hard it will be to get them working under RHL 6. And
even if they are pure Python modules, depending on how old they are, they
may be difficult to get working with something as new as Python 2.7.

Without more details of what the modules are and what errors the
infrastructure team experienced, there is no way to tell whether four weeks
to get this working was a heroic effort or a sign of utter incompetence.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#80828

On 2014-11-15, Steven D'Aprano <steve+comp.lang.python@pearwood.info> wrote:
> pythonista wrote:
>
>> I am developing a python application as a contractor.
>> 
>> I would like to know if someone can provide me with some insight into the
>> problems that then infrastructure team has been having.
>> 
>> The scope of the project was to install python 2.7.8 and 4 modules/site
>> packages on a fresh linux build.
>
> A "fresh linux build" of Red Hat Linux 6? RHL 6 was discontinued in 2000.
> That's *at least* 14 years old. Why on earth are you using something so old
> instead of a recent version of RHEL, Centos or Fedora?

I'm sure the OP meant RHEL 6, and not RH 6 [yes, I realize you know
that two and are just making a point about how it pays to include
accurate info when asking for help.]

The OP probably doesn't even _know_ that there was prior product line
called RedHat Linux with a version 6.  [Kids these days!]  IIRC, I
started with one of the "holiday" RedHat Linux releases before they
were numbered.  I think it was "Mothers Day" or "Holloween".  The
first numbered release I remember was RedHat Linux 3.something.  RHL
was pretty good up through the 6.x series, but 7.00 was utterly and
famously shoddy [at that point I switched to Mandrake].  RH 7.00 was
so notoriously bad, I'm surprised that they didn't skip "7" entirely
in the RHEL product line to avoid the bad memories...

-- 
Grant

[toc] | [prev] | [next] | [standalone]

#80842

Grant Edwards wrote:

> On 2014-11-15, Steven D'Aprano <steve+comp.lang.python@pearwood.info>
> wrote:
>> pythonista wrote:
>>
>>> I am developing a python application as a contractor.
>>> 
>>> I would like to know if someone can provide me with some insight into
>>> the problems that then infrastructure team has been having.
>>> 
>>> The scope of the project was to install python 2.7.8 and 4 modules/site
>>> packages on a fresh linux build.
>>
>> A "fresh linux build" of Red Hat Linux 6? RHL 6 was discontinued in 2000.
>> That's *at least* 14 years old. Why on earth are you using something so
>> old instead of a recent version of RHEL, Centos or Fedora?
> 
> I'm sure the OP meant RHEL 6, and not RH 6 [yes, I realize you know
> that two and are just making a point about how it pays to include
> accurate info when asking for help.]

Actually, no, the thought didn't even cross my mind. I just assumed that if
somebody is going to cast aspersions on the professionalism of others,
they'd get their facts right. If they said RHL 6, they meant RHL 6 and not
Centos 5 or Fedora 20 or Debian Squeeze. But I suppose that you're probably
right. In hindsight, given that the OP is a Windows guy and not a Linux
user, writing Red Hat Linux for Red Hat Enterprise Linux is an easy mistake
to make.

Assuming it was RHEL 6, then installing Python 2.7 from source as a separate
application from the system Python should be trivially easy, half an hour's
work. Download the source, untar, run ./configure, make, make altinstall
and you should be done. There may be a few bumps in the road to get
optional components supported, in which case a skilled Linux admin (which I
am not) might need perhaps a couple of hours. Depending on just how bad the
bumps were, an unskilled admin like me might take a day or so, not three
weeks, before giving up.

The most obvious trap is to run `make install` instead of `make altinstall`,
in which case congratulations, you've just broken the RHEL install, and why
didn't you read the README first? You can recover from it, probably, by
fixing a few sym links, or by re-installing the system Python using the
package manager. Worst case you just reinstall the whole OS. Two or three
days, tops, not four weeks.

If some foolish person insisted on upgrading the system python to 2.7
instead of installing a parallel installation, then the sky is the limit. I
cannot imagine how much effort that would take, or how fragile it would be.
Weeks? Probably. And then the first time the admin runs the package manager
to install updates, things could start breaking.

One thing which the OP hasn't told us, how much of the four weeks was
effort, as opposed to elapsed time. For all we know, it took them four
weeks to install this because for three weeks, four days and seven hours
they were doing something else.

In any case, if the OP has been billed for this time, I would insist on
justification for why it took so long before paying.



-- 
Steven

[toc] | [prev] | [next] | [standalone]

#80844

On Sun, Nov 16, 2014 at 12:08 PM, Steven D'Aprano
<steve+comp.lang.python@pearwood.info> wrote:
> Assuming it was RHEL 6, then installing Python 2.7 from source as a separate
> application from the system Python should be trivially easy, half an hour's
> work. Download the source, untar, run ./configure, make, make altinstall
> and you should be done. There may be a few bumps in the road to get
> optional components supported, in which case a skilled Linux admin (which I
> am not) might need perhaps a couple of hours. Depending on just how bad the
> bumps were, an unskilled admin like me might take a day or so, not three
> weeks, before giving up.

For a competent Linux system admin, this should be second nature.
Installing from source should be as normal an operation as writing a
shell script or scheduling a job with cron. But if the people
concerned aren't so much "POSIX system admins" as "RHEL admins", then
it's possible they depend entirely on the upstream repositories, and
aren't familiar with the dance of "can't find -lfoo so go install
libfoo-dev" (with occasionally a more exotic step, when the package
name isn't obvious, but Google helps there); in that case, it could
well take a long time, but that's like a Python programmer who's
having trouble debugging an asyncio program because s/he isn't used to
walking through the control flow of "yield from". I know I'm not
competent at that, and it doesn't stop me from being a programmer -
but I'm not going to come to python-list saying "This language sucks,
I can't find this bug", because the problem is a limitation in my own
skills.

Other RHEL people: Is there a yum equivalent to "apt-get build-dep",
which goes out and fetches all the compilers, libraries, build tools,
etc, needed to build a package from source? If so - and I wouldn't be
at all surprised if there is - that would be my recommended first
step: grab the build deps for Python 2.6 and use those to build 2.7.
Chances are that's all you need - that, plus the one little trick of
"make altinstall" rather than "make install".

ChrisA

[toc] | [prev] | [next] | [standalone]

#80845

On 11/15/2014 06:08 PM, Steven D'Aprano wrote:
> Assuming it was RHEL 6, then installing Python 2.7 from source as a separate
> application from the system Python should be trivially easy, half an hour's
> work. Download the source, untar, run ./configure, make, make altinstall
> and you should be done. There may be a few bumps in the road to get
> optional components supported, in which case a skilled Linux admin (which I
> am not) might need perhaps a couple of hours. Depending on just how bad the
> bumps were, an unskilled admin like me might take a day or so, not three
> weeks, before giving up.

In my last system administration job, we forbade installing from source,
at least in the manner you are describing.  It's a maintenance
nightmare.  Especially when it comes time to upgrade the system and get
things up and running on a new OS version.  To make the system
maintainable and re-creatable (it's easy to dump a list of installed
packages to install on anther machine), it had to be done using the
package manager.  Preferably with trusted packages (trusted
repositories) that were actively maintained.  At that time Red Hat
software collections didn't exist, so I did have to spend some
considerable time building and testing RPM packages.  Yes it's a
headache for the developer in the short term, but in the long term it
always turned out better than hacking things together from source.

The OP hasn't said that this is the case for his client of course.

[toc] | [prev] | [next] | [standalone]

#80846

On Sun, Nov 16, 2014 at 12:57 PM, Michael Torrie <torriem@gmail.com> wrote:
> In my last system administration job, we forbade installing from source,
> at least in the manner you are describing.  It's a maintenance
> nightmare.  Especially when it comes time to upgrade the system and get
> things up and running on a new OS version.  To make the system
> maintainable and re-creatable (it's easy to dump a list of installed
> packages to install on anther machine), it had to be done using the
> package manager.  Preferably with trusted packages (trusted
> repositories) that were actively maintained.  At that time Red Hat
> software collections didn't exist, so I did have to spend some
> considerable time building and testing RPM packages.  Yes it's a
> headache for the developer in the short term, but in the long term it
> always turned out better than hacking things together from source.
>

Fundamentally, this comes down to a single question: Who do you most trust?

1) Upstream repositories? Then install everything from the provided
package manager. All will be easy; as long as nothing you use got
removed in the version upgrade, you should be able to just grab all
the same packages and expect everything to run. This is what I'd
recommend for most end users.

2) Yourself? Then install stuff from source. For a competent sysadmin
on his/her own computer, this is what I'd recommend. Use the repos
when you can, but install anything from source if you feel like it. I
do this for a number of packages where Debian provides an oldish
version, or where I want to tweak something, or anything like that.
When you upgrade to a new OS version, it's time to reconsider all
those decisions; maybe there's a newer version in repo than the one
you built from source a while ago.

3) A local repository? Then do what you describe above - build and
test the RPMs and don't install anything from source. If you need
something that isn't in upstream, you compile it, package it, and
deploy it locally. Great if you have hundreds or thousands of similar
machines to manage, eliminates the maintenance nightmare, but is
unnecessary work if you have only a handful of machines and they're
all unique.

I'm responsible for maybe a dozen actively-used Linux boxes, so I'm a
pretty small-time admin. Plus, they run a variety of different
hardware, OS releases (they're mostly Debian-family distros, but I
have some Ubuntu, some Debian, one AntiX, and various others around
the place - and a variety of versions as well), and application
software (different needs, different stuff installed). Some are
headless servers that exist solely for the network. Others are
people's clients that they actively use every day. They don't all need
Wine, VirtualBox, DOSBox, or other compatibility/emulation layers; but
some need versions newer than those provided in the upsstream repos.
One box needs audio drivers compiled from source, else there's no
sound. Until a few months ago, several - but not all - needed a few
local patches to one program. (Then the patches got accepted upstream,
but that version isn't in the Debian repos yet.) And of course, they
all need their various configs - a web server is not created by "sudo
apt-get install apache2" so much as by editing /etc/apache2/*. Sure, I
*could* run everything through deb/rpm packages, but I'd be constantly
tweaking and packaging and managing upgrades, and it's much better use
of my time to just deploy from source. With so few computers to
manage, the Big-Oh benefits of your recommendation just don't kick in.

But I doubt these considerations apply to the OP. Of course, we can't
know until our crystal balls get some more to work on.

ChrisA

[toc] | [prev] | [next] | [standalone]

#80885

On 2014-11-16, Steven D'Aprano <steve+comp.lang.python@pearwood.info> wrote:
> Grant Edwards wrote:
>
>> On 2014-11-15, Steven D'Aprano <steve+comp.lang.python@pearwood.info>
>> wrote:
>>> pythonista wrote:
>>>
>>>> I am developing a python application as a contractor.
>>>> 
>>>> I would like to know if someone can provide me with some insight
>>>> into the problems that then infrastructure team has been having.
>>>> 
>>>> The scope of the project was to install python 2.7.8 and 4
>>>> modules/site packages on a fresh linux build.
>>>
>>> A "fresh linux build" of Red Hat Linux 6? RHL 6 was discontinued
>>> in 2000.  That's *at least* 14 years old. Why on earth are you
>>> using something so old instead of a recent version of RHEL, Centos
>>> or Fedora?
>> 
>> I'm sure the OP meant RHEL 6, and not RH 6 [yes, I realize you know
>> that two and are just making a point about how it pays to include
>> accurate info when asking for help.]
>
> Actually, no, the thought didn't even cross my mind. I just assumed
> that if somebody is going to cast aspersions on the professionalism
> of others, they'd get their facts right.

Nah, this is Usenet (from my POV).  Or Google Groups.  Or possibly a
mailing list.  In any case, casting aspersions is often done with
without regard to or even possesion of the facts.  :)

> Assuming it was RHEL 6, then installing Python 2.7 from source as a
> separate application from the system Python should be trivially
> easy, half an hour's work.

Yep.

> Download the source, untar, run ./configure, make, make altinstall
> and you should be done.

I think even "make install" should have worked -- since it will
install in /usr/local, right?

> There may be a few bumps in the road to get optional components
> supported, in which case a skilled Linux admin (which I am not)
> might need perhaps a couple of hours. Depending on just how bad the
> bumps were, an unskilled admin like me might take a day or so, not
> three weeks, before giving up.

All that should be involved is installing the "-devel" versions of a
few libraries -- exactly which ones depend on what features of Python
and the standard libarary you care about.  A few minutes with Google
would have ironed out any questions in that regard.

> The most obvious trap is to run `make install` instead of `make
> altinstall`, in which case congratulations, you've just broken the
> RHEL install, and why didn't you read the README first?

Doesn't 'make install' still install in /usr/local?  (I haven't
manually built built from sources for some time.)  If that's the case,
it shouldn't interfear with the installation in /usr.

But, you're right: anybody who spent more than a half hour on this is
either a flaming incompetent or a scam artist.

-- 
Grant

[toc] | [prev] | [next] | [standalone]

#80924

On Mon, Nov 17, 2014 at 3:28 AM, Grant Edwards <invalid@invalid.invalid> wrote:
> But, you're right: anybody who spent more than a half hour on this is
> either a flaming incompetent or a scam artist.

Half an hour of human time, maybe, but potentially spread across a few
hours of wall time. Building Python from source can take quite a
while, and if you then go back and get some more dev libraries and
start over, it might take a good while to get the job done. Let's be a
bit more generous and say a day. But beyond that, one must know one's
limits: it's better to give up after a day (or half a day, preferably)
and go ask for help, rather than struggle for weeks.

ChrisA

[toc] | [prev] | [next] | [standalone]

#80830

On 11/15/2014 08:15 AM, Steven D'Aprano wrote:
> A "fresh linux build" of Red Hat Linux 6? RHL 6 was discontinued in 2000.

Yes I know you're making a point about not assuming anything, but the
odds are very good that the OP meant RHEL6.

And meaning RHEL6, there are some good reasons why the infrastructure
team struggled, since Python 2.6 is a core system dependency and cannot
easily be overridden without breaking the entire system.  The answer is
either use Redhat Software Collections[1], or compile from scratch to /opt.

[1] https://www.softwarecollections.org/en/scls/rhscl/python27/

[toc] | [prev] | [next] | [standalone]

#80829

On 11/14/2014 08:01 PM, pythonista wrote:
> Can anyone provide me with insight as to the scope what the problem could have been?

Well the fact is that RHEL 6 uses Python 2.6 as a core system package.
Many system utilities depend on it, so it cannot be replaced with a
newer version.  You must install the newer version alongside the old
version.  This can be easily accomplished with Redhat's Software
Collections system.  I have Python 2.7 and 3.4 installed on my RHEL6 box
with this machanism.

[toc] | [prev] | [next] | [standalone]

#80832

On 11/14/2014 08:01 PM, pythonista wrote:
> The scope of the project was to install python 2.7.8 and 4 modules/site packages on a fresh linux build.

I neglected to put the URL for software collections in my reply to you.
Here it is.

https://www.softwarecollections.org/en/scls/rhscl/python27/

Note that the infrastructure team will have to learn how to interface
with it, since, as a software collection, it's not going to be in the
path by default (because it would conflict).  Any startup scripts or
other invocations of your app will have to be modified to acquire the
python 2.7 special environment, by sourcing the python 2.7 collection's
enable file.  I'm not entirely sure how to integrate it with Apache, but
I know it can be done.

If it's a package that won't conflict, such as Python 2.4, you can
permanently integrate it into the environment this way:
http://developerblog.redhat.com/2014/03/19/permanently-enable-a-software-collection/
-- but I can't recommend this for python 2.7 as it would break a lot of
RHEL 6's commands.

[toc] | [prev] | [next] | [standalone]

#80833

On 11/15/2014 10:13 AM, Michael Torrie wrote:
> If it's a package that won't conflict, such as Python 2.4, you can
Ahem, that should have been 3.4

[toc] | [prev] | [standalone]

Back to top | Article view | comp.lang.python

csiph-web