Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.maint.python > #9295 > unrolled thread

Adopting OpenStack packages

Back to article view | Back to linux.debian.maint.python

Contents

  Adopting OpenStack packages Barry Warsaw <barry@debian.org> - 2017-02-28 21:50 +0100
    Re: Adopting OpenStack packages Thomas Goirand <zigo@debian.org> - 2017-03-03 14:10 +0100
      Re: Adopting OpenStack packages Allison Randal <allison@lohutok.net> - 2017-03-03 16:10 +0100
        Re: Adopting OpenStack packages Thomas Goirand <zigo@debian.org> - 2017-03-04 04:40 +0100
          Re: Adopting OpenStack packages Brian May <bam@debian.org> - 2017-03-04 05:20 +0100
            Re: Adopting OpenStack packages Thomas Goirand <zigo@debian.org> - 2017-03-05 01:50 +0100
              Re: Adopting OpenStack packages Barry Warsaw <barry@debian.org> - 2017-03-06 00:20 +0100
                Re: Adopting OpenStack packages Thomas Goirand <zigo@debian.org> - 2017-03-06 01:00 +0100
                  Re: Adopting OpenStack packages Brian May <brian@linuxpenguins.xyz> - 2017-03-06 06:40 +0100
                    Re: Adopting OpenStack packages Vincent Bernat <bernat@debian.org> - 2017-03-06 07:40 +0100
                      Re: Adopting OpenStack packages Brian May <bam@debian.org> - 2017-03-06 09:20 +0100
                    Re: Adopting OpenStack packages Clint Byrum <spamaps@debian.org> - 2017-03-06 16:30 +0100
                      Re: Adopting OpenStack packages Scott Kitterman <debian@kitterman.com> - 2017-03-06 16:40 +0100
                        Re: Adopting OpenStack packages Barry Warsaw <barry@debian.org> - 2017-03-06 17:00 +0100
                        Re: Adopting OpenStack packages Simon McVittie <smcv@debian.org> - 2017-03-06 17:50 +0100
                          Transition away from git-dpm was: Re: Adopting OpenStack packages Scott Kitterman <scott@kitterman.com> - 2017-03-06 18:20 +0100
                            Re: Transition away from git-dpm was: Re: Adopting OpenStack packages Brian May <bam@debian.org> - 2017-03-06 21:30 +0100
                              Re: Transition away from git-dpm was: Re: Adopting OpenStack packages Thomas Goirand <zigo@debian.org> - 2017-03-06 22:50 +0100
                                Re: Transition away from git-dpm was: Re: Adopting OpenStack packages Brian May <brian@linuxpenguins.xyz> - 2017-03-06 23:20 +0100
                                  Re: Transition away from git-dpm was: Re: Adopting OpenStack packages Thomas Goirand <zigo@debian.org> - 2017-03-07 23:20 +0100
                                    Re: Transition away from git-dpm was: Re: Adopting OpenStack packages Brian May <bam@debian.org> - 2017-03-08 07:50 +0100
                                      Re: Transition away from git-dpm was: Re: Adopting OpenStack packages Simon McVittie <smcv@debian.org> - 2017-03-08 09:20 +0100
                          Re: Adopting OpenStack packages Brian May <bam@debian.org> - 2017-03-08 10:50 +0100
                Re: Adopting OpenStack packages Brian May <brian@linuxpenguins.xyz> - 2017-03-06 06:50 +0100
          Re: Adopting OpenStack packages Scott Kitterman <debian@kitterman.com> - 2017-03-04 06:10 +0100
            Re: Adopting OpenStack packages Thomas Goirand <zigo@debian.org> - 2017-03-05 00:50 +0100
              Re: Adopting OpenStack packages Scott Kitterman <debian@kitterman.com> - 2017-03-05 01:20 +0100
                Re: Adopting OpenStack packages Thomas Goirand <zigo@debian.org> - 2017-03-05 23:10 +0100
                  Re: Adopting OpenStack packages Scott Kitterman <debian@kitterman.com> - 2017-03-06 02:40 +0100
          Re: Adopting OpenStack packages Clint Byrum <clint@fewbar.com> - 2017-03-04 07:10 +0100
            Re: Adopting OpenStack packages Vincent Bernat <bernat@debian.org> - 2017-03-04 10:50 +0100
              Re: Adopting OpenStack packages Scott Kitterman <debian@kitterman.com> - 2017-03-04 16:10 +0100
                Re: Adopting OpenStack packages Vincent Bernat <bernat@debian.org> - 2017-03-04 21:30 +0100
                  Re: Adopting OpenStack packages Scott Kitterman <debian@kitterman.com> - 2017-03-05 01:00 +0100
                    Re: Adopting OpenStack packages Vincent Bernat <bernat@debian.org> - 2017-03-05 06:40 +0100
                      Re: Adopting OpenStack packages Scott Kitterman <debian@kitterman.com> - 2017-03-05 07:20 +0100
                Re: Adopting OpenStack packages Thomas Goirand <zigo@debian.org> - 2017-03-05 01:30 +0100
                  Re: Adopting OpenStack packages Scott Kitterman <debian@kitterman.com> - 2017-03-05 01:50 +0100
                    Re: Adopting OpenStack packages Thomas Goirand <zigo@debian.org> - 2017-03-05 23:00 +0100
            Re: Adopting OpenStack packages Thomas Goirand <zigo@debian.org> - 2017-03-05 04:00 +0100
              Re: Adopting OpenStack packages Allison Randal <allison@lohutok.net> - 2017-03-05 18:20 +0100
                Re: Adopting OpenStack packages Ondrej Novy <novy@ondrej.org> - 2017-03-05 21:20 +0100
                  Re: Adopting OpenStack packages Scott Kitterman <debian@kitterman.com> - 2017-03-05 22:10 +0100
                Re: Adopting OpenStack packages Thomas Goirand <zigo@debian.org> - 2017-03-05 22:50 +0100

Page 1 of 3  [1] 2 3  Next page →

#9295 — Adopting OpenStack packages

[Multipart message — attachments visible in raw view] — view raw

We've talked on various lists about adopting the OpenStack packages into DPMT,
and also adopting the team's standard workflows and helpers.

The way the packages have been maintained in the past isn't aligned with our
team practices, but Allison and I spent a little time today importing alembic
into DPMT git.  After looking at the existing git repo (i.e. what you get with
debcheckout), we both figured it would be better to start fresh, so I used my
import-dscs.py script[1] to import all previous releases into a git-dpm repo
(via debsnap).  Then I pushed that repo to DPMT git.

$ git clone git+ssh://git.debian.org/git/python-modules/packages/alembic.git

That looks like it matches what's in unstable/testing right now.  The package
doesn't use pybuild, and both Allison and I want to convert it to our standard
build system, but before I go hacking around in d/rules, I want to get a
sanity check on the conversion process.  Please poke around and let me know
what you think.  If the results of import-dscs.py look good enough, then the
thinking is that we'll use that to convert the rest of the packages.  If it's
not good enough, then let's work out a better conversion process before we
import more packages.

Things to not worry about, IMHO:

* Anything other than making the package maintainable in git-dpm
* Any commits to the existing git repo that weren't uploaded

I don't think there's a huge urgency here, since I'm not aware of any packages
that need updates before Stretch is released.  (Allison, please confirm.)
E.g. alembic is at 0.9.0 upstream, but it'll stay at 0.8.8 for Stretch.
Still, we'd like to have everything in place once the release happens and
freeze is lifted.

-Barry

[1] git clone https://anonscm.debian.org/git/users/barry/import-dscs.git

[toc] | [next] | [standalone]

#9303

On 02/28/2017 09:46 PM, Barry Warsaw wrote:
> We've talked on various lists about adopting the OpenStack packages into DPMT,
> and also adopting the team's standard workflows and helpers.
> 
> The way the packages have been maintained in the past isn't aligned with our
> team practices, but Allison and I spent a little time today importing alembic
> into DPMT git.  After looking at the existing git repo (i.e. what you get with
> debcheckout), we both figured it would be better to start fresh, so I used my
> import-dscs.py script[1] to import all previous releases into a git-dpm repo
> (via debsnap).  Then I pushed that repo to DPMT git.
> 
> $ git clone git+ssh://git.debian.org/git/python-modules/packages/alembic.git
> 
> That looks like it matches what's in unstable/testing right now.  The package
> doesn't use pybuild, and both Allison and I want to convert it to our standard
> build system, but before I go hacking around in d/rules, I want to get a
> sanity check on the conversion process.  Please poke around and let me know
> what you think.  If the results of import-dscs.py look good enough, then the
> thinking is that we'll use that to convert the rest of the packages.  If it's
> not good enough, then let's work out a better conversion process before we
> import more packages.
> 
> Things to not worry about, IMHO:
> 
> * Anything other than making the package maintainable in git-dpm
> * Any commits to the existing git repo that weren't uploaded
> 
> I don't think there's a huge urgency here, since I'm not aware of any packages
> that need updates before Stretch is released.  (Allison, please confirm.)
> E.g. alembic is at 0.9.0 upstream, but it'll stay at 0.8.8 for Stretch.
> Still, we'd like to have everything in place once the release happens and
> freeze is lifted.
> 
> -Barry

Could you please put this on hold? It's possible that I resume my work
on OpenStack packages (though I can't disclose anything on this yet).

Cheers,

Thomas Goirand (zigo)

[toc] | [prev] | [next] | [standalone]

#9306

On 03/03/2017 08:01 AM, Thomas Goirand wrote:
> Could you please put this on hold? It's possible that I resume my work
> on OpenStack packages (though I can't disclose anything on this yet).

Hi Thomas,

I appreciate your preferences, I really do. But the past few months have
been a pretty harsh reminder that a bus factor of one is a dangerous
place to be. OpenStack isn't the only project that depends on these
Python modules, and it's bad for Debian if maintenance drops every time
you're between jobs. You need to let go, and let us help you.
Maintaining general Python dependencies is what DPMT is here for. I
promise we will be mindful of OpenStack's needs.

Allison

[toc] | [prev] | [next] | [standalone]

#9307

On 03/03/2017 04:09 PM, Allison Randal wrote:
> On 03/03/2017 08:01 AM, Thomas Goirand wrote:
>> Could you please put this on hold? It's possible that I resume my work
>> on OpenStack packages (though I can't disclose anything on this yet).
> 
> Hi Thomas,
> 
> I appreciate your preferences, I really do. But the past few months have
> been a pretty harsh reminder that a bus factor of one is a dangerous
> place to be. OpenStack isn't the only project that depends on these
> Python modules, and it's bad for Debian if maintenance drops every time
> you're between jobs. You need to let go, and let us help you.
> Maintaining general Python dependencies is what DPMT is here for. I
> promise we will be mindful of OpenStack's needs.
> 
> Allison

Allison,

I do appreciate and welcome help. But this would IMO not help. Here's why.

1/ The OpenStack team is as much open as the DPMT. Anyone can join. Even
better, you don't need to join, you can just send patches to Gerrit. And
any DD/DM can upload. The Gerrit CI/CD is far more advanced than the Git
on Alioth. Moving back to Alioth is a regression in may ways (safety of
data, access rights vs gerrit, workflow effectiveness, automated
testing, etc.). And I'm not even addressing yet the horrible git-dpm
troubles, how many more years the team is forcibly burying every
contributor into. Plus Alioth is a security nightmare, and it's loaded
so much that even a simple git push can take hours (real life experience).

2/ Stretch is frozen, and during the freeze, I continued maintaining and
closing RC bugs. I will continue to do so. I never wrote I would stop
caring for Debian packages or Debian in general.

I wrote I cannot continue maintaining OpenStack on my free time the way
I did when I was full time, and I did so after seeing that nobody worked
on the Ocata release. But there's no meaningful RC bugs open right now,
and there's never been any for more than a few days on all the OpenStack
packages (ie: Newton release). I don't think it is giving me justice to
say I stopped caring, and that it caused trouble to Debian, the DPMT, or
any Python module/app maintainer. It did not (at least so far).

3/ The history of critical OpenStack dependencies not maintained in the
team shows the exact opposite thing that you are promising.

SQLAlchemy and it's half finished transition during the e freeze (1.1.5
is in Sid and haven't migrated, forcing some not-needed potentially
hand-crafted dangerous SQLA-related patches in OpenStack Newton for
Stretch) is a very good example of the maintainer being completely
careless of both the freeze of Stretch and OpenStack. I raised the topic
in the release list, the maintainer didn't even care to give meaningful
answers to valid points of argumentation (unless I raised the issue to a
tech-ctte bug), nobody seemed to care enough, and I didn't find enough
energy to fight yet another battle with the maintainer. The only option
given by the maintainer (ie: fight it through the tech-comittee) would
have been a waste of time for a lot of people, and probably would have
drained a lot of my energy at a moment were I didn't had much to spare.
So I gave up.

Django maintainers haven't been very careful either, giving weeks
instead of the needed months for the transition, making Horizon for
Newton in Stretch very difficult to achieve. The last patch was made
after the final release, even if both upstream and myself produced
patches during more than 3 months to achieve the result.

Nothing makes me believe this will change and maintainers in Debian will
start caring for OpenStack.

Alembic is one piece of the puzzle that may be a source of trouble in
OpenStack. Having a package inside the umbrella of the OpenStack team
has the huge advantage to tell the world that package maintainers must
care. I'm not sure it will continue to be the case if you move packages
to the DPMT. At the same time, I'm not sure if it will bring you more
help maintaining these packages anyway (I hope to be proven wrong here).

4/ Finally, I feel very much unwelcome by the team "leaders" of the DPMT
(of which the "main" person happen to also be that SQLA maintainer which
I prefer not to name). I already have, and will continue to avoid -as
much as possible- any contribution in the team, since I do fear that any
consequent work that I do will lead to another ban from git write
access, another round of undeserved public shaming, and a loss of the
remaining motivation and energy I still have.

So, to sum-up, if you move packages to the DPMT, that will be one good
reason for me to stop contributing to these packages in the future.

As a conclusion, instead of putting efforts into moving packages from
one team to another, which will achieve no meaningful result, the
efforts should be made on the maintenance of packages itself (like
having the ocata and pike debian repo up and running on OpenStack
infra). That's IMO a way more productive approach, compared to the -from
my side of things- destructive way you're trying to push for.

All this being written, yes, I will let go if you decide to go through
this path. But IMO, it's not the best option.

Cheers,

Thomas Goirand (zigo)

P.S: For many reasons, I would have prefer not having to write some of
the above, as some of its content is socially sensitive. Unfortunately,
I don't think I have a choice, and it's my duty to write these words
publicly. Hopefully, no flame war will start on these sensitive
subjects, and this thread will stay on the same single topic...

[toc] | [prev] | [next] | [standalone]

#9308

Thomas Goirand <zigo@debian.org> writes:

> And I'm not even addressing yet the horrible git-dpm troubles, how
> many more years the team is forcibly burying every contributor into.

There is discussion on changing this. The consensus seems to be we
should wait until after the next release however.

> Plus Alioth is a security nightmare, and it's loaded so much that even
> a simple git push can take hours (real life experience).

I have not noticed this. Maybe the repositories I have dealt with are
small and simple.

(I don't have any opinions on what should happen with the open stack
packages however).
-- 
Brian May <bam@debian.org>

[toc] | [prev] | [next] | [standalone]

#9318

On 03/04/2017 05:13 AM, Brian May wrote:
> Thomas Goirand <zigo@debian.org> writes:
> 
>> And I'm not even addressing yet the horrible git-dpm troubles, how
>> many more years the team is forcibly burying every contributor into.
> 
> There is discussion on changing this. The consensus seems to be we
> should wait until after the next release however.

Why waiting? The freeze is typically a time of very low activity and low
disturbance. That's a perfect moment for doing the switch.

It took *years* to switch from SVN to Git. It's taking *years* to get
out of git-dpm. How many centuries until the team realize that others
are using CI/CD, automated testing, and such, and team member accept
things changing fast on the right direction? There's always resistance
for change in this team.

I fought this for a while... Then I decided it was better to just
give-up, and maintain packages elsewhere. I very much welcome anyone
packaging Python modules to push them to the OpenStack team, and enjoy
the infrastructure, rather than moving things on the opposite direction.

>> Plus Alioth is a security nightmare, and it's loaded so much that even
>> a simple git push can take hours (real life experience).
> 
> I have not noticed this. Maybe the repositories I have dealt with are
> small and simple.

The issue isn't about having big or complicated repositories.

When you do intensive packaging from early in the morning up to late in
the afternoon (ie: the usual office schedule), and work on a dozen of
package per day, then you start feeling very bad about using Alioth.
It's clunky and slow. At any given time of the day, it has a load of at
least 5 to 10. And the I/O is quite loaded as well.

Because of Alioth's complexity, Alioth's admins now refuse to use a
distributed system, and so Alioth is a single machine. Often, cron jobs
for the Postgress db destroy any I/O responsiveness. Or Apache is just
too loaded. That's how Alioth becomes not responsive, and you begin a
very frustrating experience of restarting 5 times a "git clone" operation.

Not so long ago, there's been a huge issue with the RAID 6 array, and it
was down for 12 days (2 weeks during which I couldn't work). So on top
of this, this makes me worry for the safety of data hosted there.

This doesn't stand half a millisecond compared with OpenStack infra
which is distributed across multiple data centers, on different
providers that are sponsoring computing power. Yes, it has issues too,
but it's light-years ahead of what Alioth does.

Cheers,

Thomas Goirand (zigo)

[toc] | [prev] | [next] | [standalone]

#9329

On Mar 05, 2017, at 01:47 AM, Thomas Goirand wrote:

>Why waiting? The freeze is typically a time of very low activity and low
>disturbance. That's a perfect moment for doing the switch.

I think it's generally been the consensus, even outside of this team, that
doing vcs or other disruptive switches are bad ideas during times of release
freezes.  For example, upstream CPython recently switched to git + github, but
while an enormous amount of work went into making that as smooth as possible
beforehand, the actual switch wasn't done until after the 3.6 release.

There are lots of good reasons for that.  I think most importantly is that if
a last minute RC bug were to pop up, no one wants to have to figure out (or
worse, debug) a new maintenance workflow in order to fix that critical
problem.

But that only gates flag day.  Any switch of this nature requires a lot of
work before flag day.  Look at the switch from svn to git,  Stefano did a huge
amount of testing and development to get us to that point, with lots of test
migrations, feedback, etc.  Kudos to him for the tenacity and dedication to
the team on that.

>It took *years* to switch from SVN to Git. It's taking *years* to get
>out of git-dpm. How many centuries until the team realize that others
>are using CI/CD, automated testing, and such, and team member accept
>things changing fast on the right direction? There's always resistance
>for change in this team.

Let's look at the switch from git-dpm as an example.  We *know* there are
challenges in that conversion; I've experimented with it as have others, and
it's not a trivial operation.  So we need one or a few dedicated people to
investigate all the technical details of such a switch.  What are the steps
needed to convert an individual package?  How and when will we convert all
team packages?  What exactly will the new workflow look like?  Does the wiki
page accurate describe all the common tasks that team members will need to
perform?  Is there a test conversion that people can try out?  Where are the
scripts to do the conversion so others can contribute?  What is the process
for providing and addressing feedback as people test it?  What's the timeline,
and when is flag day?

I think one of the problems specifically with getting rid of git-dpm is that,
while the tool is deprecated and there are known problems, it actually kind of
works pretty well for us.  svn clearly was breaking down, but from a global
team point of view, git-dpm is still almost good enough, so the urgency to
switch hasn't been there.  Still, I think there's consensus that we need to be
using supported tools, and git-dpm does occasionally rear up and bite your
head off.  But we need volunteers to say "I am going to do the hard work to
make the conversion happen".  And of course we're all busy, and it's a
thankless job (but thank you Stefano for your previous work!).  So that's why,
IMHO, the git-dpm conversion hasn't happened yet.

If we're just not going to find the round tuits to do the conversion before
then, this would make for a very suitable collaboration for a team Debconf
sprint.

CI/CD, automated testing, etc. cannot just happen by fiat.  They may be great
ideas we can adopt, but *a lot* of hard work and dedicated time goes into
making sure the technology can handle things, but also, and more importantly
IMHO to make sure that everyone on the team knows how it works, understands
and can help debug any problems, knows where the well-written documentation
is, etc.  We need dedicated people to help people on IRC and email when they
get stuck or have a problem.  And we have to consider the needs of those for
whom contribution to DPMT is not a full-time job.

Yes, there is always a natural resistance to change.  But that can be greatly
alleviated by preparation and dedication.  Those things take serious amounts
of time and fortitude.  It's hard to devote that time when you Just Want To
Get Something Done and have to squeeze in an hour here and there to do it.

I look at folks like Brett Cannon doing the CPython GitHub transition as a
model for this.  I'm truly impressed at the work he's done (with lots of help
from others), and at the enormous positive outcome of the effort.  But also
keep in mind that flag day isn't the end of the process.  Things are still
getting tweaked to help smooth the post-migration pain points.  Now, maybe
DPMT won't be as complicated, but it still requires dedication and follow
through.  And faith!  You have to believe (and be convincing and convincingly
friendly in the face of all that inertia) that what you are proposing will
help make everyone's life easier, will make the process of contributions more
efficient for daily maintainers and occasional maintainers alike.

Cheers,
-Barry

[toc] | [prev] | [next] | [standalone]

#9330

On 03/06/2017 12:15 AM, Barry Warsaw wrote:
> There are lots of good reasons for that.  I think most importantly is that if
> a last minute RC bug were to pop up, no one wants to have to figure out (or
> worse, debug) a new maintenance workflow in order to fix that critical
> problem.

If such thing happened, I don't think it's such a huge issue to just fix
the RC bug and upload, then figure out how to commit later on. Because
of the freeze, the change will be minimal anyway.

> But that only gates flag day.  Any switch of this nature requires a lot of
> work before flag day.  Look at the switch from svn to git,  Stefano did a huge
> amount of testing and development to get us to that point, with lots of test
> migrations, feedback, etc.  Kudos to him for the tenacity and dedication to
> the team on that.

A huge +1 for the kudos to Stefano and yourself for this indeed.

> Let's look at the switch from git-dpm as an example.  We *know* there are
> challenges in that conversion; I've experimented with it as have others, and
> it's not a trivial operation.  So we need one or a few dedicated people to
> investigate all the technical details of such a switch.  What are the steps
> needed to convert an individual package?  How and when will we convert all
> team packages?  What exactly will the new workflow look like?  Does the wiki
> page accurate describe all the common tasks that team members will need to
> perform?  Is there a test conversion that people can try out?  Where are the
> scripts to do the conversion so others can contribute?  What is the process
> for providing and addressing feedback as people test it?  What's the timeline,
> and when is flag day?

I agree it's not trivial.

> I think one of the problems specifically with getting rid of git-dpm is that,
> while the tool is deprecated and there are known problems, it actually kind of
> works pretty well for us. svn clearly was breaking down, but from a global
> team point of view, git-dpm is still almost good enough, so the urgency to
> switch hasn't been there.

Here, I don't agree. but YMMV, I guess. Maybe there's no point
discussing the urgency! :)

> But we need volunteers to say "I am going to do the hard work to
> make the conversion happen".  And of course we're all busy, and it's a
> thankless job (but thank you Stefano for your previous work!).  So that's why,
> IMHO, the git-dpm conversion hasn't happened yet.
> 
> If we're just not going to find the round tuits to do the conversion before
> then, this would make for a very suitable collaboration for a team Debconf
> sprint.

I'm hereby volunteering for such a sprint (if I hopefully make it to
Montreal). Hopefully, migrating from git-dpm to git-pq wont be as hard
as from SVN to Git.

> CI/CD, automated testing, etc. cannot just happen by fiat.  They may be great
> ideas we can adopt, but *a lot* of hard work and dedicated time goes into
> making sure the technology can handle things, but also, and more importantly
> IMHO to make sure that everyone on the team knows how it works, understands
> and can help debug any problems, knows where the well-written documentation
> is, etc.  We need dedicated people to help people on IRC and email when they
> get stuck or have a problem.  And we have to consider the needs of those for
> whom contribution to DPMT is not a full-time job.

I already talked and wrote about it, I would very much like a packaging
CI/CD to be deployed for Debian. However, I'm scared to attempt doing it
by myself only. I don't want to become a single point of failure.

Cheers,

Thomas Goirand (zigo)

[toc] | [prev] | [next] | [standalone]

#9332

[Multipart message — attachments visible in raw view] — view raw

On 2017-03-06 10:54, Thomas Goirand wrote:

> I'm hereby volunteering for such a sprint (if I hopefully make it to
> Montreal). Hopefully, migrating from git-dpm to git-pq wont be as hard
> as from SVN to Git.

Great! The sooner (after the freeze) we can do this, the better IMHO.
git-dpm looked good at the time, however the more I think about it, the
more I think gbp is the superior solution - especially for a team
effort. It is very easy to break gbp pq unless you are careful to follow
correct procedures. Once a broken repository is pushed (e.g. manual
changes to debian/patches/*), it is not easy to fix without liberal use
of pushing a rebased repository. Or starting git-dpm from scratch. If
there was a conflict with debian patches (fortunately this hasn't
happened to me yet), I think it would be very challenging to resolve the
conflict with git-dpm (not that gbp pq is going to be great in this
situation either, however I think it would be easier to understand what
is going on, and hence find a good solution). 

The concept to convert from git-dpm to gbp pq is very very easy: 

1. Delete debian/.git-dpm 
2. Unapply all patches. 
3. Commit and push. 

(repeat for all branches on all repositories) 

Step 2 is easier said then done. I can do it easy enough on my own
packages. I think we need a documented process anyone can follow. Plus
dealing with errors as required (e.g. if unapplying patches causes
conflicts due to non-compliant git-dpm package) - if anything like this
happens - hopefully on not many packages, these packages might require
manual processing. 

The obvious way "quilt pop -a" doesn't work, because the quilt .pc
directory does not exist, and quilt thinks the patches are already
unapplied. I sent an email previously with some (untested) suggestions I
had. 
  

[toc] | [prev] | [next] | [standalone]

#9334

[Multipart message — attachments visible in raw view] — view raw

 ❦  6 mars 2017 12:30 +1100, Brian May <brian@linuxpenguins.xyz> :

>>  I'm hereby volunteering for such a sprint (if I hopefully make it to
>>  Montreal). Hopefully, migrating from git-dpm to git-pq wont be as hard
>>  as from SVN to Git.
>
> Great! The sooner (after the freeze) we can do this, the better
> IMHO. git-dpm looked good at the time, however the more I think about
> it, the more I think gbp is the superior solution - especially for a
> team effort. It is very easy to break gbp pq unless you are careful to
> follow correct procedures.

I think you mean git-dpm.
-- 
The smallest worm will turn being trodden on.
		-- William Shakespeare, "Henry VI"

[toc] | [prev] | [next] | [standalone]

#9335

Vincent Bernat <bernat@debian.org> writes:

> I think you mean git-dpm.

Whoops. Yes, of course.
-- 
Brian May <bam@debian.org>

[toc] | [prev] | [next] | [standalone]

#9337

Excerpts from Brian May's message of 2017-03-06 12:30:56 +1100:
> The concept to convert from git-dpm to gbp pq is very very easy: 
> 
> 1. Delete debian/.git-dpm 
> 2. Unapply all patches. 
> 3. Commit and push. 
> 
> (repeat for all branches on all repositories) 
> 
> Step 2 is easier said then done. I can do it easy enough on my own
> packages. I think we need a documented process anyone can follow. Plus
> dealing with errors as required (e.g. if unapplying patches causes
> conflicts due to non-compliant git-dpm package) - if anything like this
> happens - hopefully on not many packages, these packages might require
> manual processing. 
> 
> The obvious way "quilt pop -a" doesn't work, because the quilt .pc
> directory does not exist, and quilt thinks the patches are already
> unapplied. I sent an email previously with some (untested) suggestions I
> had. 
>   

Can we do this now, and push to a branch, so that when we unfreeze we
can just 'git merge --ff-only' from master and then re-do any that fail?

[toc] | [prev] | [next] | [standalone]

#9338

On Monday, March 06, 2017 07:04:41 AM Clint Byrum wrote:
> Excerpts from Brian May's message of 2017-03-06 12:30:56 +1100:
> > The concept to convert from git-dpm to gbp pq is very very easy:
> > 
> > 1. Delete debian/.git-dpm
> > 2. Unapply all patches.
> > 3. Commit and push.
> > 
> > (repeat for all branches on all repositories)
> > 
> > Step 2 is easier said then done. I can do it easy enough on my own
> > packages. I think we need a documented process anyone can follow. Plus
> > dealing with errors as required (e.g. if unapplying patches causes
> > conflicts due to non-compliant git-dpm package) - if anything like this
> > happens - hopefully on not many packages, these packages might require
> > manual processing.
> > 
> > The obvious way "quilt pop -a" doesn't work, because the quilt .pc
> > directory does not exist, and quilt thinks the patches are already
> > unapplied. I sent an email previously with some (untested) suggestions I
> > had.
> 
> Can we do this now, and push to a branch, so that when we unfreeze we
> can just 'git merge --ff-only' from master and then re-do any that fail?

I think it's reasonable to try this out on a branch for some number of 
packages and write documentation (the documentation for using git with git-dpm 
in DPMT is excellent and I don't think we should regress in that area).  Once 
that's done, someone who doesn't know anything about gbp pq should try it out 
and see if the documentation works (I'll be glad to do that).

Once we have that and have a tested migration script to do this and the later 
merge, I think it would be reasonable to go ahead.

Scott K

[toc] | [prev] | [next] | [standalone]

#9339

On Mar 06, 2017, at 10:32 AM, Scott Kitterman wrote:

>I think it's reasonable to try this out on a branch for some number of
>packages and write documentation (the documentation for using git with
>git-dpm in DPMT is excellent and I don't think we should regress in that
>area).  Once that's done, someone who doesn't know anything about gbp pq
>should try it out and see if the documentation works (I'll be glad to do
>that).

We have this wiki page for describing common workflows for gbp-pq:

https://wiki.debian.org/Python/GitPackagingPQ

It started as a copy of the git-dpm page:

https://wiki.debian.org/Python/GitPackaging

So one thing to do is to debug the instructions there.  I think the pq page
would be a good place to capture whatever foolproof conversion recipe we come
up with.  Even after a mass migration, this might be useful for other
maintainers that want to switch from git-dpm.

Cheers,
-Barry

[toc] | [prev] | [next] | [standalone]

#9340

On Mon, 06 Mar 2017 at 10:32:17 -0500, Scott Kitterman wrote:
> I think it's reasonable to try this out on a branch

Here's a maybe-stupid idea: use http://dep.debian.net/deps/dep14/ branch
naming (debian/master, debian/experimental) for that branch, and switch to
it as the default branch (edit foo.git/HEAD on alioth) when unfreezing
and "officially" switching to gbp-pq?

(You would have to stick to either upstream or upstream/latest but not
mix them, though, because file vs. directory duality applies here.)

I can offer https://anonscm.debian.org/cgit/pkg-utopia/dbus.git as an
example of a repository with semi-complex history, that uses gbp-pq and
DEP-14. ioquake3, flatpak, ostree, openjk, iortcw are all simpler examples
if you want one of those.

    S

[toc] | [prev] | [next] | [standalone]

#9341 — Transition away from git-dpm was: Re: Adopting OpenStack packages

Updated the subject, since we've drifted...

On Monday, March 06, 2017 04:47:39 PM Simon McVittie wrote:
> On Mon, 06 Mar 2017 at 10:32:17 -0500, Scott Kitterman wrote:
> > I think it's reasonable to try this out on a branch
> 
> Here's a maybe-stupid idea: use http://dep.debian.net/deps/dep14/ branch
> naming (debian/master, debian/experimental) for that branch, and switch to
> it as the default branch (edit foo.git/HEAD on alioth) when unfreezing
> and "officially" switching to gbp-pq?
> 
> (You would have to stick to either upstream or upstream/latest but not
> mix them, though, because file vs. directory duality applies here.)
> 
> I can offer https://anonscm.debian.org/cgit/pkg-utopia/dbus.git as an
> example of a repository with semi-complex history, that uses gbp-pq and
> DEP-14. ioquake3, flatpak, ostree, openjk, iortcw are all simpler examples
> if you want one of those.
> 
>     S

Personally, I don't know enough to have an opinion.  I'm interested in the 
views of DPMT members with gbp pq experience.  What's the consensus about 
branch naming (all I know is for git-dpm, it was pretty hard wired)?

Scott K

[toc] | [prev] | [next] | [standalone]

#9342 — Re: Transition away from git-dpm was: Re: Adopting OpenStack packages

Scott Kitterman <scott@kitterman.com> writes:

> Personally, I don't know enough to have an opinion.  I'm interested in the 
> views of DPMT members with gbp pq experience.  What's the consensus about 
> branch naming (all I know is for git-dpm, it was pretty hard wired)?

I tend to think that branching will easier with git pq. I seem to
remember some maintainers having problems with non-standard branch names
under git dpm. So one valid reason for not adopting different branches
will no longer apply.

I personally think that DEP14 would be a good idea. I think the only
difference people would notice is that master is renamed to
debian/master.

We would need to be careful though, having both a master and a
debian/master - even during transition planning - could be
confusing. Which one do I update? Probably no way to mark a branch read
only on alioth either (?).

After the transition, what happens to the master branch? Do we delete
it?
-- 
Brian May <bam@debian.org>

[toc] | [prev] | [next] | [standalone]

#9343 — Re: Transition away from git-dpm was: Re: Adopting OpenStack packages

On 03/06/2017 09:25 PM, Brian May wrote:
> Scott Kitterman <scott@kitterman.com> writes:
> 
>> Personally, I don't know enough to have an opinion.  I'm interested in the 
>> views of DPMT members with gbp pq experience.  What's the consensus about 
>> branch naming (all I know is for git-dpm, it was pretty hard wired)?
> 
> I tend to think that branching will easier with git pq. I seem to
> remember some maintainers having problems with non-standard branch names
> under git dpm. So one valid reason for not adopting different branches
> will no longer apply.
> 
> I personally think that DEP14 would be a good idea. I think the only
> difference people would notice is that master is renamed to
> debian/master.
> 
> We would need to be careful though, having both a master and a
> debian/master - even during transition planning - could be
> confusing. Which one do I update? Probably no way to mark a branch read
> only on alioth either (?).
> 
> After the transition, what happens to the master branch? Do we delete
> it?

I *very much* vouch for the dep14 style of branch names. I've been using
this for years. Maybe some will remember that's what I suggested in
Debconf portland where we discussed switching to Git, and for the first
time, agree on it. At the time, mostly everyone didn't like the idea
though. I'm glad it looks like mind sets have moved to the direction I
consider the best.

There's 2 issues with calling the Debian packaging branch "master".
First, it may conflict with an eventual upstream branch also called
"master" (for those who like me enjoy doing a checkout of it). Second,
the name "master" doesn't express anything, while "debian/unstable" is
very much self-explanatory.

I prefer if we use debian/unstable rather than debian/master though, so
it is more explicit where we upload that branch. Also, it'd be wise to
set that branch as default when cloning the repository. ie we shall ran
on Alioth:

git symbolic-ref HEAD refs/heads/debian/unstable

so that debian/unstable becomes the default branch. This has a good
chance to avoid confusion between the old "master" (ie: git-dpm) branch
and the new DEP14 style that we would adopt.

When doing this, we should also make sure to clearly self-document this
using:

# cat debian/gbp.conf
[DEFAULT]
debian-branch = debian/unstable

Last, I would consider it a nice improvement. Not a critical one. So if
others feel like we should keep the git-dpm old layout to avoid
confusing people, I wouldn't mind so much.

Your thoughts?

Cheers,

Thomas Goirand (zigo)

[toc] | [prev] | [next] | [standalone]

#9344 — Re: Transition away from git-dpm was: Re: Adopting OpenStack packages

[Multipart message — attachments visible in raw view] — view raw

On 2017-03-07 08:43, Thomas Goirand wrote:

> I prefer if we use debian/unstable rather than debian/master though, so
> it is more explicit where we upload that branch.

My reading of DEP-14 is that is says we should use debian/master. 

Not that I care much myself, either is fine with me. 

Why debian/unstable, and not debian/sid? 

> Also, it'd be wise to
> set that branch as default when cloning the repository. ie we shall ran
> on Alioth:
> 
> git symbolic-ref HEAD refs/heads/debian/unstable
> 
> so that debian/unstable becomes the default branch. This has a good
> chance to avoid confusion between the old "master" (ie: git-dpm) branch
> and the new DEP14 style that we would adopt.

I didn't know you could do this. Good to know. 

> When doing this, we should also make sure to clearly self-document this
> using:
> 
> # cat debian/gbp.conf
> [DEFAULT]
> debian-branch = debian/unstable
> 
> Last, I would consider it a nice improvement. Not a critical one. So if
> others feel like we should keep the git-dpm old layout to avoid
> confusing people, I wouldn't mind so much.

I would also suggest making the same change (maybe with an appropriate
comment) on the old branch. It should result in people checking out the
wrong branch being unable to build with gbp, and hopefully this should
also serve as a reference to the correct branch. 
  

[toc] | [prev] | [next] | [standalone]

#9347 — Re: Transition away from git-dpm was: Re: Adopting OpenStack packages

On 03/06/2017 11:18 PM, Brian May wrote:
> On 2017-03-07 08:43, Thomas Goirand wrote:
> 
>> I prefer if we use debian/unstable rather than debian/master though, so
>> it is more explicit where we upload that branch.
>  
> My reading of DEP-14 is that is says we should use debian/master.
>  
> Not that I care much myself, either is fine with me.
>  
> Why debian/unstable, and not debian/sid?

Because "unstable" is what we write in debian/changelog, so that's
consistent, and also consistent if we upload to experimental. But I'm
fine either ways anyway, if others would like to use debian/sid because
it's faster to type.

>> Also, it'd be wise to
>> set that branch as default when cloning the repository. ie we shall ran
>> on Alioth:
>>
>> git symbolic-ref HEAD refs/heads/debian/unstable
>>
>> so that debian/unstable becomes the default branch. This has a good
>> chance to avoid confusion between the old "master" (ie: git-dpm) branch
>> and the new DEP14 style that we would adopt.
>  
> I didn't know you could do this. Good to know.

The only annoying bit with that thing, is that you can't run it remotely
with a git command (unless there's a new command I didn't know about).
As a consequence, you need to have an interactive shell access and type
the command on the server. Lucky, Alioth provides it and it's ok in our
case. But sometimes, you don't have such access, and it's very annoying.

Cheers,

Thomas Goirand (zigo)

[toc] | [prev] | [next] | [standalone]

Page 1 of 3  [1] 2 3  Next page →

Back to top | Article view | linux.debian.maint.python

csiph-web