Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.project > #8643 > unrolled thread

third-party packages adding apt sources

Back to article view | Back to linux.debian.project

Contents

  third-party packages adding apt sources Daniel Pocock <daniel@pocock.pro> - 2016-05-19 17:20 +0200
    Re: third-party packages adding apt sources Paul Tagliamonte <paultag@debian.org> - 2016-05-19 17:50 +0200
      Re: third-party packages adding apt sources Bas Wijnen <wijnen@debian.org> - 2016-05-19 19:00 +0200
        Re: third-party packages adding apt sources "Adam D. Barratt" <adam@adam-barratt.org.uk> - 2016-05-19 19:10 +0200
        Re: third-party packages adding apt sources Mike Hommey <mh@glandium.org> - 2016-05-20 00:40 +0200
        Re: third-party packages adding apt sources Vincent Bernat <bernat@debian.org> - 2016-05-20 07:30 +0200
          Re: third-party packages adding apt sources Antonio Terceiro <terceiro@debian.org> - 2016-05-20 14:00 +0200
            Re: third-party packages adding apt sources The Wanderer <wanderer@fastmail.fm> - 2016-05-20 14:30 +0200
            Re: third-party packages adding apt sources Ole Streicher <olebole@debian.org> - 2016-05-20 15:00 +0200
              testing is a tool for the release team (Re: third-party packages  adding apt sources Holger Levsen <holger@layer-acht.org> - 2016-05-20 15:10 +0200
            Re: third-party packages adding apt sources Vincent Bernat <bernat@debian.org> - 2016-05-20 20:40 +0200
          Re: third-party packages adding apt sources Paul Wise <pabs@debian.org> - 2016-05-21 08:10 +0200
        Re: third-party packages adding apt sources Tollef Fog Heen <tfheen@err.no> - 2016-05-20 14:50 +0200
      Re: third-party packages adding apt sources Ian Jackson <ijackson@chiark.greenend.org.uk> - 2016-05-19 20:20 +0200
    Re: third-party packages adding apt sources Russ Allbery <rra@debian.org> - 2016-05-19 17:50 +0200
      Re: third-party packages adding apt sources Holger Levsen <holger@layer-acht.org> - 2016-05-19 18:00 +0200
    Re: third-party packages adding apt sources Ian Jackson <ijackson@chiark.greenend.org.uk> - 2016-05-19 19:10 +0200
      Re: third-party packages adding apt sources Daniel Pocock <daniel@pocock.pro> - 2016-05-19 19:20 +0200
        Re: third-party packages adding apt sources Bas Wijnen <wijnen@debian.org> - 2016-05-19 19:50 +0200
          Re: third-party packages adding apt sources Ian Jackson <ijackson@chiark.greenend.org.uk> - 2016-05-19 20:20 +0200
        Re: third-party packages adding apt sources Russ Allbery <rra@debian.org> - 2016-05-19 20:20 +0200
        Re: third-party packages adding apt sources Ian Jackson <ijackson@chiark.greenend.org.uk> - 2016-05-19 20:20 +0200
      Re: third-party packages adding apt sources Vincent Bernat <bernat@debian.org> - 2016-05-20 07:40 +0200
        Re: third-party packages adding apt sources Charles Plessy <plessy@debian.org> - 2016-05-20 11:40 +0200
        Re: third-party packages adding apt sources Paul Wise <pabs@debian.org> - 2016-05-21 08:10 +0200
          Re: third-party packages adding apt sources Vincent Bernat <bernat@debian.org> - 2016-05-21 08:50 +0200
            Re: third-party packages adding apt sources Paul Wise <pabs@debian.org> - 2016-05-21 09:00 +0200
              Re: third-party packages adding apt sources Vincent Bernat <bernat@debian.org> - 2016-05-21 09:10 +0200
          Re: third-party packages adding apt sources Lars Wirzenius <liw@liw.fi> - 2016-05-21 09:40 +0200
            Re: third-party packages adding apt sources Martin Steigerwald <martin@lichtvoll.de> - 2016-05-21 10:30 +0200
            Re: third-party packages adding apt sources Martin Steigerwald <martin@lichtvoll.de> - 2016-05-21 10:30 +0200
              Re: third-party packages adding apt sources Vincent Bernat <bernat@debian.org> - 2016-05-21 11:00 +0200
                Re: third-party packages adding apt sources Martin Steigerwald <martin@lichtvoll.de> - 2016-05-21 11:30 +0200
          Re: third-party packages adding apt sources Martin Steigerwald <martin@lichtvoll.de> - 2016-05-21 10:10 +0200
            Re: third-party packages adding apt sources Lars Wirzenius <liw@liw.fi> - 2016-05-21 10:40 +0200
              Re: third-party packages adding apt sources Martin Steigerwald <martin@lichtvoll.de> - 2016-05-21 11:20 +0200
        Re: third-party packages adding apt sources Ole Streicher <olebole@debian.org> - 2016-05-21 10:00 +0200
          Re: third-party packages adding apt sources Vincent Bernat <bernat@debian.org> - 2016-05-21 11:00 +0200
    Re: third-party packages adding apt sources Hakan Peker <holypeker@manlymail.net> - 2016-05-19 19:50 +0200
      Re: third-party packages adding apt sources Vincent Danjean <vdanjean.ml@free.fr> - 2016-05-20 21:40 +0200
        Re: third-party packages adding apt sources Hakan Peker <holypeker@manlymail.net> - 2016-05-22 16:50 +0200
          Re: third-party packages adding apt sources Andrew McGlashan <andrew.mcglashan@affinityvision.com.au> - 2016-05-22 17:10 +0200
    Re: third-party packages adding apt sources Charles Plessy <plessy@debian.org> - 2016-05-21 05:50 +0200
    Re: third-party packages adding apt sources Paul Wise <pabs@debian.org> - 2016-05-21 07:50 +0200
      Re: third-party packages adding apt sources Adam Borowski <kilobyte@angband.pl> - 2016-05-21 14:40 +0200
        Re: third-party packages adding apt sources Paul Wise <pabs@debian.org> - 2016-05-22 03:50 +0200

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

#8654

Daniel Pocock <daniel@pocock.pro> writes:

> Another thing comes to mind: making sure that even if the user
> explicitly allows some other repository, they are protected from package
> updates that come along and replace other things like apt itself, libc,
> bash, gnupg, ...

While this would be nice to prevent accidents, it's not clear that you can
really establish any security guarantees.  You can protect against some
very obvious things, such as wholesale replacing core packages, but
postinst scripts still run as root and can do anything they want.  So you
don't get any real security benefit here that I can see.

-- 
Russ Allbery (rra@debian.org)               <http://www.eyrie.org/~eagle/>

[toc] | [prev] | [next] | [standalone]

#8655

Daniel Pocock writes ("Re: third-party packages adding apt sources"):
> On 19/05/16 19:04, Ian Jackson wrote:
> > Debian proper has a very high bar for inclusion.  Obviously there are
> > perhaps some packages which are close to suitable for inclusion, but
> > the vast majority of things that aren't in Debian proper are outside
> > it for real, nontrivial reasons (whether of technical quality of the
> > binaries, technical quality of the source, or political/ethical
> > reasons).
> 
> Do you think that if these upstreams became involved in other ways - for
> example, if we proactively invited them to MiniDebConfs and other events
> - we might bridge the gap to help them understand our way of thinking,
> whether it is technical or otherwise?

I think you are missing my point.  You seem to have the assumption
that for many of the people providing packages in third-party repos,
they don't have what I'm calling real and nontrivial reasons.

Hell, I even have .debs I distribute myself outside Debian (although I
haven't gone to the trouble of setting up reprepro).

> Sure, some of them will never change, some of them have no capacity to
> think long-term but there are others who simply don't quite understand
> and may go the extra mile if they get to know us a little better.

The problem is not one of lack of understanding.  These out-of-Debian
repos are not going to be abolished by an outreach and education
programme.


Let me go into more detail about a few of the kinds of real,
nontrivial reasons, I'm talking about:


1. Proper source format.  I wrote:

> > Providing a proper Debian source package is also a lot more work than
> > writing some kind of ad-hoc build system that spits out a .deb or
> > three.

Packages with ad-hoc source build systems are never going to be in
Debian proper, for obvious reasons.  They wouldn't fit.  That's not
something we can fix.

But writing a proper build system for an ad-hoc Debian source package
is a fair amount of work.  I did it recently for a moderately easy
package I encountered in Raspbian, and it took /me/ most of a day.

That upstream didn't do it is quite understandable.  Doing so was not
in their `long-term' interests, as you put it: the effort for them to
learn how to do it, and then do it, and then debug it, and then deal
with the transitional pain, was simply not worth it to them.  And
probably not to their userbase, either.

We in Debian can improve this situation by making proper Debian source
packages easier to build.  For example, we can:
 * Provide build tools which require less boilerplate, are less
   fragile, and are more automatic.  Yay dh(1).
 * Provide _shorter_ intro documentation.
 * Provide better source code management systems - that are less
   prehistoric than .dsc 1.0 (non-native) and less "omg wtf bbq"
   than .dsc "3.0 (quilt)".  (non-crazy git integration, dgit)

But there are still going to be plenty of people for whom it's a
better use of their time to do something else but tart up the source
for their .deb builds.


2. ABI/API

The package I mention above didn't take the care with API/ABI, split
library packaging, and so on, that would be expected in Debian.  This
was tolerable in the original Raspbian context.

I sent the upstream patches to fix this but there are compatibility
implications with changing the packaging arrangements.  Thinking about
this stuff, and planning for it, and anticipating the needs of the
users, is nontrivial.

For quite a few packages, the levels of compatibility and
upgradeability we would expect in Debian are simply not worth the
effort.  I don't think in Debian we would want to relax this, do we ?


3. Freedom

This is the killer reason to provide a third-party apt repository.

A software provider who provides packages outside Debian does not need
our permission to do so; they do not need our help to provide new
versions; they do not need to confirm to our release rules.

Of course in Debian our focus is freedom for our users, so there are
limits to how much freedom we want to make available to upstreams and
third-party packagers: we want to avoid becoming complicit in efforts
to control users.  But users often choose (for reasons that seem good
to those users - and, often, reasons that /are/ good for those users)
to want to run non-free software.  So even when it concerns non-free
software we need to respect and support that decision, while avoiding
encouraging it.

I don't think an outreach campaign by Debian, encouraging people to do
work in Debian rather than outside, is going to significantly change
the way software providers make decisions about licensing, release
schedules, or whatever.


> Another thing comes to mind: making sure that even if the user
> explicitly allows some other repository, they are protected from package
> updates that come along and replace other things like apt itself, libc,
> bash, gnupg, ...

This has to be an optional feature.  The user might specifically want
that !

Ian.

[toc] | [prev] | [next] | [standalone]

#8659

[Multipart message — attachments visible in raw view] — view raw

 ❦ 19 mai 2016 18:04 +0100, Ian Jackson <ijackson@chiark.greenend.org.uk> :

>> b) many upstreams appear frustrated about getting their package
>> officially supported in Debian.  Sometimes there is good reason their
>> package doesn't belong in Debian but sometimes it is more about inertia
>> in Debian or the upstream isn't aware about backports and thinks their
>> package will be stuck at a particular version forever
>
> Providing a proper Debian source package is also a lot more work than
> writing some kind of ad-hoc build system that spits out a .deb or
> three.

Totally agree. Our standards are far too high for many upstreams.

I am always flabestered by the popularity of fpm to build Debian
packages (and by the increasing popularity of pleaserun by the same
author on the same concepts). It provides a way to easily build a Debian
package from a directory but produces somewhat crippled/incomplete
packages and is no help to us since it's completely outside of any of
our tools. It also handles RPM (and now other package formats), but I
don't think this would explain its popularity alone.

And there is also the lost cause of vendoring that gained even more
traction with Go but that was already a problem with the Java ecosystem.
I don't think there is much to do about this one.
-- 
Don't sacrifice clarity for small gains in "efficiency".
            - The Elements of Programming Style (Kernighan & Plauger)

[toc] | [prev] | [next] | [standalone]

#8660

Le Fri, May 20, 2016 at 07:34:59AM +0200, Vincent Bernat a écrit :
> 
> I am always flabestered by the popularity of fpm to build Debian
> packages (and by the increasing popularity of pleaserun by the same
> author on the same concepts). It provides a way to easily build a Debian
> package from a directory but produces somewhat crippled/incomplete
> packages and is no help to us since it's completely outside of any of
> our tools. It also handles RPM (and now other package formats), but I
> don't think this would explain its popularity alone.

For software using CMake, there are also CPackDeb and CPackRPM.

https://cmake.org/cmake/help/latest/module/CPackDeb.html

Were I an upstream developer, I would definitely be interested by tools like
this that leverage the build system to build installation packages for various
platforms.

Have a nice week-end,

Charles

-- 
Charles Plessy
Debian Med packaging team,
http://www.debian.org/devel/debian-med
Tsurumi, Kanagawa, Japan

[toc] | [prev] | [next] | [standalone]

#8676

On Fri, May 20, 2016 at 1:34 PM, Vincent Bernat wrote:

> Totally agree. Our standards are far too high for many upstreams.

I don't understand the disconnect here. Are upstreams not interested
in software quality to the extent we are?

> I am always flabestered by the popularity of fpm to build Debian
> packages (and by the increasing popularity of pleaserun by the same
> author on the same concepts). It provides a way to easily build a Debian
> package from a directory but produces somewhat crippled/incomplete
> packages and is no help to us since it's completely outside of any of
> our tools. It also handles RPM (and now other package formats), but I
> don't think this would explain its popularity alone.

I think we probably need to get dpkg-buildpackage to automatically run
some of these:

https://wiki.debian.org/AutomaticPackagingTools

> And there is also the lost cause of vendoring that gained even more
> traction with Go but that was already a problem with the Java ecosystem.
> I don't think there is much to do about this one.

Embedded code/data copies have been a problem since as long as I've
been a Debian user, they aren't specific to any particular language
community.

https://wiki.debian.org/EmbeddedCodeCopies

-- 
bye,
pabs

https://wiki.debian.org/PaulWise

[toc] | [prev] | [next] | [standalone]

#8678

[Multipart message — attachments visible in raw view] — view raw

 ❦ 21 mai 2016 14:07 +0800, Paul Wise <pabs@debian.org> :

>> Totally agree. Our standards are far too high for many upstreams.
>
> I don't understand the disconnect here. Are upstreams not interested
> in software quality to the extent we are?

Many of them don't consider packaging quality as important. As long as
the package does its job, they are fine. Notably, packages generated
with fpm have many downside but they mostly work and people are
happy. The main feature of fpm is being easy, not producing good quality
packages. In our cases, good quality packages are the main feature we
want. Being easy is a plus.

>> I am always flabestered by the popularity of fpm to build Debian
>> packages (and by the increasing popularity of pleaserun by the same
>> author on the same concepts). It provides a way to easily build a Debian
>> package from a directory but produces somewhat crippled/incomplete
>> packages and is no help to us since it's completely outside of any of
>> our tools. It also handles RPM (and now other package formats), but I
>> don't think this would explain its popularity alone.
>
> I think we probably need to get dpkg-buildpackage to automatically run
> some of these:
>
> https://wiki.debian.org/AutomaticPackagingTools

A meta tool "package me this" would be interesting. However note that
many of those tools are too complex for many upstreams because they
don't want to package each dependency one by one. For example,
dh-make-golang is fully automatic but you need to run it on each
dependency. That's fine because those tools are for building packages
that should be fit to go into our archive.

>> And there is also the lost cause of vendoring that gained even more
>> traction with Go but that was already a problem with the Java ecosystem.
>> I don't think there is much to do about this one.
>
> Embedded code/data copies have been a problem since as long as I've
> been a Debian user, they aren't specific to any particular language
> community.
>
> https://wiki.debian.org/EmbeddedCodeCopies

For some languages, embedded copies are a pattern. Notably Go. But there
is also the omnibus stance: the embedded copy could not be in the
source, but could be in the shipped artifact. This includes Go, JS and
Java (when using uberjars). For some other ecosystems, the embedded copy
is more the exception than the rule (C, C++, Python).

If upstream is using embedded copies, they are quite unlikely to make
any effort tu undo this aspect.
-- 
Indent to show the logical structure of a program.
            - The Elements of Programming Style (Kernighan & Plauger)

[toc] | [prev] | [next] | [standalone]

#8679

On Sat, May 21, 2016 at 2:46 PM, Vincent Bernat wrote:

> A meta tool "package me this" would be interesting.

There is debdry but it got orphaned.

> many of those tools are too complex for many upstreams because they
> don't want to package each dependency one by one. For example,
> dh-make-golang is fully automatic but you need to run it on each
> dependency. That's fine because those tools are for building packages
> that should be fit to go into our archive.

ISTR A recursive mode is what some automatic packaging tools do there.

> For some languages, embedded copies are a pattern. Notably Go. But there
> is also the omnibus stance: the embedded copy could not be in the
> source, but could be in the shipped artifact. This includes Go, JS and
> Java (when using uberjars). For some other ecosystems, the embedded copy
> is more the exception than the rule (C, C++, Python).

By shipped artifact, it sounds like you are talking about source
packages? or do you mean binary packages?

> If upstream is using embedded copies, they are quite unlikely to make
> any effort tu undo this aspect.

I see upstreams doing that on debian-mentors reasonably often,
especially when the upstream is the one doing the RFS. For upstreams
who aren't interested in getting their software into Debian, that is
definitely the case.

-- 
bye,
pabs

https://wiki.debian.org/PaulWise

[toc] | [prev] | [next] | [standalone]

#8680

[Multipart message — attachments visible in raw view] — view raw

 ❦ 21 mai 2016 14:55 +0800, Paul Wise <pabs@debian.org> :

>> For some languages, embedded copies are a pattern. Notably Go. But there
>> is also the omnibus stance: the embedded copy could not be in the
>> source, but could be in the shipped artifact. This includes Go, JS and
>> Java (when using uberjars). For some other ecosystems, the embedded copy
>> is more the exception than the rule (C, C++, Python).
>
> By shipped artifact, it sounds like you are talking about source
> packages? or do you mean binary packages?

I meant binary packages but wanted to be more general (not limited to Debian).

>> If upstream is using embedded copies, they are quite unlikely to make
>> any effort tu undo this aspect.
>
> I see upstreams doing that on debian-mentors reasonably often,
> especially when the upstream is the one doing the RFS. For upstreams
> who aren't interested in getting their software into Debian, that is
> definitely the case.

I shouldn't have made a generality. Some upstreams are more receptive
than others to this problematic.
-- 
Each module should do one thing well.
            - The Elements of Programming Style (Kernighan & Plauger)

[toc] | [prev] | [next] | [standalone]

#8681

[Multipart message — attachments visible in raw view] — view raw

On Sat, May 21, 2016 at 02:07:53PM +0800, Paul Wise wrote:
> On Fri, May 20, 2016 at 1:34 PM, Vincent Bernat wrote:
> 
> > Totally agree. Our standards are far too high for many upstreams.
> 
> I don't understand the disconnect here. Are upstreams not interested
> in software quality to the extent we are?

I don't think it's that upstreams aren't interested in quality, but
that Debian and (some) upstreams have different opinions on what
aspects of quality are more important.

* Debian: don't embed copies of libraries you use. It makes it harder
  to do security updates in the libraries, makes it harder to use the
  libraries on their own, and makes the Debian package archive
  unnecessarily larger.

  Some upstreams: we embed copies of libraries. It makes it easier to
  install our software, and guarantees us that the library doesn't
  change from underneath us, and that means we don't need to support
  many versions of the library. We're an active project, and if a
  library needs a security update, we do it quickly.

* Debian: it's important to follow Debian Policy and the Debian
  workflow of uploading to unstable and letting packages flow from
  there to testing and stable, if they don't have bad bugs. There's
  thousands of people making packages and things will break if they
  all do the same thing differently.

  Some upstreams: it's OK to cut some corners and do things simply. We
  care about getting the software into the hands of its users as soon
  as possible, and we also don't have a lot of time to spend on
  packaging. From our point of view, packaging is a necessary evil
  that is much too difficult and takes much too much time from us.
  That's effort we could spend on making the software better instead.

* Debian: it's important to have package versions that can be
  supported for many years. We produce a release every two years, and
  support it for at least three, and more if one counts the LTS
  project. Software that changes a lot, or that has an API that
  changes a lot, or that doesn't separate security updates and
  backport those to the version included in a Debian release, make
  this harder for us. We can't generally update to a new upstream
  release whenever there is a security problem, as it would negate the
  point of making Debian releases. For example, the new upstream
  version might require entirely new forests of dependencies, or newer
  versions of dependencies, all the way down to the kernel. For some
  packages that we deem sufficiently important to our users, we deal
  with that, but it is not something that generalises to all packages.

  Some upstreams: we don't support our old releases. We have only so
  much time to spend on this software, and supporting old releases
  would take a lot of effort we don't have time for. That's why we
  embed most of our dependencies into the installation packages we
  make, so that they can be installed onto the Debian releases, even
  if we decide to require more dependencies, or newer versions of
  them.

Et cetera. Debian has one set of quality factors it particularly cares
about, and some upstreams think differently.

-- 
Schrödinger's backup hypothesis: the condition of any backup is
undefined until a restore is attempted. -- andrewsh

[toc] | [prev] | [next] | [standalone]

#8684

[Multipart message — attachments visible in raw view] — view raw

On Samstag, 21. Mai 2016 10:24:22 CEST Martin Steigerwald wrote:
> I wonder about some kind of adopt an upstream within a Debian team kind of 
> approach. A landing page and mailing list where upstream can write in for 
> getting help and advice and voicing their needs. And when there are people
> in  Debian willing to work with upstream they can build a team.
[…]
> I do think spelling out an invitation to work together officially can help
> to  attract more upstreams to work closer with Debian.

Also explaining on that page to upstream why Debian has the quality and 
stability requirements it has and what benefits this can create for upstreams 
including with offers on how to meet different upstream needs where that would 
be possible.

-- 
Martin

[toc] | [prev] | [next] | [standalone]

#8685

[Multipart message — attachments visible in raw view] — view raw

On Samstag, 21. Mai 2016 10:24:06 CEST Lars Wirzenius wrote:
> Et cetera. Debian has one set of quality factors it particularly cares
> about, and some upstreams think differently.

Yes, I seen all those reasons you mentioned.

I just wonder how about if upstreams can learn easily how to work together 
with Debian developers and maintainers which basically would mean offloading 
some of the work involved?

Still, the turn around time between upstream and debian release would be quite 
high for Debian stable users, but maybe part of such a collaboration could be 
to also provide newer releases via backports. Also… if upstream wants to 
release the built packages even quicker to testers or adventurous people, why 
not allow them to put newer versions of the official packages into their own 
repo while still integrating them with the official repo? For Owncloud for 
example this could lead at least to *compatible* packaging. Right now 
switching between Debian packages and upstream packages basically destroys a 
working Owncloud installation and requires quite some manual interaction to 
get things working again. But with compatible packages people could easily 
switch between "I use stable packages", "I use backport packages" and "I don´t 
care I want the latest I add the upstream repo".


I wonder about some kind of adopt an upstream within a Debian team kind of 
approach. A landing page and mailing list where upstream can write in for 
getting help and advice and voicing their needs. And when there are people in 
Debian willing to work with upstream they can build a team.

May still not be suitable for some upstreams or some upstreams not be suitable 
for Debian, but I got the impression that Debian may be seen as elitist and 
basically unapproachable by some upstreams – and that in quite some cases like 
with the Owncloud packaging a different approach would be possible. (Actually 
that is what I thought for a long time as well, until I found sponsors for 
some package I wanted in the archive and well, until Debconf 2015 in 
Heidelberg.)

I do think spelling out an invitation to work together officially can help to 
attract more upstreams to work closer with Debian.

-- 
Martin

[toc] | [prev] | [next] | [standalone]

#8688

[Multipart message — attachments visible in raw view] — view raw

 ❦ 21 mai 2016 10:24 +0200, Martin Steigerwald <martin@lichtvoll.de> :

> Still, the turn around time between upstream and debian release would be quite 
> high for Debian stable users, but maybe part of such a collaboration could be 
> to also provide newer releases via backports. Also… if upstream wants to 
> release the built packages even quicker to testers or adventurous people, why 
> not allow them to put newer versions of the official packages into their own 
> repo while still integrating them with the official repo? For Owncloud for 
> example this could lead at least to *compatible* packaging. Right now 
> switching between Debian packages and upstream packages basically destroys a 
> working Owncloud installation and requires quite some manual interaction to 
> get things working again. But with compatible packages people could easily 
> switch between "I use stable packages", "I use backport packages" and "I don´t 
> care I want the latest I add the upstream repo".

Owncloud upstream seems quite hostile towards Debian. But your
proposition works with some other upstreams. For example, we are doing
all the packaging work for HAProxy, both official and unofficial
packages (more backports, backports to Ubuntu) and upstream is quite
happy (while in the past, upstream asked us to not ship HAProxy in
Debian because it would be too old).

 http://mozilla.debian.net/
 http://haproxy.debian.net/
 http://ganeti.debian.net/

I think those packages are ideal to keep everyone happy. People can
choose whatever they want and bear with the consequences. And the
packages are "top" quality because they are derived from the packages in
unstable.

In some cases, upstream accepts/wants to host packaging in its own git
repository as well and is happy to help (I have librdkafka/kafkacat and
ExaBGP as examples). Therefore, they can do their own releases as well.

Maybe once we have PPA, we could mainstream a bit the wizard and propose
more options.

However, the examples above are compatible with our way of
packaging. Would we want spend time on packaging stuff that would never
go to the Debian archive due to excessive vendoring or unwilling from
upstream to be in a stable release?

Now, I usually ask upstream if they would be interested to have their
software in Debian and then, I propose for them to maintain it or
comaintain it. Many are happy with that but some just say no. I don't
keep tabs, but here is one example (not my own request):

 https://github.com/jordansissel/fpm/issues/409
-- 
If you laid all of our laws end to end, there would be no end.
		-- Mark Twain

[toc] | [prev] | [next] | [standalone]

#8691

[Multipart message — attachments visible in raw view] — view raw

On Samstag, 21. Mai 2016 10:53:34 CEST Vincent Bernat wrote:
>  ❦ 21 mai 2016 10:24 +0200, Martin Steigerwald <martin@lichtvoll.de> :
> > Still, the turn around time between upstream and debian release would be
> > quite high for Debian stable users, but maybe part of such a
> > collaboration could be to also provide newer releases via backports.
> > Also… if upstream wants to release the built packages even quicker to
> > testers or adventurous people, why not allow them to put newer versions
> > of the official packages into their own repo while still integrating them
> > with the official repo? For Owncloud for example this could lead at least
> > to *compatible* packaging. Right now switching between Debian packages
> > and upstream packages basically destroys a working Owncloud installation
> > and requires quite some manual interaction to get things working again.
> > But with compatible packages people could easily switch between "I use
> > stable packages", "I use backport packages" and "I don´t care I want the
> > latest I add the upstream repo".
> 
> Owncloud upstream seems quite hostile towards Debian. But your
> proposition works with some other upstreams. For example, we are doing

Yeah, with Owncloud I still hope for changes, but I think it needs to happen 
on both sides and I saw some willingness on side of upstream, but also quite 
some concern. Also upstream project seems changing quite a bit with people 
leaving Owncloud Inc.

> all the packaging work for HAProxy, both official and unofficial
> packages (more backports, backports to Ubuntu) and upstream is quite
> happy (while in the past, upstream asked us to not ship HAProxy in
> Debian because it would be too old).
> 
>  http://mozilla.debian.net/
>  http://haproxy.debian.net/
>  http://ganeti.debian.net/
> 
> I think those packages are ideal to keep everyone happy. People can
> choose whatever they want and bear with the consequences. And the
> packages are "top" quality because they are derived from the packages in
> unstable.

Okay. I was aware of mozilla.debian.net, but not the others, well, that would 
be also a nice approach for upstream which might be nice to mention on 
upstream landing page.

I never thought this is available on a more general base.

> However, the examples above are compatible with our way of
> packaging. Would we want spend time on packaging stuff that would never
> go to the Debian archive due to excessive vendoring or unwilling from
> upstream to be in a stable release?

No. I don´t think that is a good idea and so I agree with David´s decision 
regarding Owncloud. He spend *a lot* of work which will not end up in Debian 
Stretch, at least not with the current situation.

But in some case I am not sure whether there have been any serious attempts to 
talk and find solutions that work. Maybe currently its not feasible with 
Owncloud, but I do think the current situation is a loss for both upstream and 
Debian.

> Now, I usually ask upstream if they would be interested to have their
> software in Debian and then, I propose for them to maintain it or
> comaintain it. Many are happy with that but some just say no. I don't
> keep tabs, but here is one example (not my own request):
> 
>  https://github.com/jordansissel/fpm/issues/409

*sigh*, not the first time I heard that Jordan is not fond of Debian packaging 
guidelines and more of a Fedora guy.

-- 
Martin

[toc] | [prev] | [next] | [standalone]

#8683

Hello Paul,

On Samstag, 21. Mai 2016 14:07:53 CEST Paul Wise wrote:
> On Fri, May 20, 2016 at 1:34 PM, Vincent Bernat wrote:
> > Totally agree. Our standards are far too high for many upstreams.
> 
> I don't understand the disconnect here. Are upstreams not interested
> in software quality to the extent we are?

I know one quite popular example where some upstream people even objected 
having the software packaged in Debian:

Unfit upstream / RM: owncloud -- ROM; PHP 7.0 Transition
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=816376

I think similar can be said for many web applications that are just developed 
at a very high pace. Combine that with an upstream policy not to support 
skipping updates between major versions… by supporting redoing older update 
steps and you have something that is compatible with Debian stable standards.

Now I know other web apps where this works better, like Wordpress, that now 
even has a wonderfully working backport. Still this has brought up a principal 
issue that is still unsolved.

I know of some other upstreams that provided packages themselves for… whatever 
reasons, might be a good idea to ask them.

So for example basically all log analysis and indexing software that seems to 
be so popular in the last time:

- Elasticsearch: newer versions than in unstable
- Logstash
- Kibana
- Fluentd
- Graylog

Also things like OpenNebula where I have been involved a bit failed in Debian 
due to lack of manpower combined with that some effort to work together did 
not play out probably due to lack of manpower on upstream side as well. 
Instead they just do a stuff it all in one or a few packages no matter what. 
Which is similar to how Owncloud upstream does packages. It is basically like 
a huge tarball, all just stuff in /var/html, including config files and be 
done with it. Logstash packages at least have their config files in /etc, yet 
all else in /opt/logstash or so.

I suspect one of the reasons is the perceived simplicity of just doing it 
yourself without having to abide to all the rules, to have the package lintian 
clean (I never checked those against lintian tough), and to generally get more 
work done in less time, even if that work has a lower quality than what is 
usually accepted into Debian.

> > I am always flabestered by the popularity of fpm to build Debian
> > packages (and by the increasing popularity of pleaserun by the same
> > author on the same concepts). It provides a way to easily build a Debian
> > package from a directory but produces somewhat crippled/incomplete
> > packages and is no help to us since it's completely outside of any of
> > our tools. It also handles RPM (and now other package formats), but I
> > don't think this would explain its popularity alone.
> 
> I think we probably need to get dpkg-buildpackage to automatically run
> some of these:
> 
> https://wiki.debian.org/AutomaticPackagingTools

I am very happy about Maxy´s work to automate Plasma 5 and KDE Frameworks 5 
packaging. Here there is a similar issue:

Upstream has way faster release cycles and additionally they splitted up their 
stuff into a lot more single projects. With KDE 3, a bit less with KDE SC 4 it 
was a number of larger packages that you could even still count easily enough 
with your fingers. With Plasma 5  and KDE Frameworks 5 upstream provides 
literally hundreds of tarballs and git repos to package in order to get a full 
desktop experience. According to cd /usr/share/doc/libkf<TAB><TAB> I have 213 
library packages installed on my system.

Maxy went to a great extent of automating things, which in some cases can lead 
to this:

karchive (5.22.0-1) unstable; urgency=medium

  [ Automatic packaging ]
  * Update symbols files from buildds logs (5.21.0-1).
  * Update symbols files.
  * Update build-deps and deps with the info from cmake

 -- Maximiliano Curia <[…]>  Thu, 19 May 2016 00:19:49 +0200

In other cases it is a mix of manual and automatic work:

karchive (5.21.0-1) experimental; urgency=medium

  [ Maximiliano Curia ]
  * Replace the "Historical name" ddeb-migration by its "Modern, clearer" 
replacement dbgsym-migration.
  * Add upstream metadata (DEP-12)
  * debian/control: Update Vcs-Browser and Vcs-Git fields
  * Update acc test script
  * uscan no longer supports more than one main upstream tarball being listed

  [ Automatic packaging ]
  * Update build-deps and deps with the info from cmake
  * Update symbols files.
  * Bump Standards-Version to 3.9.8

 -- Maximiliano Curia <[…]>  Thu, 05 May 2016 15:04:24 +0200


I am aware of similar efforts for a lot of different things like haskell 
packages, where Joachim Breitner and other teams upload literally hundreds of 
packages in a single day. I don´t know whether someone has an overview on 
whats available and what is used in the different packaging teams. But I do 
think the wiki page you linked is not complete.


I wonder about a landing page for upstreams interested in working with the 
Debian project to provide packages within the official Debian repos. Sure 
there is new maintainers guide and debian-mentors mailing list, but is there 
any official page where upstream can see whom to approach for help and where 
to learn how to interact efficiently with Debian developers and maintainers in 
order to learn the process for doing it within Debian?

Maybe something similar to what the KDE project has started the other way 
around: They have started a new effort to communicate and interact with distro 
people, where packagers can raise their concerns and needs, ask questions. And 
where KDE developers can also say what they want from distributions.

Maybe create a landing page and mailing list for upstreams to voice what they 
need and why they do things the way they do (package on their own)? At least 
it provided the opportunity to learn why upstreams do it the way they do.

Thanks,
-- 
Martin

[toc] | [prev] | [next] | [standalone]

#8686

[Multipart message — attachments visible in raw view] — view raw

On Sat, May 21, 2016 at 10:07:43AM +0200, Martin Steigerwald wrote:
> I wonder about a landing page for upstreams interested in working with the 
> Debian project to provide packages within the official Debian repos.

Is https://wiki.debian.org/UpstreamGuide the kind of page you mean? It
is not necessarily well known.

-- 
Schrödinger's backup hypothesis: the condition of any backup is
undefined until a restore is attempted. -- andrewsh

[toc] | [prev] | [next] | [standalone]

#8690

[Multipart message — attachments visible in raw view] — view raw

On Samstag, 21. Mai 2016 11:13:41 CEST Lars Wirzenius wrote:
> On Sat, May 21, 2016 at 10:07:43AM +0200, Martin Steigerwald wrote:
> > I wonder about a landing page for upstreams interested in working with the
> > Debian project to provide packages within the official Debian repos.
> 
> Is https://wiki.debian.org/UpstreamGuide the kind of page you mean? It
> is not necessarily well known.

Oh, its all there already, *including* a mailing list.

Ok, so maybe all whats needed for now is some promo work on this? And probably 
updating here and there.

I was totally unaware of this.

Thanks,
-- 
Martin

[toc] | [prev] | [next] | [standalone]

#8682

Vincent Bernat <bernat@debian.org> writes:
>  ❦ 19 mai 2016 18:04 +0100, Ian Jackson <ijackson@chiark.greenend.org.uk> :
>>> b) many upstreams appear frustrated about getting their package
>>> officially supported in Debian.  Sometimes there is good reason their
>>> package doesn't belong in Debian but sometimes it is more about inertia
>>> in Debian or the upstream isn't aware about backports and thinks their
>>> package will be stuck at a particular version forever
>>
>> Providing a proper Debian source package is also a lot more work than
>> writing some kind of ad-hoc build system that spits out a .deb or
>> three.
>
> Totally agree. Our standards are far too high for many upstreams.

which is a Good Thing.

I usually put quite much effort to bring the packages to the Debian
level, and at the end many upstreams are convinced that our standards
help them as well on the long run.

Best

Ole

[toc] | [prev] | [next] | [standalone]

#8689

[Multipart message — attachments visible in raw view] — view raw

 ❦ 21 mai 2016 09:40 +0200, Ole Streicher <olebole@debian.org> :

>>> Providing a proper Debian source package is also a lot more work than
>>> writing some kind of ad-hoc build system that spits out a .deb or
>>> three.
>>
>> Totally agree. Our standards are far too high for many upstreams.
>
> which is a Good Thing.
>
> I usually put quite much effort to bring the packages to the Debian
> level, and at the end many upstreams are convinced that our standards
> help them as well on the long run.

The problem is with upstreams that (sometimes loudly) complains that our
standards are useless. Users will still install their packages and
rumble against us. Of course, I am not saying that we should lower our
standards, but it is a difficult world.
-- 
Use self-identifying input.  Allow defaults.  Echo both on output.
            - The Elements of Programming Style (Kernighan & Plauger)

[toc] | [prev] | [next] | [standalone]

#8651

On 05/19/2016 06:18 PM, Daniel Pocock wrote:
>
> More and more frequently I'm encountering systems where third-party
> repositories have been added into /etc/apt/sources.list or
> /etc/apt/sources.list.d, usually put there by some .deb package that a
> user installed from some third party site.
>

Hey, This is a good thing™! I'm glad some upstreams care enough for 
Debian that they build deb packages of their releases and I'm even more 
delighted they are doing it properly by serving them in an apt repository.

This doesn't mean there is no benefit in discussing problematic areas or 
potential improvements in providing them also in Debian repository, but 
please don't pressure the poor upstreams too much making them feel like 
they are doing a disservice by providing a deb package to their users 
comfort.

>  From a technical perspective, can we do more to prevent users being
> surprised by packages putting new entries in /etc/apt/sources.list.d?
>
Please no. The system is working as intended. I don't think anybody is 
surprised after installing a 3rd party deb package and seeing it adds 
its apt repository to the system, actually they would be pleased to know 
that they won't have to manually update it later. And anybody who is 
uncomfortable with it would either disable it or not install the deb in 
the first place.



-------------------------------------------------

ONLY AT VFEmail! - Use our Metadata Mitigator to keep your email out of the NSA's hands!
$24.95 ONETIME Lifetime accounts with Privacy Features!  
15GB disk! No bandwidth quotas!
Commercial and Bulk Mail Options!  

[toc] | [prev] | [next] | [standalone]

#8672

Le 19/05/2016 19:20, Hakan Peker a écrit :
> On 05/19/2016 06:18 PM, Daniel Pocock wrote:
>>  From a technical perspective, can we do more to prevent users being
>> surprised by packages putting new entries in /etc/apt/sources.list.d?
>>
> Please no. The system is working as intended. I don't think anybody is surprised after installing a 3rd party deb package and seeing it adds its apt repository to the system, actually they would be pleased to know that they won't have to manually update it later. And anybody who is uncomfortable with it would either disable it or not install the deb in the first place.

  Please yes.

  I built some (local) packages to easily install the same sources.list
on several machines, so I agree the feature in useful and interesting.
I named my packages like *-sources and the description is explicit.

  But I've been very suprised when I discovered that the google-earth
package installed a sources.list on my parents' computer after I
downloaded and install an application deb with dpkg.
  And even more when I see that the source is silently reinstalled when
you remove it (no respect of admin modification)

  Would it be possible to have a package that install a trigger and
advertise (debconf/mail/...) when such a change occurs? I would install
it anywhere personally.

  Regards,
    Vincent

-- 
Vincent Danjean       GPG key ID 0xD17897FA         vdanjean@debian.org
GPG key fingerprint: 621E 3509 654D D77C 43F5  CA4A F6AE F2AF D178 97FA
Unofficial pkgs: http://moais.imag.fr/membres/vincent.danjean/deb.html
APT repo:  deb http://people.debian.org/~vdanjean/debian unstable main

[toc] | [prev] | [next] | [standalone]

Page 2 of 3 — ← Prev page 1 [2] 3  Next page →

Back to top | Article view | linux.debian.project

csiph-web