Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.project > #8594 > unrolled thread

shutting down httpredir.debian.org?

Back to article view | Back to linux.debian.project

Contents

  shutting down httpredir.debian.org? Peter Palfrader <weasel@debian.org> - 2016-04-12 09:10 +0200
    Re: shutting down httpredir.debian.org? Raphael Hertzog <hertzog@debian.org> - 2016-04-12 09:40 +0200
      Re: shutting down httpredir.debian.org? Peter Palfrader <weasel@debian.org> - 2016-04-12 10:00 +0200
        Re: shutting down httpredir.debian.org? Cord Beermann <cord@debian.org> - 2016-04-12 12:30 +0200
        Re: shutting down httpredir.debian.org? Christian Rohmann <crohmann@netcologne.de> - 2016-04-12 13:00 +0200
          Re: shutting down httpredir.debian.org? Raphael Hertzog <hertzog@debian.org> - 2016-04-13 10:30 +0200
            Re: shutting down httpredir.debian.org? Christian Rohmann <crohmann@netcologne.de> - 2016-04-14 10:00 +0200
              Re: shutting down httpredir.debian.org? Peter Palfrader <weasel@debian.org> - 2016-04-14 11:10 +0200
                Re: shutting down httpredir.debian.org? Bastian Blank <waldi@debian.org> - 2016-04-14 12:10 +0200
                  Re: shutting down httpredir.debian.org? David Kalnischkies <david@kalnischkies.de> - 2016-04-14 15:40 +0200
                  Re: shutting down httpredir.debian.org? Tollef Fog Heen <tfheen@err.no> - 2016-04-15 06:50 +0200
                    Re: shutting down httpredir.debian.org? Ondřej Surý <ondrej@sury.org> - 2016-04-25 14:30 +0200
                      Re: shutting down httpredir.debian.org? Tollef Fog Heen <tfheen@err.no> - 2016-04-25 14:50 +0200
                        Re: shutting down httpredir.debian.org? Ondřej Surý <ondrej@sury.org> - 2016-04-25 15:10 +0200
                          Re: shutting down httpredir.debian.org? David Kalnischkies <david@kalnischkies.de> - 2016-04-25 15:50 +0200
                            Re: shutting down httpredir.debian.org? Peter Palfrader <weasel@debian.org> - 2016-04-25 17:40 +0200
                Re: shutting down httpredir.debian.org? anarcat <anarcat@orangeseeds.org> - 2016-04-14 17:00 +0200
                  Re: shutting down httpredir.debian.org? Charles Plessy <plessy@debian.org> - 2016-04-15 03:20 +0200
                    Re: shutting down httpredir.debian.org? Paul Wise <pabs@debian.org> - 2016-04-15 04:00 +0200
                  Re: shutting down httpredir.debian.org? Tollef Fog Heen <tfheen@err.no> - 2016-04-15 07:00 +0200
            Re: shutting down httpredir.debian.org? Christian Rohmann <crohmann@netcologne.de> - 2016-04-14 14:00 +0200
              Re: shutting down httpredir.debian.org? Raphael Hertzog <hertzog@debian.org> - 2016-04-14 17:00 +0200
                Re: shutting down httpredir.debian.org? Christian Rohmann <crohmann@netcologne.de> - 2016-04-19 13:50 +0200
      Re: shutting down httpredir.debian.org? Paul Wise <pabs@debian.org> - 2016-04-12 18:30 +0200
    Re: shutting down httpredir.debian.org? Lucas Nussbaum <lucas@debian.org> - 2016-04-12 10:50 +0200
    Re: shutting down httpredir.debian.org? Raphael Geissert <geissert@debian.org> - 2016-04-12 14:00 +0200
      Re: shutting down httpredir.debian.org? Ben Hutchings <ben@decadent.org.uk> - 2016-04-12 15:10 +0200
      Re: shutting down httpredir.debian.org? Peter Palfrader <weasel@debian.org> - 2016-04-12 15:30 +0200
        Re: shutting down httpredir.debian.org? Raphael Geissert <geissert@debian.org> - 2016-04-13 14:30 +0200
      Re: shutting down httpredir.debian.org? Donald Norwood <dnorwood@portalus.com> - 2016-04-12 19:20 +0200
        Re: shutting down httpredir.debian.org? Donald Norwood <dnorwood@portalus.com> - 2016-04-12 22:50 +0200

Page 2 of 2 — ← Prev page 1 [2]

#8611

On 04/13/2016 10:23 AM, Raphael Hertzog wrote:
>>  * Intensive checks (even via rsync) regarding mirror consistency
> That's good too but the downside is that the mirrors must
> offer rsync service, either public or at least for the mirror redirector.
> It's not possible to use a mirror that cannot be scanned via rsync.

Likely wrong: http://mirrorbrain.org/docs/mirrors/#scanning-mirrors

There is FTP and HTTP fallback.



Regards

Christian

[toc] | [prev] | [next] | [standalone]

#8614

On Thu, 14 Apr 2016, Christian Rohmann wrote:
> On 04/13/2016 10:23 AM, Raphael Hertzog wrote:
> >>  * Intensive checks (even via rsync) regarding mirror consistency
> > That's good too but the downside is that the mirrors must
> > offer rsync service, either public or at least for the mirror redirector.
> > It's not possible to use a mirror that cannot be scanned via rsync.
> 
> Likely wrong: http://mirrorbrain.org/docs/mirrors/#scanning-mirrors
> 
> There is FTP and HTTP fallback.

Duh... yes there is, but have you tried scanning a Debian mirror
with FTP? It just never finishes. And with HTTP it's even worse, it's
not possible to scan unless the mirror has enabled directory listing (or
you try one by one all the files of the local mirror).

The intent of mirrorbrain is to update the file lists regularly
and with rsync I get a 2 minutes update once every hour. That's OK.
With FTP I never had the patience to see how long it actually took.

So for all practical matters, only rsync is usable.

Cheers,
-- 
Raphaël Hertzog ◈ Debian Developer

Support Debian LTS: http://www.freexian.com/services/debian-lts.html
Learn to master Debian: http://debian-handbook.info/get/

[toc] | [prev] | [next] | [standalone]

#8624

On 04/14/2016 04:49 PM, Raphael Hertzog wrote:
> So for all practical matters, only rsync is usable.

Agreed. So there will just be that requirement to provide (restricted)
rsync access for your mirror to be part of httpredr.debian.org.

How else would any other software achieve a completeness check? Sure,
there is Fedora with their mirrormanager
(https://fedorahosted.org/mirrormanager/) which has the report_mirror
tool simply do a "find" on the local mirror and then back to big mama
about the files and their checksum. But that's even more complicated to
deploy in the field or rather changes the infrastructure of Debian
mirrors even more.


Regards

Christian

[toc] | [prev] | [next] | [standalone]

#8603

On Tue, Apr 12, 2016 at 3:33 PM, Raphael Hertzog wrote:

> I'd like also to note that once we have proper by-hash package indices in
> Debian too, it's entirely reasonable to rely on MirrorBrain as HTTP
> redirector. I use it for Kali for more than 3 years already.
>
> http://mirrorbrain.org

BTW, there is a MirrorBrain instance for Debian's CD image mirrors here:

http://debian-cd.debian.net/

-- 
bye,
pabs

https://wiki.debian.org/PaulWise

[toc] | [prev] | [next] | [standalone]

#8597

On 12/04/16 at 07:04 +0000, Peter Palfrader wrote:
> Hi,
> 
> we keep getting reports of httpredir.debian.org not working correctly,
> such as intermittently just sending errors or redirecting to mirrors
> that are out of date.

This doesn't solve the issue of finding someone to maintain it, but
wouldn't it be easier if, instead of trying to use all existing Debian
mirrors, httpredir just used a smaller number of them, known to be quite
reliable?

Lucas

[toc] | [prev] | [next] | [standalone]

#8600

Hi,

On 12 April 2016 at 09:04, Peter Palfrader <weasel@debian.org> wrote:
> Hi,
>
> we keep getting reports of httpredir.debian.org not working correctly,
> such as intermittently just sending errors or redirecting to mirrors
> that are out of date.
[...]
> So, it appears as if currently nobody has time or the energy to take
> care of httpredir.debian.org properly.
>
> I suggest we shut down the service for now.  If, at some future point,
> somebody wants to maintain again we can always start it up again.

Let me summarise the current situation:
- user-visible errors happen from time to time,  possibly increasing over time
- the current code base has a high cost of maintenance
- the current infrastructure is not homogeneous, there are three
hosts, each launching the application in a different way
- the main code contributors (Simon and yours truly) have been
EBUSY/ENOTIME for a while - Simon, please correct me if I'm wrong
- the service has not been fully integrated into the mirror's team
workflow - ex. a recent change to ftpsync makes httpredir believe
mirrors do not handle some files correctly
- the service has not been fully integrated into the mirror's team
umbrella, notably contact addresses as weasel correctly noted
- the service has been widely welcome by users - a single address
- the service has superseded similar services: geomirror.d.n, cdn.d.n
- no other service exists that could supersede httpredir while
retaining control over it
- no other read-to-use solution exists that could supersede httpredir
without a significant cost (time)

(all points are TTBOMK and IMHO)

Given the above, I believe it to be in the best interest of the
project to further attempt to improve the service rather than shutting
it down.

What I propose is:
1. to fix the recent regressions and perform some maintenance on the
service in an attempt[1] to reduce the user-visible errors. Before the
end of April.
2. define the next steps towards improving the service - can be done
in an open way in the mirrors ML. Starting today, with no ending date.
3. organise a sprint in order to ensure that time is available to work
on the service. Perhaps during SunCamp, as people such as weasel also
appear to be interested in participating - so let's say end of May.

[1] as of this time there's no metric for this

Do people agree to the above?

PS. Regarding mirrorbrain, I can go into further details as to why it
doesn't fit our use case but I think it's orthogonal to the discussion
here.
It's a great project, Peter Pöml has done a great work, and httpredir
is inspired on it.

Cheers,
-- 
Raphael Geissert - Debian Developer
www.debian.org - get.debian.net

[toc] | [prev] | [next] | [standalone]

#8601

[Multipart message — attachments visible in raw view] — view raw

On Tue, 2016-04-12 at 13:52 +0200, Raphael Geissert wrote:
[...]
> - the main code contributors (Simon and yours truly) have been
> EBUSY/ENOTIME for a while - Simon, please correct me if I'm wrong
[...]

I assume this means you don't have spare time. Are either of you (or
anyone else with knowledge of the service) available to work on it in
the short term on a paid contract?  Of course Debian itself will not
pay for this, but some users might be willing to sponsor this work.

Ben.

-- 
Ben Hutchings
This sentence contradicts itself - no actually it doesn't.

[toc] | [prev] | [next] | [standalone]

#8602

On Tue, 12 Apr 2016, Raphael Geissert wrote:

> - the main code contributors (Simon and yours truly) have been
> EBUSY/ENOTIME for a while - Simon, please correct me if I'm wrong
 [*]

> What I propose is:
> 1. to fix the recent regressions and perform some maintenance on the
> service in an attempt[1] to reduce the user-visible errors. Before the
> end of April.
> 2. define the next steps towards improving the service - can be done
> in an open way in the mirrors ML. Starting today, with no ending date.
> 3. organise a sprint in order to ensure that time is available to work
> on the service. Perhaps during SunCamp, as people such as weasel also
> appear to be interested in participating - so let's say end of May.

Do you expect [*] to change in the near future?  The current status has
existed for a while, and whishful thinking hasn't helped to improve it.

What is needed here is continuous work be put into improving and
maintaining the service.  A one-off rush won't be sufficient.

[Currently SunCamp already has release move stuff on the list of things
 that need doing.]

Cheers,
-- 
                            |  .''`.       ** Debian **
      Peter Palfrader       | : :' :      The  universal
 https://www.palfrader.org/ | `. `'      Operating System
                            |   `-    https://www.debian.org/

[toc] | [prev] | [next] | [standalone]

#8607

Hi,

On 12 April 2016 at 15:25, Peter Palfrader <weasel@debian.org> wrote:
> On Tue, 12 Apr 2016, Raphael Geissert wrote:
>
>> - the main code contributors (Simon and yours truly) have been
>> EBUSY/ENOTIME for a while - Simon, please correct me if I'm wrong
>  [*]
>
>> What I propose is:
>> 1. to fix the recent regressions and perform some maintenance on the
>> service in an attempt[1] to reduce the user-visible errors. Before the
>> end of April.
>> 2. define the next steps towards improving the service - can be done
>> in an open way in the mirrors ML. Starting today, with no ending date.
>> 3. organise a sprint in order to ensure that time is available to work
>> on the service. Perhaps during SunCamp, as people such as weasel also
>> appear to be interested in participating - so let's say end of May.
>
> Do you expect [*] to change in the near future?  The current status has
> existed for a while, and whishful thinking hasn't helped to improve it.

I believe so, yes.

Cheers,
-- 
Raphael Geissert - Debian Developer
www.debian.org - get.debian.net

[toc] | [prev] | [next] | [standalone]

#8604

[Converted from HTML] — view raw

On 04/12/2016 07:52 AM, Raphael Geissert wrote:

> Hi,
> 
> 
> 
> What I propose is:
> 1. to fix the recent regressions and perform some maintenance on the
> service in an attempt[1] to reduce the user-visible errors. Before the
> end of April.
> 2. define the next steps towards improving the service - can be done
> in an open way in the mirrors ML. Starting today, with no ending date.
> 3. organise a sprint in order to ensure that time is available to work
> on the service. Perhaps during SunCamp, as people such as weasel also
> appear to be interested in participating - so let's say end of May.
> 

I would very much like to keep the service.

At this time it provides a clear source to point users to for connection to
the mirrors network, provides a great reporting tool as opposed to the static
status.html pages[1][2] which are confusing to read at times and have outdated
information and, lastly since it is already out there and in use (and
promoted) we may as well continue to use it provided we can address the needs
of it needing a dedicated maintainer or team of maintainers.

Some time ago Iain Learmonth was kind enough to take a look at the redirector
and was kind enough to created a minor status page[3] that pulls data from the
redirector, but I don't think he wanted to inherit it the overall project.

I think the service needs to be restarted every once in awhile if I am not
mistaken, which would update the available mirrors to it and keep Iain's page
in sync. 

Problems with the redirector should be have bugs opened against them, perhaps
removing the single point of contact on the service and stating 'please file a
bug against' would give us a better idea of where the visible user issues are
and allow us to implement change, modification, or documentation. More so as
those bugs are cc'd to the mirrors team.

Remove /ports/ from the redirector would be good, it just sits out there and
goes nowhere.

The mirrors team also needs to focus on the direction the tool will go in and
its implementation..even internally. 

[1][http://mirror-master.debian.org/status.html][1]
[2][http://ftp.de.debian.org/dmc/today/][2]
[3][http://debian-data.alioth.debian.org/mirrors.html][3]

-Donald


[1]: http://mirror-master.debian.org/status.html
[2]: http://ftp.de.debian.org/dmc/today/
[3]: http://debian-data.alioth.debian.org/mirrors.html

[toc] | [prev] | [next] | [standalone]

#8605

Sorry for the html earlier, my apologies.

[toc] | [prev] | [standalone]

Page 2 of 2 — ← Prev page 1 [2]

Back to top | Article view | linux.debian.project

csiph-web