Groups | Search | Server Info | Keyboard shortcuts | Login | Register [http] [https] [nntp] [nntps]

Groups > linux.debian.user > #171239 > unrolled thread

Frequent Hash Sum mismatch on jessie-backports with apt-get update

Back to article view | Back to linux.debian.user

Contents

  Frequent Hash Sum mismatch on jessie-backports with apt-get update poiuytrez <guillaume@databerries.com> - 2016-08-18 14:10 +0200
    Re: Frequent Hash Sum mismatch on jessie-backports with apt-get  update Brad Rogers <brad@fineby.me.uk> - 2016-08-18 14:40 +0200
    Re: Frequent Hash Sum mismatch on jessie-backports with apt-get update Greg Wooledge <wooledg@eeg.ccf.org> - 2016-08-18 15:00 +0200
      Re: Frequent Hash Sum mismatch on jessie-backports with apt-get  update Henrique de Moraes Holschuh <hmh@debian.org> - 2016-08-18 17:40 +0200
        Re: Frequent Hash Sum mismatch on jessie-backports with apt-get update Greg Wooledge <wooledg@eeg.ccf.org> - 2016-08-18 20:30 +0200
          Re: Frequent Hash Sum mismatch on jessie-backports with apt-get  update Curt <curty@free.fr> - 2016-08-19 12:20 +0200

#171239 — Frequent Hash Sum mismatch on jessie-backports with apt-get update

Hello, 

I have frequent errors when I do apt-get update: 
W: Failed to fetch http://http.debian.net/debian/dists/jessie-backports/main/source/Sources  Hash Sum mismatch

W: Failed to fetch http://http.debian.net/debian/dists/jessie-backports/main/i18n/Translation-en  Hash Sum mismatch

W: Failed to fetch http://http.debian.net/debian/dists/jessie-backports/main/binary-amd64/PackagesIndex  Hash Sum mismatch

Sometimes it works, sometimes it doesn't. The problem appeared on all my servers running Debian 8 (hosted on Google Cloud Platform) the last couple of days. 

What's wrong and what would you recommend?

Thank you for your help.
poiuytrez

PS: cat /etc/apt.sources.list
deb http://httpredir.debian.org/debian/ jessie main
deb http://security.debian.org/ jessie/updates main
deb http://httpredir.debian.org/debian/ jessie-updates main
deb http://http.debian.net/debian jessie main
deb-src http://http.debian.net/debian jessie main

cat /etc/apt/sources.list.d/backports.list
deb http://httpredir.debian.org/debian/ jessie-backports main
deb-src http://httpredir.debian.org/debian/ jessie-backports main
deb http://http.debian.net/debian jessie-backports main
deb-src http://http.debian.net/debian jessie-backports main

[toc] | [next] | [standalone]

#171241 — Re: Frequent Hash Sum mismatch on jessie-backports with apt-get update

[Multipart message — attachments visible in raw view] — view raw

On Thu, 18 Aug 2016 04:47:55 -0700 (PDT)
poiuytrez <guillaume@databerries.com> wrote:

Hello poiuytrez,

>What's wrong and what would you recommend?

If it persists for more than a day or two, I usually do;

# rm -r /var/lib/apt/lists/*

Not elegant but it works.

-- 
 Regards  _
         / )           "The blindingly obvious is
        / _)rad        never immediately apparent"
Watching the people get lairy
I Predict A Riot - Kaiser Chiefs

[toc] | [prev] | [next] | [standalone]

#171243

On Thu, Aug 18, 2016 at 04:47:55AM -0700, poiuytrez wrote:
> I have frequent errors when I do apt-get update: 
> W: Failed to fetch http://http.debian.net/debian/dists/jessie-backports/main/source/Sources  Hash Sum mismatch

For you, I would suggest trying a different mirror.

$ host http.debian.net
http.debian.net is an alias for httpredir.debian.org.

This redirection service works very well for some people, and very poorly
for other people.  You appear to be in the latter group.

You can either select a single mirror directly from:
https://www.debian.org/mirror/list

(the single mirrors are listed at the bottom of the page)

or you can use one of the country-code round robin services, such as:
http://ftp.us.debian.org/debian/
http://ftp.ca.debian.org/debian/

(the round robins are listed at the top of the page).

Obviously you would want to choose a country that's close to you,
network-wise.

> Sometimes it works, sometimes it doesn't. The problem appeared on all my
> servers running Debian 8 (hosted on Google Cloud Platform) the last couple of
> days. 

I have also experienced a MASSIVE problem with apt-get in the last week
or so.  I don't know if it's the same problem you're having, or totally
unrelated, but I am going to piggyback on your thread.

I have my problem at WORK only.  At work, I am trapped inside a firewalled
network.  I have no control over the firewall.  The entities who control
it are mysterious dark wizards who do not reveal themselves or their
secrets to mere mortals like me.

One of the past features of this firewall was a "transparent" (worst term
ever) HTTP proxy with forced authentication (user name and password).
I was able to plead to get an exception in place.  HTTP connections coming
from my desktop workstation's IP address weren't required to perform a
Windows Active Directory login to the HTTP proxy.  So I put a Squid proxy
on my workstation and told all of the Debian servers to use that for apt.

This worked well enough for years.

Now, starting last week, the problem is that when I try to apt-get update,
the files coming from security.debian.org HANG at 100% complete.  It
looks something like this:

root@ebase-fla:~# apt-get update
Hit http://ftp.us.debian.org wheezy Release.gpg
Hit http://ftp.us.debian.org wheezy-updates Release.gpg                        
Hit http://ftp.us.debian.org wheezy Release                                    
Hit http://ftp.us.debian.org wheezy-updates Release                            
Hit http://ftp.us.debian.org wheezy/main Sources             
Hit http://ftp.us.debian.org wheezy/contrib Sources                            
Hit http://ftp.us.debian.org wheezy/non-free Sources                           
Hit http://ftp.us.debian.org wheezy/main amd64 Packages                        
Hit http://ftp.us.debian.org wheezy/contrib amd64 Packages                     
Hit http://ftp.us.debian.org wheezy/non-free amd64 Packages                    
Hit http://ftp.us.debian.org wheezy/contrib Translation-en                     
Hit http://ftp.us.debian.org wheezy/main Translation-en                        
Hit http://ftp.us.debian.org wheezy/non-free Translation-en                    
Hit http://ftp.us.debian.org wheezy-updates/main Sources                       
Hit http://ftp.us.debian.org wheezy-updates/contrib Sources                    
Hit http://ftp.us.debian.org wheezy-updates/non-free Sources                   
Hit http://ftp.us.debian.org wheezy-updates/main amd64 Packages/DiffIndex
Hit http://ftp.us.debian.org wheezy-updates/contrib amd64 Packages             
Hit http://ftp.us.debian.org wheezy-updates/non-free amd64 Packages/DiffIndex  
Hit http://ftp.us.debian.org wheezy-updates/contrib Translation-en             
Hit http://ftp.us.debian.org wheezy-updates/main Translation-en/DiffIndex      
Hit http://ftp.us.debian.org wheezy-updates/non-free Translation-en/DiffIndex  
Hit http://security.debian.org wheezy/updates Release.gpg                      
Hit http://security.debian.org wheezy/updates Release
Hit http://security.debian.org wheezy/updates/contrib Sources
Hit http://security.debian.org wheezy/updates/non-free Sources
Get:1 http://security.debian.org wheezy/updates/main amd64 Packages [426 kB]
Hit http://security.debian.org wheezy/updates/contrib amd64 Packages           
Hit http://security.debian.org wheezy/updates/non-free amd64 Packages
Hit http://security.debian.org wheezy/updates/contrib Translation-en
Hit http://security.debian.org wheezy/updates/non-free Translation-en          
Get:2 http://security.debian.org wheezy/updates/main Sources [276 kB]
Get:3 http://security.debian.org wheezy/updates/main amd64 Packages [426 kB]   
Ign http://security.debian.org wheezy/updates/main Translation-en              
Get:4 http://security.debian.org wheezy/updates/main Sources [276 kB]
Err http://security.debian.org wheezy/updates/main amd64 Packages
  404  Not Found [IP: 128.101.240.215 80]
Fetched 69.9 kB in 22min 8s (52 B/s)
W: Failed to fetch http://security.debian.org/dists/wheezy/updates/main/source/Sources  Hash Sum mismatch

W: Failed to fetch http://security.debian.org/dists/wheezy/updates/main/binary-amd64/Packages  404  Not Found [IP: 128.101.240.215 80]

E: Some index files failed to download. They have been ignored, or old ones used instead.


So, after 22 minutes of waiting for that to fail, I try again.  And again.
And again.  And again.  Every 20 to 30 minutes, which is how long it
takes to finally give up and die.

The "hanging at 100%" symptom isn't visible at the very end.  It's only
visible during the apt-get.  It gets erased from the terminal when apt
finally gives up.  That symptom looks like:

...
Get:3 http://security.debian.org wheezy/updates/main Sources [277 kB]
...
Hit http://ftp.us.debian.org wheezy-updates/main Translation-en/DiffIndex      
Hit http://ftp.us.debian.org wheezy-updates/non-free Translation-en/DiffIndex
100% [3 Sources 276 kB/277 kB 100%]_

where _ is the terminal's cursor.  Sitting.  Waiting.  Eternally.
(Or, for 20-30 minutes, which merely feels like eternity.)

I've tried bypassing the Squid proxy.  Maybe the dark wizards dropped the
Windows AD authentication requirement based on source IP, or destination
domain name, or the alignment of the planets, or god-only-knows-what.
It makes no difference.  I get the same symptoms whether I go through
my Squid proxy or not.

I've tried Googling, which led me to look into Acquire::http::No-Cache.

Here's one of my apt.conf attempts:

root@ebase-fla:~# cat /etc/apt/apt.conf
// Acquire::http::Proxy "http://imadev.eeg.ccf.org:3128/";
Acquire::http::No-Cache "true";

This didn't change the behavior either.

Nothing I've tried has worked.  Occasionally after a few days of trying,
one of my machines will get LUCKY, but it is not reproducible.

Apparently there is also no way for me to use a SOCKS proxy (i.e. an
ssh -D tunnel) with apt, which is what I normally use for my web browsing.
Not that I would be wild about forcing all of the workplace servers to
use my private ssh -D tunnel for their legitimate business-relevant
security updates, but at least it would *work*.  Except that, y'know, it
doesn't actually work.  There isn't any apt-get-through-socks feature
that I can see.

To try to identify the dark wizards' proxy, I attempt a manual HTTP
session:

imadev:~$ telnet localhost 3128
Trying...
Connected to localhost.
Escape character is '^]'.
HEAD http://www.debian.org/ HTTP/1.0

HTTP/1.0 200 OK
Date: Thu, 18 Aug 2016 12:37:01 GMT
Server: Apache
Content-Location: index.en.html
Vary: negotiate,accept-language,Accept-Encoding
TCN: choice
Last-Modified: Thu, 18 Aug 2016 03:43:23 GMT
ETag: "39cf-53a5062360bac"
Accept-Ranges: bytes
Content-Length: 14799
Cache-Control: max-age=86400
Expires: Fri, 19 Aug 2016 12:37:01 GMT
X-Clacks-Overhead: GNU Terry Pratchett
Content-Type: text/html
Content-Language: en
Age: 0
X-Cache: MISS from imadev
Via: 1.1 wfmc-cluster, 1.0 imadev:3128 (squid/2.7.STABLE9)
Connection: close

Connection closed by foreign host.


That's all the information I have: "Via: 1.1 wfmc-cluster".  I've tried
Googling that, with no success.  Maybe "wfmc-cluster" is a hostname,
in which case I have zero information about what software it's running.
(No, it does not resolve.  I tried that.)

My Debian system at home, and my Debian VPS, do not have this issue.
Nobody on IRC seems to be having this issue.  My tentative conclusion
is that it must be related to my workplace's HTTP proxy.

If anyone has suggestions for how to get apt to work around a workplace's
"transparent" (as mud) HTTP proxy, I'd love to hear them.

[toc] | [prev] | [next] | [standalone]

#171254 — Re: Frequent Hash Sum mismatch on jessie-backports with apt-get update

On Thu, 18 Aug 2016, Greg Wooledge wrote:
> Now, starting last week, the problem is that when I try to apt-get update,
> the files coming from security.debian.org HANG at 100% complete.  It
> looks something like this:

While it really might be something else completely, the anti-virus
function of an "UTM" box (firewall+av+ips+ids+content filter+kitchen
sink) box is your first suspect.

You could try using https. It might help, or it might make matters much
worse.

-- 
  Henrique Holschuh

[toc] | [prev] | [next] | [standalone]

#171269

On Thu, Aug 18, 2016 at 12:33:27PM -0300, Henrique de Moraes Holschuh wrote:
> On Thu, 18 Aug 2016, Greg Wooledge wrote:
> > Now, starting last week, the problem is that when I try to apt-get update,
> > the files coming from security.debian.org HANG at 100% complete.  It
> > looks something like this:
> 
> While it really might be something else completely, the anti-virus
> function of an "UTM" box (firewall+av+ips+ids+content filter+kitchen
> sink) box is your first suspect.
> 
> You could try using https. It might help, or it might make matters much
> worse.

Around 10:30 this morning, it started working again all on its own.  I'm
guessing one of the networking people restarted something.

There is a phenomenon in one of the online communities that I engage with.
A sort of wink-nudge superstition, if you will: "If you want something
to happen, complain in chat about it not happening."  Posting in IRC
didn't work in this case, but apparently posting to debian-user worked.

[toc] | [prev] | [next] | [standalone]

#171292 — Re: Frequent Hash Sum mismatch on jessie-backports with apt-get update

On 2016-08-18, Greg Wooledge <wooledg@eeg.ccf.org> wrote:
>
> There is a phenomenon in one of the online communities that I engage with.
> A sort of wink-nudge superstition, if you will: "If you want something
> to happen, complain in chat about it not happening."  Posting in IRC
> didn't work in this case, but apparently posting to debian-user worked.
>

https://en.wikipedia.org/wiki/Magical_thinking

-- 
 "Until the Lion learns to write, tales of the hunt shall always glorify the Hunter."
— African proverb

[toc] | [prev] | [standalone]

Back to top | Article view | linux.debian.user

csiph-web