Bionic apt update "File has unexpected size"

MotoFckr9k · December 16, 2020

When trying to run `apt update` on Bionic I am getting a `File has unexpected size` from the armbian mirrors. This has been the case for several hours now. This is with the Bionic xfce desktop image from 12th December 2020 (just installed today).

motofckr9k@rockpro64:~$ LC_ALL=C sudo apt update
Hit:1 http://ports.ubuntu.com bionic InRelease
Hit:2 http://ports.ubuntu.com bionic-security InRelease                                           
Hit:3 http://ports.ubuntu.com bionic-updates InRelease                                            
Hit:4 http://ports.ubuntu.com bionic-backports InRelease                                          
Get:5 https://minio.k-space.ee/armbian/apt bionic InRelease [18.3 kB]                             
Get:6 https://minio.k-space.ee/armbian/apt bionic/main arm64 Packages [758 kB]
Err:6 https://minio.k-space.ee/armbian/apt bionic/main arm64 Packages
  File has unexpected size (761349 != 758336). Mirror sync in progress? [IP: 193.40.103.4 443]
  Hashes of expected file:
   - Filesize:758336 [weak]
   - SHA512:f5e27558c3a89a5c74e8cdcbdeb5ac7d9689c1323a4b90cc056a0072ee10d2a078bd755dc19193d8ee9a8ee9765dbf07b2b2522b636639498feb07cd34d2c746
   - SHA256:ecd882b1aeac7ea6b94f80e6a5bc5eedd0bfc8267d18b33c5153653edaa4b9a9
   - SHA1:e6faf5bf69ff731ea978d9c6053cfdd5c8663f2e [weak]
   - MD5Sum:d9092324c3048d8d9c8d2247c47f58a9 [weak]
  Release file created at: Sat, 12 Dec 2020 14:54:36 +0000
Get:7 https://minio.k-space.ee/armbian/apt bionic/main armhf Packages [645 kB]
Err:7 https://minio.k-space.ee/armbian/apt bionic/main armhf Packages
  
Fetched 18.3 kB in 2s (10.3 kB/s)
Reading package lists... Done
E: Failed to fetch https://minio.k-space.ee/armbian/apt/dists/bionic/main/binary-arm64/Packages.gz  File has unexpected size (761349 != 758336). Mirror sync in progress? [IP: 193.40.103.4 443]
   Hashes of expected file:
    - Filesize:758336 [weak]
    - SHA512:f5e27558c3a89a5c74e8cdcbdeb5ac7d9689c1323a4b90cc056a0072ee10d2a078bd755dc19193d8ee9a8ee9765dbf07b2b2522b636639498feb07cd34d2c746
    - SHA256:ecd882b1aeac7ea6b94f80e6a5bc5eedd0bfc8267d18b33c5153653edaa4b9a9
    - SHA1:e6faf5bf69ff731ea978d9c6053cfdd5c8663f2e [weak]
    - MD5Sum:d9092324c3048d8d9c8d2247c47f58a9 [weak]
   Release file created at: Sat, 12 Dec 2020 14:54:36 +0000
E: Failed to fetch https://minio.k-space.ee/armbian/apt/dists/bionic/main/binary-armhf/Packages.gz  
E: Some index files failed to download. They have been ignored, or old ones used instead.
motofckr9k@rockpro64:~$

dispo · December 16, 2020

Same issue with Packages.bz2 but for buster

Get:6 https://armbian.systemonachip.net/apt buster/main armhf Packages [591 kB]
Err:6 https://armbian.systemonachip.net/apt buster/main armhf Packages
File has unexpected size (591174 != 590754). Mirror sync in progress? [IP: 51.75.2.52 443]

E: Failed to fetch https://armbian.systemonachip.net/apt/dists/buster/main/binary-armhf/Packages.bz2 File has unexpected size (591174 != 590754). Mirror sync in progress? [IP: 51.75.2.52 443]

Werner · December 16, 2020

Also noticed yesterday. Something is wrong there... @lanefu ?

lanefu · December 16, 2020

1 minute ago, Werner said:

Also noticed yesterday. Something is wrong there... @lanefu ?

Blerg. The error is telling the truth. Most likely mirror sync. Mirror validation is something for us to work on next.

Igor · December 16, 2020

This might help:

sudo apt-get -o Acquire::ForceIPv4=true -o Acquire::CompressionTypes::Order::=gz -o Acquire::http::No-Cache=true -o Acquire::BrokenProxy=true -o Acquire::http::Pipeline-Depth=0 update

dispo · December 16, 2020

Sadly that didn't work for me - thank you for your efforts though.

I have one machine (NanoPi Neo2) that was built yesterday, running buster (Armbian_20.11.3_Nanopineo2_buster_current_5.9.14) that downloads 5 files under apt update and says all up to date. [I had successfully updated the machine already]

I have a second machine (NanoPi Neo2) that was built today using the same image which fails.

I have tried the default and regional mirrors and get a mix of errors. The EU is an odd one as it says the mirror is not signed (seperate issue ?). I'm assuming that NA is also the default as it's the same IP address.

apt-get -o Acquire::ForceIPv4=true -o Acquire::CompressionTypes::Order::=gz -o Acquire::http::No-Cache=true -o Acquire::BrokenProxy=true -o Acquire::http::Pipeline-Depth=0 update

...

Reading package lists... Done
E: Failed to fetch http://apt.armbian.com/dists/buster/main/binary-armhf/Packages.gz File has unexpected size (645768 != 645022). Mirror sync in progress? [IP: 193.40.103.4 443]
E: Failed to fetch http://apt.armbian.com/dists/buster/main/binary-arm64/Packages.gz File has unexpected size (761502 != 758434). Mirror sync in progress? [IP: 193.40.103.4 443]
E: Failed to fetch http://apt.armbian.com/dists/buster/main/Contents-arm64.gz File has unexpected size (11237461 != 11236778). Mirror sync in progress? [IP: 193.40.103.4 443]
E: Failed to fetch http://apt.armbian.com/dists/buster/main/Contents-armhf.gz File has unexpected size (17272323 != 17272310). Mirror sync in progress? [IP: 193.40.103.4 443]
E: Some index files failed to download. They have been ignored, or old ones used instead.

apt-get -o Acquire::ForceIPv4=true -o Acquire::CompressionTypes::Order::=gz -o Acquire::http::No-Cache=true -o Acquire::BrokenProxy=true -o Acquire::http::Pipeline-Depth=0 update
...

Reading package lists... Done
E: Failed to fetch http://apt.armbian.com/region/NA/dists/buster/main/binary-arm64/Packages.gz File has unexpected size (761502 != 758434). Mirror sync in progress? [IP: 193.40.103.4 443]
E: Failed to fetch http://apt.armbian.com/region/NA/dists/buster/main/binary-armhf/Packages.gz File has unexpected size (645768 != 645022). Mirror sync in progress? [IP: 193.40.103.4 443]
E: Failed to fetch http://apt.armbian.com/region/NA/dists/buster/main/Contents-armhf.gz File has unexpected size (17272323 != 17272310). Mirror sync in progress? [IP: 193.40.103.4 443]
E: Failed to fetch http://apt.armbian.com/region/NA/dists/buster/main/Contents-arm64.gz File has unexpected size (11237461 != 11236778). Mirror sync in progress? [IP: 193.40.103.4 443]
E: Some index files failed to download. They have been ignored, or old ones used instead.

apt-get -o Acquire::ForceIPv4=true -o Acquire::CompressionTypes::Order::=gz -o Acquire::http::No-Cache=true -o Acquire::BrokenProxy=true -o Acquire::http::Pipeline-Depth=0 update
...

Reading package lists... Done
W: GPG error: https://minio.k-space.ee/armbian/apt buster Release: The following signatures were invalid: BADSIG 93D6889F9F0E78D5 Igor Pecovnik (Ljubljana, Slovenia) <igor.pecovnik@gmail.com>
E: The repository 'http://apt.armbian.com/region/EU buster Release' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.

dispo · December 16, 2020

working now - thank you

Igor · December 16, 2020

2 minutes ago, dispo said:

working now - thank you

But I haven't change anything That's what bothers us - we don't know why this is happening and there are lots of reports of other Debian / Ubuntu users reporting the same problem. Its clearly not Armbian specifics, but we would like to know why and get control over this ...

dispo · December 16, 2020

One thing I noticed -

When it failed

Hit:1 http://security.debian.org buster/updates InRelease
Hit:2 http://deb.debian.org/debian buster InRelease
Hit:3 http://deb.debian.org/debian buster-updates InRelease
Hit:4 http://deb.debian.org/debian buster-backports InRelease
Get:5 https://armbian.systemonachip.net/apt buster InRelease [18.3 kB]
Get:6 https://armbian.systemonachip.net/apt buster/main armhf Packages [645 kB]
Ign:6 https://armbian.systemonachip.net/apt buster/main armhf Packages
Get:7 https://armbian.systemonachip.net/apt buster/main arm64 Packages [758 kB]
Get:7 https://armbian.systemonachip.net/apt buster/main arm64 Packages [758 kB]
Get:7 https://armbian.systemonachip.net/apt buster/main arm64 Packages [758 kB]
Get:7 https://armbian.systemonachip.net/apt buster/main arm64 Packages [758 kB]
Get:7 https://armbian.systemonachip.net/apt buster/main arm64 Packages [758 kB]
Get:7 https://armbian.systemonachip.net/apt buster/main arm64 Packages [758 kB]
Get:7 https://armbian.systemonachip.net/apt buster/main arm64 Packages [758 kB]
Get:7 https://armbian.systemonachip.net/apt buster/main arm64 Packages [758 kB]
Get:7 https://armbian.systemonachip.net/apt buster/main arm64 Packages [758 kB]
Ign:7 https://armbian.systemonachip.net/apt buster/main arm64 Packages
Get:16 https://armbian.systemonachip.net/apt buster/main arm64 Contents (deb) [11.2 MB]
Ign:16 https://armbian.systemonachip.net/apt buster/main arm64 Contents (deb)
Ign:16 http://apt.armbian.com/region/EU buster/main arm64 Contents (deb)
Get:7 https://minio.k-space.ee/armbian/apt buster/main arm64 Packages [758 kB]
Err:7 https://minio.k-space.ee/armbian/apt buster/main arm64 Packages
File has unexpected size (761502 != 758434). Mirror sync in progress? [IP: 193.40.103.4 443]

When it worked -

Hit:1 http://security.debian.org buster/updates InRelease
Hit:2 http://deb.debian.org/debian buster InRelease
Hit:3 http://deb.debian.org/debian buster-updates InRelease
Hit:4 http://deb.debian.org/debian buster-backports InRelease
Get:5 https://mirrors.netix.net/armbian/apt buster InRelease [18.3 kB]
Get:6 https://mirrors.netix.net/armbian/apt buster/main arm64 Packages [762 kB]
Get:7 https://mirrors.netix.net/armbian/apt buster/main armhf Packages [646 kB]
Get:8 https://mirrors.netix.net/armbian/apt buster/main arm64 Contents (deb) [11.2 MB]
Get:9 https://mirrors.netix.net/armbian/apt buster/main armhf Contents (deb) [17.3 MB]
Get:10 https://mirrors.netix.net/armbian/apt buster/buster-utils arm64 Packages [6,789 B]
Get:11 https://mirrors.netix.net/armbian/apt buster/buster-utils armhf Packages [6,801 B]
Get:12 https://mirrors.netix.net/armbian/apt buster/buster-utils armhf Contents (deb) [8,535 B]
Get:13 https://mirrors.netix.net/armbian/apt buster/buster-utils arm64 Contents (deb) [8,531 B]
Get:14 https://mirrors.netix.net/armbian/apt buster/buster-desktop armhf Packages [22.8 kB]
Get:15 https://mirrors.netix.net/armbian/apt buster/buster-desktop arm64 Packages [20.9 kB]
Get:16 https://mirrors.netix.net/armbian/apt buster/buster-desktop arm64 Contents (deb) [32.9 kB]
Get:17 https://mirrors.netix.net/armbian/apt buster/buster-desktop armhf Contents (deb) [96.1 kB]
Fetched 29.9 MB in 52s (571 kB/s)
Reading package lists... Done
Building dependency tree
Reading state information... Done
All packages are up to date.

It no longer reported IGN for get 6 and get 7 and the file sizes were different. Can you tell if a mirror sync was happening at the time ?

Igor · December 16, 2020

3 minutes ago, dispo said:

Can you tell if a mirror sync was happening at the time ?

Mirrors are pooling from our server. I don't know and I don't know how this update is working - into the details. We already tried everything we could without deep investigation. For that time is needed.

MichaIng · December 16, 2020

Could the issue be that every single file can be pulled from a different mirror? E.g. when packages list is pulled form one mirror, the content hashes from another one and the final package again from another one, it is easily possible that one does not match as for sure not all mirrors sync and have caches emptied exactly the same time. E.g. here my apt update which pulls from various different mirrors:

Get:5 https://minio.k-space.ee/armbian/apt buster InRelease [18.3 kB]
Get:6 https://minio.k-space.ee/armbian/apt buster/main armhf Packages [591 kB]
Ign:6 https://minio.k-space.ee/armbian/apt buster/main armhf Packages
Get:7 https://minio.k-space.ee/armbian/apt buster/main arm64 Packages [698 kB]
Ign:7 https://minio.k-space.ee/armbian/apt buster/main arm64 Packages
Ign:7 https://apt.armbian.com buster/main arm64 Packages
Ign:7 https://apt.armbian.com buster/main arm64 Packages
Get:8 https://minio.k-space.ee/armbian/apt buster/main arm64 Contents (deb) [11.2 MB]
Ign:8 https://minio.k-space.ee/armbian/apt buster/main arm64 Contents (deb)
Get:9 https://minio.k-space.ee/armbian/apt buster/main armhf Contents (deb) [17.3 MB]
Ign:9 https://minio.k-space.ee/armbian/apt buster/main armhf Contents (deb)
Get:6 https://us.mirrors.fossho.st/armbian/apt buster/main armhf Packages [591 kB]
Err:6 https://us.mirrors.fossho.st/armbian/apt buster/main armhf Packages
  File has unexpected size (591174 != 590754). Mirror sync in progress? [IP: 23.237.182.68 443]
  Hashes of expected file:
   - Filesize:590754 [weak]
   - SHA512:bb903949666449552cda361ee24d4d0579baa72915b5fb3d7418acede319562ce4b93a15bf7605ae270bd9f0e59ececf622fea1365e8aa7a3d8339e2a2644f56
   - SHA256:d822f1c4a63ce566baa27d1f5c523057c1d295222f097b072dc8fbeb14a29bc3
   - SHA1:0f37871329a6a28cf7c53dd6ffd952342432c7ed [weak]
   - MD5Sum:517fa48fb23e17f6c94248da6ce5b44b [weak]
  Release file created at: Sat, 12 Dec 2020 15:07:27 +0000
Get:7 https://armbian.systemonachip.net/apt buster/main arm64 Packages [3941 kB]
Err:7 https://armbian.systemonachip.net/apt buster/main arm64 Packages

Get:8 https://mirrors.dotsrc.org/armbian-apt buster/main arm64 Contents (deb) [11.2 MB]
Err:8 https://mirrors.dotsrc.org/armbian-apt buster/main arm64 Contents (deb)

Get:9 https://armbian.tnahosting.net/apt buster/main armhf Contents (deb) [17.3 MB]
Err:9 https://armbian.tnahosting.net/apt buster/main armhf Contents (deb)

Fetched 18.3 kB in 55s (332 B/s)
Reading package lists... Done
E: Failed to fetch https://us.mirrors.fossho.st/armbian/apt/dists/buster/main/binary-armhf/Packages.bz2  File has unexpected size (591174 != 590754). Mirror sync in progress? [IP: 23.237.182.68 443]
   Hashes of expected file:
    - Filesize:590754 [weak]
    - SHA512:bb903949666449552cda361ee24d4d0579baa72915b5fb3d7418acede319562ce4b93a15bf7605ae270bd9f0e59ececf622fea1365e8aa7a3d8339e2a2644f56
    - SHA256:d822f1c4a63ce566baa27d1f5c523057c1d295222f097b072dc8fbeb14a29bc3
    - SHA1:0f37871329a6a28cf7c53dd6ffd952342432c7ed [weak]
    - MD5Sum:517fa48fb23e17f6c94248da6ce5b44b [weak]
   Release file created at: Sat, 12 Dec 2020 15:07:27 +0000
E: Failed to fetch https://armbian.systemonachip.net/apt/dists/buster/main/binary-arm64/Packages
E: Failed to fetch https://mirrors.dotsrc.org/armbian-apt/dists/buster/main/Contents-arm64.gz
E: Failed to fetch https://armbian.tnahosting.net/apt/dists/buster/main/Contents-armhf.gz
E: Some index files failed to download. They have been ignored, or old ones used instead.

The reported successful update from @dispo above was one case where everything was pulled from the same mirrors.netix.net mirror.

I'm not sure how e.g. Debian handles this, but it makes IMO sense that somehow tie a client to a fixed mirror for a certain time.

Although I just compared the files of three mirrors and they all match, synced at 15-Dec-2020 09:03. So not sure if this really is the issue.

MichaIng · December 16, 2020

Sorry for the double post, I cannot edit my first post, it seems. I retried a few times and indeed the first successful update was when all got pulled from the same mirror:

Get:5 https://us.mirrors.fossho.st/armbian/apt buster InRelease [18.3 kB]
Get:6 https://us.mirrors.fossho.st/armbian/apt buster/main arm64 Packages [701 kB]
Get:7 https://us.mirrors.fossho.st/armbian/apt buster/main armhf Packages [591 kB]
Get:8 https://us.mirrors.fossho.st/armbian/apt buster/main armhf Contents (deb) [17.3 MB]
Get:9 https://us.mirrors.fossho.st/armbian/apt buster/main arm64 Contents (deb) [11.2 MB]

Igor · December 16, 2020

28 minutes ago, MichaIng said:

Although I just compared the files of three mirrors and they all match, synced at 15-Dec-2020 09:03. So not sure if this really is the issue.

Exactly that is puzzling me. And after some time things go back to normal ... like some caching on client makes troubles.

czosnekltd · December 16, 2020

it helps for me:

dpkg-reconfigure ca-certificates

Nitrax · December 16, 2020

I am getting this problem every time now on apt update on my armbian tinkerboard:

Hit:1 http://deb.debian.org/debian stable InRelease
Hit:2 http://deb.debian.org/debian stable-updates InRelease
Hit:3 http://deb.debian.org/debian-security stable/updates InRelease
Hit:4 http://ports.ubuntu.com focal InRelease
Hit:5 http://ports.ubuntu.com focal-security InRelease
Hit:6 https://download.docker.com/linux/debian buster InRelease
Hit:7 https://repos.influxdata.com/debian focal InRelease
Hit:8 http://ports.ubuntu.com focal-updates InRelease
Hit:9 http://ports.ubuntu.com focal-backports InRelease
Get:10 https://minio.k-space.ee/armbian/apt focal InRelease [18.2 kB]
Hit:11 https://packages.gitlab.com/runner/gitlab-runner/ubuntu focal InRelease
Get:12 https://minio.k-space.ee/armbian/apt focal/main armhf Packages [645 kB]
Ign:12 https://minio.k-space.ee/armbian/apt focal/main armhf Packages
Get:12 https://us.mirrors.fossho.st/armbian/apt focal/main armhf Packages [645 k                                                                    B]
Err:12 https://us.mirrors.fossho.st/armbian/apt focal/main armhf Packages
  File has unexpected size (646150 != 645411). Mirror sync in progress? [IP: 23.                                                                    237.182.68 443]
  Hashes of expected file:
   - Filesize:645411 [weak]
   - SHA512:b1c4ae525ec721387a734673069af21268482d7324822871727d7b54f49a807cac2c                                                                    e64b739b23cedbe309d3cd203af9ba297dee6042e4a9071cae5f986071c8
   - SHA256:0ba674c754337dffa93851eb62e0f4841a4a29841ea708dd6bfde44ca2eedf3f
   - SHA1:703b61507243cdddd410855b86e45d07955c53c9 [weak]
   - MD5Sum:e0745d84180e2a4deaa33c5ad469c24a [weak]
  Release file created at: Sat, 12 Dec 2020 15:43:37 +0000
Fetched 18.2 kB in 4s (4,942 B/s)
Reading package lists... Done
E: Failed to fetch https://us.mirrors.fossho.st/armbian/apt/dists/focal/main/binary-armhf/Packages.gz  File has unexpected size (646150 != 645411). Mirror sync in progress? [IP: 23.237.182.68 443]
   Hashes of expected file:
    - Filesize:645411 [weak]
    - SHA512:b1c4ae525ec721387a734673069af21268482d7324822871727d7b54f49a807cac2ce64b739b23cedbe309d3cd203af9ba297dee6042e4a9071cae5f986071c8
    - SHA256:0ba674c754337dffa93851eb62e0f4841a4a29841ea708dd6bfde44ca2eedf3f
    - SHA1:703b61507243cdddd410855b86e45d07955c53c9 [weak]
    - MD5Sum:e0745d84180e2a4deaa33c5ad469c24a [weak]
   Release file created at: Sat, 12 Dec 2020 15:43:37 +0000
E: Some index files failed to download. They have been ignored, or old ones used instead.

Nitrax · December 16, 2020

 # sudo apt-get -o Acquire::ForceIPv4=true -o Acquire::CompressionTypes::Order::=gz -o Acquire::http::No-Cache=true -o Acquire::BrokenProxy=true -o Acquire::http::Pipeline-Depth=0 update
Hit:1 http://deb.debian.org/debian stable InRelease
Hit:2 http://deb.debian.org/debian stable-updates InRelease
Hit:3 http://ports.ubuntu.com focal InRelease
Hit:4 http://deb.debian.org/debian-security stable/updates InRelease
Hit:5 http://ports.ubuntu.com focal-security InRelease
Hit:6 https://download.docker.com/linux/debian buster InRelease
Hit:7 https://repos.influxdata.com/debian focal InRelease
Hit:8 http://ports.ubuntu.com focal-updates InRelease
Hit:9 http://ports.ubuntu.com focal-backports InRelease
Get:10 https://armbian.tnahosting.net/apt focal InRelease [18.2 kB]
Hit:11 https://packages.gitlab.com/runner/gitlab-runner/ubuntu focal InRelease
Get:12 https://armbian.tnahosting.net/apt focal/main armhf Packages [645 kB]
Ign:12 https://armbian.tnahosting.net/apt focal/main armhf Packages
Get:12 https://armbian.systemonachip.net/apt focal/main armhf Packages [645 kB]
Err:12 https://armbian.systemonachip.net/apt focal/main armhf Packages
  File has unexpected size (646150 != 645411). Mirror sync in progress? [IP: 51.75.2.52 443]
  Hashes of expected file:
   - Filesize:645411 [weak]
   - SHA512:b1c4ae525ec721387a734673069af21268482d7324822871727d7b54f49a807cac2ce64b739b23cedbe309d3cd203af9ba297dee6042e4a9071cae5f986071c8
   - SHA256:0ba674c754337dffa93851eb62e0f4841a4a29841ea708dd6bfde44ca2eedf3f
   - SHA1:703b61507243cdddd410855b86e45d07955c53c9 [weak]
   - MD5Sum:e0745d84180e2a4deaa33c5ad469c24a [weak]
  Release file created at: Sat, 12 Dec 2020 15:43:37 +0000
Reading package lists... Done
E: Failed to fetch https://armbian.systemonachip.net/apt/dists/focal/main/binary-armhf/Packages.gz  File has unexpected size (646150 != 645411). Mirror sync in progress? [IP: 51.75.2.52 443]
   Hashes of expected file:
    - Filesize:645411 [weak]
    - SHA512:b1c4ae525ec721387a734673069af21268482d7324822871727d7b54f49a807cac2ce64b739b23cedbe309d3cd203af9ba297dee6042e4a9071cae5f986071c8
    - SHA256:0ba674c754337dffa93851eb62e0f4841a4a29841ea708dd6bfde44ca2eedf3f
    - SHA1:703b61507243cdddd410855b86e45d07955c53c9 [weak]
    - MD5Sum:e0745d84180e2a4deaa33c5ad469c24a [weak]
   Release file created at: Sat, 12 Dec 2020 15:43:37 +0000
E: Some index files failed to download. They have been ignored, or old ones used instead.

Hit:1 http://deb.debian.org/debian stable InRelease
Hit:2 http://deb.debian.org/debian stable-updates InRelease
Hit:3 http://deb.debian.org/debian-security stable/updates InRelease
Hit:4 http://ports.ubuntu.com focal InRelease
Hit:5 http://ports.ubuntu.com focal-security InRelease
Hit:6 https://download.docker.com/linux/debian buster InRelease
Hit:7 http://ports.ubuntu.com focal-updates InRelease
Hit:8 https://repos.influxdata.com/debian focal InRelease
Hit:9 http://ports.ubuntu.com focal-backports InRelease
Get:10 https://armbian.hosthatch.com/apt focal InRelease [18.2 kB]
Hit:11 https://packages.gitlab.com/runner/gitlab-runner/ubuntu focal InRelease
Get:12 https://armbian.hosthatch.com/apt focal/main armhf Packages [645 kB]
Ign:12 https://armbian.hosthatch.com/apt focal/main armhf Packages
Get:12 https://mirrors.dotsrc.org/armbian-apt focal/main armhf Packages [645 kB]
Err:12 https://mirrors.dotsrc.org/armbian-apt focal/main armhf Packages
  File has unexpected size (646150 != 645411). Mirror sync in progress? [IP: 130.225.254.116 443]
  Hashes of expected file:
   - Filesize:645411 [weak]
   - SHA512:b1c4ae525ec721387a734673069af21268482d7324822871727d7b54f49a807cac2ce64b739b23cedbe309d3cd203af9ba297dee6042e4a9071cae5f986071c8
   - SHA256:0ba674c754337dffa93851eb62e0f4841a4a29841ea708dd6bfde44ca2eedf3f
   - SHA1:703b61507243cdddd410855b86e45d07955c53c9 [weak]
   - MD5Sum:e0745d84180e2a4deaa33c5ad469c24a [weak]
  Release file created at: Sat, 12 Dec 2020 15:43:37 +0000
Fetched 18.2 kB in 3s (6,143 B/s)
Reading package lists... Done
E: Failed to fetch https://mirrors.dotsrc.org/armbian-apt/dists/focal/main/binary-armhf/Packages.gz  File has unexpected size (646150 != 645411). Mirror sync in progress? [IP: 130.225.254.116 443]
   Hashes of expected file:
    - Filesize:645411 [weak]
    - SHA512:b1c4ae525ec721387a734673069af21268482d7324822871727d7b54f49a807cac2ce64b739b23cedbe309d3cd203af9ba297dee6042e4a9071cae5f986071c8
    - SHA256:0ba674c754337dffa93851eb62e0f4841a4a29841ea708dd6bfde44ca2eedf3f
    - SHA1:703b61507243cdddd410855b86e45d07955c53c9 [weak]
    - MD5Sum:e0745d84180e2a4deaa33c5ad469c24a [weak]
   Release file created at: Sat, 12 Dec 2020 15:43:37 +0000
E: Some index files failed to download. They have been ignored, or old ones used instead.

Edited December 16, 2020 by Nitrax
more failing mirrors

Nitrax · December 16, 2020

And after trying around 20 times:

 # apt update
Hit:1 http://deb.debian.org/debian stable InRelease
Hit:2 http://deb.debian.org/debian stable-updates InRelease
Hit:3 http://ports.ubuntu.com focal InRelease
Hit:4 http://deb.debian.org/debian-security stable/updates InRelease
Get:5 http://ports.ubuntu.com focal-security InRelease [109 kB]
Hit:6 https://download.docker.com/linux/debian buster InRelease
Hit:7 https://repos.influxdata.com/debian focal InRelease
Get:8 http://ports.ubuntu.com focal-updates InRelease [114 kB]
Get:9 http://ports.ubuntu.com focal-backports InRelease [101 kB]
Hit:11 https://packages.gitlab.com/runner/gitlab-runner/ubuntu focal InRelease
Get:10 https://us.mirrors.fossho.st/armbian/apt focal InRelease [18.2 kB]
Get:12 https://us.mirrors.fossho.st/armbian/apt focal/main armhf Packages [646 kB]
Fetched 988 kB in 4s (243 kB/s)
Reading package lists... Done
Building dependency tree
Reading state information... Done
25 packages can be upgraded. Run 'apt list --upgradable' to see them.

And now its working every time after something like 2 days of failing.

dispo · December 16, 2020

Failed -

Get:6 https://minio.k-space.ee/armbian/apt buster/main arm64 Packages [758 kB]

Ign:6 https://minio.k-space.ee/armbian/apt buster/main arm64 Packages
Get:7 https://minio.k-space.ee/armbian/apt buster/main armhf Packages [645 kB]
Ign:7 https://minio.k-space.ee/armbian/apt buster/main armhf Packages

Worked -

Get:6 https://us.mirrors.fossho.st/armbian/apt buster/main armhf Packages [646 kB]
Get:7 https://us.mirrors.fossho.st/armbian/apt buster/main arm64 Packages [762 kB]

edit: Same machine, minutes apart but to a different mirror.

The file sizes are the same as yesterday in each case. I'm wondering where the failing mirror is getting the incorrect 645/758 from instead of the correct 646/762.

I've tried to find detail of how this process work but didnt get very far. I have been avoiding searches via google though

It also always appears to be 'main armhf Packages' that fails and occassionally 'main arm64 Packages'

MotoFckr9k · December 17, 2020

vor 17 Stunden schrieb Igor:

But I haven't change anything That's what bothers us - we don't know why this is happening and there are lots of reports of other Debian / Ubuntu users reporting the same problem. Its clearly not Armbian specifics, but we would like to know why and get control over this ...

I feel like that might be the reason why Ubuntu and Debian don't do this kind of automatic mirror redirection. (at least I have never noticed it)
Maybe we should just choose a mirror manually like we do on Debian and Ubuntu.

A good tool to do this with would probably be mintsources:
screenshot.png

minnix · December 17, 2020

$ sudo apt update
Hit:1 http://ports.ubuntu.com focal InRelease
Hit:3 http://ports.ubuntu.com focal-security InRelease
Hit:4 http://ports.ubuntu.com focal-updates InRelease
Hit:5 http://ports.ubuntu.com focal-backports InRelease
Get:2 https://armbian.systemonachip.net/apt focal InRelease [18.2 kB]
Get:6 https://armbian.systemonachip.net/apt focal/focal-utils armhf Packages [5,034 B]
Get:7 https://armbian.systemonachip.net/apt focal/main armhf Packages [645 kB]
Get:7 https://armbian.systemonachip.net/apt focal/main armhf Packages [645 kB]
Ign:7 https://armbian.systemonachip.net/apt focal/main armhf Packages
Get:7 https://us.mirrors.fossho.st/armbian/apt focal/main armhf Packages [645 kB]
Err:7 https://us.mirrors.fossho.st/armbian/apt focal/main armhf Packages
  File has unexpected size (646150 != 645411). Mirror sync in progress? [IP: 23.237.182.68 443]
  Hashes of expected file:
   - Filesize:645411 [weak]
   - SHA512:b1c4ae525ec721387a734673069af21268482d7324822871727d7b54f49a807cac2ce64b739b23cedbe309d3cd203af9ba297dee6042e4a9071cae5f986071c8
   - SHA256:0ba674c754337dffa93851eb62e0f4841a4a29841ea708dd6bfde44ca2eedf3f
   - SHA1:703b61507243cdddd410855b86e45d07955c53c9 [weak]
   - MD5Sum:e0745d84180e2a4deaa33c5ad469c24a [weak]
  Release file created at: Sat, 12 Dec 2020 15:43:37 +0000
Get:9 https://armbian.systemonachip.net/apt focal/main arm64 Packages [759 kB]
Ign:9 https://armbian.systemonachip.net/apt focal/main arm64 Packages
Reading package lists... Done 
E: Failed to fetch https://us.mirrors.fossho.st/armbian/apt/dists/focal/main/binary-armhf/Packages.gz  File has unexpected size (646150 != 645411). Mirror sync in progress? [IP: 23.237.182.68 443]
   Hashes of expected file:
    - Filesize:645411 [weak]
    - SHA512:b1c4ae525ec721387a734673069af21268482d7324822871727d7b54f49a807cac2ce64b739b23cedbe309d3cd203af9ba297dee6042e4a9071cae5f986071c8
    - SHA256:0ba674c754337dffa93851eb62e0f4841a4a29841ea708dd6bfde44ca2eedf3f
    - SHA1:703b61507243cdddd410855b86e45d07955c53c9 [weak]
    - MD5Sum:e0745d84180e2a4deaa33c5ad469c24a [weak]
   Release file created at: Sat, 12 Dec 2020 15:43:37 +0000
E: Some index files failed to download. They have been ignored, or old ones used instead.

Igor · December 17, 2020

5 hours ago, MotoFckr9k said:

I feel like that might be the reason why Ubuntu and Debian don't do this kind of automatic mirror redirection.

Could be.

5 hours ago, MotoFckr9k said:

Maybe we should just choose a mirror manually like we do on Debian and Ubuntu.

We are looking into the way of minimise maintenance and provide better service - if it would work as designed. Current way a dead mirror can be removed / added when comes back. Also when we do update, new artefacts goes to the fastest server which become the only source until others are synched. It is possible that we will only change one problem with another ... This needs to be further researched to understand the decision why Mint is having this solved this way. AFAIK debian used to have redirector running, but its down (for good) ... for some maintenance reasons. I don't recall this kind of issues.

lobot · December 17, 2020

I've been having this problem off and on for a while now as well. The only solution that I found is simply to wait longer until more mirrors are updated and hope that one of the updated ones is selected.

@IgorHave you tried this: https://apt-mirror-updater.readthedocs.io/en/latest/readme.html

I realize it's still in beta but it may be worth testing.

Igor · December 17, 2020

1 hour ago, lobot said:

@IgorHave you tried this: https://apt-mirror-updater.readthedocs.io/en/latest/readme.html

I realize it's still in beta but it may be worth testing.

This looks more like a solution for a (business) client. Not server side trouble which we are looking for. Our system is combined with (currently in round robin mode, soon geo based) router https://github.com/armbian/dl-router and a script that delivers data to the rsyncd server where mirrors are pulling off. So far we are mitigating only "file not found" problem by delaying synching indexes. We wait until files are synched, then we push out index synching. But most likely here is the catch. We should disable all mirrors and enable them one by one as soon as indexes on those are synched. We think that is the solution. Just need to test and implement. As always, time is the problem

Thanks for the input.

lanefu · December 18, 2020

7 hours ago, lobot said:

I've been having this problem off and on for a while now as well. The only solution that I found is simply to wait longer until more mirrors are updated and hope that one of the updated ones is selected.

Hi sorry for the troubles. I've added some quick documentation on how to manually choose a mirror. Hopefully this is an easy workaround for the short-term

https://docs.armbian.com/User-Guide_Fine-Tuning/#choosing-an-apt-mirror

Exilibris · December 21, 2020

update-ca-certificates

gurabli · December 21, 2020

On 12/18/2020 at 4:35 AM, lanefu said:

Hi sorry for the troubles. I've added some quick documentation on how to manually choose a mirror. Hopefully this is an easy workaround for the short-term

https://docs.armbian.com/User-Guide_Fine-Tuning/#choosing-an-apt-mirror

Thanks, this did the trick for me. A choose manually a mirror and worked.

lanefu · December 22, 2020

Update:

We Deployed GEOIP based redirecting over the weekend. The defaults should likely send most to an appropriate mirror, but choosing a mirror or region is still a good thing to do. Not all IPs match correctly.

Igor · December 27, 2020

This problem should be fixed by now.

One of our mirror is "Kubernetes Native, High Performance Object Storage Amazon S3 compatible" and there we had to use different sync method. We also synched from there further since host has a fast connection ...

tl;dr; Wrong sync parameter :rolleyes:

MichaIng · December 29, 2020

Great, many thanks for identifying and fixing it!

PP88PP · March 10, 2022

On 12/27/2020 at 9:49 PM, Igor said:

This problem should be fixed by now.

One of our mirror is "Kubernetes Native, High Performance Object Storage Amazon S3 compatible" and there we had to use different sync method. We also synched from there further since host has a fast connection ...

tl;dr; Wrong sync parameter

Still "File has unexpected size" in 20.x and 22.02.1 bulleye/focal.

I've check different mirrors in different countries , the error file have the same size between mirrors which not match the wanted size.

Sign In

Bionic apt update "File has unexpected size"

Recommended Posts

MotoFckr9k

dispo

Werner

lanefu

Igor

dispo

dispo

Igor

dispo

Igor

MichaIng

MichaIng

Igor

czosnekltd

Nitrax

Nitrax

Nitrax

dispo

MotoFckr9k

minnix

Igor

lobot

Igor

lanefu

Exilibris

gurabli

lanefu

Igor

MichaIng

PP88PP

Join the conversation

Forums

My Activity Streams

Download

Store

Important Information