Problem with apt list


RockBian
 Share

1 1

Recommended Posts

Today I updated and upgraded my system, and after a reboot 'apt list' did no longer work. It shows 'Listing... done', and then uses 100% CPU on a single core, without producing anything. According to strace it's looping on reading a file:

openat(AT_FDCWD, "/var/lib/apt/lists/deb.debian.org_debian_dists_buster_main_binary-arm64_Packages.lz4", O_RDONLY) = 28
read(28, "\4\"M\30@@\300\206z\0\0\370\377\31Package: 0ad-data\n"..., 65536) = 65536
read(28, "ta$E\17p\2+\377\21c376d3f140540d9424753d"..., 65536) = 65536
read(28, "a5180e3e2d3f503ddcb5dd7b93\201\2\2\275\0\17"..., 65536) = 65536
read(28, "\2\3700071880bc23623a4ea3664b773a1b6"..., 65536) = 65536
read(28, "\f1'da?\2Jdan_\207\1\v=\2f215438\272\t\365\0213e4c"..., 65536) = 65536
read(28, "a6f6005b836871a9cc032472bbece3b7"..., 65536) = 65536
read(28, "3783a1460d749c6f887843fdd95ec981"..., 65536) = 65536
read(28, "\4\0032 - \16\1\0022\32\t?\27\177foreign\270\0041\370\21f1e69"..., 65536) = 65536
read(28, "dR\203\1\6\4\6T\37\3\324\0\",\ng_zplaying\2759\vt\203\1\233"..., 65536) = 65536
read(28, "\20u://ftp.\24'_nu/bc\2\3\0\363\20b8da7e3f11"..., 65536) = 65536
close(28)                               = 0
openat(AT_FDCWD, "/var/lib/apt/lists/deb.debian.org_debian_dists_buster_main_binary-arm64_Packages.lz4", O_RDONLY) = 28
read(28, "\4\"M\30@@\300\206z\0\0\370\377\31Package: 0ad-data\n"..., 65536) = 65536
read(28, "ta$E\17p\2+\377\21c376d3f140540d9424753d"..., 65536) = 65536
read(28, "a5180e3e2d3f503ddcb5dd7b93\201\2\2\275\0\17"..., 65536) = 65536
read(28, "\2\3700071880bc23623a4ea3664b773a1b6"..., 65536) = 65536
read(28, "\f1'da?\2Jdan_\207\1\v=\2f215438\272\t\365\0213e4c"..., 65536) = 65536
read(28, "a6f6005b836871a9cc032472bbece3b7"..., 65536) = 65536
read(28, "3783a1460d749c6f887843fdd95ec981"..., 65536) = 65536
read(28, "\4\0032 - \16\1\0022\32\t?\27\177foreign\270\0041\370\21f1e69"..., 65536) = 65536
read(28, "dR\203\1\6\4\6T\37\3\324\0\",\ng_zplaying\2759\vt\203\1\233"..., 65536) = 65536
read(28, "\20u://ftp.\24'_nu/bc\2\3\0\363\20b8da7e3f11"..., 65536) = 65536
close(28)                               = 0
openat(AT_FDCWD, "/var/lib/apt/lists/deb.debian.org_debian_dists_buster_main_binary-arm64_Packages.lz4", O_RDONLY) = 28
read(28, "\4\"M\30@@\300\206z\0\0\370\377\31Package: 0ad-data\n"..., 65536) = 65536
read(28, "ta$E\17p\2+\377\21c376d3f140540d9424753d"..., 65536) = 65536
read(28, "a5180e3e2d3f503ddcb5dd7b93\201\2\2\275\0\17"..., 65536) = 65536
read(28, "\2\3700071880bc23623a4ea3664b773a1b6"..., 65536) = 65536
read(28, "\f1'da?\2Jdan_\207\1\v=\2f215438\272\t\365\0213e4c"..., 65536) = 65536
read(28, "a6f6005b836871a9cc032472bbece3b7"..., 65536) = 65536
read(28, "3783a1460d749c6f887843fdd95ec981"..., 65536) = 65536
read(28, "\4\0032 - \16\1\0022\32\t?\27\177foreign\270\0041\370\21f1e69"..., 65536) = 65536
read(28, "dR\203\1\6\4\6T\37\3\324\0\",\ng_zplaying\2759\vt\203\1\233"..., 65536) = 65536
read(28, "\20u://ftp.\24'_nu/bc\2\3\0\363\20b8da7e3f11"..., 65536) = 65536
close(28)                               = 0

This repeats over and over. I checked the file /var/lib/apt/lists/deb.debian.org_debian_dists_buster_main_binary-arm64_Packages.lz4, and it seems valid, as in, lz4 decompresses it without complaints. When I put the decompressed /var/lib/apt/lists/deb.debian.org_debian_dists_buster_main_binary-arm64_Packages in the same directory, 'apt list' works.

 

What is going on? It might be size related. The file is the biggest *_Packages.lz4 file in that directory. There are bigger lz4 files, but according to strace they are not accessed. The timestamp of deb.debian.org_debian_dists_buster_main_binary-arm64_Packages.lz4 is Dec 5 11:03, so I don't think it grew, at last update.

 

The upgraded files are:

Start-Date: 2021-01-17  14:07:30
Commandline: apt upgrade
Requested-By: rockbian (1000)
Install: javascript-common:arm64 (11, automatic), php-symfony-yaml:arm64 (3.4.22+dfsg-2+deb10u1, automatic), libjs-codemirror:arm64 (5.43.0-1+deb10u1, automatic), libjs-popper.js:arm64 (1.14.6+ds2-1, automatic), libjs-jquery-mousewheel:arm64 (1:3.1.13-2, automatic), php-symfony-filesystem:arm64 (3.4.22+dfsg-2+deb10u1, automatic), libjs-jquery-timepicker:arm64 (1.2-1, automatic), php-symfony-dependency-injection:arm64 (3.4.22+dfsg-2+deb10u1, automatic), libjs-jquery-ui:arm64 (1.12.1+dfsg-5, automatic), libjs-bootstrap4:arm64 (4.3.1+dfsg2-1, automatic), php-symfony-config:arm64 (3.4.22+dfsg-2+deb10u1, automatic), php-twig-i18n-extension:arm64 (3.0.0-2~bpo10+1, automatic)
Upgrade: nodejs:arm64 (10.21.0~dfsg-1~deb10u1, 10.23.1~dfsg-1~deb10u1), python-apt-common:arm64 (1.8.4.2, 1.8.4.3), phpmyadmin:arm64 (4:4.9.7+dfsg1-1~bpo10+1, 4:5.0.4+dfsg2-1~bpo10+1), php-google-recaptcha:arm64 (1.2.4-1~bpo10+1, 1.2.4-3~bpo10+1), linux-buster-root-current-helios64:arm64 (20.11.4, 20.11.6), armbian-config:arm64 (20.11.3, 20.11.6), libnode64:arm64 (10.21.0~dfsg-1~deb10u1, 10.23.1~dfsg-1~deb10u1), libp11-kit0:arm64 (0.23.15-2, 0.23.15-2+deb10u1), php-mariadb-mysql-kbs:arm64 (1.2.11-1~bpo10+1, 1.2.12-1~bpo10+1), python3-apt:arm64 (1.8.4.2, 1.8.4.3), tzdata:arm64 (2020d-0+deb10u1, 2020e-0+deb10u1), linux-u-boot-helios64-current:arm64 (20.11.4, 20.11.6)
End-Date: 2021-01-17  14:08:10

 

Link to post
Share on other sites

Donate and support the project!

If you want to speed it up, you can quick-fix it by doing the following:

 

sudo rm /etc/apt/apt.conf.d/02-armbian-compress-indexes
sudo rm -rf /var/lib/apt/lists/*
sudo apt update

 

If you don't want to remove 02-armbian-compress-indexes you can edit it and change true to false. Do note that these will, unfortunately, be overwritten/replaced by future updates to the linux-buster-root-current-helios64 package.

Link to post
Share on other sites

@RockBian my suggestion could fix your issue since it gets rid of the gzip compression. The caveat, of course, being that the files will no longer be compressed. It might be possible to edit 02-armbian-compress-indexes and changing the compression to lz4 (if that works) but I haven't tested it.

Link to post
Share on other sites

Just for anyone reading this, this might be related. Someone suggested prioritizing gzip in order to escape the slowness of LZ4 on ARM platforms.
Here is the solution, which was also merged in Armbian itself:

 

Link to post
Share on other sites

 Share

1 1