RockBian Posted January 17, 2021 Posted January 17, 2021 Today I updated and upgraded my system, and after a reboot 'apt list' did no longer work. It shows 'Listing... done', and then uses 100% CPU on a single core, without producing anything. According to strace it's looping on reading a file: openat(AT_FDCWD, "/var/lib/apt/lists/deb.debian.org_debian_dists_buster_main_binary-arm64_Packages.lz4", O_RDONLY) = 28 read(28, "\4\"M\30@@\300\206z\0\0\370\377\31Package: 0ad-data\n"..., 65536) = 65536 read(28, "ta$E\17p\2+\377\21c376d3f140540d9424753d"..., 65536) = 65536 read(28, "a5180e3e2d3f503ddcb5dd7b93\201\2\2\275\0\17"..., 65536) = 65536 read(28, "\2\3700071880bc23623a4ea3664b773a1b6"..., 65536) = 65536 read(28, "\f1'da?\2Jdan_\207\1\v=\2f215438\272\t\365\0213e4c"..., 65536) = 65536 read(28, "a6f6005b836871a9cc032472bbece3b7"..., 65536) = 65536 read(28, "3783a1460d749c6f887843fdd95ec981"..., 65536) = 65536 read(28, "\4\0032 - \16\1\0022\32\t?\27\177foreign\270\0041\370\21f1e69"..., 65536) = 65536 read(28, "dR\203\1\6\4\6T\37\3\324\0\",\ng_zplaying\2759\vt\203\1\233"..., 65536) = 65536 read(28, "\20u://ftp.\24'_nu/bc\2\3\0\363\20b8da7e3f11"..., 65536) = 65536 close(28) = 0 openat(AT_FDCWD, "/var/lib/apt/lists/deb.debian.org_debian_dists_buster_main_binary-arm64_Packages.lz4", O_RDONLY) = 28 read(28, "\4\"M\30@@\300\206z\0\0\370\377\31Package: 0ad-data\n"..., 65536) = 65536 read(28, "ta$E\17p\2+\377\21c376d3f140540d9424753d"..., 65536) = 65536 read(28, "a5180e3e2d3f503ddcb5dd7b93\201\2\2\275\0\17"..., 65536) = 65536 read(28, "\2\3700071880bc23623a4ea3664b773a1b6"..., 65536) = 65536 read(28, "\f1'da?\2Jdan_\207\1\v=\2f215438\272\t\365\0213e4c"..., 65536) = 65536 read(28, "a6f6005b836871a9cc032472bbece3b7"..., 65536) = 65536 read(28, "3783a1460d749c6f887843fdd95ec981"..., 65536) = 65536 read(28, "\4\0032 - \16\1\0022\32\t?\27\177foreign\270\0041\370\21f1e69"..., 65536) = 65536 read(28, "dR\203\1\6\4\6T\37\3\324\0\",\ng_zplaying\2759\vt\203\1\233"..., 65536) = 65536 read(28, "\20u://ftp.\24'_nu/bc\2\3\0\363\20b8da7e3f11"..., 65536) = 65536 close(28) = 0 openat(AT_FDCWD, "/var/lib/apt/lists/deb.debian.org_debian_dists_buster_main_binary-arm64_Packages.lz4", O_RDONLY) = 28 read(28, "\4\"M\30@@\300\206z\0\0\370\377\31Package: 0ad-data\n"..., 65536) = 65536 read(28, "ta$E\17p\2+\377\21c376d3f140540d9424753d"..., 65536) = 65536 read(28, "a5180e3e2d3f503ddcb5dd7b93\201\2\2\275\0\17"..., 65536) = 65536 read(28, "\2\3700071880bc23623a4ea3664b773a1b6"..., 65536) = 65536 read(28, "\f1'da?\2Jdan_\207\1\v=\2f215438\272\t\365\0213e4c"..., 65536) = 65536 read(28, "a6f6005b836871a9cc032472bbece3b7"..., 65536) = 65536 read(28, "3783a1460d749c6f887843fdd95ec981"..., 65536) = 65536 read(28, "\4\0032 - \16\1\0022\32\t?\27\177foreign\270\0041\370\21f1e69"..., 65536) = 65536 read(28, "dR\203\1\6\4\6T\37\3\324\0\",\ng_zplaying\2759\vt\203\1\233"..., 65536) = 65536 read(28, "\20u://ftp.\24'_nu/bc\2\3\0\363\20b8da7e3f11"..., 65536) = 65536 close(28) = 0 This repeats over and over. I checked the file /var/lib/apt/lists/deb.debian.org_debian_dists_buster_main_binary-arm64_Packages.lz4, and it seems valid, as in, lz4 decompresses it without complaints. When I put the decompressed /var/lib/apt/lists/deb.debian.org_debian_dists_buster_main_binary-arm64_Packages in the same directory, 'apt list' works. What is going on? It might be size related. The file is the biggest *_Packages.lz4 file in that directory. There are bigger lz4 files, but according to strace they are not accessed. The timestamp of deb.debian.org_debian_dists_buster_main_binary-arm64_Packages.lz4 is Dec 5 11:03, so I don't think it grew, at last update. The upgraded files are: Start-Date: 2021-01-17 14:07:30 Commandline: apt upgrade Requested-By: rockbian (1000) Install: javascript-common:arm64 (11, automatic), php-symfony-yaml:arm64 (3.4.22+dfsg-2+deb10u1, automatic), libjs-codemirror:arm64 (5.43.0-1+deb10u1, automatic), libjs-popper.js:arm64 (1.14.6+ds2-1, automatic), libjs-jquery-mousewheel:arm64 (1:3.1.13-2, automatic), php-symfony-filesystem:arm64 (3.4.22+dfsg-2+deb10u1, automatic), libjs-jquery-timepicker:arm64 (1.2-1, automatic), php-symfony-dependency-injection:arm64 (3.4.22+dfsg-2+deb10u1, automatic), libjs-jquery-ui:arm64 (1.12.1+dfsg-5, automatic), libjs-bootstrap4:arm64 (4.3.1+dfsg2-1, automatic), php-symfony-config:arm64 (3.4.22+dfsg-2+deb10u1, automatic), php-twig-i18n-extension:arm64 (3.0.0-2~bpo10+1, automatic) Upgrade: nodejs:arm64 (10.21.0~dfsg-1~deb10u1, 10.23.1~dfsg-1~deb10u1), python-apt-common:arm64 (1.8.4.2, 1.8.4.3), phpmyadmin:arm64 (4:4.9.7+dfsg1-1~bpo10+1, 4:5.0.4+dfsg2-1~bpo10+1), php-google-recaptcha:arm64 (1.2.4-1~bpo10+1, 1.2.4-3~bpo10+1), linux-buster-root-current-helios64:arm64 (20.11.4, 20.11.6), armbian-config:arm64 (20.11.3, 20.11.6), libnode64:arm64 (10.21.0~dfsg-1~deb10u1, 10.23.1~dfsg-1~deb10u1), libp11-kit0:arm64 (0.23.15-2, 0.23.15-2+deb10u1), php-mariadb-mysql-kbs:arm64 (1.2.11-1~bpo10+1, 1.2.12-1~bpo10+1), python3-apt:arm64 (1.8.4.2, 1.8.4.3), tzdata:arm64 (2020d-0+deb10u1, 2020e-0+deb10u1), linux-u-boot-helios64-current:arm64 (20.11.4, 20.11.6) End-Date: 2021-01-17 14:08:10
Igor Posted January 17, 2021 Posted January 17, 2021 9 hours ago, RockBian said: What is going on? Probably this https://armbian.atlassian.net/browse/AR-296 but someone needs to investigate and fix.
ShadowDance Posted January 20, 2021 Posted January 20, 2021 If you want to speed it up, you can quick-fix it by doing the following: sudo rm /etc/apt/apt.conf.d/02-armbian-compress-indexes sudo rm -rf /var/lib/apt/lists/* sudo apt update If you don't want to remove 02-armbian-compress-indexes you can edit it and change true to false. Do note that these will, unfortunately, be overwritten/replaced by future updates to the linux-buster-root-current-helios64 package.
RockBian Posted January 20, 2021 Author Posted January 20, 2021 The problem is not that it's slow, but it doesn't work at all. Or did you mean speed up the solution?
ShadowDance Posted January 20, 2021 Posted January 20, 2021 @RockBian my suggestion could fix your issue since it gets rid of the gzip compression. The caveat, of course, being that the files will no longer be compressed. It might be possible to edit 02-armbian-compress-indexes and changing the compression to lz4 (if that works) but I haven't tested it.
jpegxguy Posted April 7, 2021 Posted April 7, 2021 Just for anyone reading this, this might be related. Someone suggested prioritizing gzip in order to escape the slowness of LZ4 on ARM platforms. Here is the solution, which was also merged in Armbian itself: 1
Recommended Posts