My apt search has become super slow recently


Recommended Posts

Doing apt search took just a few seconds before, but since some days, probably after an apt update && apt upgrade, it has become so slow that a search takes many minutes (20+ minutes) instead of seconds. I don't know why, and I don't know how to fix this. Do you know what to do?

 

The slow part of the search is where it says "Full Text Search". It reaches 50% quickly, and then goes slow.

 

If it matters: ODROID-C2, Buster

Link to post
Share on other sites
Armbian is a community driven open source project. Do you like to contribute your code?

Thank you for the suggestion, xwiggen, and sorry for the delayed response. I just checked in both places, but no trace of ext4lazyinit. I noticed that an apt search causes one of the cores to ramp up to 100% for the duration of the search. There doesn't appear to be any disk activity while searching, and RAM usage only increases by ~25 MB.

Link to post
Share on other sites

Normally I'd search with apt, but that's performing poorly on my device ( e.g. 12+ minutes per search):

apt search mariadb

 

Searches with aptitude work fine, both with the ncurses interface, and from the command line:

aptitude search mariadb

 

I just tried and apt-cache search works fine as well:

apt-cache search mariadb

 

Link to post
Share on other sites

I was able to fix my slow apt searches by removing /etc/apt/apt.conf.d/02-armbian-compress-indexes as described in this thread (forum.odroid.com). This didn't immediately fix the problem, but it started working normally after I removed all of the .lz4 files from /var/lib/apt/lists/ and ran apt update. I confirmed that all files ending with "_Packages.lz4" are now stored in uncompressed form,  and my searches are nearly instantaneous.

 

Aside from lost disk space, are there any downsides to this approach?

Link to post
Share on other sites
4 minutes ago, matteo said:

I was able to fix my slow apt searches by removing /etc/apt/apt.conf.d/02-armbian-compress-indexes as described in this thread (forum.odroid.com). This didn't immediately fix the problem, but it started working normally after I removed all of the .lz4 files from /var/lib/apt/lists/ and ran apt update. I confirmed that all files ending with "_Packages.lz4" are now stored in uncompressed form,  and my searches are nearly instantaneous.

 

Aside from lost disk space, are there any downsides to this approach?

Just curious: How much is the difference in space?

Link to post
Share on other sites

It looks like the files are approximately three times larger now (roughly 35 MB vs 93).

 

I noticed that /etc/apt/apt.conf.d/02-armbian-compress-indexes specifies that files should be gzipped:

Acquire::GzipIndexes "true";
Acquire::CompressionTypes::Order:: "gz";

while mine were compressed with lz4. Not sure if this could explain the poor performance, but I thought it was curious. For what it's worth, they're compressed with .gz on my x86_64 desktop.

 

Edit: scratch that - the package files on my desktop are uncompressed, but there are some other files in that directory compressed with gz.

Edited by matteo
correcting erroneous information
Link to post
Share on other sites
3 hours ago, Werner said:

Not sure why this has been implemented. Maybe in the early days to save valuable disk space but today maybe it could be sacrificed for the sake of performance?

I'm not sure either, though I checked and the files are uncompressed on my old cubox i4 pro running Stretch server.  Whatever the case, I prefer searching with apt, so the trade off is certainly worth it for me.

 

3 hours ago, guidol said:

from my personal feeling the time-usage of apt has increased since (armbian) buster.

An apt update after a fresh install was much faster before buster (with stretch, jessie, wheezy)

This is difficult for me to judge, as I went from running Stretch on a cubox via microsd, to Buster on an n2 via emmc. This new system is so much faster that I'm afraid I can't draw any comparisons.

Link to post
Share on other sites

You're welcome, linuxfan, and thank you for confirming that it works on your setup as well. Big thanks to Igor for mentioning the compressed indexes over on the odroid forum!

 

I did some poking around, and it looks like the decision to compress the indexes was made upstream:

Quote

I'd like us to try out using LZ4 compressed index files in
/var/lib/apt/lists for the next APT release series, starting
in October, after the release of Ubuntu 17.10 "artful".

This is done by swapping the default for Acquire::gzipIndexes
from false to true.

On my system, this compresses /var/lib/apt/lists
from 1.3 GB down to 241 MB, which is a lot of space.

https://groups.google.com/forum/#!topic/linux.debian.devel/f_rlg3-cw2Y

 

I ran a test and found that a Tritium H5 running Bionic has no problem searching compressed indexes with apt. Maybe this issue is peculiar to Amlogic SBCs? Or perhaps a Debian vs Ubuntu issue?

 

Link to post
Share on other sites
5 hours ago, Tido said:

Very interesting, and thank you for sharing. Do you have any insight into why the Tritium H5 would be so much faster than the ODroid N2 (9 seconds vs 750), despite both including the NEON extension? Not that it matters, I'm happy to use uncompressed indexes.

Link to post
Share on other sites

Hi everyone, 

Interestingly enough I had one board have this problem while a fresh new install on another identical board that has been around much longer did not have the issue. 

 

At first deleted the armbian repo lz4 files first and ran apt update. That didn't seem to solve the issue.

 

I was able to find that the culprit was this file:

/var/lib/apt/lists/deb.debian.org_debian_dists_buster_main_binary-arm64_Packages.lz4

 

The freshly installed board board did not have the compressed version. I deleted that file and ran apt update and it was recreated as a lz4 file (and took a good while running cnf-update-db). 

 

I ended up simply unlz4'ing the file and now my full text searches are back to normal speed. I re-ran apt update and it didn't pull down a compressed version. 

 

Since these are the debian based packages I thought it had to do with the compress index but the two boards had identical /etc/apt/apt.conf.d/ entries. The only difference was updates that happened in between when I ran the build script - so I'm wondering if once you have a lz4 version apt continues getting that version instead of the uncompressed one.

 

If the symptoms come back I will update this thread. 

Link to post
Share on other sites
8 minutes ago, linuxfan said:

Can we somehow reach out to the Armbian team?


... which has endless resources to cover your R&D ideas, fix bugs in public software and cover 99% of the costs by doing that.
 

17 minutes ago, linuxfan said:

Maybe they don't even know about this bug?


Your support contract:

https://github.com/armbian/build#support

 

Without your help, things will probably not move anywhere:

https://armbian.atlassian.net/browse/AR-296

https://github.com/armbian/build/tree/master/packages/bsp/common/etc/apt/apt.conf.d
https://docs.armbian.com/Process_Contribute/

https://forum.armbian.com/forum/54-help-wanted/

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...