My apt search has become super slow recently

[mrjpaxton]

Now in theory, I could try to manually remove the *.lz4 files only, but that is a hack, and I'm not going to do that. It doesn't actually solve the problem in a concise, concrete way.

 

I feel like the overly intrusive lz4 files are just going to be rebuilt somehow, and there's no stopping them. It's like they are hard-coded in to ruin us ARM users.

 

I also want to correct a mistake that I made. I think `apt update` is actually building both *.gz and *lz4 archives in the `/var/lib/apt/lists/partial` directory the whole time, and not just *.gz

 

Yeah, this is starting to be a really bad problem for people who have updated their systems... worse than I thought.

 

You know what's interesting though, I actually have an Ubuntu 20.04 system, and I compared the `/var/lib/apt/lists` directories, and Ubuntu actually keeps them uncompressed with some .gz files, so the Ubuntu patch must have been changed, or been deprecated, or something. I dunno.

 

This is just weird. It could be something Debian-specific, but that's just speculation at that point. I need to check a proper Debian Live or Debian desktop or something, and find out. Then I can compare the apt.conf files from Ubuntu 20.04 and Debian Buster, and maybe find a solution for this problem on Armbian.

[mrjpaxton]

Okay, sorry for the multiple posts, by the way. I think I have found a solution for the problem.

 

For my next test case, I just tried uncompressed. I made quick a backup of my `/etc/apt/apt.conf.d/` as `/etc/apt/apt.conf.d.bak/`, and then I edited a few lines.

 

Firstly, I changed the line in `/etc/apt/apt.conf.d/02-armbian-compress-indexes` to this:

 

Acquire::GzipIndexes "false";

 

Secondly, I changed each line of "KeepCompressed" in `/etc/apt/apt.conf.d/50apt-file.conf` to this:

KeepCompressed "false";

 

Now use this command:

 rm -r /var/lib/apt/lists/

 

And run `apt update`. Wait for it to finish, then try `apt search` again.

 

It should work for now! I could provide patch files on request, but I think some simple changes to config files can be done by anyone. I'm going to make sure that daily, sequential runs of `apt update` and/or `apt clean` do not break this. But so far, it's looking good.

 

EDIT on 11 Feb 2021:

 

One more thing I noticed after browsing around in my config files was `/etc/logrotate.conf` has an uncommented line that says "compress".

 

Can anyone verify if commenting that out gets rid of LZ4 compression for APT? I already have the uncompressed solution above working for me, but maybe I can eventually test in the future to see if logrotate is causing any issues.

 

I'm going to guess not, since it really shouldn't interfere with dpkg/APT lists, but maybe I'm wrong.

[Kimberly White]

If apt-crawl is slowing the crawl or even stopping, read about the IPv6 fix that will keep your server and desktop running smoothly.

[lurdan]

Another solution (tested on armbian buster):

 

Add this line to /etc/apt/apt.conf.d/02-armbian-compress-indexes:

APT::Compressor::gzip::Cost "10";

 

and remove /var/lib/apt/lists/*.lz4.

[Igor]

If this is proven to work also on Ubuntu variant(s), welcome to submit a PR here:

https://github.com/armbian/build/blob/master/packages/bsp/common/etc/apt/apt.conf.d/02-armbian-compress-indexes

[lurdan]

/usr/share/doc/apt/NEWS.Debian says:

Quote

  If you use Acquire::gzipIndexes, or any other compressed index targets,
  those will now be compressed with the fastest supported algorithm,
  currently lz4.

 

and their default settings (same on Debian buster and Ubuntu 20.04):

 

~$ apt-config dump | grep Cost
APT::Compressor::.::Cost "0";
APT::Compressor::zstd::Cost "60";
APT::Compressor::lz4::Cost "50";
APT::Compressor::gzip::Cost "100";
APT::Compressor::xz::Cost "200";
APT::Compressor::bzip2::Cost "300";
APT::Compressor::lzma::Cost "400";

 

So, basically setting any number under 50 to gzip's Cost will make apt use gzip to compress its indexes. 

 

Anyway, I've send a PR, though I'm not sure its proven to work in ubuntu variants (not interested in their modiifications), and also "10" is good choice or not.

cf. https://github.com/armbian/build/pull/2729

[Igor]

2 hours ago, lurdan said:

Anyway, I've send a PR

Thank you! Merged.
 

2 hours ago, lurdan said:

in ubuntu variants (not interested in their modiifications)

Don't worry, we neutralise (hopefully all) of their bloatware and modifications  (Usually) more recent package base in its stable addition comes handy sometimes.

[jpegxguy]

On 3/18/2021 at 10:09 AM, lurdan said:

Another solution (tested on armbian buster):

 

Add this line to /etc/apt/apt.conf.d/02-armbian-compress-indexes:

APT::Compressor::gzip::Cost "10";

 

and remove /var/lib/apt/lists/*.lz4.

I just want to say, thanks for this. LZ4 was REALLY slow on my Renegade board, and it kept popping up. I added that line and did not touch any other file. Good to see it merged, too.

[blood]

On a rockpi 4 I found that switching from lz4 to gz by itself didn't make much of a difference, but disabling compressed indices altogether in /etc/apt/apt.conf.d/02-armbian-compress-indexes and deleting all the lz4 and gz files from /var/lib/apt/lists/ did the trick for me. I'll happily give up a bit of disk space for the interactive performance for now.

 

I didn't dig into the code yet (and maybe this is already obvious to some) but I suspect this is a mix of unoptimized code paths when dealing with compressed files as well as maybe differences in instruction set extentions between systems. With the default buster settings, this slow behavior occurs on all my arm systems except for a clearfog base which has the notable differences of not being aarch64 and including some handy instruction set extensions that aren't always present on other boards: half thumb fastmult vfp edsp thumbee neon vfpv3 tls vfpd32

 

When I strace one of the "search" operations, I see a pattern that just doesn't look right... I see lots of repeated "openat" syscalls for the same files over and over along with "reads" for the same blocks of data. Over time, the reads expand to read additional data - but they always follow reading the same data over and over again. It smells to me like whatever method is being used by apt to iterate through these compressed files is resulting in some _very_ inefficient file access patterns. None of this translates to any noticeable hit on the disks because the filesystem cache works well and so all you see is user cpu thrashing one core.

 

I'll be surprised if there isn't a way to avoid the pattern that I'm observing that would clean this up considerably and hopefully I'll have time to help figure it out.

[Fluora]

I've been dealing with this problem for a while now, and after seeing this thread, I decided to dive in a little deeper.

With LZ4-compressed lists, running an apt search on my Rock64 currently takes at least 20 minutes. As other users have reported, it appears to be bottlenecked by CPU, and is not saturating the storage I/O. Needless to say, this is not a usable speed.

 

Based on what I've found, I believe with substantial confidence that there is no performance problem with LZ4. This should not be too much of a surprise, because speed, especially decompression speed, is LZ4's primary objective; it is in fact considerably faster than LZO in that regard, which means that it's dramatically faster than DEFLATE (gzip). I cannot imagine any conceivable case where switching from LZ4 to DEFLATE would result in decompression performance improvements, and indeed it does not seem to do so here.

 

In any case, you can trivially see this for yourself by using the command-line lz4 utility to decompress one of the compressed list files in /var/lib/apt/lists. On my Rock64, decompressing the largest of these (/var/lib/apt/lists/apt.armbian.com_dists_buster_main_binary-arm64_Packages.lz4, which weighs about 1.8 MB) takes around a tenth of a second. For some perspective, my test run of an apt search took just over 2700 seconds, so if it did spend all or most of this time doing LZ4 decompression, then it could easly have decompressed every list in the directory over a thousand times each. I am not an expert on the intricacies of file compressors, but I find it to be highly unlikely that LZ4 is responsible for the slowness we're seeing here.

 

I had a look at the strace output for the search operation. I'm new to reading straces, but I think I can confirm what a previous poster said, which is that it's accessing the same files over and over again with small reads, often many times in the same place. It is unclear to me what function this serves, but this appears to be what takes up the majority of the runtime. If you would like to see for yourself, I've made the full strace output available at https://fluora.net/strace_apt-search_armbian.7z. Be warned that it's around 260 megabytes when uncompressed, but since it's extremely repetitive, it compresses efficiently, and the 7z archive is only about 96 kB. In addition, you may compare this to the strace of a sane apt-search operation that I took on an x86 laptop running Debian Bullseye with gzip-compressed package lists, available at https://fluora.net/strace_apt-search_debian86.txt (116 kB, not compressed). This isn't much more than a guess, but I wonder if apt is decompressing the entire list with each one of these repeated read operations - this would explain not only the slowness, but also why switching to uncompressed lists seems to help (since it allows the storage cache to render the repeated accesses cheap).

 

From what I've found here, I strongly suspect that something is wrong with the version of apt distributed by Armbian. I don't know enough about the code to know where to start looking, but this is the point where I would hope that someone familiar with Armbian's modifications to apt could help out.

[Werner]

Armbian does not distribute or touch apt as package at all. This is upstream, Debian or Ubuntu to say.

[haskal]

i have reviewed the strace output helpfully provided by Fluora and most of it is taken up by repeatedly opening, reading, and closing the same index file over 26 thousand times. specifically, the file is opened, one chunk is read, then it is closed, and the process repeats. eventually it gets to reading 2 chunks on each iteration, and then 3, and so on

 

that seems like the problem! there's no reason to be repeatedly opening and reading a file from the beginning tens of thousands of times just for one search. and this does sound like an apt bug -- somehow it is deciding to just spin the same file read for 20 minutes, rather than just reading it once

[Fluora]

On 4/27/2021 at 11:51 PM, Werner said:

Armbian does not distribute or touch apt as package at all. This is upstream, Debian or Ubuntu to say.

Oh, sure enough. I see this now.

I had a look at the strace dumps again, and it looks like this is in fact an upstream bug affecting all apt versions I tested.

I didn't think this was the case at first because Armbian is the only one where the bug inflates the processing time enough to make apt search actually unusable - on my Raspberry Pis running Raspbian and laptop running Debian, searching with compressed indexes only takes a few seconds, even despite the many repeated access operations. I remain perplexed as to why only Armbian seems to take around two orders of magnitude longer to complete the operation (presumably due to a similarly higher number of extraneous read operations).

 

In any case, the solution in the meantime is just to disable compression, either by commenting out the lines in /etc/apt/apt.conf.d/02-armbian-compress-indexes or by removing that file, then clearing and re-fetching the index files in /var/lib/apt/lists/. When operating on uncompressed lists, apt seems to be much better behaved, and I don't tihnk I see any signs of the vast numbers of repeated file accesses that I see when the lists are compressed. As I already mentioned, and contrary to what has been said previously in this thread, switching to gzip will not be effective. Disabling list compression entirely seems to be the only way to dodge this bug at the moment. Fortunately, it does appear to work properly; when operating on uncompressed lists, apt doesn't seem to perform the extraneous reads at all (and you're not just being saved by your disk cache).

 

I still can't quite figure out why this seems to be affecting Armbian so much more severely than any of the other Debian-derived distros I currently run. The bug clearly happens on my other systems, but only makes the search operation take around 10 seconds at most, whereas on Armbian, searches on compressed lists take upwards of 20 minutes, and don't go faster than about 16 seconds even with uncompressed ilsts. It could be that there are just more lists to search through under my Armbian install than on my others, but I don't think the differences are enough to account for that, even considering that the bug causes the time it takes to load a file to scale with the third power of the file size.

 

Anyway, a bug report clearly needs to be opened on apt upstream, and Armbian's configuration tweak to enable compression of index files should probably be removed or disabled, at least until the upstream bug is fixed.

[debianxfce]

I have this bug in the  Armbian git version and Amlogic S912. Removing gz files and running apt-get update solved the problem and apt search works OK. I am running Debian Sid Xfce.

[weyou]

Found the same bug on RockPi4, the solution "Cost: 10" does switch the compress method from "lz4" to "gz". but just like @blood said, it doesn't make any difference, the "apt search" still very slow.

[armleuchter]

Hello  new on board.

 

This issue persists and obviously a "feature" (aka. BUG) on this armbian version? This issue doesn't occure most of the normal linux distros and not even on the hardkernel ubuntu and not on rapsian as far as I observed.

 

I have an odroic-c2 with "Armbian 21.02.3 Buster" and no desktop environment just pure cli.

 

Here a simple tests and results

 

# time sudo apt search fbdev

Sorting... Done
Full Text Search... Done
kwin-wayland-backend-fbdev/stable 4:5.14.5-1 arm64
  KDE window manager fbdev plugin

libmali-xu4-x11-gbm-fbdev/buster 19.0.6-3armbian1 armhf
  Mali X11 FBDEV GBM Binary Driver and Development files

libomxil-bellagio0-components-fbdevsink/stable 0.1-2 arm64
  Frame Buffer Video Sink components for Bellagio OpenMAX IL

mali-t62x-fbdev-driver/stable 0.1-3 armhf
  Mali binary framebuffer driver for t62x

mali-t76x-fbdev-driver/stable 0.1-3 armhf
  Mali binary framebuffer driver for t76x

mplayer/stable 2:1.3.0-8+b4 arm64
  movie player for Unix-like systems

mplayer-gui/stable 2:1.3.0-8+b4 arm64
  movie player for Unix-like systems (GUI variant)

xserver-xorg-video-fbdev/stable 1:0.5.0-1 arm64
  X.Org X server -- fbdev display driver


real	0m21.671s
user	0m20.076s
sys	0m0.867s

 

# time sudo apt-cache search fbdev
kwin-wayland-backend-fbdev - KDE window manager fbdev plugin
libomxil-bellagio0-components-fbdevsink - Frame Buffer Video Sink components for Bellagio OpenMAX IL
mplayer - movie player for Unix-like systems
mplayer-gui - movie player for Unix-like systems (GUI variant)
xserver-xorg-video-fbdev - X.Org X server -- fbdev display driver
mali-t62x-fbdev-driver - Mali binary framebuffer driver for t62x
mali-t76x-fbdev-driver - Mali binary framebuffer driver for t76x
libmali-xu4-x11-gbm-fbdev - Mali X11 FBDEV GBM Binary Driver and Development files

real	0m1.844s
user	0m1.657s
sys	0m0.132s

 

It's not the worst situation because at least apt-get still works fine - but will there ever be a fix for that?

Edited June 9, 2021 by armleuchter
fixing bad english :)

[EasyNet]

HI all,

 

I have a new Armbian installed and the search on Odroid C2 is about 116 minutes ..

 

# time apt search mali
Sorting... Done
Full Text Search... Done

aewm/stable 1.3.12-3 arm64
  minimalist window manager for X11
...
zathura/stable 0.4.3-1 arm64
  document viewer with a minimalistic interface

real    116m51.972s
user    114m27.114s
sys     2m10.154s

 

On my Odroid XU4 is very fast, but is using Armbian 32 bit.

[jotapesse]

Hi, all!

 

I also confirm this issue on my Helios64 with a fresh install of the latest Armbian Buster v21.05.4 and default settings. It seems to be a persistent and global issue still. Reported by the OP more than one year now. I'm sorry to ask but why isn't the Armbian team doing anything about it? Solution as detailed above is to disable the APT lists compression entirely. It would be probably best to have this taken care of in the next Armbian releases/updates, compression should probably be optional and not the default.

 

Workaround steps:

 

1. Remove the file "/etc/apt/apt.conf.d/02-armbian-compress-indexes" or edit it to comment out all settings resulting as following:

//Acquire::GzipIndexes "true";
//Acquire::CompressionTypes::Order:: "gz";
//APT::Compressor::gzip::Cost "10";

 

2. Remove the existing compressed APT lists:

# rm /var/lib/apt/lists/*

 

3. Update APT indexes to recreate the new uncompressed index files:

# apt update

 

[armleuchter]

2 hours ago, jotapesse said:

I'm sorry to ask but why isn't the Armbian team doing anything about it?

 

Obviously because they are people like you and me and go through their daily life ... imho 

 

I just changed back to the ubuntu OS from hardkernel for my C2 and everything works fine there. I just wanted to test armbian but I guess it was not what I though it would be.

 

So for me it's solved as unsolved  and I don't care about it anymore. I am out from armbian.

[Igor]

57 minutes ago, armleuchter said:

everything works fine there

 

 I am sure HW sellers demo OS level creations are full of bugs, but perhaps not this one. Thank you for your support.

 

57 minutes ago, armleuchter said:

ubuntu OS from hardkernel

Use Ubuntu based Armbian since it seems bug is only on Debian derived builds.
https://docs.armbian.com/#what-is-armbian

[jotapesse]

4 hours ago, Igor said:

Use Ubuntu based Armbian since it seems bug is only on Debian derived builds.
https://docs.armbian.com/#what-is-armbian

 

I would rather have Debian based, Ubuntu brings me back bad memories so I stopped using anything Ubuntu quite sometime ago. By the way, I understand that Armbian is developed on Ubuntu first, is that right? As there is an "unstable" Ubuntu Hirsute based image available but none of Debian Bullseye or Sid. I wonder why?  

[Igor]

6 hours ago, jotapesse said:

By the way, I understand that Armbian is developed on Ubuntu first, is that right? As there is an "unstable" Ubuntu Hirsute based image available but none of Debian Bullseye or Sid. I wonder why?  

Armbian is a build system first, cheap ARM hardware enabling second, OS 3rd. We construct user land from Ubuntu and Debian packages since early days, could be said from the day 0. We always provided both because of things as such - bugs and to not deal with user land - let user choose what he needs / prefers. We - those few people - that are investing our resources into common software development, can't afford to fix (all) bugs that showed up. That is not possible by greater Debian community or open source community software in general. Certainly not by corporate Linux flushed with resources such as Cannonical, RedHat, ... even they could do a lot more.

 

6 hours ago, jotapesse said:

I would rather have Debian based, Ubuntu brings me back bad memories

We are using carefully selected packages. Cancer, which you are afraid from, is removed in the process - but surely it could return for short time.

 

My x86 workstation is running Linux Mint, which is also assembled from Ubuntu packages, using their repositories. My personal and also very much Armbian philosophy is - get from Ubuntu what is good / was improved, leave everything else and make a new, better OS. In our case, this is even more extreme, since most of the hardware we deal with, doesn't even boot Ubuntu and we maintain our own kernels. The most expensive part of the OS.

 

6 hours ago, jotapesse said:

Ubuntu Hirsute based image available but none of Debian Bullseye or Sid. I wonder why?

Maintenance responsibility and costs. We are trying to keep things minimal (we only need one image with latest kernel, user land is less of importance) & manageable, low entropy ... and IHMO we are already failing. I work 6-10h on this project every day and there is not much to notice. Contrary. It looks like we are not doing anything, not answering on bugs, not answering on problems, not porting hardware, I don't answer on personal request ... People, everyday Linux user, usually have absolutely no clue whatsoever how much work is needed to fix a bug, to develop this and that feature, enable some hardware ... to keep a project as ours operational, that moves on.

[armleuchter]

13 minutes ago, Igor said:

People, everyday Linux user, usually have absolutely no clue whatsoever how much work is needed to fix a bug, to develop this and that feature, enable some hardware ... to keep a project as ours operational, that moves on.

 

That is true and the sad part that most of the time the DEVs are not really honored for their work for which they sacrifice their own time for but one has to be honest enough to mention also that it's voluntary and by creating such work you attract people to use your stuff otherwise why would you publish it? One could develope it for own purpose but by making it public it's automaticaly something you give to other people and of course they will use it.

The nature of it will always have bugs or problems but this should be fixed otherwise there is no use of the software and people will dismiss this work and will get bad reputation over time.

 

It's important that users and DEVs respect each others time, effort and needs (no pun intended) and for the "normal" users remember:

This stuff is NOT free!!! yes not monetary but it costs valuable human life time which the DEVs sacrifice. Other projects have more developers so more man power.

 

I still think that Armbian is OS with good basis and possibilities but it's not for me in current state and everybody should just evaluate if they want to stick with it or just move on.

May be in few years I will try armbian again 

[Igor]

12 hours ago, armleuchter said:

why would you publish it?

 

12 hours ago, armleuchter said:

and of course they will use it.

 

There are many reasons why people produce open source software. Master skills, knowing people, serving people yes, but you risk being eaten alive for peoples egoistic and business goals.

 

Users  are welcome to use our software but "no support" is a default answer. Going out from this zone is disrespecting from the users side. You got free software, but if you want attention, well ... in our case, I have an universal answer, there were 1000 people before you asking for attention / help. I would like to help, but it will be some time to wait. Since you don't pay for support, we can't expand and since we can't expand you need to wait years to get attention. Cruel, yes. But in most cases you will understand what you are asking for. People that sales hardware will always say the opposite, just to support sales ... 

 

12 hours ago, armleuchter said:

it will always have bugs or problems but this should be fixed otherwise there is no use of the software

 

There will be bugs, yes. All Linux distributions distribute a package of application and drivers that are a work of lets say 5.000 people. Or ten times more. It doesn't matter. When you approach to us and demands that we fix a bug for you (which usually nobody else will) especially in a software we have nothing with ... why on earth a bug you found in software package number 12496 is our problem to fix? There is no way to respect a user when approaching that way and educating users (explaining this in proper way) is expensive long term never ending job - when I hear "I am new to Linux" I stop reading - I am too old for this. Again on our costs (only what we like to do is not considered as a cost) since it happens that users doesn't pay for education. Why should they? They even refuse that need, since they think they are the smartest people on earth since they know how to use Linux. Even on some weird ARM single board computer ... 

In our world we have a lot of pleads like: hey, video acceleration in Chromium is not working, display driver via usbc is not working, board stuck on reboot (Odroid boards for example and 100% Hardkernel problem to fix which they ignore since its expensive to fix and it happens they mitigate this in their cheap dirty firmware). "Can you please fix it". A week later, more anger demand please fix it! Is it point to explain that person, that fixing that will take months if not years and cost lets say 20.000, 500.000 EUR or more and which is the reasons we will not even start. I am long enough in this world to understand how much something costs, so I don't allow that users will lead us into their small world. Big projects are never covered by amateurs, professions if there is enough of resources and if there is business case to cover. Not always is (in Linux rare is), while people / users have big eyes / demands / ideas. Just not providing any cash.

Economic relations are always in actions - it is a difference when users ask politely and with respect, but the problem is that a lot of users think it is our duty to serve them and invest our money to fix software they use. Why don't you just fix it? Its open source, provide a fix. In most cases costs for fixing are too big. Also for those big groups you have in mind. They don't even come close to this world if there is a bag of cash on the table.

Bugs on all Linux distributions can easily be opened for years before anything happens.

 

12 hours ago, armleuchter said:

but it's not for me in current state and everybody should just evaluate if

I know our product is not perfect, but product you are referencing is not even close. Vendors software support is without exceptions full of quick dirty and non standard solutions. Ofc they will make everything for you to persuade you to believe into something different.

 

12 hours ago, armleuchter said:

Other projects have more developers so more man power.

Armbian is community projects and have broader support. Developers you see around are also maintainers and those are doing the visible job very little will. Developing is fun, while this is also hard but boring job.

Vendor usually forward what it comes with the SoC and are dealing with everything so they look big (Ubuntu, Debian, Android, Librelec, ...). In Hardkernel's case, they used to stick to junk private firmware (on C2 probably still) while everything else is a forward what community projects are producing. In most cases vendor support is not on very good level. Similar other projects are one or two persons in size.
 

12 hours ago, armleuchter said:

May be in few years I will try armbian again 

 

From the surface, everything looks the same. Maybe in a few years you will learn enough to see the difference? Perhaps not.

[Igor]

Important and critical problems, which have much higher priority, can be sometimes solved fast. If you have already spent weeks to understand what the problem is. Most likely, since it still needs testing.

[jotapesse]

15 hours ago, Igor said:

My x86 workstation is running Linux Mint which is also assembled from Ubuntu (...) My personal and also very much Armbian philosophy is - get from Ubuntu what is good / was improved, leave everything else and make a new, better OS. (...) Maintenance responsibility and costs. We are trying to keep things minimal (we only need one image with latest kernel, user land is less of importance) & manageable, low entropy ... and IHMO we are already failing. I work 6-10h on this project every day and there is not much to notice. Contrary. It looks like we are not doing anything, not answering on bugs, not answering on problems, not porting hardware, I don't answer on personal request ... People, everyday Linux user, usually have absolutely no clue whatsoever how much work is needed to fix a bug, to develop this and that feature, enable some hardware ... to keep a project as ours operational, that moves on.

 

Hi @Igor! Thank you for taking the time to reply to me and for your detailed explanation. I certainly appreciate it and yours and the remaing Armbian team good work on this project as well. Thank you!  In my case I do prefer and use Debian Sid + KDE Plasma on my x86 workstations. But I do understand where you are coming from. Personal preference apart I was also under the expectation, as the name "Armbian" suggests to imply by the "...bian" part, that it would be Debian base focused firstly - therefore developed from it - and then Ubuntu secondly and that it would keep it as close to default Debian base while changing whatever necessary to boot and work on the arm hardware. I'm also not so sure on those possible "Ubuntu improvements over Debian" - but again that's my personal preference. Anyway keep up the good work, great job so far and hopefully it will get even better in the future. 

[jamespo]

Well for me

 

1. Comment out all lines in /etc/apt/apt.conf.d/02-armbian-compress-indexes with //
2. rm /var/lib/apt/lists/*.lz4 /var/lib/apt/lists/*.gz
3. apt update

fixed it

 

 

[Junkman]

FYI, it worked for me:

1. Backup the `/etc/apt/apt.conf.d/02-armbian-compress-indexes`

2. Change `/etc/apt/apt.conf.d/02-armbian-compress-indexes`

```diff

-Acquire::GzipIndexes "true";

+Acquire::GzipIndexes "false";

```

3. `sudo rm -rf /var/lib/apt/lists/*`

4. apt search ...

[jpegxguy]

Ok, it turns out that lz4 is actually much faster. /etc/apt/apt.conf.d/02-armbian-compress-indexes has to be removed from the official packages, it's a bad idea.

The gz indexes are slooow

[revoman]

2 hours ago, jpegxguy said:

Ok, it turns out that lz4 is actually much faster. /etc/apt/apt.conf.d/02-armbian-compress-indexes has to be removed from the official packages, it's a bad idea.

The gz indexes are slooow

I don't understand why, but it seems when compression backends are mixed is when the slowness occurs.  When trying this on my Rock64, RockPro64 and x86_64 VM with Armbian bullseye it seemed to be fast for a while, and then degrade...   disabling the compression made them all similarly quick.