tkaiser Posted October 6, 2017 Posted October 6, 2017 When building fresh jessie images the last days I always got somewhere in between: W: Failed to fetch http://apt.armbian.com/dists/jessie/InRelease Unable to find expected entry 'jessie-utils/binary-armhf/Packages' in Release file (Wrong sources.list entry or malformed file) Such an image then also reports this as a problem when doing an 'apt update': ... W: Failed to fetch http://apt.armbian.com/dists/jessie/InRelease Unable to find expected entry 'jessie-utils/binary-armhf/Packages' in Release file (Wrong sources.list entry or malformed file) E: Some index files failed to download. They have been ignored, or old ones used instead. root@odroidxu4:~# cat /etc/apt/sources.list.d/armbian.list deb http://apt.armbian.com jessie main jessie-utils jessie-desktop root@odroidxu4:~# I searched for this in the forum and found Zador mentioning he had the problem but deleting cached rootfs solved the problem. Did this right now but compile.sh still complains as above (quoted from most recent run right now). Any ideas? Edit: forgot to provide 'armbianmonitor -u' output from running installation: http://sprunge.us/iXJP
Igor Posted October 6, 2017 Posted October 6, 2017 I think that means I need to manually rebuild the main repository ... which is today and tomorrow running on (HC1) my desk until the main server is transferred to new location.
mpmc Posted October 6, 2017 Posted October 6, 2017 I had a similar problem, thought it was my network, but re-ran the compile after deleting the cache (It needed doing anyway), that seemed to fix it. . Wasn't sure whether to report it or not, so thanks @tkaiser!
tkaiser Posted October 17, 2017 Author Posted October 17, 2017 Hmm... downloaded https://dl.armbian.com/orangepiplus2e/Debian_jessie_default.7z, booted OPi Plus 2E and get W: Failed to fetch http://apt.armbian.com/dists/jessie/InRelease Unable to find expected entry 'jessie-desktop/binary-armhf/Packages' in Release file (Wrong sources.list entry or malformed file) What happened to both jessie-utils and jessie-desktop packages in the repo? Armbianmonitor -u and contents of /etc/apt/sources.list.d/armbian.list: deb http://apt.armbian.com jessie main utils jessie-desktop BTW: Wouldn't be switching to https://apt.armbian.com a good idea?
zador.blood.stained Posted October 17, 2017 Posted October 17, 2017 26 minutes ago, tkaiser said: BTW: Wouldn't be switching to https://apt.armbian.com a good idea? Not exactly necessary, package signing provides a good security by default: https://askubuntu.com/a/146117
tkaiser Posted October 17, 2017 Author Posted October 17, 2017 6 minutes ago, zador.blood.stained said: package signing provides a good security by default: https://askubuntu.com/a/146117 I was thinking about DNS spoofing attacks (not that realistic but at least possible if I understood the current KRACK implications correctly)
zador.blood.stained Posted October 17, 2017 Posted October 17, 2017 34 minutes ago, tkaiser said: I was thinking about DNS spoofing attacks (not that realistic but at least possible if I understood the current KRACK implications correctly) Packages are signed, package lists are signed and public key is imported at the image creation stage. Adding HTTPS will only change the list or package signature error into invalid HTTPS certificate error in case apt.armbian.com is redirected to a 3rd party server.
Igor Posted October 17, 2017 Posted October 17, 2017 Huh. Do we have a problem or not? I mean after last repository rebuild? On a related topic. Since now we have .uk server online, shall we implement some "load balancing" into first-run or armhwinfo script?
tkaiser Posted October 17, 2017 Author Posted October 17, 2017 15 minutes ago, Igor said: Do we have a problem or not? I mean after last repository rebuild? We still have one since 1 or 2 weeks (but it seems no one is using Jessie any more ). Currently jessie-utils and jessie-desktop packages throw errors. 19 minutes ago, Igor said: Since now we have .uk server online, shall we implement some "load balancing" into first-run or armhwinfo script? Hmm... if we stay with http deb sources (which I'm fine with) what about better using round robin DNS?
zador.blood.stained Posted October 17, 2017 Posted October 17, 2017 35 minutes ago, Igor said: Since now we have .uk server online, shall we implement some "load balancing" into first-run or armhwinfo script? Or at the server side (nginx). As a bonus it's easy to add/remove servers at any point without breaking the clients or forcing them to update server addresses. https://nginx.org/en/docs/http/load_balancing.html Edit: the proxy implementation doesn't really solve the limited traffic problem, so I meant the redirect approach: https://serverfault.com/questions/597671/load-balancing-in-nginx-with-redirect-rather-than-proxy I guess DNS round-robin will work too
botfap Posted October 17, 2017 Posted October 17, 2017 Round Robin DNS and nginx cluster balancing are probably not the correct solutions here for a number of reasons. Primarily they both break session consistency for the client which messes with downloads, leaving corrupt incomplete archives and downloading other archives multiple times. The bigger the file size and and frequency, the bigger the problem becomes. The nginx load balancing is great in theory but the whole world wide cluster goes down if the primary load balancer goes offline. I would have thought you would want to silently direct people to their closest geographical mirror (IP based) from an initial download perspective and then do the same at initial boot time (but obviously with a mechanism to swap mirrors). Same goes for new builds, set the mirror location at initialisation time and produce an image with that mirror config. If you want, we can help out with this.
Bingo Posted October 17, 2017 Posted October 17, 2017 I'm getting this error on 2 of my OPIZ's Quote W: Failed to fetch http://apt.armbian.com/dists/jessie/InRelease Unable to find expected entry 'jessie-desktop/binary-armhf/Packages' in Release file (Wrong sources.list entry or malformed file)
Igor Posted October 17, 2017 Posted October 17, 2017 30 minutes ago, Bingo said: I'm getting this error on 2 of my OPIZ's 3 hours ago, tkaiser said: What happened to both jessie-utils and jessie-desktop packages in the repo? Working on it ...
zador.blood.stained Posted October 17, 2017 Posted October 17, 2017 2 hours ago, botfap said: I would have thought you would want to silently direct people to their closest geographical mirror (IP based) from an initial download perspective and then do the same at initial boot time (but obviously with a mechanism to swap mirrors) For apt packages I was thinking about something similar to httpredir.debian.org (even though it's listed as discontinued here, but we don't have to use this exact software implementation). You don't have to worry about large files or parallel/multithreaded downloads with apt, and while both current mirrors are located more or less in Europe GeoIP won't make a significant improvement yet. 2 hours ago, botfap said: Same goes for new builds, set the mirror location at initialisation time and produce an image with that mirror config. Again, IMO having a single address (apt.armbian.com) on clients and dealing with CDNs, proxies or redirects at the server side is easier and more reliable.
Bingo Posted October 17, 2017 Posted October 17, 2017 Ohh yes Preparing to unpack .../hostapd_2%3a2.6-4~armbian5.34.171017+1_armhf.deb ... Thanx Igor /Bingo 1
Igor Posted November 14, 2017 Posted November 14, 2017 On 17. 10. 2017 at 7:17 PM, zador.blood.stained said: and dealing with CDNs, proxies or redirects at the server side is easier and more reliable And how to implement this? For example - user clicks on: https://uk.armbian.com/bananapi/Debian_jessie_next.7z and Nginx randomly redirects to https://dl.armbian.com/bananapi/archive/Armbian_5.31_Bananapi_Debian_jessie_next_4.11.5.7z or https://dl.uk.armbian.com/bananapi/archive/Armbian_5.31_Bananapi_Debian_jessie_next_4.11.5.7z If this is possible to achieve in some trivial way, it is good enough.
botfap Posted November 14, 2017 Posted November 14, 2017 If you install geoip-database then define a map with different regions, ie. west-eu and not west-eu nginx will redirect downloads to the closest server based on ip. Obviously you probably only want it on the dl.* vhost not on the others. It takes 10-15 mins to set up for 2 regions. If you want I can set it up on the uk server and generate config patches for other regions.
Igor Posted November 14, 2017 Posted November 14, 2017 I seek some dirty and quick solution at this moment. Or no change at all. Random balancing is O.K. since I only want to share bandwidth ... later we can make some geo-based.
tkaiser Posted November 14, 2017 Author Posted November 14, 2017 43 minutes ago, Igor said: I seek some dirty and quick solution at this moment. If I understood correctly using GeoIP is a nice attempt to do it quick&dirty (at least after quickly reading through https://www.digitalocean.com/community/tutorials/how-to-use-nginx-as-a-global-traffic-director-on-debian-or-ubuntu). Configure 3 servers (two times Lauri's sitting in the US and in Russia, and uk.armbian.com sitting in the EU), done. Should then work for both images and apt traffic.
botfap Posted November 14, 2017 Posted November 14, 2017 4 hours ago, tkaiser said: If I understood correctly using GeoIP is a nice attempt to do it quick&dirty (at least after quickly reading through https://www.digitalocean.com/community/tutorials/how-to-use-nginx-as-a-global-traffic-director-on-debian-or-ubuntu). Configure 3 servers (two times Lauri's sitting in the US and in Russia, and uk.armbian.com sitting in the EU), done. Should then work for both images and apt traffic. Thats what I would suggest, pretty much in the same way as that article. Its very simple and straight forward to set up and is very easy to adapt if you need to add / change servers. Its more complicated to put in place simple redirects with nginx that to do proper geoip based distribution. To do simple random redirects you could use a combination of rewrite and random rules from the rewrite and misc nginx addons with region maps to achieve the simple solution, but its a lot more effort than doing it properly. 1
botfap Posted November 14, 2017 Posted November 14, 2017 Its now set up on the UK server for both dl and apt. Test it, if you visit from the US it redirects you to main. Clone the setup on to any webserver with the following patch against /etc/nginx. Add more regions in the map if needed and make sure the server name directive for each vhost contains all the regional host names. You will also need the max mind geoip database from here: http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz. Save it to /usr/share/GeoIP. It literally takes less than 2 mins per server then its done. Dont fanny about with bad solutions when the proper solution is so easy. diff -r bkp/etc/nginx/nginx.conf /etc/nginx/nginx.conf 60a61,64 > #Updated maxmind geoip data > geoip_country /usr/share/GeoIP/GeoIP.dat; > geoip_city /usr/share/GeoIP/GeoLiteCity.dat; > diff -r bkp/etc/nginx/sites-available/apt.uk.armbian.com /etc/nginx/sites-available/apt.uk.armbian.com 0a1,6 > map $geoip_city_continent_code $closest_server { > default apt.armbian.com; > EU apt.uk.armbian.com; > US apt.armbian.com; > } > 5c11 < server_name apt.uk.armbian.com; --- > server_name apt.uk.armbian.com apt.armbian.com; 7a14,18 > > #GeoIP redirect > #if ($closest_server != $host) { > # rewrite ^ $scheme://$closest_server$request_uri break; > #} diff -r bkp/etc/nginx/sites-available/dl.uk.armbian.com /etc/nginx/sites-available/dl.uk.armbian.com 26a27,32 > map $geoip_city_continent_code $closest_server { > default dl.armbian.com; > EU dl.uk.armbian.com; > US dl.armbian.com; > } > 31c37 < server_name dl.uk.armbian.com; --- > server_name dl.uk.armbian.com dl.armbian.com; 33a40,44 > > #GeoIP redirect > if ($closest_server != $host) { > rewrite ^ $scheme://$closest_server$request_uri break; > } If you need vpn's for testing I can give you and ipsec, l2tp or pptp vpn connection that terminates in the US, UK, Germany, Italy, France, Australia, Russia, Taiwan, Japan or Holland. 1
rainforest1155 Posted December 4, 2017 Posted December 4, 2017 On 10/17/2017 at 7:13 PM, Igor said: It should be fixed now. I'm seeing the same issue now for jessie-utils: "W: Failed to fetch http://apt.armbian.com/dists/jessie/InRelease Unable to find expected entry 'jessie-utils/binary-armhf/Packages' in Release file (Wrong sources.list entry or malformed file)" The APT sources list file looks like this: "deb http://apt.armbian.com jessie main jessie-utils jessie-desktop" Is there any change that I would need to make to resolve this issue? Thanks, Sebastian
tmihai20 Posted December 4, 2017 Posted December 4, 2017 I am getting this error on my oDroid running OMV (based on Debbian, built here by @tkaiser). What is the fix? Do we need to modify a repo manually?
Igor Posted December 4, 2017 Posted December 4, 2017 28 minutes ago, tmihai20 said: I am getting this error on my oDroid running OMV (based on Debbian, built here by @tkaiser). What is the fix? Do we need to modify a repo manually? This means we have some trouble with the repository. You don't need to do anything. It will be fixed asap / eventually. 1
Вячеслав Зубакин Posted December 4, 2017 Posted December 4, 2017 Hi Igor! Can not make access point with Asus USB N10 nano ... Linux orangepipc 3.4.113-sun8i #4 SMP PREEMPT Wed Nov 22 13:45:28 CET 2017 armv7l GNU/Linux sudo apt-get -y install git cd ~ git clone https://github.com/igorpecovnik/hostapd chmod +x ./hostapd/go.sh cd hostapd ... root@orangepipc:~/hostapd# ./go.sh [ o.k. ] Building hostapd [ o.k. ] Downloading dependencies. E: Unable to correct problems, you have held broken packages. Outside chroot [ o.k. ] Compiling v2.5 for Realteks. [ o.k. ] Compiling v2.5. [ err. ] hostapd not built. and apt-get says: root@orangepipc:~# apt-get install hostapd-realtek Reading package lists... Done Building dependency tree Reading state information... Done The following packages will be REMOVED: hostapd The following NEW packages will be installed: hostapd-realtek 0 upgraded, 1 newly installed, 1 to remove and 5 not upgraded. Need to get 271 kB of archives. After this operation, 170 kB disk space will be freed. Do you want to continue? [Y/n] Err http://apt.armbian.com/ jessie/jessie-utils hostapd-realtek armhf 2:2.5-1~armbian5.34.171017+1 404 Not Found E: Failed to fetch http://apt.armbian.com/pool/jessie-utils/w/wpa/hostapd-realtek_2.5-1~armbian5.34.171017+1_armhf.deb 404 Not Found E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing? root@orangepipc:~# What I need to do for success?
Igor Posted December 4, 2017 Posted December 4, 2017 30 minutes ago, Вячеслав Зубакин said: What I need to do for success? Wait that we fix repository which was (not correctly) updated yesterday or manual way.
Igor Posted December 4, 2017 Posted December 4, 2017 1 hour ago, Вячеслав Зубакин said: Can not make access point with Asus USB N10 nano ... The repository is fixed. 1. apt update & apt upgrade 2. armbian-config -> network -> hotspot
Recommended Posts