Jump to content

Create a Method to Select Repo Mirror by Country


Recommended Posts

If you

curl -I http://apt.armbian.com

over and over again, you will receive a 302 redirect to what seems to be a random mirror. I am wondering how I can help add some sophistication to that randomness. As someone downloading updates from the US, it's a big exercise of patience to download at a rate of 20-60 KBps. As each package request goes to the redirector every time (since it's not a 301 redirect), some packages download at a good rate but plenty are very slow for me. In contrast, when I download an Armbian image from bittorrent, I can get 9MBps.

 

I know that everything here is provided for free and I am grateful! I am asking how I can help make this better myself. I have two ideas:

1) Someone teach me about the web server (or load balancer) that's currently handling the redirects... source code, documentation, etc. Maybe I can add geo-location based replies to the web clients' requests. I posted this issue thinking maybe it's the tool involved.

2) Maybe a simpler solution is more agreeable like a user picking a mirror or mirror set themselves. If this is already possible in a reliable way, I will write up the documentation page for it after someone explains it to me. :)

2a) Perhaps this method could be written into the armbian-config tool.

 

I personally like option 1 better.

 

 

A little about me since this is my first post:

I'm a long-time IT professional. I am not a developer, much more an infrastructure guy, but I work with modern systems which means I am good at scripting and configuration languages like Ansible, Terraform, and Helm Charts. I have a lot of web services and web traffic experience.

 

I would like to contribute to the effort of maintaining and enhancing Armbian!

Link to comment
Share on other sites

5 hours ago, greypilgrim said:

A little about me since this is my first post:

I'm a long-time IT professional. I am not a developer, much more an infrastructure guy, but I work with modern systems which means I am good at scripting and configuration languages like Ansible, Terraform, and Helm Charts. I have a lot of web services and web traffic experience.

 

I would like to contribute to the effort of maintaining and enhancing Armbian!


Lately we made many enhancements on infrastructure but a lot of things are still WIP. Re-director stuck somewhere in the half-way - geolocation is planned since start but requires more work and and better testings. Before jumping ahead I would welcome you to check https://armbian.atlassian.net/browse/AR-444 and comment there (PM to me or @lanefu) for write access. Then we decide which part suits you best.

 

We will do the best to support you but sometimes it can take time - we all have daytime work and families and we are also just in the final round to push out a release. 

 

Welcome!

Link to comment
Share on other sites

6 hours ago, greypilgrim said:

know that everything here is provided for free and I am grateful! I am asking how I can help make this better myself. I have two ideas:

1) Someone teach me about the web server (or load balancer) that's currently handling the redirects... source code, documentation, etc. Maybe I can add geo-location based replies to the web clients' requests. I posted this issue thinking maybe it's the tool involved.

2) Maybe a simpler solution is more agreeable like a user picking a mirror or mirror set themselves. If this is already possible in a reliable way, I will write up the documentation page for it after someone explains it to me

 

Yep that repo is the python code.  Technically apt is a branch in that repo.

 

It's a flask app that runs in WSGI containers.  2 instances running one for redirect.armbian.com and one for apt.armbian.com.  both reverse proxies behind nginx

 

I entertained just doing it with nginx but didn't want to go down the wormhole of exotic nginx configs wormhole or figure out lua

 

Anyway moving the redirect code to using a yaml config to indicate regions and other granual info is like the next step to making other improvements and I'd love a PR for that.

 

Then we can have an endpoint that list available mirrors.  And have a endpoint for regional grouping.  Ex apt.armbian.com/na or /eu

 

After that we can work on geoip support.  I also like the idea of setting mirror with armbian-config. That would pair well with having an endpoint that publishes all mirrors.

Link to comment
Share on other sites

6 hours ago, greypilgrim said:

A little about me since this is my first post:

I'm a long-time IT professional. I am not a developer, much more an infrastructure guy, but I work with modern systems which means I am good at scripting and configuration languages like Ansible, Terraform, and Helm Charts. I have a lot of web services and web traffic experience.


As Igor Mentioned in https://armbian.atlassian.net/browse/AR-444 there's some on-going stuff.    Log shipping would be an ansible task.  So could use a role, or favorite galaxy role and collaborate on options for strategy there.    I think I just want to go old school and just do rsyslog shipping and have an intelligent file layout.  But certainly open to logstash or fluentd solutions as well.

Link to comment
Share on other sites

This thread is quite old. Please consider starting a new thread rather than reviving this one.

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Loading...
×
×
  • Create New...

Important Information

Terms of Use - Privacy Policy - Guidelines