Why are armbian images not in a format that Etcher understands?

[hinkley]

Etcher understands zip, gzip, bzip2, and I don't know what all else.  But it has no idea what a 7z file is. 

 

So I have link in proximity for two of the three binaries I need to get, and the third buried in grey on black text at the bottom of the page where nobody will see it and instead they will feel compelled to go out to find it on their own. Are 7z files really enough smaller than bz2 that it's worth this?

Edited November 3, 2019 by hinkley

[chwe]

https://lmgtfy.com/?q=7zip+image+compression+site%3Aforum.armbian.com&s=g

 

 

search engine might do it as well.. cause it's google.

[guidol]

and also:
https://github.com/balena-io/etcher/issues/711

[Igor]

From my POW main showstopper is proper image authentication. If that can now be solved somehow, we could use gz or xz.

 

BTW. Images names will anyway be changed due to 

 

From Armbian_5.91_Bananapi_Debian_buster_dev_4.19.59.7z -> Armbian_19.12_Bananapi_buster_current_5.3.8.7z

Debian_buster_next.7z -> Buster_legacy

[guidol]

39 minutes ago, Igor said:

From my POW main showstopper is proper image authentication. If that can now be solved somehow, we could use gz or xz.

When you mean with "proper image authentication" to store multiple files (.asc/.txt/.sha) in a .xz-archive - so how about this page?:

How do I compress multiple files into a .xz archive?

https://askubuntu.com/questions/610002/how-do-i-compress-multiple-files-into-a-xz-archive

 

[Igor]

3 minutes ago, guidol said:

How do I compress multiple files into a .xz archive?

Is Etcher capable to write an image and checks if its legit out of that?

[guidol]

1 hour ago, Igor said:

Is Etcher capable to write an image and checks if its legit out of that?

No because this uses tar which isnt supported by etcher

etcher would only support .img.xz - so no multiple files with .xz *sigh*

[hinkley]

14 hours ago, chwe said:

https://lmgtfy.com/?q=7zip+image+compression+site%3Aforum.armbian.com&s=g

 

 

search engine might do it as well.. cause it's google.

I’d already found the github conversations from 2017 and. 2018 that were inconclusive. Your sarcastic reply of a conversation from 2016 isn’t helpful and reinforces my initial impression.

 

Don’t make people install two pieces of software they’ve never used before to try out your stuff. And if you must, put the instructions in *one* place.  It’s hostile to new and prospective users if you don’t. 

[hinkley]

8 hours ago, guidol said:

No because this uses tar which isnt supported by etcher

etcher would only support .img.xz - so no multiple files with .xz *sigh*

So if I understand what you guys are saying, it’s not the compression format that’s the problem, it’s the payload.  There is no way to sign the kernel image that Etcher would accept (and that humans could accept) ?
 

That... really seems like a feature I might like to have in Etcher itself. I’ll keep poking around their issue database and see if they have anything more to say on that.

 

Thanks. 

[hinkley]

8 hours ago, guidol said:

No because this uses tar which isnt supported by etcher

etcher would only support .img.xz - so no multiple files with .xz *sigh*

Im not sure https://github.com/balena-io/etcher/issues/711 says tarballs couldn’t be supported. I’ll double check that they still aren’t. It clearly has a way to locate .img files in the top level directory of some archives. 

[Igor]

2 hours ago, hinkley said:

it’s not the compression format that’s the problem, it’s the payload.

 

Let's say its both ... but perhaps today - I didn't check Etcher progress - we have a better solution than years ago?

Now we have one 7z file where raw image is compressed together with its signature. Other option is to provide .xz and signature in a separate file or properly implement 7z, build own utility (we have no resources for that) ... or 3rd something like this https://github.com/pine64dev/PINE64-Installer (but with up2date version).

[hinkley]

52 minutes ago, Igor said:

 

Let's say its both ... but perhaps today - I didn't check Etcher progress - we have a better solution than years ago?

Now we have one 7z file where raw image is compressed together with its signature. Other option is to provide .xz and signature in a separate file or properly implement 7z, build own utility (we have no resources for that) ... or 3rd something like this https://github.com/pine64dev/PINE64-Installer (but with up2date version).

 

Just ran a test, and it's still both.

 

Quote

foo.tar.gz is not a supported image type.

 

That's just a flat tarball of the contents of the 7z file I extracted.

I'll see what it looks like to get tarball support into Etcher.  It's likely I could put together a PR for that bit. From recent experience, working with tarballs in Node seems easier than zip files, so I'm surprised it isn't in there.  But I also may be missing some architectural detail that is peculiar to the burning process.

 

Also it looks like this guy just resurrected his 7zip implementation: https://github.com/quentinrossetti/node-7z but the release and commit history are pretty sparse. I'd be surprised if it didn't need some more QA. Etcher is probably not the right project to be a guinea pig.

 

That still leaves the issue of "Will people check the code signature if they didn't extract the archive? (or do people even check the signatures?)" I can see an argument for adding that functionality to Etcher too, maybe as an extension of the metadata format. But I don't think I have the sort of free time required to do that properly.

 

[chwe]

3 hours ago, hinkley said:

Don’t make people install two pieces of software they’ve never used before to try out your stuff. And if you must, put the instructions in *one* place.  It’s hostile to new and prospective users if you don’t. 

 

then feel free to try out someone else's stuff. The instructions are in one place: https://docs.armbian.com/User-Guide_Getting-Started/ or in the FAQ section on the downloadpage.

 

3 hours ago, hinkley said:

I’d already found the github conversations from 2017 and. 2018 that were inconclusive. Your sarcastic reply of a conversation from 2016 isn’t helpful and reinforces my initial impression.

then it would probably make sense that you link the conversations you find. This would give a first impression that you actually did your part. Is it a satisfying solution we have right now? Probably not, but around all the stuff which isn't in a perfect shape using 2 programs for image decompression and writing is probably for most people here on the lower end of priorities.

 

7 minutes ago, hinkley said:

But I don't think I have the sort of free time required to do that properly.

and exactly that's probably the reason nobody touches stuff which works "good enough".

 

9 minutes ago, hinkley said:

That still leaves the issue of "Will people check the code signature if they didn't extract the archive? (or do people even check the signatures?)"

probably not, I'm okay when they at least read the getting started and follow the recommendations. Otherwise chances are higher that they end here:

https://forum.armbian.com/forum/36-board-doesnt-start/

 

 

 

[Igor]

12 minutes ago, chwe said:

probably not, I'm okay when they at least read the getting started and follow the recommendations

The other day I got an email (which I have less and less time to deal with) saying why we don't provide SHA numbers for 7z archive. He is afraid to open our archive and my answer was not satisfying him. Since I can't afford to educate people in person (or change anything in this regard) I didn't proceed.

 

Where can I get the file hashes (MD5, SHA1, SHA256, or SHA512) for each download so that I can verify that my image download is pristine and not corrupted in any way?  I've searched quite a bit and couldn't find this important info on your site.

 

Me:

They are inside 7z file.
https://docs.armbian.com/User-Guide_Getting-Started/#how-to-check-download-integrity

 

Thanks. I think it would be a good idea to publish the hash also for the 7z file the same way raspbian does for their compressed file (https://www.raspberrypi.org/downloads/raspbian/).  When I download any file from the web, I calculate the hash before interacting it with. If the hash doesn't match what the vendor has published on their website, I don't even attempt to extract or use it in any way. It's a good a security practice in my opinion.

[martinayotte]

14 hours ago, Igor said:

I don't even attempt to extract or use it in any way.

He is a bit paranoid ...

[chwe]

On 11/4/2019 at 12:21 AM, Igor said:

vendor has published on their website

happily we're not a vendor right. would it be nice to have the hashes published somewhere.. of course it would.. it would also be nice to have a unified bootloader on arm which doesn't suck give me headache every time I look at it. or wifi which just works or device tree which actually describes the devices properly and not copy paste gone or boardmaker cares about mainlining their products etc.

 

If you don't trust our images you can still build them yourself. Then you just have to trust that we didn't hide something in the x lines of code to create an image (have fun to review that ).

 

it would also be nice if someone rewrites nand-sata-install.. My attempts so far just made it worse..

 

15 hours ago, martinayotte said:

On 11/4/2019 at 12:21 AM, Igor said:

I don't even attempt to extract or use it in any way.

He is a bit paranoid ...

 

well 42.zip is still a thing? https://en.wikipedia.org/wiki/Zip_bomb

 

well we had fun to send each other shutdown commands over network in school when I was young (nothing was more insecure than our schools network back then - that's why no teacher trusted it and never had exams on the school computer )..

 

[Werner]

Quote

well we had fun to send each other shutdown commands over network in school when I was young (nothing was more insecure than our schools network back then - that's why no teacher trusted it and never had exams on the school computer )..

Right. Memories coming back from school. Yes, we had fun too.

They would not stop playing music on their computers. Seconds later their system stopped responding. I really had no idea why....

[Igor]

23 minutes ago, chwe said:

If you don't trust our images you can still build them yourself. Then you just have to trust that we didn't hide something in the x lines of code to create an image (have fun to review that ).

 

... and you have to trust that compilers aren't fake  Or that rootfs was not somehow tempered in the build process ... or package in the upstream repository is somehow not security problematic.

But some are later mislead by flashy ads/words promising wonder upgrades and convenience ... and you get what you were afraid to get 
 

27 minutes ago, chwe said:

it would also be nice if someone rewrites nand-sata-install.. My attempts so far just made it worse..

Someday  I wasn't sure that I will manage to RFC package naming 

but its almost done by now.

 

31 minutes ago, chwe said:

well we had fun to send each other shutdown commands over network

 We were not that bad, but it was fun sending pop-up messages around to X terminals. 

[Technicavolous]

On 11/5/2019 at 1:16 AM, Igor said:

We were not that bad

We were ... I had this marvelous trojan that could be sent over chat that would cause a little smiley face to bounce around the screen every time the control button got pressed. After a while there were several faces bouncing around the screen. If the victim hit the right shift all the faces would go in clockwise circles, and counter clockwise with the left ... after a while it got too cluttered so all the faces would just 'drip' off the screen and start over. God I miss terminate and stay resident on command line windows ;]