w0ndersp00n

Thanks for this Too bad in no cases my C2 wants to boot up I guess I should burn a new image all together. I still don't understand what went wrong during that last update... Guess I'm happy that I still can access the files on the eMMC card.

A few weeks ago, after a firmware-upgrade my C2 bricked. It's blue led didn't flash any more and no serial console was available. Since I had everything setup just right I wanted to recover it using these steps here: https://docs.armbian.com/User-Guide_Recovery/#replacing-boot Off course I replaced the packages with the ones for the C2. Specifically: https://mirrors.dotsrc.org/armbian-apt/pool/main/l/linux-5.9.14-meson64/linux-dtb-current-meson64_20.11.3_arm64.deb https://mirrors.dotsrc.org/armbian-apt/pool/main/l/linux-5.9.14-meson64/linux-image-current-meson64_20.11.3_arm64.deb https://mirrors.dotsrc.org/armbian-apt/pool/main/l/linux-buster-root-current-odroidc2/linux-buster-root-current-odroidc2_20.11.6_arm64.deb https://mirrors.dotsrc.org/armbian-apt/pool/main/a/armbian-firmware/armbian-firmware_20.11.3_all.deb After following all the steps, I don't seem to have an uInitrd. I don't know if that would help booting up again, but from the symlinks section it's the only file missing after following all the steps: Does anybody know what went wrong or guide me towards the right steps to recover my C2?

Hi, No haven't had any progress as an Armbian firmware upgrade broke my C2 :( Currently still working on getting it back up and running without a new install but I don't seem to succeed...

Well, my C2 doesn’t seem to have a console so I guess I need to recover it another way. My N2 thankfully didn’t crash after the update. It had a power outage... I’ll check out the autotests later. Need to deal with some private stuff first, then recover the C2 and then I’ll be able to get tinkering again.

It seems I was too late to notice this topic. My C2 “died” as well, as it no longer boots. I didn’t have a backup script in place yet. I’ve ordered an UART cable, which should arrive this week. Hopefully I understand how this work and be able to resurrect Armbian. I’ll just need to make sure not to reboot my N2, since I updated it as well, but didn’t reboot yet.

That's strange - it only happens to my new Armbian installs. But I've assigned static IP Addresses for now, to see if that fixes this issue

With my OPi Zero, I noticed that at random times the ethernet connection somehow gets disabled. The only way to recover is by power cycling. I use the device as Pi-Hole (so when ethernet gets disconnected, I don't have internet anymore), print server and UniFi controller. I decided to swap it for my Odroid C2. Odd enough the C2 has the same issue. Maybe someone can help me with this? When powercycled, the Odroid with the latest Armbian (installed two days ago) is all up and running fine. It can connect to the internet, I can SSH into it. The ethernet led is flashing orange, so the gigabit connection is available. However, at rondom times the ethernet leds are both disabled. At this point, the C2 is still alive, but there's no way to access it without power cycling. I've tried removing the ethernet cable and connecting it again, hoping Armbian would recognize the connection - what happens in this case is that the green led will flash a few times (100Mbit) but the device is still offline. I've added some information from the log here before rebooting. This information keeps repeating in the log files... Hopefully someone can guide my as how to get the connection stable.

I never heard of Teedy before... It has a beautiful UI though. But considering it's written in Java, I can't call it lightweight. Every Java app I run uses about 6 times the amount of memory compared to Python apps. Rust or C/C++ would be best, considering most ARM boards don't get more than 4GB's of RAM. I might try it out on an old C2, to get a feeling of how it compares to the other ones.

In general, a lot of things are not documented or are documented as comments within the code and config files. So besides the documentation, you need to read the code and config files to figure out how everything works. This is a small package, but probably not worthwile since it's only helpful for storing text-based documents, which isn't the core. True, but it doesn't fix the fact that the database is locked during operations, making it sometimes annoying to work with Paperless. I'm going to find out if it is possible to use MariaDB or PostgreSQL, since they only lock records and not the complete database for writing. I didn't, it's not documented. I've read the docker documentation and found out how I was able to use my own locations for volumes. I'll upload my docker-compose.yml sometime. Yes, the scanned documents are still 300dpi. I actually think it is 600dpi in my case. Yes, some people are trying this, but it isn't very easy since Paperless in it's core wasn't designed for this. You should probably place the script in a folder on your host and then use file mapping to map a file from the host to the container: https://docs.docker.com/compose/compose-file/#volumes

In my case `disable-notify` is needed, because my consume folder is on a NFS-network location. inotify doesn't work on network locations, so new files would never be processed. Because I'm using NFS, rights are not an issue (if you set the UID and GID to the same from the NFS server.) Regarding the date: in the Netherlands we use DMY dates. I noticed that Paperless wouldn't detect any correct date when this was set to YMD. I guess this idea is nice, but I still don't understand how that script would be able to define the correct filename. E.g. my scanner scans to my NAS and it sets a default filename, which always is the serial number of the scanner, appended with a number. So ABC7483393939_000001.pdf. There would be no way for any software to know what this file is. So that's where Paperless and OCR come in, reading and automatically tagging it. I guess the only way to make this work, is to indeed scan to a 'pre-Paperless' location and pre-process it. Paperless however does support POST-OCR scripts, so maybe it would be worthwile to check that out. I assume you can change the Title (Filename) of the consumed file, based on the OCR results: https://paperless.readthedocs.io/en/latest/consumption.html#hooking-into-the-consumption-process https://github.com/the-paperless-project/paperless/blob/master/scripts/post-consumption-example.sh Yes, so in the match field I've added `mediamarkt "media markt" redcoon`, so all scans containing those strings will automatically be assigned to correspondent MediaMarkt.

This is the same with Paperless as it is with Mayan. By setting the restart policy in the docker-compose file or in the docker run command. I use “unless-stopped”.

I haven’t tested exporting documents yet. I did notice that when downloading a file, the filename will become ‘Date of the file_Title of document’. Title of document is the original filename, if you didn’t change it. Documentation regarding exporting all files: https://paperless.readthedocs.io/en/latest/utilities.html#the-exporter I’ve been using Paperless for a few moments now and I have to say it really seems to be what I was looking for. On a technical level I’ve seen that the documents are saved with numeric filename (00000010.pdf), while Mayan uses UIDs. I don’t know which is better, but since Paperless seems to have quite a large community of home users, I assume that any issues regarding this would’ve been noticed by now.

So after playing around with Paperless for a few hours and with a few testruns, I have to admit I'm starting to get more positive. The downside still is that documentation is lacking and that development seems slowed down quite a bit. On the plus side, when you know what to do, this tool can work very well for you. Some less positive points: It doesn't support Office-type of files. So no docx, odt, xlsx, etc. This issue isn't too big, since I'm using it mostly for files I received/scanned. And if I receive an Office-type of document, it isn't very hard converting it to PDF. It does support TXT, CSV and MD files. However, the default Dockerfile doesn't account for this functionality. So you'll need to add 'ghostscript-fonts' to the added packages, or add it to the consumer container manually afterwards if you want to use this. While consuming, it's impossible to add new tags or correspondents due to SQLite being locked. Some more positive points: By additionally mounting paperless.conf via the volumes in both containers, a lot of configuration changes can be made. And easier then sh-ing into the containers. E.g. encryption of saved files can be enabled You can set the date order (default YMD, I set it to DMY) This is very worthwile, because with this, Paperless will read the OCR'ed file and detect the date of the file. I successfully tested this with some invoices. You can set the default language for tesseract. I set it to Dutch, and this really speeds up the process of consuming documents, instead of trying English first, before any other languages. Also I've set the OCR quality to 150dpi instead of 300dpi, which also helps speeding up the OCR process. Since my scanner is high quality, I don't assume this will become an issue. And most important: you can enable "forgiving OCR", which means that a file is consumed, even if OCR failed! In tags and correspondents you can set some Match keywords to search for. If Paperless finds those, the correspondent and or tags will automatically be added to the consumed document. I added an invoice from a retailer, and after consuming, the date was correct, the correspontent was correct and the right tags were added I'm going to scan some files and upload them to my development box to test if everything will still work as expected. If it does, I might replace Mayan wih Paperless. Edit: In the image you can see two POS-tickets I scanned, which were from 2015. Paperless automatically found the correct correspondent, MediaMarkt. In the first run I tried to see if it would automatically find the correct date (July 2), which was correct. On the second run I wanted to know if automatic tagging worked (bon), and that worked. It isn't flawless, especially with these POS-tickets degrading over time. E.g. for some IKEA tickets Paperless was unable to determine the correct date. I guess that this issue will be smaller with recent tickets without the amount of degradation these have.

It’s true that initial setup of Paperless is quite ok. However, as you noticed, you have to use the command line in order to change settings and setup the user. Basically everything to setup happens via the CLI. In my case I also had to disable inotify. This is once again a specific setting which needs to be changed before starting the container, because otherwise it would never detect any of my files in the consume directory. The consumption process on my side is also less reliable than with Mayan. It’s very slow. Where Mayan processes a PDF including OCR with in seconds I need to wait minutes on Paperless When Paperless is unable to execute OCR or detects a language for which you haven’t installed tesserae-ocr, than the consumption fails and your document is never added. There is no way to override it. Because of some reason Paperless thought one of my documents was in Portuguese, so it simply doesn’t process it. It’s impossible to add files other that PDF, JPG and TIFF. Mayan will convert DOCX and ODT to PDF while processing the file. On the other hand: Paperless gives me the ability to quickly search on correspondent, date and text in the file which is the basic functionality I’m looking for. This works out of the box, while you need to configure this with Mayan. So I’m still very much on the fence regarding Paperless... I still prefer=er Mayan, but it’s resource requirements are out of the roof... So neither Mayan or Paperless will (by default) rename files. Mayan will not do that, because it’s Metadata-driven. You should use metadata and I think you can setup rules to add specific metadata when some text has been found in the OCR data. Paperless can’t do that, but when reading the manual, you probably can write a hook function in order to do that after the consumption. Also most DMS systems can export all files, I’d desired, using specific naming conventions. I have seen this application as well indeed. But my biggest gripe is that it tries to be a file system, with conventions such as folders. For my the file names are irrelevant. My requirements are: Categorise on sender (correspondent) Categorise on type of document (tagging) OCR Search on date, correspondent, tag and document text I don’t want another file system. I want to use metadata in order to find my files. So with these requirements Paperless should be a fit, but I’m still not liking the small issues I’ve found until now. Edit: I just found out about https://github.com/the-paperless-project/paperless/blob/1c956652f360e58409c8fca148b7662585dd1087/paperless.conf.example. Maybe this file should be added to volumes in order to change settings somewhat more easy.

I don't know why it failed. But here are the steps I took to get Paperless running next to Mayan on my dev/test Odroid: git clone https://github.com/the-paperless-project/paperless.git cd paperless cp docker-compose.yml.example docker-compose.yml cp docker-compose.env.example docker-compose.env docker-compose up -d I edited docker-compose.env to change variables for Timezone and OCR Languages: TZ=Europe/Amsterdam PAPERLESS_OCR_LANGUAGES=nld deu eng With these steps, Paperless will be build and run on port 8000. This process takes about 15 minutes on my Odroid C2. Maybe you can try again with these steps. One thing I did notice already: the consumption of PDF files takes waaaay longer with Paperless then with Mayan. But in my short test this application actually seems worthwile. I'm going to test it further as well.

No problem! I want it for private use. It’s true wat you say about paperless, that it supports web upload (HTTP POST). The big difference is that you need to create/write/develop the form to upload a document yourself. A lot of functionality lacks out of the box. In Mayan, you simply go to the settings and enable a web form for the upload and it works. So Mayan has a larger footprint, but is also more complete in my opinion. Actually, there is about nothing you can do regarding settings from the Web UI with Paperless, in contrary to Mayan. Maybe I haven’t looked at it long enough, but if such features would be user friendly to enable and work with, then I’d probably opt for paperless. Still some functions that make me feel that Mayan also is a great solution for personal use: Custom metadata (such as who send a document, which category, date, etc) Custom indexes: it’s possible to navigate documents using the custom metadata (e.g. I want to look through my invoices which I received in January of 2020) Besides these configuration steps everything works out of the box Have you installed Paperless and compared it to Mayan? I’m very curious if you may find it different.

The changes made are in no way valuable for upstream. The original script simply pulls the AMD64 image from Docker Hub, which is way quicker and easier than building the image during setup. Probably creating a second script to only update the Mayan image and container would be a simpler solution. However, recreating the containers for Postgres and Redis barely takes up measurable time on my C2. So it's not worth the effort. I assume higher end boards, such as N2 and Rock Pi 4 wouldn't even notice this part of the script. The more interesting question would be: how to know if a new version was released and trigger a new build? Best practice obviously is not to do that on production systems, but otherwise you'd have to monitor the Mayan releases yourself and then decide when to trigger the new build. Unless you schedule the build. I've looked at Paperless myself as well. About a year ago the developer announced he wouldn't be very active anymore. I just noticed that the community took over, so I quickly did a test and these are my findings: Paperless, just like Mayan, only offers an AMD64 image, so ARM images should be build by yourself On my C2, Building Paperless takes about 10 - 15 minutes, compared to the 1 hour from Mayan Memory consumption out of the box is about 50 MiB, versus Mayan's 350 MiB on my C2 Building the image is way more fiddly, and scripting it would require too much effort You need to clone the Git repo You need to edit some config files And from that moment you can start building My conclusion would be that anyone who would like to use Paperless should build the image with the instructions from Paperless manual. After you've build the image, you probably can build new versions again after refreshing from Git. After playing arount with Paperless, I found it less suitable for me. I also am looking for a DMS, and Mayan offers way more flexibility and more features regarding sorting files, searching, metadata etc. Paperless doesn't even allow for a web upload of a file, which is something that I need.

This script doesn't update the system, it only installs docker when the variable is set. But great to know the build also worked for the Rock Pi! Maybe... I could add an additional variable where you can enter the tesseract-ocr languages. But if this script would eventually be called from Softy, the question would never be asked, so a default install is performed. The Mayan containers can be updated independently, but this script was made for easy set-up. So it'll remove all the containers and create them again on every build. The used images for Postgres and Redis stays the same, the container just gets a new id. I think the yellow line is the cache and is available for other applications when they need it. Mine shows almost the same:

Yes, I guess updating would be quicker. But I personally wouldn't want to try; if a new version in a container would fail, it's easier to roll back. I also changed the script a little bit. I figured that the version number is stored in the Git repo, so I'm using that in the script. So the current script is able to always build the newest Mayan version and start a container with that version.

No, installing on bare metal wouldn't lower memory usage. Maybe 1 or 2MiB from the Docker overhead, but isn't worth the trouble of installing Mayan. I'd still recommend a container (Docker or podman). Mayan has high system requirements: https://docs.mayan-edms.com/parts/installation.html#minimum-hardware-requirements Considering this it simply doesn't make sense to install it on lower end ARM devices. It should have at least 1GiB of RAM, but that is the bare minimum. 2GiB is recommended, so a Raspberry Pi 3 would be the bare minimum for running Mayan EDMS. But a Pi 4 or Odroid C2 with 2GB would be a better choice.

Considering the complexity and all the dependencies I doubt it. The buildfile for Mayan EDMS uses Debian Slim and removes a lot of packages during the build in order to keep the resulting image as small as possible. It theoretically would be possible to to switch to Alpine, but considering all dependencies for Mayan I don't think it's worth the trouble. For every new version of Mayan a new image should be build. So that would mean an hour of time from a C2 or RPi3. For anything lower end I wouldn't even consider Mayan EDMS, it's way to heavy. Right now I'm trying to figure out if I can extend the script in order to automatically get the latest version number from here: https://docs.mayan-edms.com/releases/index.html or here: https://pypi.org/project/mayan-edms/ If I'm able to do that, then this script will be able to automatically build the latest version and use it to create the container. I've never installed Mayan using Softy, so I don't know if the user can change the Environmental Variables before the script actually starts.

So with a few modifications in the script from Mayan I was able to build the image on my C2. Building on the C2 with Class 10 SD Card takes about 50 minutes, so I assume that a Raspberry Pi 3 might need more than 1 hour. However there are some issues which I haven't figured out, since I usually don't write or edit shell scripts. The default Installation script pulls mayanedms:latest, however during the build a versioned image is created. This means that the variable DOCKER_MAYAN_IMAGE should be updated for every new version. I don't know if there is some automatic way to do this or if the script needs updating after every release. Right now it's not possible to add additional tesseract-ocr languages. They should be added when creating the container, but since that happens in the script the only way to do that is to remove the container after the script finishes and create it again with the right environmental variable. Another small gripe is the memory usage. On my C2 it uses about 900MiB of memory and on my N2 about 1300MiB, so that's quite a lot. Besides those bullets it pretty much works ok on my Odroid C2 and N2. https://gist.github.com/w0ndersp00n/86cd913616d64eb90674aedf4b6f6d0d

Building on the N2 takes about 5 to 10 minutes, so it's not that bad. The resulting image is about 1.2 GB in size. I'm not really someone who can write great scripts, but I'll get out my old C2 and try to make a script that builds the image and then starts Docker with it.

Actually, the current Dockerfile is working for aarch64. I've tested it on my N2. The issue is that the GitLab CI right now seems to lack the option to build ARM images, that's why they're not supplied. You have to build on the platform for which the image should be. On my N2 I followed this guide: https://docs.mayan-edms.com/chapters/docker/building.html I only changed the tag in the Makefile, because I didn't know if it would conflict. After building locally, you can start the container. As of now I haven't noticed any issues, but haven't been using it longer than a few hours. Basically, the script in Armbian Config to install Mayan EDMS will never work, because it tries to use an AMD64/x64 image. It could be possible to change the script, so that the SBC will build the image itself. Another option is, if anyone has experience with it, to automatically build the aarch64 image and publish it on the Docker Hub.

The default Mayan EDMS docker image is only built for AMD-64. It doesn't work on ARM, so this will never work. Someone might write a Dockerfile for armhf and or aarch64, which we can use for this, or Mayan should be installed on bare metal. See: https://gitlab.com/mayan-edms/mayan-edms/issues/422 https://hub.docker.com/r/mayanedms/mayanedms/tags