I followed @Pita Bread's steps but there were some things I had to do differently so I am noting it down for others' reference.
I had to plug the SD card and USB stick into a USB hub and plug the hub into the box. I also had to use an SD card smaller than 32GB. With these I was able to boot into TWRP using the hold reset button during power on method. The H96 splash screen shows first then TWRP comes up shortly after.
Proceeded in TWRP with specified steps but my system partition showed as 0MB for some reason. I was only able to back up my boot and recovery. There were some mounting errors for /system and even though it said [install complete] my tv box firmware did not downgrade. Ultimately I ended up formatting the system partition as exFAT and EXT4 using Wipe > Advanced wipe > change file system > Format (off the top of my memory). Note this removes the entire operating system.
After wiping it i shut down the system from the main menu.
i used burncard maker on my micro SD card with the desired version of firmware (2019 firmware for this box in my case). Then I inserted the card with the image directly into the SD card port of the tv box.
The box was then powered on WITHOUT holding the reset button. It showed "updating android" and successfully downgraded the firmware. Hooray!
Part 2: Installing armbian
I installed an armbian image from this repository https://github.com/ophub/amlogic-s9xxx-armbian which is being developed specifically for TV boxes.
I used balenaEtcher to burn the image onto the micro SD card, then inserted the card into my USB hub and plugged the hub into the powered off tv box. While holding the reset button inside the AV port, I inserted the power cable to boot the box and it booted into armbian. I forget if I needed to hold that reset button or not though. After that i followed the instructions from the repository and on screen to copy armbian into the eMMC to permanently install armbian onto it (ie. Use the armbian-install command in the terminal)
Currently wifi and ethernet seem to work. I am continuing to test the box.
