Finally I got my lcd (some of them) to work with armbian on mainline kernel (5.4) using device tree overlay. It should be working on 4.19 kernel too.
I test mine on orange pi zero. The one I am able to get working is ili9341 and st7735 based lcd.
I'll add more detail later but for now I'd like to get this out.
edit 1 :
Adding ssd1306 and nokia 5110 overlay to the repo
device tree overlay