hjc

It's too early to use the mainline kernel as "next". In fact, RK3399's situation is more or less similar to RK3328's 4.17 kernel right now, so a single mainline branch should be used as "dev". Question here, does a dev has all those RK3399 boards at hand to test them with a single kernelsource? I would prefer a 'not boardmaker related' kernelsource, therefore ayufan's or RKs own kernelsource might be the best way to go (hope that the missing bits can be added properly by patching the kernel). But I think it's time to discuss/decide it now, before all those boards get added into the buildsystem. I'd prefer ayufan's kernel, too. Rockchip's kernel development seems to be focusing on their newer low-end SoCs right now, and they usually break things (and maybe fix them in a few weeks after they found older SoCs are broken). Actually I don't understand why they're doing all active development in a branch called release-4.4. What Armbian needs right now is a solid branch optimized specifically for RK3328/3399.

The reason is that, those more expensive RK3399 boards have eMMC and Wi-Fi/BT module, or even heat sink and power supply included. In Pine64 store, these components have an extra cost of $15.95 (16G eMMC) + $15.99 (WiFi/BT) + $3.49 (heat sink) + $8.99 (power supply) = $44.42. If you compare carefully between NanoPC T4 ($129 = 4GB RAM + all above components included) and RockPro64 ($79.99 + $44.42 = $124.41), you will find that RockPro64 is not a lot cheaper. It's just eliminating those component that you may not use at all, or those you may already have one and don't need another (e.g. power supply).

Create a bootable (but not optimized) headless Armbian image for a new Rockchip-based board is not difficult, if you are familiar with both Rockchip's boot process and Armbian build scripts. I've been using Rockchip-based boards since last year, and recently only spent a few hours to create images for my Firefly-RK3399 and NanoPC-T4, both legacy 4.4 and mainline 4.17 kernel. Rockchip provides relatively better BSP and upstream support (compared to AllWinner), sometimes they just need to be put together properly. But board vendors are not always willing to do the job, many of them just use an old u-boot (2014.x) and the "Android way" to load the kernel. (i.e. pack kernel into an .img, flash them to a fixed offset on emmc/sd, load them into memory so that u-boot does not even need fs drivers. At least Firefly and NanoPC T4 have official "Linux" images in this form) Board/SoC vendors maintain u-boot/kernel sources, Armbian create build scripts to pack them up, with patches and optimizations.

Overclock isn't really recommended, and I'm just doing that for fun. Though, 2.2/1.8GHz is the max supported clock speed in Rockchip's driver code, so it's not really "over"clocking. There's a slight performance gain, but it does not mean that you can run workloads that you can't do before the overclock. (In other words, it does not change the board's use case) It just made the numbers a little better, by significantly sacrificing power consumption and cooling. No, I don't think that it could be more efficient than a fan.

With DT modified, I got 5GT/s x4 working. (lspci shows LnkCap/LnkSta is x4, but I suspect that's still only x2, see results below) Now the disk performance is a lot better, though it's still not as fast as that on Intel platform. Tested with scaling governor set to "performance", pinned to core 4-5 using taskset. random random bkwd record stride kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread 102400 4 79329 110933 67335 67356 34010 109116 102400 16 237860 300516 144350 144549 104765 294656 102400 512 576148 566193 739793 743689 742523 577145 102400 1024 566945 565336 905777 914270 912475 575646 102400 16384 551399 563214 1032516 1041588 1042083 553401

On NanoPC T4, it seems to be x4, but only 2.5GT/s, PCIe 1.0 speed. root@hjc-nanopc:/home/hjc# lspci -vvv 00:00.0 PCI bridge: Device 1d87:0100 (prog-if 00 [Normal decode]) Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort+ <TAbort+ <MAbort+ >SERR+ <PERR+ INTx- Latency: 0 Interrupt: pin A routed to IRQ 230 Bus: primary=00, secondary=01, subordinate=01, sec-latency=0 I/O behind bridge: 00000000-00000fff Memory behind bridge: fa000000-fa0fffff Prefetchable memory behind bridge: 00000000-000fffff Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity- SERR- NoISA- VGA- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [80] Power Management version 3 Flags: PMEClk- DSI- D1+ D2- AuxCurrent=0mA PME(D0+,D1+,D2-,D3hot+,D3cold-) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME+ Capabilities: [90] MSI: Enable+ Count=1/1 Maskable+ 64bit+ Address: 00000000fee30040 Data: 0000 Masking: 00000000 Pending: 00000000 Capabilities: [b0] MSI-X: Enable- Count=1 Masked- Vector table: BAR=0 offset=00000000 PBA: BAR=0 offset=00000008 Capabilities: [c0] Express (v2) Root Port (Slot+), MSI 00 DevCap: MaxPayload 256 bytes, PhantFunc 0 ExtTag- RBE+ DevCtl: Report errors: Correctable+ Non-Fatal+ Fatal+ Unsupported+ RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop+ MaxPayload 128 bytes, MaxReadReq 512 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend- LnkCap: Port #0, Speed 2.5GT/s, Width x4, ASPM L1, Exit Latency L0s <256ns, L1 <8us ClockPM- Surprise- LLActRep- BwNot+ ASPMOptComp+ LnkCtl: ASPM L1 Enabled; RCB 64 bytes Disabled- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt+ AutBWInt+ LnkSta: Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt- SltCap: AttnBtn- PwrCtrl- MRL- AttnInd- PwrInd- HotPlug- Surprise- Slot #0, PowerLimit 0.000W; Interlock- NoCompl- SltCtl: Enable: AttnBtn- PwrFlt- MRL- PresDet- CmdCplt- HPIrq- LinkChg- Control: AttnInd Off, PwrInd Off, Power+ Interlock- SltSta: Status: AttnBtn- PowerFlt- MRL+ CmdCplt- PresDet- Interlock- Changed: MRL- PresDet- LinkState- RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible- RootCap: CRSVisible- RootSta: PME ReqID 0000, PMEStatus- PMEPending- DevCap2: Completion Timeout: Range B, TimeoutDis+, LTR+, OBFF Via message ARIFwd+ DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled ARIFwd- LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis- Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete-, EqualizationPhase1- EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest- Capabilities: [100 v2] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn- Capabilities: [274 v1] Transaction Processing Hints Interrupt vector mode supported Device specific mode supported Steering table in TPH capability structure Kernel driver in use: pcieport 01:00.0 Non-Volatile memory controller: Intel Corporation Device f1a5 (rev 03) (prog-if 02 [NVM Express]) Subsystem: Intel Corporation Device 390a Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+ Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Interrupt: pin A routed to IRQ 229 Region 0: Memory at fa000000 (64-bit, non-prefetchable) [size=16K] Capabilities: [40] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-) Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME- Capabilities: [70] Express (v2) Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited ExtTag- AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0.000W DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported- RlxdOrd+ ExtTag- PhantFunc- AuxPwr- NoSnoop- FLReset- MaxPayload 128 bytes, MaxReadReq 512 bytes DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr+ TransPend- LnkCap: Port #0, Speed 8GT/s, Width x4, ASPM L1, Exit Latency L0s <1us, L1 <8us ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+ LnkCtl: ASPM L1 Enabled; RCB 64 bytes Disabled- CommClk- ExtSynch- ClockPM+ AutWidDis- BWInt- AutBWInt- LnkSta: Speed 2.5GT/s, Width x4, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR+, OBFF Via message DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis- Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance De-emphasis: -6dB LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1- EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest- Capabilities: [b0] MSI-X: Enable+ Count=16 Masked- Vector table: BAR=0 offset=00002000 PBA: BAR=0 offset=00002100 Capabilities: [100 v2] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- NonFatalErr+ AERCap: First Error Pointer: 00, GenCap+ CGenEn- ChkCap+ ChkEn- Capabilities: [158 v1] #19 Capabilities: [178 v1] Latency Tolerance Reporting Max snoop latency: 0ns Max no snoop latency: 0ns Capabilities: [180 v1] L1 PM Substates L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+ L1_PM_Substates+ PortCommonModeRestoreTime=10us PortTPowerOnTime=10us L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- T_CommonMode=0us LTR1.2_Threshold=0ns L1SubCtl2: T_PwrOn=10us Kernel driver in use: nvme kernel root@hjc-nanopc:/home/hjc# uname -a Linux hjc-nanopc 4.4.132-rk3399 #8 SMP Wed Jun 13 13:03:44 HKT 2018 aarch64 GNU/Linux Somehow it seems to be capped at around 500MB/s, and has much lower 4k IOPS (Intel 600p 256G) Iozone: Performance Test of File I/O Version $Revision: 3.429 $ Compiled for 64 bit mode. Build: linux Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins Al Slater, Scott Rhine, Mike Wisner, Ken Goss Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, Vangel Bojaxhi, Ben England, Vikentsi Lapa. Run began: Wed Jun 13 05:29:26 2018 Include fsync in write timing O_DIRECT feature enabled Auto Mode File size set to 102400 kB Record Size 4 kB Record Size 16 kB Record Size 512 kB Record Size 1024 kB Record Size 16384 kB Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2 Output is in kBytes/sec Time Resolution = 0.000001 seconds. Processor cache size set to 1024 kBytes. Processor cache line size set to 32 bytes. File stride size set to 17 * record size. random random bkwd record stride kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread 102400 4 46753 69928 44747 44325 23204 58273 102400 16 124122 169052 92119 91352 73088 106009 102400 512 420699 451433 389454 390496 390295 480645 102400 1024 465004 524627 464479 466948 466706 490374 102400 16384 507718 461558 546822 545124 524262 503565 iozone test complete. For comparison, here are results on Intel NUC7i5BNK Iozone: Performance Test of File I/O Version $Revision: 3.429 $ Compiled for 64 bit mode. Build: linux-AMD64 Contributors:William Norcott, Don Capps, Isom Crawford, Kirby Collins Al Slater, Scott Rhine, Mike Wisner, Ken Goss Steve Landherr, Brad Smith, Mark Kelly, Dr. Alain CYR, Randy Dunlap, Mark Montague, Dan Million, Gavin Brebner, Jean-Marc Zucconi, Jeff Blomberg, Benny Halevy, Dave Boone, Erik Habbinga, Kris Strecker, Walter Wong, Joshua Root, Fabrice Bacchella, Zhenghua Xue, Qin Li, Darren Sawyer, Vangel Bojaxhi, Ben England, Vikentsi Lapa. Run began: Wed Jun 13 13:13:09 2018 Include fsync in write timing O_DIRECT feature enabled Auto Mode File size set to 102400 kB Record Size 4 kB Record Size 16 kB Record Size 512 kB Record Size 1024 kB Record Size 16384 kB Command line used: iozone -e -I -a -s 100M -r 4k -r 16k -r 512k -r 1024k -r 16384k -i 0 -i 1 -i 2 Output is in kBytes/sec Time Resolution = 0.000001 seconds. Processor cache size set to 1024 kBytes. Processor cache line size set to 32 bytes. File stride size set to 17 * record size. random random bkwd record stride kB reclen write rewrite read reread read write read rewrite read fwrite frewrite fread freread 102400 4 177910 195270 73608 74533 35979 175159 102400 16 346863 395238 153802 156536 110934 382846 102400 512 585384 597408 854478 873586 867914 590035 102400 1024 588475 188750 1071468 1097910 1081047 594930 102400 16384 490908 592000 1452265 1466850 1466042 584065 iozone test complete.

Update: Tried to boot the 4.17 kernel on NanoPC T4. Works, but same issues as Firefly RK3399 above. It seems that RK3399's upstream support still needs some time to complete, especially for newer boards.

Yes, with some patches, in the same repo. Only succeeded to boot the 4.4 legacy kernel, though. Copy the device tree from FriendlyELEC's kernel, and it boots with Rockchip u-boot and kernel. Their kernel modifications are mostly for their accessories, like several LCD screens, touch screens, etc. It's a rough port, especially for u-boot. Since T4 does not have upstream u-boot support, and I only spent a few minutes creating the dts file from the Linux one. (Still a lot of bugs, like USB is not functional, so you cannot load kernel from a USB device. but that's sufficient for booting linux kernel from emmc/sd) As for 4.17 mainline kernel, it's still far from complete. 1. CPU-bound tasks are not pinned to the Cortex-A72 core automatically 2. HDMI not working 3. USB3/USB-C not working so I did not spend time investigating on that running on T4. It may be a lot better later this year, though.

https://git.kernel.org/pub/scm/linux/kernel/git/mmind/linux-rockchip.git/ Got this kernel booted up successfully (tag v4.18-rockchip-drivers-2). Haven't confirmed what's working and what's not.

Managed to boot rockchip kernel with two patches (this and this), so far it worked fine (headlessly. Mali Android drivers are causing compiler errors). These two patches are just to revert two commits in rockchip kernel (first and second). The first change is causing strange clock issues on Firefly board, and the second one causes a kernel panic. These may be board specific issues. AFAIK RockPro64's kernel source included these changes (Odroid N1 and NanoPC T4 did not). Maybe I should keep my fork and wait for a few more months, until it is decided which specific rk3399 kernel source will be used by other boards.

Recently I've been trying to build Armbian for Firefly RK3399, since I've purchased this board last year, and it has been lacking proper software support for a long time. Here is my fork https://github.com/hjc4869/build/tree/development Basically, It is a copy & paste of what's already done to support Odroid N1, by replacing the Hardkernel's kernel to what Firefly provides (an older 4.4.77 kernel), and replacing the Hardkernel u-boot with rockchip u-boot. I created a new 'firefly' board family for convenience. Also tried to build the image with some other kernel variants (Rockchip 4.4 bsp and mainline 4.17-rc6) without success, still investigating reasons. The image boots and runs flawlessly as a headless OS, but that's far from complete. Remaining work: Kernel 1. Replace mali r13 driver with r14. Since r14 supports fbdev and xf86-video-armsoc while r13 does not. Fbdev performance is terrible, though. 2. (Maybe) Use the latest rockchip 4.4.126 kernel instead of Firefly's old (and buggy) 4.4.77, since this branch seems to be a port-and-forget one, while Rockchip kernel is still been actively developed. Userland 1. Rockchip xserver and libmali This makes the desktop experience much better (glamor, and enables GLES for X11 window programs), but there are some problems related to desktop composition. In Firefly's official Ubuntu release, the desktop composition works just fine, but I haven't figure out how to enable that with my own build. Firefly team never provide any scripts/docs to build a usable rootfs. Even without this, the desktop is definitely usable, including Chromium/Firefox, with mesa llvmpipe properly disabled. 2. Firmware Rockchip's rootfs-build repo contains several firmware required by the board (WiFi/BT/DP/etc). These firmware are currently not packed into the image. 3. Rockchip Gstreamer for hardware video decoding 4. Bluetooth This is troublesome even with Firefly's own Ubuntu image. They put some sort of magic in a `/usr/local/bin/enable_bt` script, and it must be executed right after user logon, or Bluetooth simply won't work. Really terribly documented, never want to mess up with it again. 5. Sound The board uses a rt5640 chip, which needs some tweaks related to ALSA. Most userland tweaks are likely to be the same as other RK3399 boards, given the only difference should be bootloader/kernel related stuff. Will it be possible to merge these into armbian/build as a community supported board in the future when RK3399 board family gets stable? (Of course with cleanup, e.g. use the rk3399 board family instead of a dedicated firefly family)

It's the baud rate of rk33xx SoC. Though you can specify a different baud rate in kernel args, it will not affect the bootrom/spl/u-boot/etc.

Well, I refreshed the page a few more times just now, and the issue is gone. Maybe DNS cache or something else.

Seems to be down again?

I'd rather recommend ROCK64 with some USB3 multi-bay HDD enclosure. Since GbE is the bottleneck anyway, unless you've got something like Ethernet link aggregation, but that's beyond the topic of 'cheap' NAS.

Well, I tried a QCA9994 card and it worked on Armbian without any kernel patch, if the correct firmware is loaded. (Debian firmware-atheros package is out-dated and does not support the card I use)

I saw someone mentioned here that ath10k works after patching the kernel, is that worth a try?

Are there any 802.11ac cards that can be confirmed working? I'm interested in building a wireless router with this board (Dual band, 3x3 or 4x4 is ideal), but based on what is mentioned above, it seems to be impossible now?