Igor Posted September 26, 2018 Posted September 26, 2018 8 hours ago, chwe said: so, enough off topic here... We were off topic all the time so I did change the name of it. 10 hours ago, freak said: But to be fair... all 85 of my boards were working fine till I ran apt upgrade. Regressions happen and will happen in the future. https://www.google.com/search?q=how+many+people+work+on+the+kernel "There are over 13,500 developers working on the Linux kernel, most of them will know the general workings of the kernel. No one knows the exact details of what every line of code does in the monster kernel. Even Linus Torvalds himself does not know every detail of the kernel, it's not the pet project he started in 1991 anymore" There are tens of thousands of Tinkerboard users and since it only bricked a few of your boards (a problem with some of your monitors, cablings, DVI adaptors), the issue is not critical. That changes a view on the problem for 360. If you want us or me to solve your problem, its possible, but our resources are so tiny that waiting time is currently measured in years. If donations bump for 100x, I will hire a few professionals to help out. Until we have to earn money elsewhere to keep this project running ... this remains as is. There are many kernel hackers out there who are fixing problems for money. You can also just switch faulty hardware. It's your call.
freak Posted September 26, 2018 Author Posted September 26, 2018 7 hours ago, Igor said: There are many kernel hackers out there who are fixing problems for money. You can also just switch faulty hardware. It's your call. I'll ship you a brand new http://a.co/d/2YbJdo1 and if it turns out to not be a software problem you can keep it. Deal?
Igor Posted September 26, 2018 Posted September 26, 2018 7 hours ago, Igor said: out there 11 minutes ago, freak said: I'll ship you a brand new http://a.co/d/2YbJdo1 For what we already do for you? Deal.
Igor Posted September 26, 2018 Posted September 26, 2018 12 minutes ago, freak said: PM me a shipping address. I personally don't need it and have no time - it will be donated directly to one of the project volunteers (living in US or Canada). Will send you info ASAP. Thanks.
freak Posted September 26, 2018 Author Posted September 26, 2018 Another update... Fresh install of Bionic... Won't boot with monitor plugged in. Remove power, Unplug hdmi, replace power, allow time to boot, plug hdmi, system is up and running. (yes I'm still working on getting you logs Igor but I'll be out of town for the next few days)
TonyMac32 Posted September 26, 2018 Posted September 26, 2018 So the update you performed, I assume is still to a 4.14 kernel?
freak Posted September 26, 2018 Author Posted September 26, 2018 By update I meant and update to the thread not to the system. That was using a fresh install of bionic from the download page.
TonyMac32 Posted September 26, 2018 Posted September 26, 2018 By update I meant whatever change you have done. I am generally uninterested in this thread other than a potential bug that can keep the board from booting. My query is because it is possible (although unlikely, since 4.14 has some crust developing on it) that @Myy may have an idea what's going on without putting too much time into it.
freak Posted September 26, 2018 Author Posted September 26, 2018 update again... I connected two tinkers... Installed picocom. Using fresh install of armbian 5.59 4.14.67 I can watch the tinker in question boot just fine over the tty... IF the hdmi cable is not plugged in. With HDMI plugged in it will not boot at all. The boot is not visible through the tty because it's not booting so I cannot get logs. Next... Using fresh install of armbian 5.41 4.14.23 on exact same tinker and monitor and cable and power supply, the system boots fine. HOWEVER, if I take this fresh install of 5.41 4.14.23 and run apt update/upgrade the system no longer boots unless you remove the HDMI cable (just like above using 5.59 4.14.67). And as it doesn't even attempt to boot there are no logs to be had. Not sure what to try next.
chwe Posted September 26, 2018 Posted September 26, 2018 1 minute ago, freak said: The startup is not visible through the tty so I cannot get logs. sure? where did you connect it? UART2 or UART3? for new images it should definitively be UART3 and from the 'console video' I made (https://asciinema.org/a/IJhwnKjfp0fyOmeZyKKccHKW5) you should get some sort a bootlog. Obviously the console must be started before you power the second board.. otherwise you'll be to late.. It's possible that we hat UART2 during 5.41 (can't remember anymore) so it might be not as easy to monitor this update if it switches the console during update.. (well you could monitor both uarts in two consoles )
freak Posted September 26, 2018 Author Posted September 26, 2018 Like I said the startup is visible if the hdmi cable is disconnected.
freak Posted September 26, 2018 Author Posted September 26, 2018 Here's a cool update... I took an extra hdmi cable I had lying around and cut what I believe to be the CEC line and the system boots fine.
Myy Posted September 26, 2018 Posted September 26, 2018 Logging through the serial console is configured through systemd and boot parameters... But, yeah, one of the default behaviour is to output the kernel messages through the serial console ONLY IF no screen is available. So I guess that HDMI CEC support is broken ? Does the freeze happen with other screens too ? The default UART for debugging purposes on RK3288 is the UART2 (/dev/ttyS1). It might be changed though. The RK3288 Datasheet name UART2 "uart2dbg". UART3 is for plugging GPS systems (which is not used very often, I concur). Just for information : 1. When the HDMI cable is plugged, do you have any SSH access ? If you wait for 5 minutes, is it still frozen ? 2. If you start without an HDMI cable, get a serial console and *then* plug the HDMI screen, what happens ? Do you have any crash or freeze message ? I remember some bug a long time ago, about 4.14 kernels... That would make the system unbootable if you plugged the HDMI cable before booting : https://github.com/Miouyouyou/RockMyy-Build/issues/1 I don't remember how I was able to get a log for this though...
TonyMac32 Posted September 26, 2018 Posted September 26, 2018 41 minutes ago, Myy said: The default UART for debugging purposes on RK3288 is the UART2 (/dev/ttyS1). It might be changed though. The RK3288 Datasheet name UART2 "uart2dbg". UART3 is for plugging GPS systems (which is not used very often, I concur). Yes, but the Tinker moves it from 2 to 3 to keep common with what ASUS is doing. I think the UART 2 might be blocking an IO range that periogerals could use, but pure speculation. 1
freak Posted September 26, 2018 Author Posted September 26, 2018 2 hours ago, Myy said: Logging through the serial console is configured through systemd and boot parameters... But, yeah, one of the default behaviour is to output the kernel messages through the serial console ONLY IF no screen is available. So I guess that HDMI CEC support is broken ? Does the freeze happen with other screens too ? The default UART for debugging purposes on RK3288 is the UART2 (/dev/ttyS1). It might be changed though. The RK3288 Datasheet name UART2 "uart2dbg". UART3 is for plugging GPS systems (which is not used very often, I concur). Just for information : 1. When the HDMI cable is plugged, do you have any SSH access ? If you wait for 5 minutes, is it still frozen ? 2. If you start without an HDMI cable, get a serial console and *then* plug the HDMI screen, what happens ? Do you have any crash or freeze message ? I remember some bug a long time ago, about 4.14 kernels... That would make the system unbootable if you plugged the HDMI cable before booting : https://github.com/Miouyouyou/RockMyy-Build/issues/1 I don't remember how I was able to get a log for this though... 1. No ssh access. Left it sit for an hour or more. 2. My guess is it works but I just messed up my test environment. I'll have to redo everything next week. Today's my last day till Monday.
freak Posted September 26, 2018 Author Posted September 26, 2018 ...in the meantime I'm going to walk around with cutters and clip the red wire on all my terminals. Lol!
Myy Posted September 26, 2018 Posted September 26, 2018 Just let a few ones intact, just in case (´・ω・` )
freak Posted September 26, 2018 Author Posted September 26, 2018 Does anyone know why kernel options don't save in armbian-config? I've set hdmi_ignore_cec=1 but changes always disappear upon reboot. Not that I know that would fix it but there's a number of hdmi options I thought I would try.
freak Posted September 27, 2018 Author Posted September 27, 2018 Who do I report this problem to? Looks like a bug to me. Who creates the kernel that armbian uses?
TonyMac32 Posted September 27, 2018 Posted September 27, 2018 Well, it's a clean 4.14 from kernel.org with patches that can be found in our build system. If CEC is killing it, then that driver could be blacklisted. Out of curiosity, can you give me the output of armbianmonitor -u on a board you start without the monitor plugged in, and of one with a cut wire? Sent from my Pixel using Tapatalk
freak Posted September 27, 2018 Author Posted September 27, 2018 I can do that but I won't be back at the office till Monday.
freak Posted October 1, 2018 Author Posted October 1, 2018 uname -a Linux tinkerboard 4.14.23-rockchip #55 SMP PREEMPT Thu Mar 1 10:56:06 CET 2018 armv7l armv7l armv7l GNU/Linux armbian 5.41 4.14.23 as downloaded. No upates - plugged in to monitor - CEC wire connected... http://ix.io/1o15 armbian 5.41 4.14.23 as downloaded. No upates - NOT plugged in to monitor - CEC wire connected... http://ix.io/1o17 armbian 5.41 4.14.23 as downloaded. No upates - plugged in to monitor - CEC wire DISconnected... http://ix.io/1o19 armbian 5.41 4.14.23 as downloaded. apt upgrade finished - plugged in to monitor - CEC wire connected... (no boot cannot provide logs) uname -a Linux tinkerboard 4.14.70-rockchip #270 SMP PREEMPT Wed Sep 19 11:40:40 CEST 2018 armv7l armv7l armv7l GNU/Linux armbian 5.41 4.14.23 as downloaded (4.14.70 now). apt upgrade finished - plugged in to monitor - CEC wire DISconnected... http://ix.io/1o1j armbian 5.41 4.14.23 as downloaded (4.14.70 now). apt upgrade finished - monitor plugged in AFTER boot - CEC wire connected http://ix.io/1o1l armbian 5.41 4.14.23 as downloaded (4.14.70 now). apt upgrade finished - monitor plugged in AFTER boot - CEC wire DISconnected http://ix.io/1o1m (testing finished) I should note that with the CEC wire disconnected only two resolutions are available - 800x600 and 1024x768. With CEC connected and monitor plugged after boot I can get up to 1920x1080 (even though the monitor supports up to 1920x1200.
freak Posted October 1, 2018 Author Posted October 1, 2018 Ok I switched to ttyS2 instead of ttyS3 and I get this at boot with CEC connected... (via picocom) U-Boot 2018.07-armbian (Sep 19 2018 - 13:23:29 +0200) Model: Tinker-RK3288 DRAM: 2 GiB MMC: dwmmc@ff0c0000: 1, dwmmc@ff0f0000: 0 Loading Environment from EXT4... Card did not respond to voltage select! ** Bad device mmc 0 ** Failed (-5) ...and it just stops there.
freak Posted October 1, 2018 Author Posted October 1, 2018 (edited) Here's the full boot with CEC disconnected... Spoiler U-Boot 2018.07-armbian (Sep 19 2018 - 13:23:29 +0200) Model: Tinker-RK3288 DRAM: 2 GiB MMC: dwmmc@ff0c0000: 1, dwmmc@ff0f0000: 0 Loading Environment from EXT4... Card did not respond to voltage select! ** Bad device mmc 0 ** Failed (-5) In: serial Out: serial Err: serial Model: Tinker-RK3288 Net: eth0: ethernet@ff290000 Hit any key to stop autoboot: 0 switch to partitions #0, OK mmc1 is current device Scanning mmc 1:1... Found U-Boot script /boot/boot.scr 1485 bytes read in 1 ms (1.4 MiB/s) ## Executing script at 00000000 U-boot loaded from eMMC 155 bytes read in 3 ms (49.8 KiB/s) 43206 bytes read in 6 ms (6.9 MiB/s) 4600547 bytes read in 203 ms (21.6 MiB/s) 8546016 bytes read in 378 ms (21.6 MiB/s) ## Loading init Ramdisk from Legacy Image at 21000000 ... Image Name: uInitrd Image Type: ARM Linux RAMDisk Image (gzip compressed) Data Size: 4600483 Bytes = 4.4 MiB Load Address: 00000000 Entry Point: 00000000 Verifying Checksum ... OK ## Flattened Device Tree blob at 01f00000 Booting using the fdt blob at 0x1f00000 Loading Ramdisk to 0fb9c000, end 0ffff2a3 ... OK Loading Device Tree to 0fb8e000, end 0fb9b8c5 ... OK Starting kernel ... Loading, please wait... starting version 229 Begin: Loading essential drivers ... done. Begin: Running /scripts/init-premount ... done. Begin: Mounting root file system ... Begin: Running /scripts/local-top ... done. Begin: Running /scripts/local-premount ... Scanning for Btrfs filesystems done. Begin: Will now check root file system ... fsck from util-linux 2.27.1 [/sbin/fsck.ext4 (1) -- /dev/mmcblk0p1] fsck.ext4 -a -C0 /dev/mmcblk0p1 /dev/mmcblk0p1: clean, 114086/1869120 files, 689803/7712800 blocks done. done. Begin: Running /scripts/local-bottom ... done. Begin: Running /scripts/init-bottom ... done. Welcome to Ubuntu 16.04.5 LTS! [ OK ] Reached target Remote File Systems (Pre). [ OK ] Reached target User and Group Name Lookups. [ OK ] Started Trigger resolvconf update for networkd DNS. [ OK ] Listening on Journal Socket (/dev/log). [ OK ] Created slice System Slice. [ OK ] Created slice system-serial\x2dgetty.slice. [ OK ] Listening on Journal Socket. Starting Nameserver information manager... Starting Create list of required st... nodes for the current kernel... Starting Restore / save the current clock... Starting Load Kernel Modules... Mounting POSIX Message Queue File System... [ OK ] Reached target Remote File Systems. [ OK ] Reached target Encrypted Volumes. Mounting Debug File System... [ OK ] Created slice User and Session Slice. [ OK ] Started Dispatch Password Requests to Console Directory Watch. Starting Set console keymap... [ OK ] Started Forward Password Requests to Wall Directory Watch. [ OK ] Listening on fsck to fsckd communication Socket. Starting Remount Root and Kernel File Systems... [ OK ] Reached target Swap. [ OK ] Listening on udev Control Socket. [ OK ] Reached target Slices. [ OK ] Listening on udev Kernel Socket. [ OK ] Listening on /dev/initctl Compatibility Named Pipe. [ OK ] Listening on Syslog Socket. [ OK ] Mounted Debug File System. [ OK ] Mounted POSIX Message Queue File System. [ OK ] Started Create list of required sta...ce nodes for the current kernel. [ OK ] Started Restore / save the current clock. [FAILED] Failed to start Load Kernel Modules. See 'systemctl status systemd-modules-load.service' for details. [ OK ] Started Set console keymap. [ OK ] Started Remount Root and Kernel File Systems. [ OK ] Started Nameserver information manager. [ OK ] Reached target Network (Pre). Starting udev Coldplug all Devices... Starting Load/Save Random Seed... Starting Apply Kernel Variables... Mounting Configuration File System... Starting Create Static Device Nodes in /dev... [ OK ] Mounted Configuration File System. [ OK ] Started Load/Save Random Seed. [ OK ] Started Apply Kernel Variables. [ OK ] Started Create Static Device Nodes in /dev. [ OK ] Reached target Local File Systems (Pre). Mounting /tmp... Starting udev Kernel Device Manager... [ OK ] Mounted /tmp. [ OK ] Reached target Local File Systems. Starting Armbian ZRAM config... Starting Raise network interfaces... Starting Set console font and keymap... [ OK ] Started Entropy daemon using the HAVEGE algorithm. Starting Journal Service... [ OK ] Started udev Coldplug all Devices. [ OK ] Started udev Kernel Device Manager. [ OK ] Started Armbian ZRAM config. Starting Armbian memory supported logging... [ OK ] Started Journal Service. Starting Flush Journal to Persistent Storage... [ OK ] Started Flush Journal to Persistent Storage. Starting Create Volatile Files and Directories... [ OK ] Started Create Volatile Files and Directories. Starting Update UTMP about System Boot/Shutdown... [ OK ] Reached target System Time Synchronized. [ OK ] Started Update UTMP about System Boot/Shutdown. [ OK ] Started Set console font and keymap. [ OK ] Found device /dev/ttyS2. [ OK ] Created slice system-getty.slice. [ OK ] Reached target Sound Card. [ OK ] Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch. Starting Load/Save RF Kill Switch Status... [ OK ] Started Load/Save RF Kill Switch Status. [ OK ] Started ifup for eth0. [ OK ] Found device /sys/subsystem/net/devices/eth0. [ OK ] Started Raise network interfaces. [ OK ] Started Armbian memory supported logging. [ OK ] Reached target System Initialization. [ OK ] Listening on D-Bus System Message Bus Socket. [ OK ] Listening on Avahi mDNS/DNS-SD Stack Activation Socket. [ OK ] Started CUPS Scheduler. [ OK ] Reached target Paths. Starting Armbian hardware monitoring... Starting Armbian hardware optimization... [ OK ] Listening on CUPS Scheduler. [ OK ] Reached target Sockets. [ OK ] Started Daily Cleanup of Temporary Directories. [ OK ] Started Daily apt download activities. [ OK ] Started Daily apt upgrade and clean activities. [ OK ] Reached target Timers. [ OK ] Started Armbian hardware optimization. [ OK ] Started Armbian hardware monitoring. [ OK ] Reached target Basic System. Starting System Logging Service... [ OK ] Started D-Bus System Message Bus. Starting Accounts Service... [ OK ] Started Regular background program processing daemon. Starting LSB: Speech Dispatcher... Starting Login Service... Starting LSB: Load kernel modules needed to enable cpufreq scaling... Starting Realtek H5 bluetooth support... [ OK ] Started CUPS Scheduler. Starting Avahi mDNS/DNS-SD Stack... Starting LSB: No Display Manager... Starting Network Manager... Starting Permit User Sessions... Starting LSB: Start/stop sysstat's sadc... Starting Save/Restore Sound Card State... Starting Initializes zram swaping... [ OK ] Started System Logging Service. [ OK ] Started Realtek H5 bluetooth support. [ OK ] Started Permit User Sessions. [ OK ] Started Save/Restore Sound Card State. [ OK ] Started LSB: Speech Dispatcher. [ OK ] Started LSB: No Display Manager. [ OK ] Started Initializes zram swaping. [ OK ] Started LSB: Load kernel modules needed to enable cpufreq scaling. [ OK ] Started LSB: Start/stop sysstat's sadc. [ OK ] Started Avahi mDNS/DNS-SD Stack. [ OK ] Started Login Service. Starting Authenticate and Authorize Users to Run Privileged Tasks... Starting LSB: set CPUFreq kernel parameters... Starting Set console scheme... [ OK ] Started Set console scheme. [ OK ] Started Network Manager. Starting Network Manager Script Dispatcher Service... Starting Network Manager Wait Online... [ OK ] Reached target Network. Starting OpenVPN service... Starting OpenBSD Secure Shell server... [ OK ] Started Unattended Upgrades Shutdown. [ OK ] Started LSB: set CPUFreq kernel parameters. [ OK ] Started OpenVPN service. [ OK ] Started Authenticate and Authorize Users to Run Privileged Tasks. [ OK ] Started Accounts Service. [ OK ] Started OpenBSD Secure Shell server. [ OK ] Started Network Manager Script Dispatcher Service. Starting Hostname Service... Starting LSB: Set sysfs variables from /etc/sysfs.conf... [ OK ] Started Hostname Service. [ OK ] Started LSB: Set sysfs variables from /etc/sysfs.conf. Starting WPA supplicant... [ OK ] Started WPA supplicant. [ OK ] Created slice User Slice of armbian. Starting User Manager for UID 1000... [ OK ] Started Session c1 of user armbian. Starting Load/Save RF Kill Switch Status... Starting Bluetooth service... [ OK ] Started Load/Save RF Kill Switch Status. [ OK ] Started Bluetooth service. [ OK ] Reached target Bluetooth. [ OK ] Started User Manager for UID 1000. Mounting FUSE Control File System... [ OK ] Mounted FUSE Control File System. Starting Disk Manager... [ OK ] Started Disk Manager. [ OK ] Started Network Manager Wait Online. [ OK ] Reached target Network is Online. Starting LSB: Start NTP daemon... Starting LSB: disk temperature monitoring daemon... Starting /etc/rc.local Compatibility... Starting LSB: Advanced IEEE 802.11 management daemon... [ OK ] Started /etc/rc.local Compatibility. [ OK ] Started LSB: Advanced IEEE 802.11 management daemon. [ OK ] Started LSB: disk temperature monitoring daemon. [ OK ] Started Serial Getty on ttyS2. [ OK ] Started Getty on tty1. [ OK ] Reached target Login Prompts. [ OK ] Started LSB: Start NTP daemon. [ OK ] Reached target Multi-User System. [ OK ] Reached target Graphical Interface. Starting Update UTMP about System Runlevel Changes... [ OK ] Started Update UTMP about System Runlevel Changes. Ubuntu 16.04.5 LTS tinkerboard ttyS2 tinkerboard login: Edited November 10, 2018 by Tido tooo long, added spoiler for comfortable thread reading
freak Posted October 1, 2018 Author Posted October 1, 2018 Ok... a co-worker with better eyes than me says I cut pin 15 on the hdmi cable not 13. If that's right that's DDC? Not sure what the implications are of that. I ordered an hdmi cable tester which should be here tomorrow.
TonyMac32 Posted October 1, 2018 Posted October 1, 2018 Well, U-boot appears to be the issue, at least initially. That's at least something. @Igor, the tty for Tinker OS was moved to 3, I thought I had u-boot and kernel set to that, however in all the mess it could have been mixed. I saw your commit to move back to 2, is that your direction moving forward? Sent from my Pixel using Tapatalk
Recommended Posts