Jump to content

debugging fails with illegal instruction


ChrisArena52

Recommended Posts

In the "next" Vanilla version of Armbian for CB1:

 

cubie@cubieboard:~/Apps/test-gdb$ uname -a
Linux cubieboard 4.5.4-sunxi #1 SMP Wed May 11 16:51:05 EDT 2016 armv7l GNU/Linux
cubie@cubieboard:~/Apps/test-gdb$ lsb_release -a
No LSB modules are available.
Distributor ID: Debian
Description: Debian GNU/Linux 8.4 (jessie)
Release: 8.4
Codename: jessie

 

The following option is not set in the kernel configuration:

 

CONFIG_OABI_COMPAT=y

 

When this is commented out, debugging fails with an illegal instruction if you do anything other than run, just about.

 

Possibly a coincidence  is that "less /proc/config.gz" stops working when that option is set --- dunno. Don't have time to elaborate the options right now.

 

See https://github.com/raspberrypi/linux/issues/766for a discussion.

 

Unless other things break with that option, I prefer being able to debug than to have the convenience of just typing "less /proc/config.gz".

 

Chris

Link to comment
Share on other sites

Possibly a coincidence  is that "less /proc/config.gz" stops working when that option is set --- dunno. Don't have time to elaborate the options right now.

 

See https://github.com/raspberrypi/linux/issues/766for a discussion.

 

Unless other things break with that option, I prefer being able to debug than to have the convenience of just typing "less /proc/config.gz".

config OABI_COMPAT
	bool "Allow old ABI binaries to run with this kernel (EXPERIMENTAL)"
	depends on AEABI && !THUMB2_KERNEL
	help
	  This option preserves the old syscall interface along with the
	  new (ARM EABI) one. It also provides a compatibility layer to
	  intercept syscalls that have structure arguments which layout
	  in memory differs between the legacy ABI and the new ARM EABI
	  (only for non "thumb" binaries). This option adds a tiny
	  overhead to all syscalls and produces a slightly larger kernel.

	  The seccomp filter system will not be available when this is
	  selected, since there is no way yet to sensibly distinguish
	  between calling conventions during filtering.

	  If you know you'll be using only pure EABI user space then you
	  can say N here. If this option is not selected and you attempt
	  to execute a legacy ABI binary then the result will be
	  UNPREDICTABLE (in fact it can be predicted that it won't work
	  at all). If in doubt say N.

What GDB package are you using?

dpkg -S $(which gdb)

There are 2 possibilities - gdb and gdb-arm-none-eabi. This output is from Debian testing:

➜  ~  % apt-cache show gdb gdb-arm-none-eabi
Package: gdb
Source: gdb (7.10-1)
Version: 7.10-1+b1
Installed-Size: 5010
Maintainer: Héctor Orón Martínez <zumbi@debian.org>
Architecture: armhf
Replaces: gdb
Depends: libbabeltrace-ctf1 (>= 1.2.1), libbabeltrace1 (>= 1.2.1), libc6 (>= 2.15), libexpat1 (>= 2.0.1), liblzma5 (>= 5.1.1alpha+20110809), libncurses5 (>= 6), libpython3.5 (>= 3.5.0~b1), libreadline6 (>= 6.0), libtinfo5 (>= 6), zlib1g (>= 1:1.2.0)
Recommends: libc-dbg, gdbserver
Suggests: gdb-doc
Conflicts: gdb
Description-en: GNU Debugger
 GDB is a source-level debugger, capable of breaking programs at
 any specific line, displaying variable values, and determining
 where errors occurred. Currently, gdb supports C, C++, D,
 Objective-C, Fortran, Java, OpenCL C, Pascal, assembly, Modula-2,
 Go, and Ada. A must-have for any serious programmer.
Description-md5: 4f2b8eb95df2ba7a5b11e0301c48b8e4
Homepage: http://www.gnu.org/s/gdb/
Tag: devel::debugger, devel::lang:c, devel::lang:c++, devel::lang:fortran,
 devel::lang:java, implemented-in::c, interface::text-mode,
 role::program, scope::utility, suite::gnu, uitoolkit::ncurses,
 works-with::software:running
Section: devel
Priority: optional
Filename: pool/main/g/gdb/gdb_7.10-1+b1_armhf.deb
Size: 2268812
MD5sum: 6abf17d834dbd1ef3ef14d4507e35acd
SHA1: 914fb61acc8dc3a2f5d7a2b83b9f74156934ca8d
SHA256: ba508018ef304d8f1714d04c23a96a2d260bf5a3e913d108e06278b9c3f9671d

Package: gdb-arm-none-eabi
Source: gdb-arm-none-eabi (9)
Version: 7.10-1+9
Installed-Size: 4592
Maintainer: Agustin Henze <tin@debian.org>
Architecture: armhf
Depends: libc6 (>= 2.11), libexpat1 (>= 2.0.1), libncurses5 (>= 6), libpython2.7 (>= 2.7), libreadline6 (>= 6.0), libtinfo5 (>= 6), zlib1g (>= 1:1.2.0)
Recommends: gcc-arm-none-eabi
Description-en: GNU debugger for ARM Cortex-A/R/M processors
 Bare metal GNU debugger for embedded ARM chips using Cortex-M0/M0+/M3/M4,
 Cortex-R4/R5/R7 and Cortex-A* processors. GDB is a source-level debugger,
 capable of breaking programs at any specific line, displaying variable values,
 and determining where errors occurred.
Description-md5: c034a489da482847d1aaa1c6157f9f71
Homepage: http://www.gnu.org/s/gdb/
Built-Using: gdb (= 7.10-1)
Tag: uitoolkit::ncurses
Section: devel
Priority: extra
Filename: pool/main/g/gdb-arm-none-eabi/gdb-arm-none-eabi_7.10-1+9_armhf.deb
Size: 1784102
MD5sum: 1eccfacfc0f1e20479cc5311dd32f4ba
SHA1: ef55f18fbbc416f5de515bfb9300931b0af3aec9
SHA256: 1c1bb17c820a2f4ac234f8d263df5d065f369516fd525cfb525d11f361ec0a24
Link to comment
Share on other sites

OK, this is really killing me now. My gdb is new enough:

cubie@cubieboard:~/$ apt-cache show gdb gdb-arm-none-eabi
Package: gdb
Priority: optional
Section: devel
Installed-Size: 5051
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Original-Maintainer: Héctor Orón Martínez <zumbi@debian.org>
Architecture: armhf
Version: 7.11-0ubuntu1
Replaces: gdb, gdb-doc (<< 7.8-1~)
Depends: libbabeltrace-ctf1 (>= 1.2.1), libbabeltrace1 (>= 1.2.1), libc6 (>= 2.15), libexpat1 (>= 2.0.1), liblzma5 (>= 5.1.1alpha+20110809), libncurses5 (>= 6), libpython3.5 (>= 3.5.0~b1), libreadline6 (>= 6.0), libtinfo5 (>= 6), zlib1g (>= 1:1.2.0)
Recommends: libc-dbg, libcc1-0, gdbserver
Suggests: gdb-doc
Conflicts: gdb
Filename: pool/main/g/gdb/gdb_7.11-0ubuntu1_armhf.deb
Size: 2229164
MD5sum: a9d02baef1a64fc61b1689a9fed59b0f
SHA1: a8581e2ed7ebba9f17eec24cee180d6017d54365
SHA256: c3dc84d8115d41b742189d95aff8168e3cacaa25ee877846990fd16cfaafd947
Description-en: GNU Debugger
 GDB is a source-level debugger, capable of breaking programs at
 any specific line, displaying variable values, and determining
 where errors occurred. Currently, gdb supports C, C++, D,
 Objective-C, Fortran, Java, OpenCL C, Pascal, assembly, Modula-2,
 Go, and Ada. A must-have for any serious programmer.
Description-md5: 4f2b8eb95df2ba7a5b11e0301c48b8e4
Multi-Arch: allowed
Homepage: http://www.gnu.org/s/gdb/
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Origin: Ubuntu
Supported: 9m
Task: ubuntu-desktop, ubuntu-usb, kubuntu-desktop, kubuntu-full, edubuntu-desktop, edubuntu-usb, xubuntu-desktop, mythbuntu-frontend, mythbuntu-desktop, mythbuntu-backend-slave, mythbuntu-backend-master, mythbuntu-backend-master, ubuntustudio-desktop, ubuntu-gnome-desktop, ubuntu-touch-core, ubuntu-touch, ubuntu-sdk, ubuntukylin-desktop, ubuntu-mate-desktop

Package: gdb-arm-none-eabi
Priority: extra
Section: universe/devel
Installed-Size: 5326
Maintainer: Ubuntu Developers <ubuntu-devel-discuss@lists.ubuntu.com>
Original-Maintainer: Agustin Henze <tin@debian.org>
Architecture: armhf
Source: gdb-arm-none-eabi (9)
Version: 7.10-1ubuntu3+9
Depends: libc6 (>= 2.15), libexpat1 (>= 2.0.1), libncurses5 (>= 6), libpython2.7 (>= 2.7), libreadline6 (>= 6.0), libtinfo5 (>= 6), zlib1g (>= 1:1.2.0)
Recommends: gcc-arm-none-eabi
Filename: pool/universe/g/gdb-arm-none-eabi/gdb-arm-none-eabi_7.10-1ubuntu3+9_armhf.deb
Size: 2406174
MD5sum: 4b3656ea81660478e0ed5f27fda67f86
SHA1: 738f1f4016c03108fd74d68f1ce4bd398f17c526
SHA256: 07442d48b1c27953087d8639b90316c2fc8c1044fbee8fe5ded4be09557775f4
Description-en: GNU debugger for ARM Cortex-A/R/M processors
 Bare metal GNU debugger for embedded ARM chips using Cortex-M0/M0+/M3/M4,
 Cortex-R4/R5/R7 and Cortex-A* processors. GDB is a source-level debugger,
 capable of breaking programs at any specific line, displaying variable values,
 and determining where errors occurred.
Description-md5: c034a489da482847d1aaa1c6157f9f71
Built-Using: gdb (= 7.10-1ubuntu3)
Homepage: http://www.gnu.org/s/gdb/
Bugs: https://bugs.launchpad.net/ubuntu/+filebug
Origin: Ubuntu


So what's doing this? I'll rebuild with the OABI_COMPAT set in this new image. What's the long term solution that's not deprecated?

 

Thanks,

 

Chris

 

 

Link to comment
Share on other sites

I have:

cubie@cubieboard:/sys/class/gpio/gpio139$ gdb --version
GNU gdb (Ubuntu 7.11-0ubuntu1) 7.11
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "arm-linux-gnueabihf".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word".

This gdb crashes with:

Program received signal SIGSEGV, Segmentation fault.
0xb6fd9822 in ?? () from /lib/ld-linux-armhf.so.3

A "hello world" program crashes exactly the same way -- AFAICS.

 

Doing "apt-get install gdb-arm-none-eabi" does not install a binary executable by that name on my CB1.

 

Thanks

 

Chris

Link to comment
Share on other sites

OK, that file is there. But it is behaving as though the target should be a remote ... "run" gives me a "Don't know how to run. Try help target". But it's not SIGSEGV-ing at least.

 

This is weird ... why does THIS gdb behave differently from any other native gdb? I can't get it the program to start. Is this a proxy program or something?

 

I'm not connected via a serial line. I'm trying to run a native program on the CB1 under the debugger.

 

Thanks

Chris

Link to comment
Share on other sites

Hm. I just installed "normal" gdb on cubietruck (Debian stretch, kernel 4.6.2) and it doesn't crash for me at least in simple scenarios

 

 

➜  ~  % gdb axp20x-calibrate-battery
GNU gdb (Debian 7.10-1.1) 7.10
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "arm-linux-gnueabihf".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from axp20x-calibrate-battery...(no debugging symbols found)...done.
(gdb) run
Starting program: /home/zador/axp20x-calibrate-battery
axp20x battery calibration tool v1.1

Usage: axp20x-calibrate-battery [-h | -p | -e | -l <file> | -s <file>]
Arguments:
-h:             Show this help message and exit
-p:             Print configuration
-e:             Edit configuration
-s <file>:      Save OCV table to <file>
-l <file>:      Load and apply OCV table from <file>
[Inferior 1 (process 28435) exited normally]
(gdb) quit
➜  ~  % gdb free
GNU gdb (Debian 7.10-1.1) 7.10
Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "arm-linux-gnueabihf".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from free...(no debugging symbols found)...done.
(gdb) run
Starting program: /usr/bin/free
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".
              total        used        free      shared  buff/cache   available
Mem:        2061280      181728      146868        2196     1732684     1815424
Swap:             0           0           0
[Inferior 1 (process 28473) exited normally]
(gdb) quit
➜  ~  %

 

 

 

 

Program received signal SIGSEGV, Segmentation fault.
0xb6fd9822 in ?? () from /lib/ld-linux-armhf.so.3

 

Since it crashes in the loader, can you check your system for corrupted packages?

sudo apt-get install debsums
sudo debsums -cs
Link to comment
Share on other sites

The only thing that debsums -cs complained about was the sun4i-a10-cubieboard.dtb I modified to enable SPI.

I'm surprised that the check covered that file.

 

Is there a link option that makes the image "modern" so that the deprecated OABI_COMPAT isn't needed? I'm compiling like:

g++ main.cpp -g -o main

It's more elaborate with multiple .o files when I link, but essentially like that.

Link to comment
Share on other sites

An actual compile line is:

g++ -c -std=c++11 -fno-rtti -D__packed="__attribute__((__packed__))"  -Iinc -I `fltk-config --includedir`  -g -Wall  -g  -MD -MP -MF .dep/CGuiWindow.o.d src/CGuiWindow.cpp -o build/CGuiWindow.o

I do have -std=c++11 set which might be unusual. Nothing else out of the norm. But I didn't have that set for the "hello world" program that failed.

Link to comment
Share on other sites

My link line:

g++  build/CCircularBuffer.o build/CGuiWindow.o build/CMessageHandler.o build/CSpiMaster.o build/CHoldInterrupts.o \
build/CSpiMessage.o build/CSetPoint.o build/guiMain.o build/MsgQueueType.o build/CVentilatorTestMsg.o build/CVentilatorParmMessages.o \
build/CGpioInputBit.o build/CGpioBit.o build/CPeriodic.o build/COnDemand.o build/CTrigger.o build/FlowDataPointType.o build/crcpy.o  \
-L/usr/lib/arm-linux-gnueabihf -lm -lc `fltk-config --ldflags` -lpthread  -o guiMain
Link to comment
Share on other sites

Good day,

 

I'm developing on the Allwinner A20 & A33 CPUs, which contains ARM cores.

I've first noticed the segfault during remote debugging, using gdbserver,

from my host PC to the target. Debugging the the application directly on

the target did not cause this problem. Performing the remote debugging from

a 32-bit host to the 32-bit target made no difference. That seemed to 

eliminate the possibility of a 64/32 bit compatibility issue.

 

My setup:

Host PC:

  Kubuntu 16.04 64-bit

  Toolchain: gcc-linaro-6.1.1-2016.08

           : gdb version (Linaro_GDB-2016.08) 7.11.1.20160801-git

            Obtained from: https://releases.linaro.org/components/toolchain/binaries         

Target:

  Ubuntu 16.04 ARMHF

  : GDB version: (Ubuntu 7.11.1-0ubuntu1~16.04) 7.11.1

 

Illustrating the problem

------------------------------

Program: main.c

  #include <stdio.h>

  int main (void)

  {

    printf("Hallo world!\n");

    return 0;

  }

Compile on host PC:

  $ arm-linux-gnueabihf-gcc -g main.c -o main_32

File signature:

  $ file main_32

    main_32: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), dynamically linked,

          interpreter /lib/ld-linux-armhf.so.3, for GNU/Linux 2.6.32, 

          BuildID[sha1]=cc1a4a1ff17da63f9f8362c84dffe54ee7fff4ff, not stripped

 

Remote target:

The executable (main_32) needs to be copied to the remote target

 

$ gdbserver :5555 main_32  

  Process main_32 created; pid = 904

  Listening on port 5555

  Remote debugging from host 192.168.7.2

 

Host PC:

$ arm-linux-gnueabihf-gdb ./main_32

 

(gdb) target remote 192.168.7.1:5555 

  Remote debugging using 192.168.7.1:5555

  Reading /lib/ld-linux-armhf.so.3 from remote target...

  warning: File transfers from remote targets can be slow. Use "set sysroot" to access files locally instead.

  Reading /lib/ld-linux-armhf.so.3 from remote target...

  Reading symbols from target:/lib/ld-linux-armhf.so.3...Reading /lib/ld-2.23.so from remote target...

  Reading /lib/.debug/ld-2.23.so from remote target...

  (no debugging symbols found)...done.

  0xb6fd7a40 in ?? () from target:/lib/ld-linux-armhf.so.3

 

(gdb) continue

  Continuing.

  Program received signal SIGSEGV, Segmentation fault.

  0xb6fd9822 in ?? () from target:/lib/ld-linux-armhf.so.3

 

To try and figure out where the segfault occurred I figured that the debug symbols needed to be loaded.

The gdb command 'set debug-file-directory' can be used to accomplish this. It seems that this command 

is only relative to the host PC filesystem and not according to the remote target file system.

With 'set sysroot' the debugger can also be instructed to load all data locally instead of transferring it

runtime over the network.

 

I've copied the full contents of the SD card residing on the Allwinner A20 rootfs to a local directory on

the host PC: /opt/gnueabihf_rootfs

 

Illustrating the solution

-------------------------------

Required packages: apt-get install libc6-dbg

 

Remote target:

$ gdbserver :5555 main_32 

  Process main_32 created; pid = 912

  Listening on port 5555

  Remote debugging from host 192.168.7.2

  

  Hallo world!

  Child exited with status 0

 

Host PC:

$ arm-linux-gnueabihf-gdb ./main_32

  (gdb) set debug-file-directory /opt/gnueabihf_rootfs/usr/lib/debug

  (gdb) set sysroot /opt/gnueabihf_rootfs

  (gdb) target remote 192.168.7.1:5555

 

  Remote debugging using 192.168.7.1:5555

  Reading symbols from /opt/gnueabihf_rootfs/lib/ld-linux-armhf.so.3...done.

  Reading symbols from /opt/gnueabihf_rootfs/usr/lib/debug/lib/arm-linux-gnueabihf/ld-2.23.so...done.

  0xb6fd7a40 in _start ()

   

  (gdb) continue

  Continuing.

  Cannot parse expression `.L966 4@r4'.

  warning: Probes-based dynamic linker interface failed.

  Reverting to original interface.

 

Even though there is a warning regarding "Probes-based dynamic linker interface failed", the program

keeps on running correctly. The thread describing the GDB patch about the linker interface can be


 

It appears the work around was introduced around August 2015, but if you do not specify the path to

the debug symbols, the debugger still segfaults.

 

The 'set debug-file-directory' and 'set sysroot' can be added to the .gdbinit file to be applied

automatically when you start a remote debug session.

 

Regards,

Frikkie Thirion

Link to comment
Share on other sites

Guest
This topic is now closed to further replies.
×
×
  • Create New...

Important Information

Terms of Use - Privacy Policy - Guidelines