User Space IO is Python 3 and Java 8 bindings for user space GPIO, SPI, I2C, PWM and Serial interfaces

[sgjava]

Edit: User Space IO depracated. Use Java Periphery instead https://github.com/sgjava/java-periphery

 

https://github.com/sgjava/userspaceio

 

After a lot of work and testing I have produced the User Space IO project! It provides Python 3 and Java 8 bindings for Linux user space GPIO, SPI, I2C, PWM and Serial interfaces. This allows cross SBC and cross language development using a common API. I took two best of breed C APIs for Linux user space libgpiod and c-periphery and produced CFFI bindings for Python and JNA bindings for Java. Since all the bindings closely match the C API it's easy to move from language to language. The side effect of using the Java library is that it should work with many different JVM based languages. How about creating your programs in Kotlin or Scala for instance?

 

GPIO access uses the new gpiod interface and not the deprecated sysfs interface using libgpiod v1.1 (head from git repo). GPIO, SPI, I2C and serial interfaces expose all the low level functionality. I have added some helper methods for handling repetitive functions like building I2C messages, reading words from two registers, etc. You can of course go as low level as you need to. Please use Github to post any issues, pull requests, etc. I have tested Nano Pi Duo for 32 bit and NanoPi Neo +2 for 64 bit compatibility. I'll test more SBCs as I have time. Also, I have deleted https://github.com/sgjava/libgpiod-extra since User Space IO supersedes it.

 

Based of some of the questions I had in the past please note the following:

• gpiod_ctxless_event_loop_multiple can handle GPIO interrupts
• Miscellaneous GPIO request flags GPIOD_LINE_REQUEST_FLAG_OPEN_DRAIN, GPIOD_LINE_REQUEST_FLAG_OPEN_SOURCE, GPIOD_LINE_REQUEST_FLAG_ACTIVE_LOW
• Non-root access using rc.local.

Demo program using hardware PWM to flash LED:

 

[TonyMac32]

Pinned as this library has come up repeatedly in last weeks, topic was buried entirely.

[gufmar]

Sorry to jump in on this only now (because I only started some weeks ago in trying to bring some gpio-spi stuff to work.

This general SBC library approach looks very interesting to me, and I wonder how experienced GPIO/HW/SW technicians see it? Would it deserve more attention? Or had it some drawbacks because it can't be so low level and direct as device tree, libmraa ecc... ?

 

I saw the request to post issues on the github repo, but I believe this is not an issue but more a request for better understanding.

[sgjava]

The main thing for me is I wanted something more portable than RPi.GPIO, so that I can run  the same code on multiple SBCs without coding to a Pi specific API. Plus I wanted to cover the JVM in addition to Python 3 and C. Now I can pick what language makes sense in the context of the project instead of the API driving that (i.e. RPi.GPIO == Python only). Secondly, by using libgpiod for GPIO I'm insulating myself from all the low level coding and making GPIO portable.

 

To me it depends on the project. If I'm doing something from scratch (which I usually am) then I will gravitate to my library since I can port it to a different language or SBC easily. If something I'm working on has a RPi.GPIO or WiringPi dependency then I'll probably use those to save the effort and not reinvent the wheel. If I use luma.oled for instance then It's going to be a Python project. I can use Userspace IO's Python API for GPIO in addition to driving a OLED display with luma.oled.

[sgjava]

@TonyMac32 I just flash a nanopi duo v1.1 and will go through the install. I know for a fact the Java JDK will need to be updated. I'll keep you updated.

[sgjava]

@TonyMac32 OK the cffi stuff is failing, but that's all packaging (no compiles), so I hope that's easy to fix. The Java/JNA and libgpiod work so far (what's committed). libgpiod master branch requires >= 5.5.0 kernel to build.  The nanopi duo distro uses 5.3.9, so I used v1.4.x branch! Let me see if I can finish up the rest.

sudo gpiodetect
gpiochip0 [1c20800.pinctrl] (224 lines)
gpiochip1 [1f02c00.pinctrl] (32 lines)

 

[TonyMac32]

I see you were busy today, I had no intention of giving you work to do. 

[Tido]

Quote

@sgjava  April 22, 2018

If you want to get ahead of the curve a little I've been testing libgpiod master with User Space IO. This version includes Python bindings, so I'm no longer generating CFFI bindings for this library. I've converted all the example code to use the new bindings.

 

User Space IO is Python 3 and Java 8 bindings for Linux user space GPIO, SPI, I2C, PWM and Serial interfaces. User Space IO is a consistent API for C, Python 3 and most JVM languages. No more hacked up RPi.GPIO and WiringPi for each SBC model.

I am reading here and on Github, before I spend time on try & error.

You wrote:  no mor CFFI (2018),  but the introduction here (first post) and your readme.md still  go for CFFI.

armbianIO has some kind of gpio-matching-layer_of-different-SBCs, does yours have that as well?

 

After refactoring your code, it would be very nice of you to spend ten minutes to your documentation - the most boring stuff  but on the other hand fewer boring questions :-)

 

While trying to read your code (just for training), is it C?   Where is the main.c ?

 

[sgjava]

@Tido Fundamentally, I wrap libgpiod and c-periphery as to provide Java and Python wrappers. Since libgpiod and c-periphery are already C/C++ there no need for a new C/C++ API. I used CFFI to generate Python interface to libgpiod, but there was no need after the author released his own Python API. I still use CFFI for c-periphery and pwm Python wrappers.

 

The documentation is pretty clear. Look at the example code  libgpiod and c-periphery are documented in their own projects, so there no need to duplicate that in my project. For instance, the libgpiod Python LED test https://github.com/sgjava/userspaceio/blob/master/libgpiod/python/src/ledtest.py. I have examples for each subproject.

 

There's really no need for a matching layer. Larry did that to keep the pin numbers consistent. If you really wanted to do something like that it would be a layer above Userspace IO. I rely on kernel userspace, so there's no reliance on things like a hacked up RPi.GPIO, etc.

[sgjava]

@TonyMac32 I needed to bring things up to date any ways. A lot changed in a couple years  5.x kernel, library upgrades, Ubuntu updates, etc.  It's amazing though that the install scripts held up pretty good. Oracle was always a pain in how they lock down downloading the JDK. Once they no longer had ARM releases after JDK 8, I switched to Zulu which is based on OpenJDK.

[sgjava]

OK, I have c-periphery bindings working, but it looks like the project has been refactored, so the demo code broke. Fixing that now.

[sgjava]

@TonyMac32 @Tido Userspace IO has been brought up to date. I still need to do individual component tests, but everything generates/compiles fine now.  c-periphery API changed slightly, but that broke all the wrapping code. c-periphery now has a separate step besides open/close which is new/free. I just baked new/free into my open/close wrapper, so existing code will not break. To summarize you can now:

• Use 5.x kernels
• Use latest Zulu JDK
• Use latest JNA jars (for Java wrappers)
• Updated CFFI for c-periphery

Use github issues for build/runtime related stuff https://github.com/sgjava/userspaceio/issues

[TonyMac32]

6 hours ago, sgjava said:

Userspace IO has been brought up to date

 

Excellent!  I have moved the convo from the ArmbianIO thread to the Userspace IO thread so as to not confuse anyone. 

[sgjava]

GPIO tested (led blink, button presses, callbacks, etc). SPI should be easy to test with a loopback and I'll test hardware PWM with the same LED.

 

I had to remove section from dtb that used the built in button for kernel shutdown, so I didn't have to wire up one  

 

[sgjava]

@TonyMac32 @Tido I'm building a frankenboard to test all the code (I'll need to take a picture soon). Since there were a lot of things brought up to date a regression type test is required (at least in my mind). So far GPIO, PWM and SPI are working fine. I even added logic to turn on/off LED based on HC-SR501 input in Java. The only issue so far was on libgpiod chip close https://github.com/sgjava/userspaceio/issues/5, but this may go away once I can test libgpiod master branch on 5.5 kernel.

[turtius]

any benchmarks for this?

[sgjava]

@fourtyseven I haven't really had time for this, but if you understand the underlying frameworks used it should be easy to determine relative performance. Let's look at GPIO for instance. There are basically three modes (fastest to slowest):

• Board-specific Linux drivers that access GPIO addresses in /dev/mem for fasted performance at the trade-off of being able to run on very specific versions of single-board-computers. In the future, the board-specific Linux drivers may be removed in favor of only supporting libgpiod and sysfs Linux interfaces.
• libgpiod for fast full-featured GPIO access on all Linux distros since version 4.8 of the Linux kernel. 
• Slower and limited-functionality GPIO access via the deprecated Sysfs interface (/sys/class/gpio) when running on older Linux distro versions with a Linux kernel older than version 4.8.

I've never had to go the /dev/mem route and would only do that as a last resort. Remember the idea behind User Space IO is cross SBC/language support.  Now let's talk about the wrappers. Obviously Python wrappers are going to be faster than the JNA wrappers generated for the JDK. Again, the trade off here is I can use a similar API to C and Python in Java and other JDK languages. Others have done benchmarks https://blog.adafruit.com/2018/11/26/sysfs-is-dead-long-live-libgpiod-libgpiod-for-linux-circuitpython "So we’re pretty psyched about libgpiod. From our experiments, its much much faster than sysfs. Its not as fast as mem twiddling, but that’s not too surprising, there’s still kernel messages and error checking done. A Pi 3 got us 400Khz pin output toggles in a loop in C, 100KHz in Python examples, and that’s pretty good for bitbanging."

[turtius]

I tried the /dev/mem route and got around 1.6MhZ (on a H3 based board)  at best which i arrogantly concluded as 'slow'. 

Quote

Board-specific Linux drivers that access GPIO addresses in /dev/mem for fasted performance at the trade-off of being able to run on very specific versions of single-board-computers. In the future, the board-specific Linux drivers may be removed in favor of only supporting libgpiod and sysfs Linux interfaces.

Well that may hold true for a lot of SBC's but the major differences come down to the base address and the control/data registers address. The overflow for the BCM2835(faster gpio) with  H3  is quite similar when it comes down to the interaction with the GPIO. I however cannot say the same for PWM and anything other then that.

 

Quote

 A Pi 3 got us 400Khz pin output toggles in a loop in C, 100KHz in Python examples, and that’s pretty good for bitbanging."

honestly i am quite impressed as i have not seen a implementation without sysfs until now. Great Work! 

 

Quote

libgpiod for fast full-featured GPIO access on all Linux distros since version 4.8 of the Linux kernel. 

i wonder how well will this work with the RT patch. Only if the pin state's gets buffers before outputting to compensate for jitter.

[sgjava]

@fourtyseven slow is relative  Precision timing might be best left to a dedicated micro controller. I know ligpiod has less of a chance of losing an event than sysfs, plus events are timestamped. Check out for more details 

 

[sgjava]

@TonyMac32 @Tido I've gone through all the demo code and everything is working with the updated libraries. non-root access works with everything except PWM which has been a bear to figure out. Also added system LED library which includes triggers. I still need to work on some demos for LED triggers. 

[Tido]

Something like this ?  3-color LED   https://docs.google.com/document/d/1Q1zczUVP7Wb-ghuZkwOoSSn0F2znoARt9nx36JTjESc/edit  or did I misunderstand your comment?

 

[sgjava]

@Tido It's an API for system LEDs. https://fabiobaltieri.com/2011/09/21/linux-led-subsystem

 

An example in User Space IO https://github.com/sgjava/userspaceio/blob/master/c-periphery/python/src/sysled.py

[Tido]

1 hour ago, sgjava said:

It's an API for system LED

your message to me: hello! this stuff is already built in the kernel   or at least the trigger. Now all you need to do is point it to some GPIO to get the LEDs to lit up?

Would be helpful for the crazy ones who put their SBC in tight, closed box

 

[sgjava]

@Tido Just get a list of system LEDs and do what you wish:

cd /sys/class/leds
ls -1F

 

[sgjava]

@fourtyseven I threw together a non-scientific libgpiod read/write test and the results are 71 KHz write and 85 KHz read on a H2+ (Duo). Maybe not good enough to simulate a protocol, but plenty good for lots of other things. For the most part I'll take the slowness over lack of portability. https://periph.io/ claims 80 MHz, but I installed it on the Duo and it only detects sysfs stuff, thus it will be slower than libgpiod. There's really no way I found to stay portable and get register level performance.

[turtius]

Quote

 https://periph.io/ claims 80 MHz, but I installed it on the Duo and it only detects sysfs stuff, thus it will be slower than libgpiod.

Was unable to get 2Mhz stable using /dev/mem. I wonder where they got the 80Mhz number from. If they are talking about the BCM2835 chip then it's a different story.

[sgjava]

@fourtyseven Yeah with interrupts and other OS overhead it seems unbelievable. Plus I'm not switching to golang to attain this speed or pick a particular board. 80 MHz is a corner case that most devs will never need. Even 80 KHz is overkill for most needs except bit banging possibly.

[turtius]

Quote

80 MHz is a corner case that most devs will never need. Even 80 KHz is overkill for most needs except bit banging possibly.

Nothing is overkill for stepper motors 

[sgjava]

@fourtyseven I'd probably use hardware PWM for that, but I hear what you are saying.

[turtius]

I'm even more curious with the I2S bus where you can use it as a pseudo SPI bus in the ESP32 platform and control a 74hc595. Is this possible with these SBC's?