Sunday, March 4, 2018

Yocto Recipe for Accelerated Video Playback with Qt on Raspberry Pi

Raspberry has a huge community that creates a considerable amount of material in a large number of projects. A relevant support for Yocto cannot therefore be missing for it. Months ago I started to work on using Yocto to create images for the Raspberry Pi. Unfortunately I had to stop for a few months to work on other projects, so the result could not be published immediately: the project is based on Pyro while current Yocto release is Rocko. The concept is however similar.

I created a test image for the Raspberry Pi 3 board, using the best compilation flags I found available, and added a recipe for POT to include hardware acceleration in Qt. The result is good: just invoke qmlscene on a little QML code and you have a hardware accelerated OpenGL scene rendering video with Qt API (QML) in your Yocto image.

You can find the layer here: https://github.com/carlonluca/meta-pot. It is based on the POT project that can be found here: https://github.com/carlonluca/pot.

Images

In the repo I also added a couple of examples of images I created for me. I'll improve those in case I'll find the time to do it.

All you have to do is include the recipe and POT will be installed in the image (satisfy the dependencies if some were missing).

Example

I created some images using this setup (I included many things that are not mandatory at all for POT):

BBLAYERS ?= " \
   ${WORKSPACE}/poky-pyro/meta \

   ${WORKSPACE}/poky-pyro/meta-poky \
   ${WORKSPACE}/poky-pyro/meta-openembedded/meta-oe \
   ${WORKSPACE}/poky-pyro/meta-openembedded/meta-multimedia \
   ${WORKSPACE}/poky-pyro/meta-openembedded/meta-networking \
   ${WORKSPACE}/poky-pyro/meta-openembedded/meta-python \
   ${WORKSPACE}/poky-pyro/meta-raspberrypi-crypto \
   ${WORKSPACE}/poky-pyro/meta-qt5 \
   ${WORKSPACE}/poky-pyro/meta-oracle-java \
   ${WORKSPACE}/poky-pyro/meta-java \
   ${WORKSPACE}/poky-pyro/meta-office \
   ${WORKSPACE}/meta-pot \
   "

in my layer.conf I placed:

LICENSE_FLAGS_WHITELIST = "commercial oracle_java"
DISTRO_FEATURES = "ext2 ext3 ext4 pam gles2 usbhost ${DISTRO_FEATURES_LIBC}"
DISTRO_FEATURES_BACKFILL_CONSIDERED += "pulseaudio"
PACKAGE_CLASSES = "package_deb"
IMAGE_FEATURES += "package-management ssh-server-openssh"
DISTRO_FEATURES_remove = "X11 wayland"
EXTRA_IMAGE_FEATURES = "debug-tweaks dev-pkgs tools-sdk tools-debug"
PREFERRED_PROVIDER_jpeg = "libjpeg-turbo"
PREFERRED_PROVIDER_jpeg-native = "libjpeg-turbo-native"
PREFERRED_PROVIDER_udev = "eudev"
PREFERRED_PROVIDER_virtual/java-initial-native = "cacao-initial-native"
PREFERRED_PROVIDER_virtual/java-native = "cacao-native"
PREFERRED_PROVIDER_virtual/javac-native = "ecj-bootstrap-native"
VIRTUAL_RUNTIME_init_manager = "sysvinit"
MACHINE_FEATURES_remove = "apm"
IMAGE_FSTYPES ?= "rpi-sdimg"
MACHINE = "raspberrypi3"
DISTRO = "poky"


At this point you can use something like pot-minimal (https://github.com/carlonluca/meta-pot/blob/master/images/pot-minimal.bb) and you should have a working image.

You can test it pretty simply by using a trivial QML like this:

import QtQuick 2.2
import QtMultimedia 5.5

Rectangle {
   color: "orange"

   Video {
      autoLoad: true
      autoPlay: true
      source: "file:///home/root/bbb_3m.mov"
      anchors.fill: parent
   }
}


and you should see your video in the QML scene right away with the command:

qmlscene -platform eglfs main.qml

Optimization

I wanted to also create an image fully optimized for the Raspberry Pi 3 board, which is a Cortex-A53 armv8-a architecture which includes crc and new A64, A32, and T32 instructions to Advanced SIMD that accelerate Advanced Encryption Standard (AES) encryption and decryption, and the Secure Hash Algorithm (SHA) functions SHA-1, SHA-224, and SHA-256. Still I wanted a fully 32-bit system as hardware libraries are only supported for this bitness. To do this I had to apply a few patches to the Yocto repos. I uploaded every patch to these forks:

https://github.com/carlonluca/poky.git (branch: pyro_cortexa53)
https://github.com/carlonluca/meta-raspberrypi (branch: pyro_cortexa53)

Bye! ;-)

Tuesday, January 2, 2018

POT 5.7.0 with Qt 5.10.0 built for armv8 with GCC 6.3.0 on Raspbian Stretch

This was a good test for my cross toolchain with gcc 6.3.0 I uploaded some days ago: the new Qt 5.10.0 built with -march=armv8-a -mtune=cortex-a53 -mfpu=crypto-neon-fp-armv8 -mfloat-abi=hard, including POT for video acceleration on Raspbian Stretch. The build includes most Qt modules plus Qt WebKit (the modernized QtWebKit from
Konstantin Tokarev, version 5.212, https://github.com/annulen/webkit) plus QtWebEngine 5.9.1, a bit patched to build here. I didn't test browsers much, you'll have to work on those yourself if you need them.

By using the Qt build and the cross toolchain you can build your own applications.

Download here the build for Qt 5.10.0 with POT 5.7.
Download here the cross toolchain.

Friday, November 24, 2017

Raspbian Stretch and Cross Toolchain for Linux and Mac OS

Raspbian Stretch is out for Raspberry Pi. I needed a cross toolchain with the same exact version used to build the system so I built my own. As I had my script ready, I also tried to build one for Mac OS and it seems to work for the moment. I tested these cross toolchains for my projects and seemed to be able to build everything that I tried: Qt libs, apps, omxplayer, ffmpeg etc...

Download Linux x64 cross toolchain GCC 6.3.0 for Stretch here.
Download Mac OS cross toolchain GCC 6.3.0 for Stretch here.

NOTE: The Mac OS version requires some packages to be installed from macports.
NOTE: The toolchain is built to crossbuild for armv7-a. I could also build for armv8 but not sure if it also works for armv6.

Have fun! Bye! ;-)

Saturday, June 3, 2017

Mounting Shared Directories on Ubuntu Running on VMWare

This post is mostly for myself :-) This was very simple years ago; actually it was automatic. Now, I have no idea why, but shared directories are not automatically mounted anymore. By searching online it is incredibly difficult to find a proper answer: I tried a million answers but none was working properly. The only one that worked is simply the shortest and the one I never remember for some reason (and also difficult to find) :-):

vmhgfs-fuse /mount/dir

this is how I do it, and it works for me. No idea why the other techniques are not.

Sunday, March 19, 2017

Experimenting with Pi3 optimisations - POT 5.6.0-beta1 with Qt 5.8.0 built for armv8 with GCC 4.9.4

At the moment I still don't want to test custom firmwares or 64 bit arch builds, but I started to test a couple of new features: a new compiler from Linaro (the one provided by the foundation keeps giving me headaches), version 4.9.4 instead of 4.8, and optimised compiler flags for the Rapsberry Pi 3, which is an armv8.
In this build, Qt, ffmpeg and POT are all built with 4.9.4 Linaro toolchain and optimised compiler flags for Pi3. This will only work on Pi3.
You won't probably see much difference in GPU intensive apps, but it is a step on the road of optimisation!

Have fun! Bye! ;-)

Download the toolchain here.
Download POT 5.6.0-beta1 for Raspbian Jessie Lite Pi3 here (md5: 0eec41ef02e9369fc7e569030b8ff868).

Saturday, January 28, 2017

Hardware Decoding in Chromium through QtWebEngine on Raspberry Pi

A few months back I decided to try to implement hardware decoding in WebKit. Unfortunately this task is always pretty long and complex for many reasons. I found the time to draft an implementation for WebKit1, which is pretty useless as WebKit1 in Qt is only used outside QML and JS is executed in the main thread. Unfortunately I never found the time to implement this in WebKit2, which runs in QML and is suitable for more fluid UIs. This was the result:
This is how YouTube was running with this implementation:




Now Qt has deprecated QtWebKit and is working heavily on QtWebEngine, which is built on Chromium, so I wanted to try this road. Unfortunately these kind of things always claim a lot of time, and I don't typically have that much, but I was able to start and get something done already.

Writing a complete solution in Chromium to decode and render video takes much time, so I thought of a shortcut: creating a custom VDA (Video Decode Accelerator) that loads the POT library and reusing its entire codebase to implement decode and rendering with little modifications. This proved to be possible and now I get something on the screen.

So, to summarise: a little patch to Chromium is needed to create a VDA that dynamically loads POT library into memory and uses it with a common interface. Data and calls are translated to POT structures and are sent to POT, which then processes the buffers properly. The result of the decode operation is then sent back to the VDA through the same interface and textures are then sent to Chromium for rendering.

Still many problems remain open, there is much to be done yet as you can see from the video, but something is drawn. Have a look at the demo:


Have fun! ;-)

Tuesday, December 27, 2016

Binaries for POT 5.5.0-beta1 on Qt 5.8.0-rc1 and Experimental Accelerated QtWebKit

A new version of Qt is out and some interesting changes are being developed. Interesting work was done on WebEngine. This package contains an experimental build of Qt 5.8.0-rc1 with the POT driver to provide hardware acceleration to video decoding and rendering. POT was updated to build on the new plugin architecture of Qt and includes ffmpeg 3.2.2. For info on the content of the package refer to this article. The youtube player still runs on WebKit 1 and so is still hardly useful.

Please report issues on github. Have fun ;-)

Download POT 5.5.0-beta1 for Raspbian Jessie Lite Pi2 (also tested on Pi3) here (md5: 7ca1b961ab8c70176f6aeb13d0bc4f9a).