Skip to content
Snippets Groups Projects
New look: Off

QML: Crashes with "illegal instruction" on armhf / JIT is disabled

    • Closed Imported Issue created by Administrator @root

    Starting kwin_wayland or any QML example (like qmlscene-qt5 with minimer, see https://public.etherpad-mozilla.org/p/faq-hadk#lineNumber=40) on armhf device currently results in crash with:

    JIT is disabled for QML. Property bindings and animations will be very slow. Visit https://wiki.qt.io/V4 to learn about possible solutions for your platform.
Illegal instruction

    I investigated this and found out that qt5-qtdeclarative is built without JIT support due to usage of armv6 toolchain for armhf by Alpine. Would it be possible to build and use armv7 toolchain for it instead?

    To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information

    Child items ...

    • Activity

    • Administrator
      Administrator @root · Imported
      Author Owner

      Created by: PureTryOut

      Are you sure JIT being disabled has anything to do with the crash? Like it says in the next sentence, it should only be slow, not completely break. Illegal instruction seems to be caused by something different.

      Furthermore, I thought JIT being disabled for QML was because of licensing issues, but I may recall that wrong.

      By * postmarketOS Bot on 2017-11-05T13:01:14

    • Administrator
      Administrator @root · Imported
      Author Owner

      Created by: ollieparanoid

      I thought the same thing, regarding that this message reads like it should not cause the crash.

      armv7 might come at some point for Alpine, see #238 (closed). But I don't think this will happen in the next few months, because they are focusing on the 3.7 release now, and the armv7 architecture integration surely needs a big amount of work.

      By * postmarketOS Bot on 2017-11-05T13:52:29

      Edited by Ghost User
    • Administrator
      Administrator @root · Imported
      Author Owner

      Are you sure JIT being disabled has anything to do with the crash? Like it says in the next sentence, it should only be slow, not completely break. Illegal instruction seems to be caused by something different.

      Yes, I used gdb to inspect the crash, and it crashes in intepreter (which wouldn't be used if JIT was enabled). Understanding why intepreter crashes is an option, but I'd prefer to get JIT working, since we don't want to use slow path for already pretty heavy stuff (Qt Quick).

      By NotKit on 2017-11-05T13:57:38

    • Administrator
      Administrator @root · Imported
      Author Owner

      Created by: ollieparanoid

      As I understand the terminology from the V4 wiki page linked in the error message, using the interpreter is the opposite of using JIT.

      There seems to be a flag to disable the JIT (also from that page): export QV4_FORCE_INTERPRETER=1

      I guess when you run it with that, it still crashes the same way?

      It's not that I'm against getting the JIT running. But I doubt that it will solve the problem and AFAIK that would only be possible in Alpine if we compile the whole armv7 arch ourselves (because we can't cherry-pick some packages to be compiled with a armv7 toolchain - and even if we could, that sounds like a horrible hack and would break everything for actual armv6 devices, such as the first ipod).

      So my recommendation would be debugging why it crashes with the interpreter. It might be because of libhybris?

      By * postmarketOS Bot on 2017-11-05T20:54:25

    • Administrator
      Administrator @root · Imported
      Author Owner

      So my recommendation would be debugging why it crashes with the interpreter. It might be because of libhybris?

      This would be my first guess too, but I tested and it crashes without libhybris as well with xcb platform and X server started.

      By NotKit on 2017-11-06T06:28:41

    • Administrator
      Administrator @root · Imported
      Author Owner

      This might be a stupid idea but can't we run ARMv7 binaries in our ARMv6 system?

      By Martijn Braam on 2017-11-06T12:34:03

    • Administrator
      Administrator @root · Imported
      Author Owner

      Created by: ollieparanoid

      I think that would work, but the question is, how we would build them. I am not sure if our gcc-armhf cross-compiler can create armv7 packages (passing -march as CFLAG?)

      If it can't, we'd need some ugly workarounds in pmbootstrap to use the armv7 cross-compiler for some packages.

      And even if we get that right, would we ship the armv7 package of qt5-qtdeclarative for all armhf devices? Because then we'd break real armv6 devices (there are not that many, but it would be annoying for them), such as the first ipod.

      By * postmarketOS Bot on 2017-11-06T21:38:59

      Edited by Ghost User
    • Administrator
      Administrator @root · Imported
      Author Owner

      And even if we get that right, would we ship the armv7 package of qt5-qtdeclarative for all armhf devices?

      We can create qt5-qtdeclarative-armv7 as quick and dirty solution. Is it about cross compiler though, or native gcc installed into buildroot? I suppose Alpine armhf gcc package needs to be rebuilt with armv7 flags.

      By NotKit on 2017-11-07T17:12:18

      Edited by Ghost User
    • Administrator
      Administrator @root · Imported
      Author Owner

      Created by: ollieparanoid

      It is possible to do pmbootstrap aportgen binutils-armv7 gcc-armv7. But then you will be missing the musl libc and gcc's stdlibc++ (not sure about the right file name).

      I think the cleanest fastest approach is running Alpine's scripts/bootstrap.sh with armv7 as architecture (that should work when you clone the whole aports repository into an alpine chroot, as in pmbootstrap chroot). This one will bootstrap you a armv7 architecture, from which you should in theory be able to create armv7 binaries.

      By * postmarketOS Bot on 2017-11-07T22:57:36

      Edited by Ghost User
    • Administrator
      Administrator @root · Imported
      Author Owner

      Created by: ollieparanoid

      Regarding the illegal instruction issue from the first post in this issue, @magmastonealex wrote in https://github.com/postmarketOS/pmbootstrap/issues/629#issuecomment-346744695:

      I've also had QT issues - getting an illegal instruction error from QtQml, even on QT examples. I resolved that by building it from scratch in debug mode, but that brought a host of other issues.

      By * postmarketOS Bot on 2017-11-25T03:12:50

    • Administrator
      Administrator @root · Imported
      Author Owner

      Created by: ollieparanoid

      @PureTryOut wrote:

      in regards to JIT being disabled for QML on armhf. QML can work without it, but would be terribly slow (according to others), and people seem to say it's not worth it. it would need to be compiled to tell it to not use JIT, which I guess didn't happen on Alpine armhf

      ...and @NotKit replied:

      it's compiled to work without JIT, but non-JIT route crashes for some reason

      By * postmarketOS Bot on 2017-12-11T21:18:58

      Edited by Ghost User
    • Administrator
      Administrator @root · Imported
      Author Owner

      Created by: ollieparanoid

      I can reproduce the issue as follows:

      • Install weston, xfce4 or hildon (any desktop that's working for you already), and choose qt5-qtdeclarative,gdb as extra packages
      • Connect via SSH
      • Save the example code from here as hello.qml on the device
      • export DISPLAY=:0 for X11, or set the XDG_RUNTIME_DIR as in the /etc/profile.d/ script
      • qml-qt5 hello.qml (or to run it in gdb: gdb --args qml-qt5 hello.qml, then type run)

      When I run it in X11 with gdb, I get the following:

      Starting program: /usr/bin/qml-qt5 hello.qml
[New LWP 3280]
libEGL warning: DRI2: failed to authenticate
[New LWP 3288]
[New LWP 3289]
[New LWP 3291]
JIT is disabled for QML. Property bindings and animations will be very slow. Visit https://wiki.qt.io/V4 to learn about possible solutions for your platform.
[New LWP 3292]
JIT is disabled for QML. Property bindings and animations will be very slow. Visit https://wiki.qt.io/V4 to learn about possible solutions for your platform.
[LWP 3292 exited]

Thread 5 "QQmlThread" received signal SIGILL, Illegal instruction.
[Switching to LWP 3291]
0x40166b54 in QV4::Moth::InstructionSelection::getQmlContextProperty(QV4::IR::Expr*, QV4::IR::Member::MemberKind, int, bool, QV4::IR::Expr*) () from /usr/lib/libQt5Qml.so.5
(gdb) bt
#0  0x40166b54 in QV4::Moth::InstructionSelection::getQmlContextProperty(QV4::IR::Expr*, QV4::IR::Member::MemberKind, int, bool, QV4::IR::Expr*) () from /usr/lib/libQt5Qml.so.5
#1  0x40139a80 in QV4::IR::IRDecoder::visitMove(QV4::IR::Move*) () from /usr/lib/libQt5Qml.so.5
#2  0x4016a1f0 in QV4::Moth::InstructionSelection::run(int) () from /usr/lib/libQt5Qml.so.5
#3  0x401391c8 in QV4::EvalInstructionSelection::compile(bool) () from /usr/lib/libQt5Qml.so.5
#4  0x4015d420 in ?? () from /usr/lib/libQt5Qml.so.5
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
(gdb)

      By * postmarketOS Bot on 2017-12-16T12:59:03

      Edited by Ghost User
    • Administrator
      Administrator @root · Imported
      Author Owner

      Created by: ollieparanoid

      I know most of you would prefer to have armv7 working, and I would also like that. Nevertheless, I've reported this armv6 issue upstream as I have successfully reproduced it and can assist with debugging this on the device: https://bugreports.qt.io/browse/QTBUG-65246

      By * postmarketOS Bot on 2017-12-16T03:19:35

      Edited by Ghost User
    • Administrator
      Administrator @root · Imported
      Author Owner

      Created by: PureTryOut

      Besides getting armv7 working, I really think this should be fixed. There are several older armv6 only devices we support (or can support), and it would be a shame to just have to dismiss any QML application/environment. I'm glad you reported this upstream, hopefully they can find how to fix it!

      By * postmarketOS Bot on 2017-12-16T09:35:27

    • Administrator
      Administrator @root · Imported
      Author Owner

      I spent a little time debugging this further (I'll post this over on the QT bug as well).

      Following this stackoverflow post, I got a core dump of qt5-qml, and examined it in gdb.

      With the layout asm command, I looked at the assembly instruction causing the error: udf #0, an instruction often used for debugger breakpoints according to this issue. As noted there, udf #0 will cause an illegal instruction & core dump, which is exactly what we're seeing. The compiler is inserting that instruction directly after what looks like an optimized call to getParams. (Some more asm context here).

      By Alex Roth on 2017-12-16T16:31:27

      Edited by Ghost User
    • Administrator
      Administrator @root · Imported
      Author Owner

      @ollieparanoid

      I built a debug version of qtdeclarative (using this patch).

      As before though, this causes the SIGILL to disappear, being replaced with a SIGFPE/Arithmetic Exception. From the backtrace:

      #0  0xb6fab18c in __restore_sigs () from /lib/ld-musl-armhf.so.1
#1  0xb6fab2ec in raise () from /lib/ld-musl-armhf.so.1
#2  0xb5a243d0 in __aeabi_ldiv0 () from /usr/lib/libgcc_s.so.1
#3  0xb1c96408 in QSGSoftwareRenderer::render (this=0x7f637900) at scenegraph/adaptations/software/qsgsoftwarerenderer.cpp:115
#4  0xb1be88d8 in QSGRenderer::renderScene (this=0x7f637900, bindable=...) at scenegraph/coreapi/qsgrenderer.cpp:243
#5  0xb1c962a0 in QSGSoftwareRenderer::renderScene (this=0x7f637900) at scenegraph/adaptations/software/qsgsoftwarerenderer.cpp:95
#6  0xb1c7af84 in QSGSoftwareRenderContext::renderNextFrame (this=0x7f6373e0, renderer=0x7f637900, fbo=0)
    at scenegraph/adaptations/software/qsgsoftwarecontext.cpp:160
#7  0xb1cf1e98 in QQuickWindowPrivate::renderSceneGraph (this=0x7f60f2a0, size=...) at items/qquickwindow.cpp:470
#8  0xb1c97450 in QSGSoftwareRenderLoop::renderWindow (this=0x7f637280, window=0x7f60f230, isNewExpose=true)
    at scenegraph/adaptations/software/qsgsoftwarerenderloop.cpp:164
#9  0xb1c97a18 in QSGSoftwareRenderLoop::exposureChanged (this=0x7f637280, window=0x7f60f230)
    at scenegraph/adaptations/software/qsgsoftwarerenderloop.cpp:213
#10 0xb1cf0d1c in QQuickWindow::exposeEvent (this=0x7f60f230) at items/qquickwindow.cpp:215
#11 0xb61d5eb8 in QWindow::event(QEvent*) () from /usr/lib/libQt5Gui.so.5
#12 0xb1cf7000 in QQuickWindow::event (this=0x7f60f230, e=0xbeffdec8) at items/qquickwindow.cpp:1610
#13 0xb5edcdf4 in QCoreApplication::notify(QObject*, QEvent*) () from /usr/lib/libQt5Core.so.5
#14 0xb5edcf78 in QCoreApplication::notifyInternal2(QObject*, QEvent*) () from /usr/lib/libQt5Core.so.5
#15 0xb61c99f8 in QGuiApplicationPrivate::processExposeEvent(QWindowSystemInterfacePrivate::ExposeEvent*) () from /usr/lib/libQt5Gui.so.5
#16 0xb61cab14 in QGuiApplicationPrivate::processWindowSystemEvent(QWindowSystemInterfacePrivate::WindowSystemEvent*) () from /usr/lib/libQt5Gui.so.5
#17 0xb61a17ec in QWindowSystemInterface::sendWindowSystemEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib/libQt5Gui.so.5
#18 0xb61a1b0c in QWindowSystemInterface::flushWindowSystemEvents(QFlags<QEventLoop::ProcessEventsFlag>) () from /usr/lib/libQt5Gui.so.5
#19 0xb61b6be0 in QPlatformWindow::setVisible(bool) () from /usr/lib/libQt5Gui.so.5
#20 0xb392f080 in ?? () from /usr/lib/qt5/plugins/platforms/libqlinuxfb.so
Backtrace stopped: previous frame identical to this frame (corrupt stack?)

      it does, however, look like this is because of the linuxfb/software renderer, which I hadn't noticed when I did this the first time.

      So, then I ran it with the xcb backend & weston as the compositor: hello calq

      and both the hello world QML app, and the Calqlator demo open. (eventually... it's rather slow, quite possibly because of both my lack of hardware accelleration & the JIT compiler)

      E: FPE/Arithmetic exception/Div0 appears to be caused by the line setBackgroundSize(QSize(m_paintDevice->width() / m_paintDevice->devicePixelRatio() within qtdeclarative. Setting pixel size & physical size as QT_QPA_PLATFORM arguments didn't fix the issue, nor did setting QT_DEVICE_PIXEL_RATIO, QT_SCALE_FACTOR, etc.

      By Alex Roth on 2017-12-17T06:53:55

      Edited by Ghost User
    • Administrator
      Administrator @root · Imported
      Author Owner

      I also was able to build qtdeclarative so that the JIT compiler was working on armhf - I didn't need an armv7 specific compiler or anything, just some compiler flags: -mthumb-interwork -mthumb -march=armv7.

      (If you want to do this yourself, you have to patch src/qml/qml.pro with those QMAKE_CFLAGS/QMAKE_CXXFLAGS, the top level .pro file won't propagate the flags down to the other files)

      Unfortunately, this does make the qt5-qtdeclarative package armv7 specific.

      Without specifying -march=armv7, the code is compiled in thumb-1 mode, which the JIT compiler doesn't support, so it gets disabled. I believe that the thumb2 instruction set was introduced with armv6t2, which the compiler confirms, so at least some armv6 devices may be able to use the JIT as well. Packaging that will be difficult though.

      Also, running the JIT resolves the SIGILL issue. You don't need the package to have been compiled in debug mode as long as you're using it.

      As soon as you export QV4_FORCE_INTERPRETER=1, the issue reappears - so it's definitely a problem with the interpreter somewhere.

      By Alex Roth on 2017-12-17T22:57:02

      Edited by Ghost User
    • Administrator
      Administrator @root · Imported
      Author Owner

      Created by: ollieparanoid

      Thanks for all the research you have done on this!

      How big is the speed difference with JIT and without it? Depending on that it might make sense to always ship your armv7 hack when compiling qt5-qtdeclarative for armhf, until there's proper armv7 support in Alpine. (This means, that we can't support qml for some armv6 devices in postmarketOS until then.)

      It's interesting to see that the crash is resolved by running inside weston... looks like we need to do that anyway to get QTWayland based compositors working with most devices (#987 (closed)).

      By * postmarketOS Bot on 2017-12-18T18:41:32

      Edited by Ghost User
    • Administrator
      Administrator @root · Imported
      Author Owner

      @ollieparanoid The SIGILL/Illegal Instruction crash isn't fixed by running in Weston. That issue disappeared as soon as I used a debug build, or used the JIT. Something different happens when in debug mode that doesn't ever hit that udf #0 instruction in the interpreter. That's probably an interpreter bug, not a software renderer bug.

      What Weston did resolve was the arithmetic exception caused by the linuxfb/software renderer combo, which you only hit after you've passed the point that generated the SIGILL before. That's a separate issue, and seems to be related to it not picking up screen sizes from the framebuffer properly.

      The JIT is noticeably faster for animations and transitions, especially on my old i747 (Galaxy S3), but I think the bigger bottleneck there is the lack of hardware opengl. I'd be curious to see someone try it on an armhf device that has OpenGL. I agree it's probably worth packaging for now, because it unblocks a lot of testing & development. Plus, it's a much better workaround for the SIGILL issue than shipping a debug build, which is massive & slow (500+mb!).

      By Alex Roth on 2017-12-18T21:19:19

      Edited by Ghost User
    • Administrator
      Administrator @root · Imported
      Author Owner

      LuneOS supports qt5-qpa-hwcomposer. I can package that into feature/libhybris branch and test on Motorola Droid 4.

      By NotKit on 2017-12-18T19:00:30

      Edited by Ghost User
    • Loading
    • Loading
    • Loading
    • Loading
    • Loading
    • Loading
    • Loading
    • Loading
    • Loading
    • Loading
    Please register or sign in to reply