26 Jun 2016

How To Setup JTAG with Galileo (the modern version)

A recent blog post from Olimex pointed to a document [1] showing how to debug the Intel Galileo board using a JTAG. The nice thing about the document is that it assumed the user would be building their own image using Bitbake/OpenEmbedded. The unfortunate part is that the Galileo BSP downloads from Intel are so ancient they have next-to-no chance of working on a recent, modern distro. Their instructions, however, do point this out (i.e. ...this procedure was performed on <some old version of> Ubuntu...), leaving the user little choice but to start by preparing a VM in which to perform the build!

Back when the Galileo board was released, Intel did a great job of supporting it by creating various layers to be used with OpenEmbedded: meta-clanton, meta-galileo, meta-intel-iot-devkit, meta-intel-iot-middleware, meta-intel-quark-fast, meta-intel-quark. But, as you can see, that support was a bit "scattered". On top of that, it doesn't look like meta-clanton was ever placed somewhere public; the only way to get it (and to build for the Galileo) was to download a massive BSP from Intel which included it. Over time this massive download was replaced by a smaller download, which then required you to run a script which would pull in all the sub-components as a separate step (which performed the massive download). Additionally, a fixup script needed to be run in order to clean up some of the build area before you could start your build. Attempting any of this procedure on a modern Linux development host is very likely to fail.

Fast-forward to today (June 26, 2016) and all that's needed to create an image for the Galileo are a distro layer, the basic OE meta layer, and meta-intel. Or, if you're using poky as your distro, you'll get the meta data as part of it.

Building An Image for the Galileo

$ mkdir /some/place
$ cd /some/place

$ mkdir layers
$ pushd layers
$ git clone git://git.yoctoproject.org/poky meta-poky
$ git clone git://git.yoctoproject.org/meta-intel
$ popd

$ . layers/meta-poky/oe-init-build-env galileo

Now, edit conf/local.conf so that
MACHINE ?= "intel-quark"
EXTRA_IMAGE_FEATURES ?= "debug-tweaks tools-debug tools-profile"

And edit conf/bblayers.conf to replace the part that says "meta-poky/meta-yocto-bsp" with "meta-intel".

Now run:
$ bitbake core-image-minimal

When bitbake starts it prints some build configuration information. For my build I saw:

Build Configuration:
BB_VERSION        = "1.31.0"
BUILD_SYS         = "x86_64-linux"
TARGET_SYS        = "i586-poky-linux"
MACHINE           = "intel-quark"
DISTRO            = "poky"
DISTRO_VERSION    = "2.1+snapshot-20160622"
TUNE_FEATURES     = "m32 i586-nlp"
TARGET_FPU        = ""
meta-poky         = "master:6f0c5537e02c59e1c8f3b08f598dc049ff8ee098"
meta-intel        = "master:1b98ae6d7e10390c9ecb383432593644a524f9c8"

If your build fails, one thing you could try is to go to each of the layers and checkout the commits specified in the above information; then restart the build.

At the end of a successful build, continue with the following to create an SDcard image:

$ bitbake parted-native
$ wic create mkgalileodisk -e core-image-minimal

Look through the wic output, it will tell you where it has placed its artifact. Use dd to create your SDcard with the wic artifact:

# dd if=/var/tmp/wic/build/mkgalileodisk-<datetime>-mmcblk0.direct of=/dev/sdX bs=1M


Eventually you're going to use GDB, via openOCD, to debug your target. In order for this to work (in addition to openOCD) you're going to need two things:
  1. a gdbserver "stub" running on your target
  2. a cross-GDB running on your development machine
A cross-GDB is required because your native GDB will only understand your native host's machine code and other CPU-specific information. A cross-GDB is built to run on your native host, but understand a different CPU architecture. A gdbserver stub is necessary on the target because you need some device-specific software running on the target which is able to interrupt the CPU, set breakpoints, etc. The cross-GDB program is large, capable of doing all the work required to perform source-level debugging, and presets the interface to the user. The stub is quite small and has just the minimum target-CPU-specific functionality required on the target.

Above, as part of your first build, I mentioned that you needed to adjust the EXTRA_IMAGE_FEATURES variable of your conf/local.conf file. One of the things that change does is to include the gdbserver stub in your target image.

In order to build a native cross-GDB for your development host you'll need to generate an SDK for your image:

$ bitbake core-image-minimal -c populate_sdk

Once built, you then need to install the SDK. To do so, simply run the resulting SDK script which you'll find in ${TMPDIR}/deploy/sdk. The install script will ask you where you want to install the SDK; type in a path and press Enter, or simply press Enter to accept the default.

Once installed, source the SDK environment file:

$ . <SDK_INSTALL_LOCATION>/environment-setup-i586-nlp-32-poky-linux


My recommendation is to get, build, and install the latest OpenOCD from sources:

$ git clone git://git.code.sf.net/p/openocd/code openocd
$ cd openocd
$ ./bootstrap
$ ./configure

At the end of ./configure'ing, the script will print out a list of all the dongles for which it can include support. Reasons why it can't include support for a particular dongle may include the lack of additional required libraries. If a particular dongle is marked as un-buildable and you want to build support for that dongle, you'll need to figure out the reason(s) why it can't presently be built (i.e. figure out which library it needs) and fix the deficiency (i.e. use your host distribution's package manager to install that library's -dev/-devel package). The ./configure script is pretty good at telling you which library/libraries are missing.

Once the configuration is done:

$ make -j
$ sudo make install

Connecting to the Galileo via JTAG and GDB

Two terminals are required for this part. In one terminal you'll run OpenOCD and in the other you run the cross-GDB (or telnet).

To run OpenOCD you'll need to tell it to which board you're connecting, and which dongle you're using. Obviously the board part will remain the same, but the dongle part for you might be different depending on whether or not you're using the same dongle(s) as me. Also, the order in which this information is given to OpenOCD is important. Apparently you need to specify the dongle first, then the board.

In the following example I'm using the Segger j-link EDU:

# openocd -f interface/jlink.cfg -f board/quark_x10xx_board.cfg
Open On-Chip Debugger 0.10.0-dev-00322-g406f4d1 (2016-06-22-09:29)
Licensed under GNU GPL v2
For bug reports, read
Info : auto-selecting first available session transport "jtag". To override use 'transport select <transport>'.
adapter speed: 4000 kHz
trst_only separate trst_push_pull
Info : No device selected, using first device.
Info : J-Link V9 compiled Apr 15 2014 19:08:28
Info : Hardware version: 9.00
Info : VTarget = 3.354 V
Info : clock speed 4000 kHz
Info : JTAG tap: quark_x10xx.cltap tap/device found: 0x0e681013 (mfg: 0x009 (Intel), part: 0xe681, ver: 0x0)
enabling core tap
Info : JTAG tap: quark_x10xx.cpu enabled

In this example I'm using the ARM-USB-OCD-H from Olimex:

# openocd -f interface/ftdi/olimex-arm-usb-ocd-h.cfg -f board/quark_x10xx_board.cfg
Open On-Chip Debugger 0.10.0-dev-00322-g406f4d1 (2016-06-22-09:29)
Licensed under GNU GPL v2
For bug reports, read
Info : auto-selecting first available session transport "jtag". To override use 'transport select <transport>'.
adapter speed: 4000 kHz
trst_only separate trst_push_pull
Info : clock speed 4000 kHz
Info : JTAG tap: quark_x10xx.cltap tap/device found: 0x0e681013 (mfg: 0x009 (Intel), part: 0xe681, ver: 0x0)
enabling core tap
Info : JTAG tap: quark_x10xx.cpu enabled

Now, to communicate with and control the board via OpenOCD you need to open a second terminal. If you want to simply send commands to OpenOCD (such as to check or flash the board) you can simply use telnet:

$ telnet localhost 4444
Trying ::1...
telnet: connect to address ::1: Connection refused
Connected to localhost.
Escape character is '^]'.
Open On-Chip Debugger

If you want to debug the target via GDB then you need to startup the cross-GDB and connect it to OpenOCD from within GDB itself (note: the cross-GDB should be already on your $PATH, it comes from the SDK we built and installed earlier; if it's not on your PATH you may have forgotten to source the SDK's environment file, see above):

$ i586-poky-linux-gdb
Python Exception <class 'ImportError'> No module named 'operator':
i586-poky-linux-gdb: warning:
Could not load the Python gdb module from `sysroots/x86_64-pokysdk-linux/usr/share/gdb/python'.
Limited Python support is available from the _gdb module.
Suggest passing --data-directory=/path/to/gdb/data-directory.

GNU gdb (GDB)
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "--host=x86_64-pokysdk-linux --target=i586-poky-linux".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
Find the GDB manual and other documentation resources online at:
For help, type "help".
Type "apropos word" to search for commands related to "word".

(gdb) target remote localhost:3333
Remote debugging using localhost:3333
Python Exception <class 'NameError'> Installation error: gdb.execute_unwinders function is missing:
0x00000000 in ?? ()

[1] Source Level Debug using OpenOCD/GDB/Eclipse on Intel Quark SoC X1000, sourcedebugusingopenocd_quark_appnote_330015_003-2.pdf

16 Jun 2016

Overriding Repositories In repo Manifests With Personal Github Forks

Here is the situation: you're working on a development project that uses repo[1] but you'd like to replace one or more of those repositories with ones of your own which you have forked on github and would like to be able to push your changes back into your github fork.

The good news is repo already has a mechanism for this. The only complicated part is getting the github URL correct such that you can push your changes via ssh.

 In the project's manifest will be a repository you want to override; it has a name. In your project go into the .repo directory and create a directory called local_manifests. In .repo/local_manifests create a file using any filename, just make sure it ends with .xml ; for some, unknown reason, it is traditional to name this file roomservice.xml . Within .repo/local_manifests/roomservice.xml you can remove and add repositories to your heart's content.

For example, the project I'm working on has a manifest that looks like:
  1 <?xml version="1.0" encoding="UTF-8"?>
  2 <manifest>
  3     <default revision="master" sync-j="4"/>
  5     <remote fetch="http://github.com/" name="github"/>
  6     <remote fetch="http://github.com/insane-adding-machines/" name="frosted"/>
  7     <remote fetch="git://crosstool-ng.org/" name="crosstool-ng"/>
  9     <!-- utilities -->
 10     <project remote="github" name="texane/stlink" path="toolchain/stlink"/>
 12     <!-- toolchain -->
 13     <project remote="crosstool-ng" name="crosstool-ng" path="toolchain/crosstool-ng"/>
 14     <project remote="frosted" name="elf2flt" path="toolchain/elf2flt"/>
 15     <project remote="frosted" name="newlib" path="toolchain/newlib" revision="frosted">
 16         <linkfile src="../../.repo/manifests/ctng-custom-elf2flt.patch" dest="toolchain/ctng-custom-elf2flt.patch"/>
 17         <linkfile src="../../.repo/manifests/arm-frosted-eabi.config.in" dest="toolchain/arm-frosted-eabi.config.in"/>
 18         <linkfile src="../../.repo/manifests/buildtoolchain.sh" dest="buildtoolchain.sh"/>
 19     </project>
 21     <!-- kernel + userspace -->
 22     <project remote="frosted" name="frosted" path="frosted"/>
 23     <project remote="frosted" name="libopencm3.git" path="frosted/kernel/libopencm3"/>
 24     <project remote="frosted" name="busybox.git" path="frosted/apps/busybox"/>
 25     <project remote="frosted" name="picotcp" path="frosted/kernel/net/picotcp"/>
 26     <project remote="frosted" name="frosted-userland.git" path="frosted/frosted-userland"/>
 28 </manifest>
I want to replace the repositories described at lines 22 and 26 with my own repositories. I start off by going to github, finding those repositories, and clicking on the Fork button in the web interface for both. Then I create my .repo/local_manifests/roomservice.xml file with the following contents:
  1 <?xml version="1.0" encoding="UTF-8"?>
  2 <manifest>
  3         <remote fetch="ssh://git@github.com/twoerner/" name="origin"/>
  4         <remove-project name="frosted"/>
  5         <remove-project name="frosted-userland.git"/>
  6         <project remote="origin" name="frosted.git" path="frosted" revision="contrib/twoerner-master"/>
  7         <project remote="origin" name="frosted-userland.git" path="frosted/frosted-userland" revision="contrib/twoerner-master"/>
  8 </manifest>
 At this point I simply invoke "repo sync" (you might need to add --force-sync) and you're all set! Those repositories will now be sync'ed from your github copies, you can work in the code of those repositories, make commits, and push them up to github.

Notice that the names that are used in the <remove-project.../> lines are the same names as the lines from the original manifest that I want to replace.

Did you notice that I used "origin" as the remote name? When repo clones this repository, it will use this name as git's remote name, and since git assumes the primary remote's name is "origin", this configuration simply makes life a touch easier. You're free to use whatever remote name you want, but in this case you'll just need to run add your remote's name the first time you "git push..." .

"What is repo?" repo is a tool created by (and very popular with) Android developers to manage building software comprised of sets of git repositories. Many software projects fit into one git repository, but sometimes a project likes to keep separate git repositories for different parts of the project. This, then, creates a burden on the developer to make sure the correct repositories, are cloned to the right places, and checked out at the right commits. repo is a tool that lists which repositories are used, where to place them relative to one starting directory, and checks out a specific revision/commit. Therefore it removes the extra burdens caused by multi-repository projects. repo also has functionality to allow your project to integrate with gerrit (a code review system), but that's another topic. For more information on repo try: