In the previous article, how to run OKL4 based application with android emulator was introduced. Here this time I will introduce another virtualization series, EmbeddedXen running on top of qemu.
Before diving deep into this virtualization world, I knew there are only two XEN implementations on ARM architecture , one is XEN-ARM, don't know what exact name for this project, maintained by samsung and the other is EmbeddedXen. When I investigated two candidates to test, I could not find the source code for XEN-ARM but for EmbeddedXen, there is an official web site and source code is also available so I picked up Embedded Xen to test the feasibility and performance of virtualized embedded system.
Testing environment
Host OS: Windows XP or above
Cross compiler to compile EmbeddedXen for ARM: android NDK toolchain version arm-eabi-gcc-4.2.1
Compiler to compile qemu: latest MinGW gcc.
Building EmbeddedXen
There is already a good wiki page explaining to build and test but in Windows machine, some parts should be fixed and I will explain these below. It should be noted that this image, at default, can be built to be runnable on a virtual mainstone board which can be setup with qemu.
- Download source codes: Open cygwin shell and install git if you don't have git installed using cygwin setup application.
$ git clone git://embeddedxen.git.sourceforge.net/gitroot/embeddedxen/embeddedxen
- Download a patch for compiling cygwin version:
embeddedxen-cygwin-patch.diff
$ cd embeddedxen
$ patch -p1 < ../embeddedxen-cygwin-patch.diff
$ patch -p1 < ../embeddedxen-cygwin-patch.diff
- Compile the sources:
$ build-embeddedxen -u -0 -x
This script will generate an u-boot compatible image which can be loaded in the virtual machine known as mainstone. This generated image is flashable and has no debugging information. If you want to debug Xen, then there is a symbolic image named as vmlinux.xen containing full debugging information. Later I will show how to debug xen code with help of gdb.
Building custom qemu executable
qemu-0.12.5 version can be compiled under MinGW shell so open MinGW shell. configure may require to install zlib and optional SDL library. In this case, here is a useful link.
Original qemu source seems to have a bug which cannot start boot loader code when this embeddedXen rom is loaded so a patch file for this should be applied. Now let's do it.
- Download source and untar it.
$ cd {work_directory}
$ wget http://download.savannah.gnu.org/releases/qemu/qemu-0.12.5.tar.gz
$ tar xvfz qemu-0.12.5.tar.gz
$ wget http://download.savannah.gnu.org/releases/qemu/qemu-0.12.5.tar.gz
$ tar xvfz qemu-0.12.5.tar.gz
- Download a patch file enable to load and run the above xen image and apply it.
$ patch -p0 < qemu-xen-mingw.diff
- Compile qemu
$ cd qemu-0.12.5
$ configure --target-list=arm-softmmu --enable-sdl
$ make
$ configure --target-list=arm-softmmu --enable-sdl
$ make
Running EmbeddedXen
In order to simulate mainstone board on qemu, two flash files should be provided and here is the command to create them
$ cd {work_directory}
$ mkdir images
$ dd if=/dev/zero of=./images/flash1 bs=32M count=1
$ dd if=/dev/zero of=./images/flash2 bs=32M count=1
$ mkdir images
$ dd if=/dev/zero of=./images/flash1 bs=32M count=1
$ dd if=/dev/zero of=./images/flash2 bs=32M count=1
Now it is time to launch qemu with embeddedXen image
$ qemu-0.12.5/arm-softmmu/qemu-system-arm -localtime -M mainstone -kernel embeddedxen/xen/arch/arm/boot/uImage.mainstone -pflash ./images/flash1 -pflash ./images/flash2 -m 256 -serial telnet::4444,server
Debugging EmbeddedXen with gdb
As you can see in the above picture, there seems to be some bug in the xen code around mapping I/O of mainstone. You want to debug it? No problem, here is the instruction.
1. Add -s and -S option in the above command line. "-s" option make gdb enable remote debugging through TCP::1234 and "-S" option make qemu wait until gdb connection is established.
$ qemu-0.12.5/arm-softmmu/qemu-system-arm -localtime -M mainstone -kernel embeddedxen/xen/arch/arm/boot/uImage.mainstone -pflash ./images/flash1 -pflash ./images/flash2 -m 256 -serial telnet::4444,server -s -S
2. Open telnet session with address 127.0.0.1:4444 and connect.
3. After telnet connection for serial console is made, open another shell windows and run gdb and set break point.
$ cd {work_directory}
$ arm-eabi-gdb embeddedxen/vmlinux.xen
(gdb) br __start_xen_arm
(gdb) target remote :1234
(gdb) c
$ arm-eabi-gdb embeddedxen/vmlinux.xen
(gdb) br __start_xen_arm
(gdb) target remote :1234
(gdb) c
Then after few second, gdb will be waiting for user input at the break point.
(gdb) br __start_xen_arm
Breakpoint 1 at 0xff0089bc: file xen/arch/arm/hypervisor/setup.c, line 550.
(gdb) c
The program is not being run.
(gdb) target remote :1234
Remote debugging using :1234
[New Thread 1]
0xa0000000 in ?? ()
(gdb) c
Continuing.
Breakpoint 1, __start_xen_arm () at xen/arch/arm/hypervisor/setup.c:550
550 mbi.mods_count = 1;
(gdb)
