Zaurus ÖÐÎÄÂÛ̳

[ccpaging]

ARM EABI Port

ARM EABI port is now available. See ArmEabiHowto and ArmEabiTodo


Contents
ARM EABI Port
In a nutshell
GCC view
ARM floating points
Struct packing and alignment
Stack alignment
64-bit data type alignment
Enum sizes
System call interface
Choice of minimum CPU
Thumb interworking suggests armv4t
Other scenarios
Why a new port
0. Not an option!
1. Rename all library packages
2. New arch
3. ABI: field in control file
4. conflicting libc packages
Roadmap
EABI status
Naming
Strategy
Links

In a nutshell

EABI is the new "Embedded" ABI by ARM ltd. EABI is actually a family of ABI's and one of the "subABIs" is GNU EABI, for Linux. The effective changes for users are:

Floating point performance, with or without an FPU is very much faster, and mixing soft and hardfloat code is possible
Structure packing is not as painful as it used to be
More compatibility with various tools (in future - currently linux-elf is well supported)

A more efficient syscall convention
At present (with gcc-4.1.1) it works with ARMv4t, ARMv5t processors and above, but supporting ARMv4 (e.g., StrongARM) requires modification to GCC. See "Thumb interworking" below.

GCC view

New ABI is not only a new ABI field, it is also a new GCC target.

Legacy ABI
ABI flags passed to binutils: -mabi=apcs-gnu -mfpu=fpa
gcc -dumpmachine: arm-unknown-linux
objdump -x for compiled binary:

private flags = 2: [APCS-32] [FPA float format] [has entry point]
"file" on compiled Debian binary:

ELF 32-bit LSB executable, ARM, version 1 (ARM), for GNU/Linux 2.2.0, dynamically linked (uses shared libs), for GNU/Linux 2.2.0, stripped
"readelf -h | grep Flags""

Flags: 0x0

Arm EABI:
ABI flags passed by gcc to binutils: -mabi=aapcs-linux -mfloat-abi=soft -meabi=4
gcc -dumpmachine: arm-unknown-linux-gnueabi
objdump -x for compiled binary:

private flags = 4000002: [Version4 EABI] [has entry point]
"file" on compiled binary (under Debian):

ELF 32-bit LSB executable, ARM, version 1 (SYSV), for GNU/Linux 2.4.17, dynamically linked (uses shared libs), for GNU/Linux 2.4.17, stripped
"readelf -h | grep Flags""

Flags: 0x4000002, has entry point, Version4 EABI

Furthermore, as well as the usual __arm__ and maybe also __thumb__ symbols, the C preprocessor symbol __ARM_EABI__ is also defined when compiling into EABI.

ARM floating points

The current Debian port creates hardfloat FPA instructions. FPA comes from "Floating Point Accelerator". Since the FPA floating point unit was implemented only in very few ARM cores, these days FPA instructions are emulated in kernel via Illegal instruction faults. This is of course very inefficient: about 10 times slower that -msoftfloat for a FIR test program. The FPA unit also has the peculiarity of having mixed-endian doubles, which is usually the biggest grief for ARM porters, along with structure packing issues.

ARM has now introduced a new floating point unit, VFP (Vector Floating Points), which uses a different instruction set than FPA and stores floats in natural-endian IEEE-754 format. VFP is implemented in new some ARM9/10/11 cores, like in the new TI OMAP2 family. It seems likely that ARM cores without VFP will remain popular, as in many places ARM is used floats are unnecessary.

To complicate thing further, ARM processors are being integrated with many other FPUs and DSPs, each of which adds its own set of instructions to the ARM set:
Cirrus Logic's EP93XX series integrate an ARM920T core with a Maverick Crunch FPU. This also uses IEEE-754, though uses a different instruction set to VFP. Current ARM-Debian users cannot use their Maverick FPUs at all except by programming in assembler or using an alternative compiler. GCC has flags to generate Maverick FP instructions (-mfpu=maverick), but the .o files cannot be linked with the standard Debian GCC startup files or libraries.
Intel's iWMMXt unit is used in their PXA270 processor with an XScale main core. This adds integer SIMD and some other instructions but there is currently no iWMMXt processor with hardware floating point capabilities. iWMMXt processors are incompatible with FPA due to opcode overlap, while they could have an VFP coprocessor in principal. That said, iWMMXt instructions should make softfloat fairly quick anyway. Again, GCC support exists (-march=iwmmxt) for this but is also currently unusable within standard Debian.

Texas Instruments' OMAP, OMAP2, DaVinci DM644x series and numerous other products integrate a ARM9/ARM11 core with their own DSP core for multimedia acceleration and/or telecommunication signal processing. Most Dsp's do fixed-point math. DSP code is completely separated from ARM code. In Linux DSP Gateway or proprietary solutions are used to load code for execution on the c55x/c6xx and provide a way to for ARM and DSP code to communicate.

For a generic-purpose distribution like Debian, targeting binary compatibility, EABI lets us have the cake and eat it. We can make a soft-float distribution using IEEE-754 with FPU-specific versions of packages (linux-kernel-2.6.x-vfp, libc6-iwmmxt, mediaplayer-maverick, etc) where needed. This also enables individual packages to do runtime FPU detection and call code compiled for different FP scenarios (in liboil for example).

The major FP variants worth support as alternative versions of FP-critical packages seem to be
the current arm arch supporting armv3 with or without FPA and armv4 processors.

EABI for generic ARM (>= armv4t) using IEEE softfloat
EABI for lowest common denominator VFP (there are now more than one VFP "extended" variant)
EABI for Maverick FPU
EABI for iWMMXt using iWMMXt-specific softfloat

Struct packing and alignment

With the new ABI, default structure packing changes, as do some default data sizes and alignment (which also have a knock-on effect on structure packing). In particular the minimum size and alignment of a structure was 4 bytes. Under the EABI there is no minimum and the alignment is determined by the types of the components it contains. This will break programs that know too much about the way structures are packed and can break code that writes binary files by dumping and reading structures.

Stack alignment

The ARM EABI requires 8-byte stack alignment at public function entry points, compared to the previous 4-byte alignment.

64-bit data type alignment

"One of the key differences between the traditional GNU/Linux ABI and the EABI is that 64-bit types (like long long) are aligned differently. In the traditional ABI, these types had 4-byte alignment; in the EABI they have 8-byte alignment. As a result, if you use the same structure definitions (in a header file) and include it in code used in both the kernel and in application code, you may find that the structure size and alignment differ."

-- from the Codesourcery ARM GNU toolchain FAQ

Enum sizes

The EABI defines an optional system for controlling the size of C enumerated types. For arm-linux it was decided to keep the existing behaviour (enums are at least the same size as an int) for consistency with other Linux systems.

This is also reflected in the -mabi=aapcs or -mabi=aapcs-linux switches to GCC: aapcs defines enums to be a variable sized type, while with aapcs-linux they are always ints (4 bytes).

System call interface

Two things change in the system call interface: alignment of 64-bit parameters passed in registers and the way the system call number itself is passed.

With EABI, 64-bit function parameters passed in registers are aligned to an even-numbered register instead of using the next available pair.

Here's an explanation from Russell King, 12 Jan 2006:
We have r0 to r6 to pass 32-bit or 64-bit arguments. With EABI,
64-bit arguments will be aligned to an _even_ numbered register.
Hence:

long sys_foo(int a, long long b, int c, long long d);

will result in the following register layouts:

EABI Current
r0 a a
r1 unused \_ b
r2 \_ b /
r3 / c
r4 c \_ d
r5 /
r6 ... out of space for 'd' ... room for one other int.
r7 syscall number

Since this already causes an incompatible change in the system call interface, the opportunity has been taken to slip in a more efficient, totally incompatible way of doing system calls: instead of using the swi __NR_SYSCALL_BASE(==0x900000)+N instruction, where N is the number of the system call, swi 0 is always used with the system call number stashed in register r7. This is more efficient because the kernel no longer has to go and fish N out of the instruction stream(*), which used to have a negative impact on the efficiency of processors with separate text and data caches (i.e. most ARMs).

Fortunately, the two schemes can coexist and EABI kernels have an option to support the old syscall interface (including old structure layout rules) for running old-EABI binaries. However some features (e.g., ALSA, MD (RAID) and system calls from Thumb mode) do not work correctly from old-ABI binaries.

Some third party EABI toolchains (e.g., CodeSourcery 2005q3) use the old kernel interface via userspace shims in glibc. This is now obsolete and no longer supported by glibc.

(*) This is only true if the old-ABI compatibility option is disabled.

See this article for more details.

Choice of minimum CPU

Thumb interworking suggests armv4t

The EABI includes thumb interworking, which means that 16-bit Thumb and normal 32-bit ARM instructions can be mixed at function-level granularity.

Thumb interworking is mandatory according to the ARM EABI spec and requires every return and indirect function call to execute a BX instruction to set the core to the correct state, which is only present in armv4t cores and above. Gcc, too, only supports thumb interworking for armv4t and above.

So Debian armel runs on a minimum CPU of ARMv4t and by default the Debian armel GCC generates code for armv4t (rather than the usual default ARM target of armv5t).

Other scenarios

However a lower entry-level CPU is possible to do using different function return sequences which are of various speeds, and that work and/or allow Thumb interworking on different selections of the ARM architectures.

0. mov pc,lr

Is what GCC currently emits for -march=armv4. It works on ARMv4 and above but is only Thumb interworking-safe from ARMv7.

1. bx lr

Is what GCC emits for -march=armv4t. It works on ARMv4t and above and Thumb interworking is possible on ARMv4t and above. Excludes armv4, the StrongARM which are very common and some armv5 users, but armv5 with no t seems a rare processor. CC needs modifying to implement any of the following choices.

2. tst lr, #1; moveq pc, lr; bx lr

was suggested by Paul Brook as an alternative to BX. It works on ARMv4 and above and Thumb interworking is possible on ARMv4t and above, with the extra cost of two instructions per indirect call/function return, though in line with the run-on-minimum-hardware Debian way.

Here is a patch by Richard K. Pixley which illustrates what is needed, but is not (April 2007) tested: http://lists.debian.org/debian-arm/2007/05/msg00015.html

This is problematic because hand written assembly has to be manually fixed.

3. ldm/ldr

Works on ARMv4 and above but Thumb interworking is only possible on ARMv5t and above, excluding ARMv4t users from using Thumb code with Debian. Gcc currently emits this for non-leaf functions on ARMv4 and ARMv5 (but not ARMv4t, where it uses BX, the only way to do interworking on v4t). Although a single instruction, this method may be slower than the three-instruction sequence because of the memory accesses it requires.

4. Drop Thumb interworking

A final option would be simply to compile the standard Debian repo --with-arch=armv4 --with-no-thumb-interwork. This would work on all processors without the dangers inherent in modifying GCC and, according to the GCC manual page, saves a slight size and speed overhead caused by being thumb-interworkable.

There is significant discussion of the technical merits of these various schemes in the debian-arm mailing list thread Re: ARM EABI port: minimum CPU choice of which the above is a partial summary.

5. tat says that simply compile for armv3 would work, though the code would be relatively slower on the most common, later CPUs. armv3 is fairly rare: Psion 5

6. Kernel emulation traps

It may be possible to catch illegal instruction in the kernel generated by the missing "BX" instruction, in the same way as missing hard floating point instructions can be emulated. It wouldn't be that fast on armv4 hardware (causing a context switch per function call/return) but such a kernel hack would allow the current repository to be used unmodified on armv4 hardware.

7. Linker fixups

The EABI provides machanisms (R_ARM_V4BX relocations) for the linker to fixup bx instructions. Currently the linker only knows how to convert these to mov pc instructions, so you have to choose between armv4 or interworking at static link time. However the linker could be taught how to convert these into branches to a tst;moveq;bx stub. This has the advantage of also working for hand written assembly. It may be desirable to get the compiler to also generate this triplet inline for performance reasons.

Paul Brook has a partial implementation of this.

Why a new port

In Debian, we want to assure complete binary compatibility. Since the old ABI is not compatible with the new one, we can't allow packages built with old ABI to link against new-abi libs, or the other direction. So the options are:

0. Not an option!

Under no circumstances distribute EABI binaries as .arm.deb depending on current library package names!!!

1. Rename all library packages

This is an ABI transition that affects all architectures, and it has been done before (aout -> elf, c++ ABI)
+ apt-get dist-upgrade for users is possible
- Requires insane amounts of work - every single library package needs to be renamed
- Requires a very long transition period, in which unstable will be broken for all archs.
c++ ABI transition takes about half an year, full transition could thus take around 2 years
- Achieving Consensus for such transition on debian-devel would be very hard.
Non-ARM developers will object doing such amount of work only for a minor arch. If arm gets dropped from Release Arch's, we can't even file RC bugs for the migration.
- Very invasive change, affecting every user and developer of Debian.

2. New arch
+ Technically, since we drop FPA instruction support, and gcc dumpmachine triplet is different, we can argue we have a new arch
+ Does not affect non-ARM users
+ we can target EABI for armv4(t?)+ while we can can keep oldabi port for ARMv3 (RiscPC) and maybe armv4 (StrongARM) users.
+ Allows using new instructions (thumb) and drops the old FPU instruction set
+ Can be done quickly, does not affect other arch's release cycle
+ requires less archive space during migration
- Current ARM users don't have a easy upgrade path

For the last point, a statically compiled ArchUpgrade tool could be created. This would also allow i386->amd64 style migrations.

3. ABI: field in control file

This was suggested as part of Multiarch proposal. It is unknown if it would actually become part of Debian or not
+ Reflects the packages ABI correctly, would help other transitions as well
- no working implementation
- no consensus on how to do it (apt developers want something more generic instead)
- might be hard to fit into current archive infrastructure
- make dependency resolving hard

From these choices, we believe a new port is the best compromise.

4. conflicting libc packages

In this scenario, we create a libc6-eabi(-dev) package that has eabi glibc and ld-linux.so.3. This package will conflict with libc6(-dev), and thus you can mix and match eabi and non-eabi binaries and libs.
+ similar to the libc6.1 style packages on some archs
+ requires modifying only glibc
- ugly
- most of ARM port will remain uninstallable for long time
- apt-get dist-upgrade will still not work, since it gives up quickly when lots of packages conflict

Let's not make perfect an enemy of good!!

Roadmap

Armel (EABI) will be released with etch+1 as it should be in good shape by then. That release will thus contain arm and armel. Arm will be dropped in etch+2, assuming that the above gcc changes to support armv4 CPUs in armel prove practical. If we cannot support armv4 in armel then arm will remain around until we drop v4/StrongARM support, i.e. the port falls into general disuse.

EABI status

The commercial ARM RealView C/C++ compiler was the first to support EABI, and usable EABI support came into GCC from version 4.1.0.

CodeSourcery provide [GNU ARM toolchains]. The 2005Q3 release is a modified version of gcc-3.4.4 while 2006Q1 is from gcc-4.1.0. These toolchains produce EABI object code and the 2006Q1 release also uses the EABI Linux kernel interface.

EABI is supported in the ARM Linux kernel from version 2.6.16 and there is an optional compatibility feature to allow the running of old-ABI binaries with an EABI kernel. The inverse mechanism, to run EABI binaries in an ABI kernel, is not implemented and is said to be non-trivial to do.

Riku Voipio has built a booting EABI root filesystem up to X as proof of concept, which seems stable, built with codesourcery gcc 3.4 toolchain.

Koen Kooi has used OpenEmbedded to build a pure EABI root filesystem including native toolchain, visible under http://dominion.kabel.utwente.nl/koen/cms/working-native-eabi-toolchain The compiler is gcc-4.1.1 with glibc-2.4: the exact versions we need. The system boots and runs fine on armv5t and the C compiler seems to work well. However the C++ compiler is not working because libstdc++ is not installed and perl does not execute because libperl.so.5 is not installed as well. Both problems can be solved by using ipkg to install them.

The Angstrom distro of OpenEmbedded has a public repository of ARM EABI ipackages compiled for armv5te and visible under http://angstrom-distribution.org/unstable/feed/

Lennert Buytenhek and ADS have built an EABI root filesystem, and some ten thousand (probably many more by now) packages. For download locations and other details, see the announcement

QEMU 0.8.1 can run ARM EABI systems, though when running with the 2.6.16 kernel it is mind-bogglingly slow on x86 processors of a few hundred megahertz. Using 2.6.17-rc3 or later fixes this anomaly. To run a single ARM EABI executable in qemu-user mode, some patches are required, though these are not complete yet.

Minimum versions of components with the first working EABI support are:
binutils - from 2.16.92 - already in Debian
gcc - gcc 4.1.0 (Thumb interworking on armv4t needs 4.1.1)
glibc - fully upstream in 2.4. Will also be in 2.3.7
Earlier glibcs (2.3.6?) support EABI userspace but had old-style syscalls to work with older kernels (2.6.8-2.6.13ish).
kernel - eabi support is present from 2.6.16.
dpkg, apt - patches will be submitted when port name consensus is achieved

Naming

At the Extremadura emdebian meeting, 12-16 April 2006, the name "armel" was chosen. If a bigendian arm EABI port will be created, it will be called "armeb", and it will replace the previous oldabi-based "armeb" port effort.

Strategy

The ultimate aim is a new standalone architecture, composed of three concrete components:
At least two online repositories of precompiled binary packages

At least two online buildd's: native ARM EABI machines that will compile Debian packages from source into ARM EABI packages automatically. See Autobuilder network and Setting up a buildd
Debian installer support for the new arch.


The following is outdated but left here for historical interest. Lennert Buytenhek booted his Debian EABI port from the Angstrom distribution of OpenEmbedded.

The chronological steps to bootstrap the new arch seem to be:

1) Make Debian packages of a cross-compiler targeting ARM EABI. This means gcc-4.1, glibc-2.4+glibc-ports-2.4, binutils-2.17 and linux-2.6.16. This can be compiled using crosstool-0.42 and the patches and control files at http://freaknet.org/martin/crosstool, and packaged with the scripts at the same location.

2) Make a package for the existing Debian experimental ARM arch of the Linux kernel compiled to use EABI internally, with run-old-ABI-binaries enabled, and test it with existing old-ABI Debian userland.

3) Cross-build essential and required EABI userland packages using dpkg-cross. A parallel effort is the slind project, which is busy improving dpkg-cross support within the Emdebian framework.

4) Make the Debian installer debootstrap work for the new arch.

5) Populate the new-arch repository(-ies) with the rest of the Debian packages.

Is there a "HOWTO Create a New Debian Arch" document? No.

On the crossgcc list, Michael K. Edwards says of the procedure to bootstrap a new Debian arch by doing all building in a QEMU chroot on a fast host:
You build your crosstools, you build your minimal chroot and a Canadian cross, then you qemu-chroot in and build a fully native toolchain. Instantiate a dpkg database with the host's dpkg and use equivs to fake up dpkg entries for your toolchain. Build your Required packages and real packages for build-essential, then construct a fresh qemu-chroot, this time with debootstrap and your pile o' packages. If you're paranoid, rebuild all your packages in this chroot and debootstrap afresh -- it's your first chance to test that your glibc built the Debian Way really works. Then start layering on applications, without worrying about whether they cross-compile easily.

Links

ELF for the ARM Architecture (PDF by ARM Ltd)

GNU Toolchain for ARM FAQ by CodeSourcery

[ccpaging]

ÔÚzaurusÉÏ£¬debianÖ÷ÒªÊÇ°²×°ÔÚCF / SD¿¨ÉÏ£¬ÌرðÊÇSD¿¨£¬ËùÒÔÈçºÎ·ÖÇøÊÇ°²×°debian±ØÐëÒªÁ˽âµÄÄÚÈÝ¡£

Multiple partitions on a CF / SD card

Setting up multiple partitions on a CF or SD card may be quite useful. For example, to flash a new ROM image to a 5000/5500 you need a FAT partition on your CF card. On the other hand, installing programs to a card often requires a partition with symbolic linking support (i.e. not FAT). With a multi-partition setup, you can have your cake and eat it too - provided your cake is big enough (at least 64 MB).

Here is a step-by-step guide for setting up two partitions on a CF or an SD card: a FAT one and an ext2 one. It is based on the excellent Step-by-step CF/SD fdisk/formatting for newbies Howto. See the bottom of this page for notes on differences in an SD setup. (Note: 3.5.3 calls the second SD partition "card1". The original instructions were for calling it "card2", so this becomes a problem when the system is rebooted. It's probably more complicated for CF cards because of the second CF that is included in the SL6000 hardware. If you know how it works, update this page.)

The 3.5.3 distribution includes the required programs fdisk and mkfs.ext2, but does not include mkfs.msdos. You can get that by installing the dosftools package.

Then, as root, type the following from the console:
cd /root
umount /dev/hda1 [or /dev/mmcda1 if using SD]
fdisk /dev/hda [or /dev/mmcda] (it will ask you for a command)
d (this will delete a partition)
1 (this will delete partition #1)
n (this will create a new partition)
p (this will create it as a Primary partition)
1 (Choses partition 1; it will tell you the available cylinder range)
Enter (to take default start cylinder)
N (integer, determines partition size: 1 < N < number of cylinders on the card)
Enter (to confirm N as end cylinder)
v (this will verify that the partition has been created)
t (to change partition type)
1 (to select partition 1)
6 (to select FAT-16)
n (this will create a new partition)
p (this will create it as a Primary partition)
2 (Choses partition 2)
Enter (to take default start cylinder which will be N + 1)
Enter (to take default end cylinder)
v (this will verify that the partition has been created)
t (to change partition type)
2 (to select partition 2)
83 (to select ext2)
w (this will save the table and exit the fdisk program)

When you're done with fdisk, enter one of these sets of commands depending on which card you are using:
mkfs.msdos /dev/hda1 (this will format the 1st partition with the FAT-16 file system)
mkfs.ext2 /dev/hda2 (this will format the 2nd partition with the ext2 file system)
mkdir /mnt/cf2 (this will create the directory where the 2nd partition will be mounted)
...or... (if SD)
mkfs.msdos /dev/mmcda1 (this will format the 1st partition with the FAT-16 file system)
mkfs.ext2 /dev/mmcdda2 (this will format the 2nd partition with the ext2 file system)
mkdir /mnt/card2 (this will create the directory where the 2nd partition will be mounted)

Now you need to edit /etc/fstab to associate the 2nd partition with its mount point. The /etc/fstab file should already contain a line for the first partition and it should look something like this:
/dev/hda1 /mnt/cf auto defaults,noauto,noatime,user,exec,suid 0 0
...or... (if SD)
/dev/mmcda1 /mnt/card auto defaults,noauto,noatime,user,exec,suid 0 0

Use your favorite text editor to copy and tweak that line; you should end up with:
/dev/hda1 /mnt/cf auto defaults,noauto,noatime,user,exec,suid 0 0
/dev/hda2 /mnt/cf2 auto defaults,noauto,noatime,user,exec,suid 0 0
...or... (if SD)
/dev/mmcda1 /mnt/card auto defaults,noauto,noatime,user,exec,suid 0 0
/dev/mmcda2 /mnt/card1 auto defaults,noauto,noatime,user,exec,suid 0 0

If you want programs installed to the CF card to go to the ext2 partition rather than to the FAT one (and you do want that), you need to edit /etc/ipkg.conf, changing the line
dest cf /mnt/cf to dest cf /mnt/cf2
...or... (if SD)
dest cf /mnt/card to dest cf /mnt/card2

Now the instructions get entirely different for CF and SD cards.

Here's the CF version:

Finally, you want to __ensure that both partitions get auto-mounted__ when you reboot your Zaurus or eject / re-insert the CF card. For that, you need to edit /etc/pcmcia/ide.opts. This is not the place to explain how the file works (it contains explanatory comments if you care). I'll just list what you should end up with, omitting comments for brevity:
case "$ADDRESS" in
*,*,*,1)
DO_FSCK="y"
DO_MOUNT="y"
;;
*,*,*,2)
DO_FSCK="y"
DO_MOUNT="y"
;;
*,*,*)
PARTS="1 2"
NO_CHECK=n
NO_FUSER=n
;;
esac

The easiest way to apply all these changes is to restart your Zaurus whichever way your ROM allows. That's that.

Here's the SD version:
(Log in as root)
mount -n -o remount,rw / (make the file system writable)
cp /etc/sdcontrol /etc/sdcontrol2 (make a control file for the new partition)

Edit the file /etc/sdcontrol2 and change three adjacent lines
DEVICE=/dev/mmcda1 to DEVICE=/dev/mmcda2
MOUNT_POINT=/mnt/card to MOUNT_POINT=/mnt/card1
SMB_MOUNT=/home/samba/SD_Card to SMB_MOUNT=/home/samba/SD_Card1


Take your sd card out and then put it back in and both partitions should now mount.