Understand and decompress ULI files

[Go4IT]

When looking at the update-CD their are files with suffix *.uli on it. In this article i'll explain and discuss how i find out about their content and how to decompress them. For the example i'll take the "plattapp.uli" from "dnl/bin/system/arion/plattapp.uli". This is the main HMI application running on the OMAP-based system.

Looking at the file shows a "magic" in front of it ("ULI "), then random bytes seem to follow, nothing readable:

Code: Select all

Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

00000000  55 4C 49 20 44 82 5C B0 03 00 00 00 00 00 00 00  ULI D‚\°........
00000010  00 00 00 B0 54 00 00 00 BC 4B 6B 00 61 61 41 00  ...°T...¼Kk.aaA.
00000020  B2 D2 4F A7 02 00 00 00 BC 5B 6B B0 B5 61 41 00  ²ÒO§....¼[k°µaA.
00000030  34 06 0A 00 55 9D 02 00 DD C7 1D 5A 04 00 00 00  4...U...ÝÇ.Z....
00000040  F0 61 75 B0 00 00 00 00 A0 85 1E 00 00 00 00 00  ðau°.... …......
00000050  FF FF FF FF 00 21 F8 B5 05 1C CA F1 D7 FC FC 48  ÿÿÿÿ.!øµ..Êñ×üüH
00000060  28 60 28 1C 26 30 0E F0 BC FF 28 1C C8 30 0F F0  (`(.&0.ð¼ÿ(.È0.ð
....

The only readable text in the file is at the end of it, and it is in German (because Bosch is a German developer) and that gave us a hunch:

Code: Select all

Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

0043FE20  00 00 00 00 00 00 00 BF A8 00 00 06 64 61 73 20  .......¿¨...das 
0043FE30  69 73 74 20 65 69 6E 20 74 65 78 74 20 64 65 6E  ist ein text den
0043FE40  20 69 63 68 60 02 00 0F 67 65 66 FC 67 74 20 68   ich`...gefügt h
0043FE50  61 62 65 20 75 6D 20 7A 75 20 74 65 73 74 65 6E  abe um zu testen
0043FE60  20 6F 62 20 65 73 20 6D 69 74 05 00 1E 6D 20 70   ob es mit...m p
0043FE70  61 63 6B 65 6E 20 62 65 73 73 65 72 20 67 65 68  acken besser geh
0043FE80  74 20 77 65 6E 6E 20 6D 65 68 72 20 69 6E 69 74  t wenn mehr init
0043FE90  69 61 6C 69 73 69 65 72 74 65 20 64 61 7C 07 04  ialisierte da|..
0043FEA0  76 6F 72 6C 69 65 67 44 0D 09 75 6E 64 20 74 61  vorliegD..und ta
0043FEB0  74 73 E4 63 68 6C 74 0E 02 73 74 69 6D 6D 5C 04  tsächlt..stimm\.
0043FEC0  0D 6D 65 69 6E 65 20 76 65 72 6D 75 74 75 6E 67  .meine vermutung
0043FED0  3A 7C 0D 03 67 69 62 74 20 6B 80 03 03 66 65 68  :|..gibt k€..feh
0043FEE0  6C 65 72 A4 0C 07 62 65 69 6D 20 65 72 7A 65 75  ler¤..beim erzeu
0043FEF0  60 0A 0E 64 65 73 20 2E 75 6C 69 2D 66 69 6C 65  `..des .uli-file
0043FF00  73 21 21 21 39 C4 42 11 00 00 FF FF 00 00 00 00  s!!!9ÄB...ÿÿ....

I've tried to add the unreadable parts of the text to this: "das ist ein text den ich eingefügt habe um zu testen ob es mit dem packen besser geht wenn mehr initialisierte daten vorliegen und tatsächlich stimmt meine vermutung es gibt fehler beim erzeugen des .uli-files!!!"

So this gave me the idea that ULI is some kind of compression algorithm. Also the lack of repeating data, missing 0x00 give the impression of a compressed file. Those algos are made to save a much space as possible.

I also always look at file trailer to get something about it. Most often we find CRC checksums here, so what i always try is to calc checksum, except of the last 2 or 4 bytes with some often used algos like CRC16 or CRC32 to find a match:

Code: Select all

Offset(h) 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

0043FF40  00 00 01 00 CE DE AD CE 31 21 52 31 54 FF 43 00  ....ÎÞ.Î1!R1TÿC.
0043FF50  57 46 B0 55                                      WF°U

And BINGO, the last 4 Bytes is simply a CRC32 checksum of the whole file in little-endian format.

So now we know about our file:

Code: Select all

HEADER = "ULI " (magic)
TRAILER = CRC32 checksum

When i compare some trailers of other ULI files i found some static and some changing codes. This byte sequence seems to be static over all uli files:

Code: Select all

00 00 01 00 CE DE AD CE 31 21 52 31

When looking for strings in the Flash image containing keywords like "compress" "packed" i found "u32GetSizeOfUnpackedLZOFile", another hunch for LZO as packing algo.

[Go4IT]

Some portions ob the ULI-packed "*reg" files where readable, which looks like a dictionary based algorith as RLE, LZW. After some research i found out that it is LZO which is used. A algo specialized on decompression speed to place data in Flash and decompress it on runtime into RAM. This fit's perfectly here as i found the compressed ULI-files as-they-are inside the readout of the Flash (image).

So one thing is for sure right now: The packed data stays packed in Flash and is decompressed into RAM on runtime.

Now let's get back to some information about LZO i could gather.

This is the offical website for it:
http://www.oberhumer.com/opensource/lzo/

This is what the author is telling about LZO:
* LZO is a portable lossless data compression library written in ANSI C.
* Offers pretty fast compression and *extremely* fast decompression. One of the fastest compression and decompression algorithms around.
* Includes slower compression levels achieving a quite competitive compression ratio while still decompressing at this very high speed.

And here is the source:
http://www.oberhumer.com/opensource/lzo ... .10.tar.gz

By reading the original LZO source of "lzopack.c" we find:
- First, it expects a 7 byte magic: { 0x00, 0xe9, 0x4c, 0x5a, 0x4f, 0xff, 0x1a }
- After that it reads "flags" as an uint32 (note that LZO completely uses Little Endian)
- The "method" follows as a uint8 (must be set to "1", all other values would fail)
- Next it reads the "compression_level" also as uint8 (here it seems to only expect compression level "1")
- The "block_size" follows as uint32 (blocksize must be equal or bigger 1024 Bytes)

Then it processes the uncompressed data in blocks. Each block starts with:
- The size of the uncompressed as uint32
- The size of the compressed as uint32
Then it reads the uncompressed data, decompress it in memory and write it out to the target file.
If bit0 of the "flags" is set, it reads an additional checksum after the compressed data and also compares it with the decompressed data.

For the ULI-files, things seem to be different. One fact could be that compressed data is not available as files, and also it would not create uncompressed files, but everything is done in memory. So the uncompressor reads from Flash and writes into RAM.

After fiddling around a while i found the header of the "plattapp.uli" is like this:

Code: Select all

Addr         Size   Meaning
0x0000	0x04	Magic "ULI "
0x0004	0x04	???
0x0008	0x04	Number of parts in file (1...)
----------------- part 1 --------------------
0x000C	0x04	??? index ???
0x0010	0x04	Target memory address to store uncompressed data
0x0014	0x04	Offset to compressed data in file
0x0018	0x04	Size of uncompressed data
0x001C	0x04	Size of compressed data
0x0020	0x04	Checksum of uncompressed data
----------------- part 2 --------------------
0x0024	0x04	??? index ???
0x0028	0x04	Target memory address to store uncompressed data
0x002C	0x04	Offset to compressed data in file
0x0030	0x04	Size of uncompressed data
0x0034	0x04	Size of compressed data
0x0038	0x04	Checksum of uncompressed data
----------------- part 3 --------------------
0x003C	0x04	??? index ???
0x0040	0x04	Target memory address to store uncompressed data
0x0044	0x04	Offset to compressed data in file
0x0048	0x04	Size of uncompressed data
0x004C	0x04	Size of compressed data
0x0050	0x04	Checksum of uncompressed data

So the smaller ULI files (e.g. plattreg.uli) only contains one part and uint32 at 0x000C of them is always 0x0000_0000. Also the target-memory address there is always 0x0000_0000 because they may not be loaded and executed in RAM. So whatever this extra entry is meant for, it has no use for the decompression as it don't point to any usefull.