Let’s consider the clamp of an integer to the interval [0, 255].

int clamp(int x) {
    return x<0 ? 0 : x > 255 ? 255 : x;
}

JPEG decoding requires three such clamp operations per decoded pixel, so we’d like to find a fast way to do it. (I’m mainly concerned with the ARM architecture as often found on Android).

A faster way to do it is:

int clamp(int x) {
    return !(x & 0xffffff00) ? x : ((~x >> 31) & 0xff);
}

The ~x>>31 works because it’s an ASR, Arithmetic Shift Right, which feeds from the left copies of the sign bit, not zero bits.

An even faster way to do it, as discussed here, is:

int clamp(int x) {
    unsigned y;
    return !(y=x>>8) ? x : (0xff ^ (y>>24));
}

Of course, you wouldn’t like to pay the price of a function call for such a tiny operation, so let’s make the clamp() inlinable by declaring it static inline.

static inline int clamp(int x) {
    unsigned y;
    return !(y=x>>8) ? x : (0xff ^ (y>>24));
}

That’s about as fast as it gets while staying at the C level. Perhaps we can make it even faster by using some cool ARM instruction?

The USAT (Unsigned SATurate) ARM instruction does exactly what we want, and here’s how to put it cleanly in code:

static inline int clamp(int x) {
    int ret;
    asm("USAT %0, #8, %1": "=r" (ret): "r" (x));
    return ret;
}

This way, we cut the clamp() down to a single processor instruction! (yet losing the portability of the pure-C code).

Anything faster? it turns out, the USAT can do at the same time (in the same instruction) an optional shift left/right (ASR or LSL). So if it happens that you anyway needed a fixed shift before the clamp(), you can integrate the two in a shiftAndClamp(), and this way you achieve both the shift and the clamp with a single instruction. We could say that the clamp() alone now takes less than one instruction

It looks like the initial task was too hard (nobody was solving it), and after the give-away hint it become too easy. Let’s look forward to the next challenges.

PS: in task #1, each android represents a word, making up “what did j cook call hawaii”, and the answer is “the sandwich islands”.

We have two 4-byte values representing two pixels, in format RGBA_8888. Let’s ignore the Alpha channel, assuming both have alpha=0xff (opaque). We want to produce a new 4-byte value with the per-channel average of the two pixels.

Below we see the byte structure of the two pixels and of the desired average:
pixel1: r1|g1|b1|a1
pixel2: r2|g2|b2|a2
average: (r1+r2)/2 | (g1+g2)/2 | (b1+b2)/2 | (a1+a2)/2

To implement this operation in C, we’d need to extract the individual bytes (R,G,B,A) of each pixel, average them (four additions and four shifts) and finally recompose the bytes to form the result (e.g. with shifts and ORs).

It turns out there is a much faster way to do it, which takes exactly.. one CPU instruction!

The magic instruction is UHADD8, which adds the corresponding bytes of two registers, in pairs, halves the result and recomposes the bytes — exactly the operation we needed for our RGBA average.

But how to access this ARM instruction from C++? It turns out, again, that there is a clean and simple way to do it:

extern inline unsigned averageBytes(unsigned a, unsigned b) {
    unsigned ret;
    asm("uhadd8 %0, %1, %2": "=r" (ret): "r" (a), "r" (b));
    return ret;
}

GCC is smart enough that invoking this inline function compiles down to just one CPU instruction (the UHADD8) without any other overhead.

So, problem solved: we have this neat C++ function for averaging two RGBA values, with a total cost of just one CPU instruction!

What I needed to do is convert image pixel data from RGB to RGBA format — from 3bytes/pixel to 4bytes/pixel; fullscreen in real time during animation. But the general situation is anytime you need to process a large amount of RAM data really fast.

while (n--) {
    *dest++ = *src++;
}

This loop is plain, it just copies data from a memory source to a destination. It is used here just a as a placeholder for some processing of src data. (of course, if you only need to copy the data you should use memcpy instead)

Let’s time this loop over 1MB of data, on a Samsung Galaxy Tab 10.1 with a Tegra 2 processor — it takes about 25ms. What slows the loop down is waiting for data that is not in the processor cache to be fetched from the main memory, which is slow. We can fix this by directing the CPU to prefetch data ahead of the read. We modify the loop adding the PLD magic line:

while (n--) {
    asm ("PLD [%0, #128]"::"r" (src));
    *dest++ = *src++;
}

That asm line starts preloading data from memory to the CPU cache, 128 bytes ahead of the current src location, without blocking the CPU.

We measure again, and the same loop over 1MB of data now takes only 8ms instead of 25ms — it is three times faster! Amazing for that 1-liner, I say. By the way, this is now very close to the performance of memcpy, which is itself implemented in highly-optimized ARM assembly.

You may observe that our loop may be optimized a little bit further by doing partial unrolling — processing more than a single element at each iteration.

With partial loop unrolling:

n /= 4; //assume it's multiple of 4
while (n--) {
    asm ("PLD [%0, #128]"::"r" (src));
    *dest++ = *src++;
    *dest++ = *src++;
    *dest++ = *src++;
    *dest++ = *src++;
}

The conclusion is that if you find yourself optimizing to death some piece of C/C++ code on Android that reads a lot of memory, you should try using the PLD and profile again to see if it helps.. Enjoy!

asm ("PLD [%0, #128]"::"r" (src));


PS:
If you’re curious about the RGB_888 to RGBA_8888 conversion speed, it is possible to do a fullscreen conversion (1280×752 px) on the Tab in about 7ms, which is quite impressive IMO. This is faster than the corresponding memcpy() RGBA to RGBA which takes about 8ms, and thus makes the case for the introduction of the RGB_888 (3bytes/pixel) Bitmap format in the Android Java API (as it saves RAM and memory bandwidth when the Alpha channel isn’t needed).

The question is what range of integers can be represented exactly by these floating point formats?
In other words, what is the maximum value for which such a statement holds:

long v;
(long)(float) v == v;
(long)(double) v == v;


float represents integers exactly up to 2^24 (16,777,216), while double represents them exactly up to 2^53 (9,007,199,254,740,992). These numbers are consistent with the mantissa size being 23bits for single precision and 52bits for double precision.

I find the patent situation in the US profoundly sick. Microsoft is making money on the back of Android with patent fees.
Apple and Microsoft team up together to buy the Nortel patents for 4.5billion, to be used against competitor Android. Apple suing HTC and Motorola over Android. Oracle suing Google over Java patents used in Android.

Really, the software patents as they are used now in the US are a cancer. They are not good in any way, and most of all they do not promote innovation — quite the opposite, they clearly are used to stifle innovation when it happens.

The real fix is to abolish the software patents in the US. I deeply hope this will happen, but I don’t quite hold my breath for it.

In the meantime, the least I can do is vote with my money, as a humble consumer. I’ll never buy an Apple product again!

PS: Of course, there is quite some time since last I used any Microsoft product. But with Apple it’s new.

Instead of using the three color channels (RGB) you may chose to use only one, let’s say Red. White becomes Red, while the disabled channels (Green, Blue) become black. The colors are very weird, but the screen remains sort-of readable in general. Why would anybody want to do something so horrible to a display?

The first reason is to save power. A phone’s display uses a significant portion of the battery. For OLED displays, making pixels (or subpixels) black saves power. Turning color channels off saves power. For example using only the Red channel instead of all three RGB cuts the display power to less than 50%.

Automatic Brightness

Android phones have a Display Brightness setting, which is usually set to “Automatic Brightness”. This reduces the brightness of the display in dark indoors (when the screen is easy to read), while increasing the brightness in sunny outdoors, when the screen is barely readable.

On OLED displays, which have deep blacks, increased brightness usually looks better. So why there exists this adaptive brightness at all, instead of keeping the display simply set to maximum brightness all the time?
Mainly for two reasons:

• lower brightness saves battery power
• lower brightness increases the display lifetime (for OLED).

Yes, that’s right: the “brightness” setting is not there to improve the visual quality — it is there to increase the battery life. Try setting your phone to “maximum brightness” for one day, it’ll look great but will empty the battery much faster.

OLED

Traditional LCD displays need a backlight, a source of white light that is filtered through the liquid cristal layer to produce the colors. The power usage of an LCD display is independent of the colors displayed (e.g. redardless whether the screen is white, red or blue, the power usage of the display is the same).

OLED displays do not have a backlight. In OLED the pixels themselves emit light, and each individual subpixel uses power in doing so. OLED power usage depends on the colors displayed. A white screen (which has all subpixels Red, Green, Blue lighten up at maximum intensity) uses the most power, while a black screen uses almost no power at all.

In addition, OLEDs have a general difficulty with Blue. The Blue subpixels use much more power (compared to the Red or Green channels). In addition to the increased power usage (or because of it) the Blue subpixels have a much shorter lifetime. If the OLED display is kept on, bright white, for a long time, the Blue subpixels will wear off visibly and irreversibly.

So a way to increase the battery lifetime of the phone is to disable the Blue channel. Of course, this is a pretty high price to pay for power as the colors don’t look normal anymore. A more extreme is to disable two channels, e.g. Blue and Green, resulting in more battery life and a Red-only display.

But hacking is fun, and Jeff Sharkey first implemented a proof-of-concept of the color-reduction. It is implemented as an OpenGL transform in SurfaceFlinger. This means that the pixel colors undergo a final modification (e.g. zeroing-out the Blue and Green channels) just before they’re sent to the display panel. The problem is that this solution uses CPU or GPU resources (due to the additional OpenGL processing), and this means more power usage. So while some power is gained by reducing OLED colors, some power is lost because of the additional GPU usage.

Now the question presents itself: is it possible to implement this color warping, a.k.a. RenderFX, without the burden of the additional OpenGL transform? Well.. yes, there is a cool trick that does it, but it only works for OLED displays. But anyway we’re only interested in RenderFX on OLEDs (because only there it saves power).

Colored backlight

The trick concerns the backlight color. Traditional LCDs have a real backlight, a source of white light, whose intensity can be changed (more or less bright) but the color stays white.

On the other hand OLEDs do not have a real backlight. The brightness setting for OLED is not implemented by changing the backlight intensity, but simply by remapping the intensity of the pixels in the display controller. You can think of an OLED backlight, but that’s an imaginary concept simulated by the display controller.

It turns out, the OLED panel can control the intensity of the backlight color channels (RGB) independently. This can be represented as if having a backlight which can be set to any color and intensity, not only grayscale. And this is the solution: by setting the backlight to various non-white colors it is possible to implement RenderFX without the OpenGL overhead.

For example, to disable the Blue channel, it is enough to set the backlight to Yellow (Red+Green). To disable both Blue and Green we set the backlight to Red. And so on, many intermediate variations are possible.

I tested this in practice on Nexus One with CyanogenMod by modifying the kernel file
arch/arm/mach-msm/board-mahimahi-panel.c
which implements the “backlight intensity setting” for the OLED panel, to allow for non-white backlight colors — and it works nicely!

For a general and clean solution, a bit more work is needed throughout the framework in order to expose in the API the concept of “LCD backlight color” instead of “LCD backlight intensity” as it is now. This is a generalization, as an RGB color can easily be mapped to a grayscale intensity, but not the other way around (i.e. RGB is a superset of grayscale).

Yet until yesterday I didn’t know it has an FM receiver. Yes, the Nexus One has a perfectly good functional FM receiver.

The hardware (FM radio) is there in the phone. It’s just that the software is not enabling the radio. It’s such a pity: you pay for the “hard” part, the electronics, but the “soft” part is not making any use of it.

But yesterday, when I installed CyanogenMod for the first time, suddenly the FM receiver was there, and I was shocked: FM radio? working? my phone has an FM receiver?!

CyanogenMod is a fork of the Android project. It is so much better than the “stock Android” that comes with your phone.

And CyanogenMod is open. Really OPEN. You can take the source code, compile/hack it and install it on the phone, and you get a working phone. This is unlike the AOSP (android open source project), where you can get the source code, compile it, but it won’t work on any device.

And CyanogenMod is accepting contributions from external developers. And is open to source-code change, and to improvement. CyanogenMod is putting into life the open that Google is only talking about.

So, if you have an Android phone, put CyanogenMod on it and be enthrilled! If you are a developer, and you want to propose a change/improvement to Android source code, send it to CyanogenMod for inclusion.

On the other hand.. thinking about sending a change to Google AOSP? think twice.. Most likely your change is not good enough, and anyway nobody cares — the Google android developers are too busy to bother.

Kudos to CyanogenMod for showing us all what an open Android really can be.

What’s more, the memory used by Bitmaps is included in the limit. For an application manipulating images it is pretty easy to reach this limit and get the process killed with an OOM exception:

E/dalvikvm-heap(12517): 1048576-byte external allocation too large for this process.
E/GraphicsJNI(12517): VM won't let us allocate 1048576 bytes
D/AndroidRuntime(12517): Shutting down VM
W/dalvikvm(12517): threadid=1: thread exiting with uncaught exception (group=0x4001d7f0)
E/AndroidRuntime(12517): FATAL EXCEPTION: main
E/AndroidRuntime(12517): java.lang.OutOfMemoryError: bitmap size exceeds VM budget

This limit is ridiculously low. For a device, like the Nexus One, with 512MB of physical RAM, setting the per-process memory limit for the foreground activity to only 5% of the RAM is a silly mistake. But anyway, that’s how things are and we have to live with it — i.e. find how to work around it.

There are two ways to allocate much more memory than the limit:

One way is to allocate memory from native code. Using the NDK (native development kit) and JNI, it’s possible to allocate memory from the C level (e.g. malloc/free or new/delete), and such allocations are not counted towards the 24 MB limit. It’s true, allocating memory from native code is not as convenient as from Java, but it can be used to store some large amounts of data in RAM (even image data).

Another way, which works well for images, is to use OpenGL textures — the texture memory is not counted towards the limit.

To see how much memory your app has really allocated you can use android.os.Debug.getNativeHeapAllocatedSize().

Using either of the two techniques presented above, on a Nexus One, I could easily allocate 300MB for a single foreground process — more than 10 times the default 24 MB limit.