Android application backwards compatibility

11th
Mar. × ’10

Say you are an Android application developer. You have a last-generation Nexus or Droid phone, and you use for development a recent Android SDK version 2.1. Yet more than a half of your potential users have older phones with older versions of Android: 1.5 (Cupcake) or 1.6 (Donut).

You can simply ignore the not-up-to-date user base, and write your application exclusively for Android 2.0 (Eclair) or more. For this you set minSdkVersion=5 in your AndroidManifest, and you’re done. This is the simplest solution from the developer point of view, but also the worst solution from the user point of view (since half the users don’t have access to the application at all).

Or you can think about making your application backwards compatible, so that it can still be used on Cupcake and Donut. How to do this?

First, you need to look carefully at the API version for the SDK classes and methods that you use. This is indicated in the Android SDK documentation, on the right. For example looking at MotionEvent.getPointerCount() you read Since API level 5, which means that this method is available starting with Eclair — it is not available in Cupcake or Donut, although the class MotionEvent itself is available.

If you mark your application minSdkVersion=3 (Cupcake) but still use the method MotionEvent.getPointerCount(), the application will crash at runtime with a Dalvik VerificationError exception, because the method is not found. Note that you’ll only see this error at runtime and not earlier — not at compile time, and not at Market upload time. So you really have to test your application on Cupcake and Donut in order to catch these crashes. If you don’t test, users with Cupcake will download and try out the application, which will crash on them, and they’ll give you 1-star rating in exchange. Sometimes a helpful user may even drop you an email informing that your app is having a FC (forced close) on G1 with Android 1.5.

Let’s say your application wants to use multiple-touch where available, otherwise the application is still usable with single touch. For multi-touch you use, among others, the method MotionEvent.getPointerCount(). Multiple finger support was introduced in 2.0, Eclair.

You test on Cupcake and you realize that MotionEvent.getPointerCount() does not exist in Cupcake, although MotionEvent with most of the other methods is available. What to do to take advantage of getPointerCount() and multi-touch on Eclair, while not crashing on Cupcake? There is a trick related to how Dalvik loads the Java classes.

Dalvik uses a delayed class loading. The class is only verified when it is used for the first time in the application, i.e. when one of its methods or members is called/accessed. Simply referencing the class name somewhere in the code does not count as class use, and does not produce a class verification.

The second element that we use is android.os.Build.VERSION.SDK_INT. This is an integer giving the OS version installed on the phone (the same as minSdkVersion, 3 for Cupcake, 4 for Donut, 5 Eclair). We use this information to know which of the recent APIs are available and to adapt the code behavior on different OS versions depending on the available APIs.

And now you hit the API joke. By testing on Cupcake, you realize that the app crashes when accessing Build.VERSION.SDK_INT! You double-check the documentation, and indeed SDK_INT is not available on Cupcake, it was introduced in Donut. In other words, the mechanism for version-checking itself is not available on older versions, nice eh?

The workaround is to use Build.VERSION.SDK, which is a String containing the same integer as SDK_INT (e.g. “3″ on Cupcake), and this is available on Cupcake. You wonder why some developer thought that using a String for storing an integer is a good API choice… an API mistake, in other words.

Anyway, let’s go back to the code.

class WrapNew {
    static int getPointerCount(MotionEvent event) {
        return event.getPointerCount();
    }
}

You create a class that wraps the calls to the new Eclair API. Instead of calling the MotionEvent.getPointerCount() directly, you call through the wrapper class. You only call the new method after you verified that the Build.VERSION.SDK is hight enough that the new API exists.

public boolean onTouchEvent(MotionEvent event) {
    boolean hasMultiTouch = Integer.parseInt(Build.VERSION.SDK) >= 5;
    int nPointer = hasMultiTouch ? WrapNew.getPointerCount(event) : 1;
    // ...
}

The Dalvik trick is: as long as we don’t call WrapNew.getPointerCount(), the WrapNew class is not loaded by Dalvik and does not generate a VerificationError by referencing the non-existing API MotionEvent.getPointerCount().

You test again the application on Cupcake, and it does not crash anymore with VerificationError! It also supports multi-touch on Eclair, and the solution is pretty clean and simple code.

One more note: if you use Proguard to optimize your Android application, be sure to mark the wrapper class as non-inlinable, because otherwise Proguard would optimize it out of existence, denying it’s purpose.
-keep ,allowobfuscation class package.WrapNew { *; }

You can see all this in practice, real-life working code, in the Arity Calculator application, which is Open Source so you can freely explore the code. Arity Calculator is available on Market on Android phones, for Cupcake, Donut, Eclair and on.

And here is the missing table from the Android documentation:

Codename   Version   API level 
Cupcake 1.5 3
Donut 1.6 4
Eclair 2.0 5
Eclair update 2.0.1 6
Eclair MR1 2.1 7
Posted in Uncategorized | Leave a comment

The free AppEngine: how much does it cost?

17th
Feb. × ’10

AppEngine is a Google product that allows to write web applications in Python or Java and host them on the Google infrastructure. AppEngine takes care of the typically difficult tasks of distribution, scalability and fail-over. It offers easy to use APIs for web request handling, data storage, memcache, etc. One of the biggest benefits is the integration with Google accounts — a user simply signs in to an AppEngine application with their existing Google account.

Perhaps prototyping a web application, together with hosting/publishing, has never been as easy as with AppEngine. Add to that the Python language, and it’s a lean-mean web application machine.

AppEngine is free up to some quotas. If the application is successful and you need more resources, you have to pay. Here is the table of the free quota, and the incremental cost:

Free quota per day Cost above quota
CPU Time 6.50 CPU hours $0.10/CPU hour
Bandwidth Out 1.00 GBytes $0.12/GByte
Bandwidth In 1.00 GBytes $0.10/GByte
Stored Data 1.00 GBytes $0.005/GByte-day
Recipients Emailed 2,000.00 Emails $0.0001/Email



The question is, how much would it cost if there were no free quota, and you’d have to pay for the free quota at the same incremental cost. In other words, what is the dollar value of the quota that Google is offering for free?

Per day: CPU $0.65, Bandwidth Out $0.12, Bandwidth In $0.1, Storage $0.005, Emails $0.2. Summing them comes to $1.07 / day.

So the “cost” of the free quota on AppEngine is about $1/day/application. Consider that Google is offering up to 10 free applications per account, so the cost could be argued is $10/day/account.

Posted in Uncategorized | Leave a comment

Nexus One display and subpixel pattern

7th
Jan. × ’10

According to the Nexus One specs, it has a 800×480 AMOLED display.

According to Wikipedia, this is what makes an AMOLED display different from standard LCD:

  • It does not need a backlight, the pixels emit light themselves. This allows for thinner display.
  • Only the turned-on pixels consume power. In average AMOLED uses less power than LCD. The power usage is dependent on the color displayed — a black screen uses much less power than a white screen.
  • The contrast is very good. The black is very deep (as there’s no backlight leakage).
  • The viewable angle is very large; this is often a problem with LCD, particularly TN (twisted-nematic), which have small viewing angles.
  • OLED has better reponse rate than LCD.
  • The lifetime of OLED is shorter than LCD. Only the period a pixel is lightened (turned-on) counts towards the lifetime. The blue subpixels have the shortest lifetime.

Glossing over the technical details, the result is that the Nexus One has a *great* display. It is the highest quality display I’ve ever seen (but I can’t say how it compares to the Droid display as I have not seen a Droid phone yet). The Nexus display is large and high-density, is very crisp, the colors are very saturated, and the black is indeed very deep. WVGA, 800×480 is quite a high resolution for a display of this size (a more typical resolution for this size would be HVGA, 480×320 as on the G1).

After getting used to the Nexus display, when you look back to your previous phone’s display you’ll be shocked by how low-quality the old LCD seems now.

The AMOLED display on Nexus has one more surprise on hold: On Nexus, each pixel is composed of only two subpixels instead of the usual three (Red, Green, Blue) subpixels of an LCD. The following picture should help understand the subpixel pattern on a Nexus: (source)

OLED subpixel matrix

So every pixel contains a green subpixel, and alternating a red or a blue subpixel. A pixel has either red or blue subpixels but not both.

Why is it done this way? This technique allows for a larger physical area for the blue and red subpixels, thus increasing their lifetime. It also allows to implement the high pixel count (800×480) by using only two thirds of the subpixels that would normally be needed (2subpixels/pixel instead of 3subpixels/pixel).

What is the impact of this unusual subpixel pattern? Natural images (such as pictures, movies etc) are very well reproduced and likely the subpixel impact is not perceivable. Synthetic images that contain pixel-aligned thin and saturated lines (red or blue) allow to discern the “alternating” subpixel pattern. But as the display is very high density and the pixels are very small, most likely you would never become aware of the subpixel pattern if not looking explicitly for it.

What is the impact of AMOLED for application developers:

  • First of all, you may not care at all about AMOLED vs LCD, and everything will work just fine.
  • You can use darker colors to save power. You may prefer white-text-on-black-background to black-on-white-background, as the dominant black color uses less power on AMOLED. (the dominant color makes no power difference on classical LCD).
  • You may prefer using darker colors to increase the display lifetime. You may prefer avoiding displaying intense blue (as it has the lowest lifetime) for long periods of time.
  • If you want to draw highest-resolution thin lines, they are best rendered in green — because the green subpixels have double the resolution of red and blue subpixels.

It seems the choice of subpixel pattern is also related to human eye physiology, as the human eye is more sensible to green than to blue, and keeping green at full resolution gives rich image information to the eye.

Posted in Uncategorized | 15 Comments

Android package name

16th
Dec. × ’09

There are two independent notions of “package” in Android. One is the usual Java package, which is how the Java classes are organized inside your project. The other is the Application package, which is an unique application identifier used by Android to manage the installed applications. The two package names are completely independent and serve different purposes.

Java package

The Java package organizes the classes inside the application. Usually you have multiple Java packages in an application. You may put your own classes in a package that’s specific to your app (e.g. “com.domain.myapp”) and you keep the library classes that your project uses in their predefined Java package (e.g. “org.apache.lucene”). The Java packages in your app are mostly the internal business of your app and have little visibility outside of the application — they are visible only for the classes that are exposed to the system such as Activities, Views, Services etc you implement and specify in your app’s AndroidManifest.

What’s more, the Java packages of an application are local to that app. There can be multiple independent applications that implement the same class in the same package, and there is no conflict whatsoever.

The Java package is always scoped inside the application. Two Java packages with the same name (e.g. “org.apache.lucene”) located in different apps are in fact different packages and there is no identity or relation between them. This allows applications to use well-known libraries (such as our lucene example) without any possible interference with other apps that may use the same package/classes (or different versions of). The drawback is that there is duplicated code if many apps use the same classes over again (there is no reuse in-between the apps based on the Java package name).

The observation that on Android the Java package is local to the application means that we don’t have to worry about package name conflicts with other applications. Avoiding package name conflicts with unforeseen third-party apps was the main reason for the Java package name format “com.mydomain.myapp”, i.e. prefixing with your domain name avoids conflict — but there’s no possible conflict on Android to avoid.

This leads to the possibility of using liberal Java package names that violate the well-established Java convention. For example if your app is called “Calculator”, you may place classes in the Java package “calculator” instead of “com.mydomain.calculator” — how cool is that? Of course, that fact that you can does not mean that you should, and respecting the established convention is a good thing even though it’s not required on Android.

Application package

The Application package declared in the AndroidManifest.xml is specific to Android and is an identifier of the application. This identifier is unique among all the apps installed on the phone at a given moment — there can’t be two apps with the same Application package installed at the same time.

The Application package is also unique on the Android Market — there can’t be two apps with the same Application package on the Market.

On the other hand it is possible for two independent developers to create two different apps with the same Application package. Of course not both apps can be hosted on the Market — the Market would reject the second one due to the “unique App package name across Market” rule.

So conflict over the Application package with unforeseen third-party apps is possible, and that’s why it is recommended to use the Java package name convention (“com.mydomain.myapp”) for the Application package name as it avoids conflict.

Some developers chose to disregard the guideline “com.mydomain.myapp” and use fancy Application package names, e.g. “marcone.toddlerlock” for the ToddlerLock application — this liberal use is likely not recommended but certainly possible.

If you use unorthodox Application package names please be sure to make them very specific in order to minimize the chance of conflict (in the previous example, the “marcone.” prefix is specific to the developer and has little chance of conflict (hint: it’s derived from the developer’s name)).

Identifying Java classes across an Android system

To identify a Java class across an Android system (a class that can be part of any application) you have to specify both the Application package and the Java package and class name. It’s like you first specify the application with the App package, and then you specify the class inside the application with the Java package and class name. This is used for example when constructing an Intent to a specific class.

Because often the Application package is the same as the Java package (i.e. the same string), it appears as if you have to specify the package name twice! This can be a bit confusing in the beginning, but it becomes clear why both package names are needed as soon as you understand the difference between the Application package and Java package.

When specifying activity names in the AndroidManifest.xml you need to indicate the Java package and class name (e.g. “com.mydomain.myapp.MyActivity”). Android also allows the short-hand form “.MyActivity”, which indicates that the Java package for the given activity class is the same as the Application package, which is often the case. Of course if your app uses different Application package and Java package, you can’t use the short-hand form and need to always use the full Java package in the manifest.

Posted in Uncategorized | 1 Comment

Android: allocation-free code avoids the GC freeze

10th
Nov. × ’09

When a Java program creates new objects (using new), it results in the subsequent invocation of the Garbage Collector to free up memory. On Android an invocation of the GC takes of an order of 100ms, so it produces a user-visible short freeze of the application. Especially when the application is doing some form of animation the freeze is visible. In addition the GC is expensive (CPU & battery).

In order to avoid the GC invocations one can use allocation-free code. This is code that allocates most of the needed objects up-front, once, and then keeps reusing the existing objects while in the animation loop. Of course, be careful to balance the pre-allocation and object reuse (which speeds things up by avoiding GC) with the size of the allocated objects which may increase the memory usage of the application. In general, do not keep large objects alive longer than necessary.

For example in a View you can create a Paint object once and keep reusing it for all the onDraw() calls, instead of creating one or more “new Paint()” for each onDraw(). And similarly for a Point or an array of Points, etc.

String is immutable and thus can’t be reused, but you can use StringBuilder instead, which is pretty similar to String and can be reused.

Of course, it is easier to avoid the object creations that happen in the application code and thus are visible (look for “new”); it is a bit harder to detect and avoid the allocations that happen in the framework code as they are not so obvious. You can use the ddms Android tool to track down the object creations done by an application.

After taking care to remove the object creations from the hot areas of code you may be surprised by how fast and smoothly the application runs.

Posted in Uncategorized | Leave a comment

Android Proguard with useful stack-traces

23rd
Oct. × ’09

As I said in my previous post about using Proguard with Android apps, the only drawback is that the stack-traces (following an application crash) become illegible. But that is not entirely true because Proguard allows you to shrink while maintaining useful stack-traces. After this finding, there isn’t really any drawback to using Proguard anymore!

The updated Proguard config I use with Android is:

-keep public class * extends android.app.Activity
-keep public class * extends android.view.View { public (...); }
-dontskipnonpubliclibraryclasses
-optimizationpasses 2
-flattenpackagehierarchy
-keepattributes SourceFile,LineNumberTable
-printmapping map.txt

The line “-keepattributes SourceFile,LineNumberTable” instructs Proguard to not strip the file name and line number information, which results in readable stack-traces. Keeping this information makes the shrinking a bit less efficient, but it still reduced my test app from 56KB to 40KB which is impressive.

Posted in Uncategorized | Leave a comment

Kreactive calculator uses the Arity library in disguise

21st
Oct. × ’09

I installed today the Kreactive Calculator from the Android Market and I realized that it uses my Arity library. As I released Arity under the Apache 2.0 License (a very permissive license), a new user of the library is usually good news. But Kreactive does not seem to be a good open-source citizen: they repackaged Arity under their own package “com.kreactive.arithmatic” instead of the original “org.javia.arity”, and they don’t acknowledge anywhere their use of the library. Basically it looks as if they imply it is their own code. Given Arity’s Apache 2.0 license, they really didn’t have to do this ’stealing’ — but I’ll take it as a compliment.

Anyway, good luck Kreactive with your Android apps, and don’t forget to be a good citizen next time!

Posted in Uncategorized | Leave a comment

Android and Proguard, the shrink recipe

20th
Oct. × ’09

Proguard is a Java shrinker and obfuscator. It reads java .jar files and writes them back with a significantly smaller size. This is achieved by various optimizations, elimination of unused code, and renaming of classes, methods and members to shorter (1-letter) names. Proguard is a good and very useful tool for general Java projects.

What about using Proguard with Android applications? Well.. it works great!

Android compiles Java source files to Java bytecode. The Java bytecode is next converted to Dalvik bytecodes by the “dx” tool. In between these two step we can apply Proguard to shrink the Java bytecodes. The result is impressive: a reduction of about 30% or more in .apk size! This results in the application downloading faster, taking up less space on the limited flash storage on the phone and starting up faster. The only drawback? it makes analyzing an application crash more difficult, as the stack trace of an exception has only the new short names and thus is pretty obscure.

I prepared this simple shell script which illustrates the commands needed for a complete Android build including proguard shrinking. It was tested with Android SDK 1.6.

#!/bin/bash

NAME=Calculator
SRCS=src/calculator/Calculator.java
#optional, if you use external.jar libraries
LIBS=libs/arity-1.3.4.jar

SDK=/path/to/android/sdk
PROGUARD=/path/to/proguard/lib/proguard.jar
KEYSTORE=/path/to/android.keystore
KEYALIAS=andkey

PLATFORM=$SDK/platforms/android-1.6/
AAPT=$PLATFORM/tools/aapt
DX=$PLATFORM/tools/dx
AJAR=$PLATFORM/android.jar
PKRES=bin/resource.ap_
OUT=$NAME-unalign.apk
ALIGNOUT=$NAME.apk

set -e #exit on error
mkdir -p bin/classes gen

$AAPT package -f -m -J gen -M AndroidManifest.xml -S res -I $AJAR -F $PKRES
javac -d bin/classes -classpath bin/classes:$LIBS -sourcepath src:gen -target 1.5 -bootclasspath $AJAR -g $SRCS
java -jar $PROGUARD -injars $LIBS:bin/classes -outjar bin/obfuscated.jar -libraryjars $AJAR @proguard.txt
$DX --dex --output=bin/classes.dex bin/obfuscated.jar
apkbuilder bin/$OUT -u -z $PKRES -f bin/classes.dex
jarsigner -keystore $KEYSTORE bin/$OUT $KEYALIAS
zipalign -f 4 bin/$OUT bin/$ALIGNOUT

The contente of the file “proguard.txt” referenced above, which contains the Proguard options, is:

-keep public class * extends android.app.Activity
-dontskipnonpubliclibraryclasses
-optimizationpasses 2
-printmapping map.txt
-flattenpackagehierarchy

Treat this script more like a guideline to the series of commands needed for a full android build, and adapt it to your needs. Disclaimer: use at your own risk and re-test your proguarded apk thoroughly to find any problems introduced by shrinking.

Building a shrinked, optimized, signed and zipaligned .apk is only useful for the release version of an application. For debug builds where size and optimization is not important you should use the normal Android build without proguard.

Given the great usefulness of applying Proguard on Android apps, I would really like to see Proguard integrated with a future version of the Android SDK — that would assure a much wider adoption of apk shrinking.

As an example, Proguard shrinked my already tiny application Arity Calculator from 45KB to 30KB.. Enjoy Proguard on Android!

Posted in Uncategorized | 4 Comments

Synergetic Stupidity in Java

21st
Jan. × ’09

In the Java “class” file format specification, the length of the bytecode of any method is limited to 64KB. This arbitrary limitation is bad in itself, but see below what it does in conjunction with the other stupider Java ‘feature’.

In any normal language, a literal array constant (used for array initialization) is represented as a compact block of data. For example when initializing an array with 1000 byte values, you’d expect the initialization data to take up 1KB (1000 x 1B). But not so in Java, not at all.

Let’s consider a long array of bytes, initialized with some data.
static final byte B[] = {10, 20, 30, 40, 50, 60, 70, 80, …};

In the “class” format there is no provision for storing the array initialization data in a compact format. Instead, bytecode instructions are generated for filling-in the array, a small block of bytecodes for each entry of the array data:

dup          //1 byte; duplicate the array address on the stack
sipush 300 //3 bytes; the array index where to store the value
bipush 99   //2 bytes; the value to store
bastore      //1 byte; do the store

This adds up to 7 bytes for each entry of the array. So if you initialize an array of 5000 bytes, this would generate 35KB of bytecode instead of the 5KB that would normally be enough to store the data.

For an array of int, float or larger types, the bytecode structure is a bit different, in that the integer value is not inlined in the bytecode, but instead read from a constant pool.

dup          //1 byte
sipush 300 //3 bytes; the array index
ldc_w  400 //3 bytes; load int from constant pool position
iastore      //1 byte

Let’s put in a small table the amount of data Java uses for initializing arrays of different types:

Type Bytes / entry Stupidity boost
byte[] 7 7 x
short[] 8 4 x
int[], float[] 8 + 4 3 x
long[] 8 + 8 2 x

This form of doing array initialization is in my opinion the stupidest miss of the “class” file format.

But lets see where Stupidity Synergy comes into play. Synergy means that sometimes, when you put together one stupid feature with another stupid feature, you get a result that is larger than the sum of the parts. Java achieves synergy by combining the 64KB method size limit with the peculiar way of compiling the array initialization.

As we’ve seen, bytecode (i.e. method code) is needed for any array initialization. Even initializing a static array at the class level produces an invisible method that contains the code for array initialization. Combining this with the method size limit produces very restrictive limits on the size of array initializations.

Type Array initialization size limit
byte 9K
short 8K
int, float 8K

Nice powerful language, where you get a “code too large” compilation error when you try to initialize a byte array with more than 9K entries…

Posted in Uncategorized | 3 Comments

Mobile Monday Warsaw

28th
May. × ’08

I just participated to the first MoMo event to take place in Poland, Mobile Monday Warsaw. There were about eight presentations, of which I most enjoyed the NFC talk by Florian Resatsch from Servtag, and the MRIA (Mobile Rich Internet Application) talk by Alex Nerst of fring (both are start-up companies). The event was well organized, and represented a great opportunity to introduce a new product or make some local industry contacts. It seems MoMo will happen again in Warsaw in the coming months, and hopefully it will also get adopted in other Polish cities (like Krakow).

Congratulations to the organizers!

Posted in Uncategorized | Leave a comment