SoftReference problem in Android

Looks like Android VM does not honor the soft reference properly.

To improve the UI performance, I had written a two level cache for our Android and J2ME app. One cache stores the downloaded data (mostly images) to disk, I call it persistent cache. Another cache keeps the Bitmaps in heap, intuitively called runtime image cache.

When an UI element needs an image, app first checks in the runtime cache, if not found, checks the persistent cache (and puts that to runtime cache when found). If it still does not find, it will download it from server. LRU item is automatically removed after caches are full. Cache becomes full when item count reaches some predefined limit.

I had written this first for j2me apps (and reused it in android project), and used hard references. That worked nicely.. until some large images started coming. Bitmaps were too huge and it pretty soon filled up the heap.

So I thought I would use SoftReference for Android and WeakReference for J2ME implementations (unfortunately CLDC does not support soft reference). When I quickly wrote few test codes, my cache started behaving very weirdly. UI elements almost never found the images from runtime cache... cache was totally pointless!!

Looks like dalvik has some bug. It removes the soft references even when there is plenty of memory. Well, this is not really a bug, because it is quite legal to remove the soft references whenever, there cannot be any guarantee, however, the expected general behavior is that soft references should not be cleared unless there is pressure from heap usage. Dalvik developers seem to be aware of this issue and it still exists in 2.3.4. http://code.google.com/p/android/issues/detail?id=20015

After digging a bit more, I found that when dalvik intends to collect 'some' soft references, in reality it removes all. So basically, either all soft references are collected or nothing is collected during mark and sweep.

Now I am implementing some logic to cache so that it can limit the items by heap usage also. But would be nice to use soft reference in runtime cache, and I would be relaxed knowing whenever heap is running low, images will be cleared from cache automatically.