{"id":314,"date":"2012-07-06T11:07:00","date_gmt":"2012-07-06T09:07:00","guid":{"rendered":"https:\/\/www.fussylogic.co.uk\/blog\/?p=314"},"modified":"2012-08-25T14:36:34","modified_gmt":"2012-08-25T13:36:34","slug":"android-spring-clean","status":"publish","type":"post","link":"https:\/\/www.fussylogic.co.uk\/blog\/?p=314","title":{"rendered":"Android Spring Clean"},"content":{"rendered":"<p>For some insane reason, Google seem to think that the market app should get constantly larger and larger with every update. In essence, it\u00e2\u20ac\u2122s a customised web browser \u00e2\u20ac\u201d it\u00e2\u20ac\u2122s making requests to a server and displaying the appropriate page. Why on earth it\u00e2\u20ac\u2122s twice the size of an actual browser, I do not know.<\/p>\n<p>Here\u00e2\u20ac\u2122s the top five space users on my HTC Desire<\/p>\n<ul>\n<li>Google Play Store: 9.24MB (9.03MB application; 220kB data)<\/li>\n<li>k9-mail: 6.75MB (4.09MB application; 2.67MB data)<\/li>\n<li>Google+ : 5.95MB (5.4MB application; 564kB data)<\/li>\n<li>Browser: 5.84MB (348kB application; 5.5MB data)<\/li>\n<li>Maps: 5.09MB (3.95MB application; 1.95MB data)<\/li>\n<\/ul>\n<p>I am willing to forgive the mail, the browser and the maps \u00e2\u20ac\u201d they actually do have a good reason to use lots of data. I don\u00e2\u20ac\u2122t even use google+ on my phone; and what in the name of all that is holy does google play need 9MB for?<\/p>\n<p>All it does is make a request to the app server, show the list then download the APK file for installation (an APK being the file that represents an Android App package). I remember when I got the phone the Market app (which is what this Play Store used to be called) was a few MB, and even that was excessive.<\/p>\n<p>Normally a large app you can simply not upgrade, or not use. You don\u00e2\u20ac\u2122t get the choice with the market app \u00e2\u20ac\u201d it forcibly upgrades itself and cannot be uninstalled.<\/p>\n<p>The HTC desire\u00e2\u20ac\u2122s main fault is that the internal phone storage is very small; and the nature of Android is that it stores apps on the internal storage (don\u00e2\u20ac\u2122t believe the lies about moving to SD card, Android makes a cached, precompiled version of every app and stores it on the internal storage).<\/p>\n<p>Annoyingly, all this wastefulness has pushed my phone into the \u00e2\u20ac\u0153less than 15MB available\u00e2\u20ac\u009d state. At which point, Google Talk stops working. DAMNIT. So, spring cleaning time. I\u00e2\u20ac\u2122ve got a rooted phone thanks to CyanogenMod so there is scope for a bit more space finding than is normally possible. This article is my notes of what I\u00e2\u20ac\u2122ve done.<\/p>\n<hr \/>\n<p>Let\u00e2\u20ac\u2122s look at the state of play to start. I\u00e2\u20ac\u2122ve got the Android SDK installed which gives me access to the <code>adb<\/code> program, and hence <code>adb shell<\/code>; I\u00e2\u20ac\u2122ve also activated <em>USB debugging<\/em> in the Applications\u00e2\u20ac\u201dDevelopment menu. I\u00e2\u20ac\u2122ve run that to get a shell on the USB-connected phone (it\u00e2\u20ac\u2122s pretty cool that one can do this to a phone). Everything I\u00e2\u20ac\u2122ll do will therefore be at the android command line.<\/p>\n<pre><code># df -h\nFilesystem                Size      Used Available Use% Mounted on\ntmpfs                   202.8M     32.0K    202.7M   0% \/dev\ntmpfs                   202.8M         0    202.8M   0% \/mnt\/asec\ntmpfs                   202.8M         0    202.8M   0% \/mnt\/obb\n\/dev\/block\/mtdblock3    250.0M    170.3M     79.7M  68% \/system\n\/dev\/block\/mtdblock5    147.6M    136.3M     11.3M  92% \/data\n\/dev\/block\/mtdblock4     40.0M     13.7M     26.3M  34% \/cache\n\/dev\/block\/vold\/179:1    14.8G     12.8G      2.0G  86% \/mnt\/sdcard\n<\/code><\/pre>\n<p>The important partitions are clear here. <code>\/system<\/code> is the readonly operating system. This is the bit that gets upgraded when you do an over-the-air update. It can only (easily) be written to by the bootloader (I might resort to that). <code>\/data<\/code> is the read-write internal phone partition, this is what shows as \u00e2\u20ac\u0153phone memory\u00e2\u20ac\u009d in the application manager. You can see I\u00e2\u20ac\u2122ve got 11.4M free; hence the annoying \u00e2\u20ac\u0153phone storage space is getting low\u00e2\u20ac\u009d message. <code>\/cache<\/code> is also read-writable but isn\u00e2\u20ac\u2122t considered available for any kind of permanent storage. I think HTC made a mistake with that choice. It would be far more flexible to have combined <code>data<\/code> and <code>cache<\/code> on one partition to maximise space. You can see that I\u00e2\u20ac\u2122ve got another 26.3M free on <code>cache<\/code> going completely unused. <code>\/mnt\/sdcard<\/code> is the SD card (unsurprisingly), it\u00e2\u20ac\u2122s not much use to use for my purposes here, it\u00e2\u20ac\u2122s holding a lot of the uncompiled versions of apps I have installed so is saving a lot of internal space, but that\u00e2\u20ac\u2122s about all it\u00e2\u20ac\u2122s good for (in this context), so we\u00e2\u20ac\u2122ll have to forget about that 2G that would have solved all our problems.<\/p>\n<p>Let\u00e2\u20ac\u2122s go poking in <code>\/data<\/code> then\u00e2\u20ac\u00a6<\/p>\n<pre><code># du \/data -d 1 -h\n17.0K   \/data\/tombstones\n85.5K   \/data\/app-private\n2.0K    \/data\/dontpanic\n85.4M   \/data\/dalvik-cache\n9.9M    \/data\/app\n4.0K    \/data\/secure\n5.5K    \/data\/property\n13.5K   \/data\/backup\n6.0K    \/data\/local\n34.4M   \/data\/data\n50.5K   \/data\/anr\n53.5K   \/data\/misc\n343.5K  \/data\/system\n2.0K    \/data\/lost+found\n130.3M  \/data\n<\/code><\/pre>\n<p>The major users of space then are <code>\/data\/dalvik-cache<\/code> and <code>\/data\/data<\/code>. Dalvik is the Android Java engine, and <code>dalvik-cache<\/code> is where Android stores the compiled versions of your installed apps. When you download an APK you\u00e2\u20ac\u2122re getting the Java Bytecode form of an application; the first time you run it it is compiled by Dalvik into a native form (they have a <code>.dex<\/code> extension, so I\u00e2\u20ac\u2122ll call them DEX files). To save time the next time you want to run that application it makes and stores a copy of that compiled version. Here\u00e2\u20ac\u2122s a huge design fault in Android:<\/p>\n<ul>\n<li>The app won\u00e2\u20ac\u2122t run if the APK isn\u00e2\u20ac\u2122t present. That is to say that any app stored on the SD card won\u00e2\u20ac\u2122t run when the SD card isn\u00e2\u20ac\u2122t mounted.<\/li>\n<li>The DEX file is <em>always<\/em> stored on internal phone memory, regardless of where the APK is.<\/li>\n<\/ul>\n<p>Why couldn\u00e2\u20ac\u2122t the DEX version be stored right next to the APK? So if the APK is on SD card, so is the compiled version. Further, why can\u00e2\u20ac\u2122t I trade the speed increase from permanently keeping these DEX files gives for the space they take up? A little tick box in the manage applications page to say \u00e2\u20ac\u0153hey, I don\u00e2\u20ac\u2122t use this app often, I\u00e2\u20ac\u2122d rather have the space\u00e2\u20ac\u009d. Finally, since Android is calling this a <code>cache<\/code> why isn\u00e2\u20ac\u2122t it managed like a cache \u00e2\u20ac\u201d temporary files, oldest unused get sacrificed; and maybe store it on the <em>damned cache partition<\/em>?<\/p>\n<p>So.. that\u00e2\u20ac\u2122s exactly what I\u00e2\u20ac\u2122ll do. That 23M on the <code>\/cache<\/code> partition can have some of the crap from <code>\/data\/dalvik-cache<\/code>. Note: deleting them would do us no good, as Android will simply remake them exactly where they already are, so any space gain would be very temporary.<\/p>\n<pre><code># ls \/data\/dalvik-cache -s | sort -n | tail\n  1768 system@app@com.fsck.k9-f2d27042983a87ee3f098df3696fb5bf.apk@classes.dex\n  1874 data@app@com.fsck.k9-1.apk@classes.dex\n  2290 mnt@asec@com.bskyb.android.skyplus-2@pkg.apk@classes.dex\n  2301 system@app@Gmail.apk@classes.dex\n  3612 mnt@asec@piuk.blockchain-1@pkg.apk@classes.dex\n  4042 data@app@com.android.vending-1.apk@classes.dex\n  4047 system@app@Maps.apk@classes.dex\n  4422 system@app@Vending.apk@classes.dex\n  4686 system@framework@core.jar@classes.dex\n  7865 system@framework@framework.jar@classes.dex\n<\/code><\/pre>\n<p>It should be no surprise at all to see all the big apps in this list (although they\u00e2\u20ac\u2122ve got weird internal filenames instead of the ones we\u00e2\u20ac\u2122re used to). What I\u00e2\u20ac\u2122m going to do is move some of them to <code>\/cache\/dalvik-cache<\/code> and leave symbolic links in their place. That is, essentially, what Android itself should be doing. (I\u00e2\u20ac\u2122ve actually already done this before, so my <code>\/cache\/<\/code> partition probably has a little less space than yours).<\/p>\n<pre><code># mkdir \/cache\/dalvik-cache\n<\/code><\/pre>\n<p>I think I\u00e2\u20ac\u2122ll leave the <code>framework<\/code> items. To avoid busting something fundamental. Two steps for each app: move the <code>dex<\/code> to <code>\/cache\/<\/code>, then create a symbolic link from that new location back to the original location.<\/p>\n<p>I\u00e2\u20ac\u2122m using a variable to store the filename for clarity and speed of typing:<\/p>\n<pre><code># export DEX=system@app@Vending.apk@classes.dex\n# mv \/data\/dalvik-cache\/$DEX \/cache\/dalvik-cache\/\n# ln -s \/cache\/dalvik-cache\/$DEX \/data\/dalvik-cache\/\n<\/code><\/pre>\n<p>You repeat this process for every DEX you want to move, changing <code>$DEX<\/code> each time. Here\u00e2\u20ac\u2122s what I ended up with:<\/p>\n<pre><code># ls \/data\/dalvik-cache -s | sort -n | tail\n  1646 mnt@asec@com.zegoggles.smssync-2@pkg.apk@classes.dex\n  1656 mnt@asec@com.imdb.mobile-1@pkg.apk@classes.dex\n  1706 system@app@GoogleDocs.apk@classes.dex\n  1768 mnt@asec@com.fsck.k9-1@pkg.apk@classes.dex\n  1768 system@app@com.fsck.k9-f2d27042983a87ee3f098df3696fb5bf.apk@classes.dex\n  1874 data@app@com.fsck.k9-1.apk@classes.dex\n  2290 mnt@asec@com.bskyb.android.skyplus-2@pkg.apk@classes.dex\n  2301 system@app@Gmail.apk@classes.dex\n  4686 system@framework@core.jar@classes.dex\n  7865 system@framework@framework.jar@classes.dex\n<\/code><\/pre>\n<p>How are we looking now:<\/p>\n<pre><code># df -h\nFilesystem                Size      Used Available Use% Mounted on\ntmpfs                   202.8M     32.0K    202.7M   0% \/dev\ntmpfs                   202.8M         0    202.8M   0% \/mnt\/asec\ntmpfs                   202.8M         0    202.8M   0% \/mnt\/obb\n\/dev\/block\/mtdblock3    250.0M    170.3M     79.7M  68% \/system\n\/dev\/block\/mtdblock5    147.6M    124.5M     23.1M  84% \/data\n\/dev\/block\/mtdblock4     40.0M     29.4M     10.6M  74% \/cache\n\/dev\/block\/vold\/179:1    14.8G     12.8G      2.0G  86% \/mnt\/sdcard\n<\/code><\/pre>\n<p>Better. The <code>\/data<\/code> and <code>\/cache<\/code> free space has reversed. That\u00e2\u20ac\u2122ll do me for now. I\u00e2\u20ac\u2122ve got a working system again. My temptation for the future will be to do what I said Android should be doing: put the DEX next to the APK. Then the appropriate files will be on the SD card.<\/p>\n<p>A quick check shows me that the moved apps are all still working and that phone is reporting 22.95MB free.<\/p>\n<p>There is another option, which I won\u00e2\u20ac\u2122t do this time: moving some applications to the <code>\/system<\/code> partition. That\u00e2\u20ac\u2122s a bit tougher though because we would have to mess around with bootloader scripts. I\u00e2\u20ac\u2122m satisfied for now, and I\u00e2\u20ac\u2122ve left enough space for Google to bloat that damned market app a bit more without killing my phone again.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>For some insane reason, Google seem to think that the market app should get constantly larger and larger with every update. In essence, it\u00e2\u20ac\u2122s a customised web browser \u00e2\u20ac\u201d it\u00e2\u20ac\u2122s making requests to a server and displaying the appropriate page. Why on earth it\u00e2\u20ac\u2122s twice the size of an actual browser, I do not know.\u2026 <span class=\"read-more\"><a href=\"https:\/\/www.fussylogic.co.uk\/blog\/?p=314\">Read More &raquo;<\/a><\/span><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[1],"tags":[9,6],"_links":{"self":[{"href":"https:\/\/www.fussylogic.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/posts\/314"}],"collection":[{"href":"https:\/\/www.fussylogic.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.fussylogic.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.fussylogic.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.fussylogic.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=314"}],"version-history":[{"count":2,"href":"https:\/\/www.fussylogic.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/posts\/314\/revisions"}],"predecessor-version":[{"id":316,"href":"https:\/\/www.fussylogic.co.uk\/blog\/index.php?rest_route=\/wp\/v2\/posts\/314\/revisions\/316"}],"wp:attachment":[{"href":"https:\/\/www.fussylogic.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=314"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.fussylogic.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=314"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.fussylogic.co.uk\/blog\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=314"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}