Category: English

Backup And Restore Your Android Phone With ADB (And rsync)

Based on my previous scripts and inspired by two blog posts that I stumbled upon I tackled the “backup all my apps, settings and data” problem for my Android devices again. The “new” solutions both use 

rsync

  instead of 

adb pull

  for file transfers. They both use ADB to start a rsync daemon on the device, forward its ports to localhost and run rsync against it from your host.

Simon’s solution assumes your phone has rsync already (e.g. because you run CyanogenMod) and can become root via 

adb root

. It clones all files from the phone (minus 

/dev

 , 

/sys

 , 

/proc

  etc.). He also configures udev to start the backup automatically when the phone is plugged in.

pts solves the setup without necessarily becoming root. He also has a way of providing a rsync binary to phones that don’t have any (e.g. when running OxygenOS). He also has a few tricks on how to debug the rsync daemon setup on the phone.

I’ve tried to combine both methods. My approach doesn’t require adb or rsync to be run as root. It’ll use the the system’s rsync when available or temporarily upload and use a backup one extracted from Cyanogen OS (for my OnePlus One). Android won’t allow you to 

chmod +x

a file uploaded to 

/sdcard

, but in 

/data/local/tmp

it works. ?

The scripts will currently only backup and restore all of your 

/sdcard

directory. Assuming you’re also using something like Titanium Backup you’ll be able to backup and restore all your apps, settings and data. To reduce the amount of data to copy it uses rsync filters to exclude caches and other files that you definitely don’t want synced (

.DS_Store

  files anyone?).

At the moment there’s one caveat: I had to disable restoring modification times (i.e. use 

--no-times

 ) because of an obnoxious error (they will be backuped fine, only restoring is the problem): ?

mkstemp “…” (in root) failed: Operation not permitted (1)

Additionally if you’re on the paranoid side you can also build your own rsync for Android to use as the backup binary.

The code and a ton of documentation can be found on GitHub. Comments and suggestions are welcome. ?

Build Rsync for Android Yourself

To build rsync for Android you’ll need to have the Android NDK installed already.

Then clone the rsync for android source (e.g. from CyanogenMod LineageOS) …

git clone https://github.com/LineageOS/android_external_rsync.git
cd android_external_rsync
# checkout the most recent branch
git checkout cm-14.1

… create the missing

jni/Application.mk

build file (e.g. from this Gist) and adapt it to your case …

… and start the build with

export NDK_PROJECT_PATH=<code>pwd</code> ndk-build -d rsync

You’ll find your self-build rsync in

obj/local/*/rsync

. ?

Update 2017-10-06:

  • Updated sources from CyanogenMod to LineageOS.
  • Added links to Gist and Andoid NDK docs
  • Updated steps to work with up-to-date setups

If you get something like the following warnings and errors …

[...]
./flist.c:454:16: warning: implicit declaration of function 'major' is invalid in C99
      [-Wimplicit-function-declaration]
                        if ((uint32)major(rdev) == rdev_major)
                                    ^
./flist.c:458:41: warning: implicit declaration of function 'minor' is invalid in C99
      [-Wimplicit-function-declaration]
                        if (protocol_version < 30 && (uint32)minor(rdev) <= 0xFFu)
                                                             ^
./flist.c:467:11: warning: implicit declaration of function 'makedev' is invalid in C99
      [-Wimplicit-function-declaration]
                        rdev = MAKEDEV(major(rdev), 0);
                               ^
./rsync.h:446:36: note: expanded from macro 'MAKEDEV'
#define MAKEDEV(devmajor,devminor) makedev(devmajor,devminor)
                                   ^
3 warnings generated.
[...]
./flist.c:473: error: undefined reference to 'makedev'
./flist.c:454: error: undefined reference to 'major'
./flist.c:457: error: undefined reference to 'major'
./flist.c:458: error: undefined reference to 'minor'
./flist.c:467: error: undefined reference to 'major'
./flist.c:467: error: undefined reference to 'makedev'
./flist.c:617: error: undefined reference to 'major'
./flist.c:619: error: undefined reference to 'minor'
./flist.c:621: error: undefined reference to 'minor'
./flist.c:788: error: undefined reference to 'makedev'
./flist.c:869: error: undefined reference to 'makedev'
./flist.c:1027: error: undefined reference to 'minor'
clang++: error: linker command failed with exit code 1 (use -v to see invocation) 
make: *** [obj/local/armeabi-v7a/rsync] Error 1

… you probably need to update

config.h

and change

/* #undef MAJOR_IN_SYSMACROS */

to

#define MAJOR_IN_SYSMACROS 1

.

Oh, The Losses … All Those Losses

It seems pirates have found the ultimate weapon to kill the music industry: copying music to /dev/null … all the time! ?

Money Quote:

Last week, Sunde told TorrentFreak that he’d already made 120 million copies and “cost” the music industry $150 million in losses, at least by the music industry’s preferred accounting practices counting the dollar value of any copied song as lost revenue.

 

RECONSIDER

DHH has a sneering piece about why you can be content by just “making a dent in the universe” instead of joining the circus to become the next shiny startup unicorn.

In fact, it’s hard to carry on a conversation with most startup people these days without getting inundated with odes to network effects and the valiance of deferring “monetization” until you find something everyone in the whole damn world wants to fixate their eyeballs on.

In this atmosphere, the term startup has been narrowed to describe the pursuit of total business domination. It’s turned into an obsession with unicorns and the properties of their “success”. A whole generation of people working with and for the internet enthralled by the prospect of being transformed into a mythical creature.

He describes how the VC-backed startup circus is built on wrong premises and gives wrong incentives:

  • “Get Big or GTFO” … build monopolies by any means necessary.
  • Disrupt everything!
  • Buy an audience if necessary, instead of growing it!
  • Have “plans for world domination, complete capture of market and customers!”
  • “Raise more, burn faster, and grow at supersonic speeds yesterday!”
  • “Moving fast” is a license to “treating contractors like a repugnant automatron class of secondary citizens.”
  • Instead of “aligning your financial motivations with the service of your users” you’re incentivized to “capture eyeballs and sell their attention, privacy, and dignity in bulk to the highest bidder.”
  • You need VC, because “going into the cold, unknown world of business without their money in your mattress is a fool’s errand.”

Not only the premises are flawed, but also the process is:

  • San Francisco!!! “Because while that area north of Silicon Valley is busy disrupting everything, it still hasn’t caught up with the basic disruption of geography.”
  • Angels will fund anything, because they don’t know who the next unicorn will be.
  • They don’t care if you’re the unicorn or someone else is, because they’ve funded a bunch of other startups as well.
  • Every step along this scripted, “multi-level investment scheme” you’ll accumulate more bosses “dictating YOUR INCREDIBLE JOURNEY.”

Which probably also sums up my biggest reservations about the prevailing startup culture (although I’ve been working for startups for years now ?).

Ramen Code

The plaintiffs in Toyota’s Unintended Acceleration lawsuit had someone with knowledge in building embedded software had a look at Toyota’s source code:

possible bit flips, task deaths that would disable the failsafes, memory corruption, single-point failures, inadequate protections against stack overflow and buffer overflow, single-fault containment regions, thousands of global variables. The list of deficiencies in process and product was lengthy.