Android system programming : porting, customizing, and debugging Android HAL

Build, customize, and debug your own Android system Key Features Master Android system-level programming by integrating, customizing, and extending popular open source projects Use Android emulators to explore the true potential of your hardware Master key debugging techniques to create a hassle-free development environment Book Description Android system programming involves both hardware and software knowledge to work on system level programming. The developers need to use various techniques to debug the different components in the target devices. With all the challenges, you usually have a deep learning curve to master relevant knowledge in this area. This book will not only give you the key knowledge you need to understand Android system programming, but will also prepare you as you get hands-on with projects and gain debugging skills that you can use in your future projects. You will start by exploring the basic setup of AOSP, and building and testing an emulator image. In the first project, you will learn how to customize and extend the Android emulator. Then you'll move on to the real challenge―building your own Android system on VirtualBox. You'll see how to debug the init process, resolve the bootloader issue, and enable various hardware interfaces. When you have a complete system, you will learn how to patch and upgrade it through recovery. Throughout the book, you will get to know useful tips on how to integrate and reuse existing open source projects such as LineageOS (CyanogenMod), Android-x86, Xposed, and GApps in your own system. What you will learn Set up the Android development environment and organize source code repositories Get acquainted with the Android system architecture Build the Android emulator from the AOSP source tree Find out how to enable WiFi in the Android emulator Debug the boot up process using a customized Ramdisk Port your Android system to a new platform using VirtualBox Find out what recovery is and see how to enable it in the AOSP build Prepare and test OTA packages Table of Contents Introduction to Android System Programming Setting up the Development Environment Discovering Kernel, HAL and Virtual Hardware Customizing the Android Emulator Enabling ARM Translator and Introducing Native Bridge Debugging the Boot up Process Using a Customized Ramdisk Enabling Wi-Fi in the Android Emulator Creating Your Own Device on VirtualBox Booting Up x86vbox Using PXE/NFS Enabling Graphics Enabling VirtualBox-Specific Hardware Interfaces Introducing Recovery Creating OTA Packages Customizing and Debugging Recovery Cover Copyright Credits About the Author About the Reviewers About the Reviewers www.PacktPub.com Customer Feedback Table of Contents Preface Chapter 1: Introduction to Android System Programming What is system programming? What is the scope of this book? Overview of the Android system Kernel HAL Android system services Binder IPC Application framework Recovery The third-party open source projects derived from AOSP LineageOS (CyanogenMod) Android-x86 CWM/CMR/TWRP Strategy of integration Virtual hardware reference platforms Introduction to the x86-based Android emulator Introduction to ranchu VirtualBox-based Android emulators Summary Chapter 2: Setting Up the Development Environment Summary of Android versions Installing Android SDK and setting up an Android Virtual Device Creating AVD in an older version of SDK Creating AVD in the latest version of SDK Testing the goldfish emulator Testing ranchu emulator The AOSP build environment and the Android emulator build The AOSP build environment Installing the required packages Installing Open JDK 7 and 8 Downloading the AOSP source Installing repo Initializing a repo client and downloading the AOSP source tree Building AOSP Android emulator images Testing AOSP images Creating your own repository mirror Repo and manifest Using a local mirror for AOSP Creating your own mirror of GitHub Fetching Git repositories outside GitHub Creating your own manifest for client download Summary Chapter 3: Discovering Kernel, HAL, and Virtual Hardware What is inside the AOSP? Android emulator HAL Calling sequence Goldfish lights HAL The system service and hardware manager Android emulator kernel and hardware Android emulator hardware Goldfish platform bus QEMU pipe device Goldfish audio device Goldfish serial port Goldfish kernel QEMU pipe Summary Chapter 4: Customizing the Android Emulator Why customize the Android emulator Understanding build layers Build variants Creating a new x86emu device Checking out from the AOSP Checking out from a local mirror Creating x86emu device AndroidProducts.mk BoardConfig.mk device.mk Building and testing x86emu Building x86emu Testing x86emu Integrating with Eclipse Summary Chapter 5: Enabling the ARM Translator and Introducing Native Bridge Introducing Native Bridge Setting up Native Bridge as part of the ART initialization Pre-initializing Native Bridge Initializing Native Bridge Loading a native library Integrating Houdini to the x86emu device Changing the configuration of the x86emu build Extending the x86emu device Changes to BoardConfig.mk Changes to x86emu_x86.mk Changes to device.mk Using the Android-x86 implementation Analyzing libnb.so Using binfmt_misc Building and testing Testing the command-line application Testing the Android JNI application Summary Chapter 6: Debugging the Boot Up Process Using a Customized ramdisk Analyzing the Android start up process Bootloader and the kernel Analyzing the init process and ramdisk Actions Services Device-specific actions and services Source code and manifest changes The Android-x86 start up process The first-stage boot using initrd.img Inside initrd.img Inside install.img Building x86emu with initrd.img Creating a filesystem image Kernel changes Booting a disk image on the Android emulator Summary Chapter 7: Enabling Wi-Fi on the Android Emulator Wi-Fi on Android The Wi-Fi architecture QEMU networking and wpa_supplicant in Android Adding Wi-Fi to the emulator Enabling wpa_supplicant in BoardConfig.mk Providing a proper wpa_supplicant configuration Creating services in init scripts Initializing network interface eth1 Starting up wpa_supplicant Building the source code Getting the source code Enabling boot with initrd.img Testing Wi-Fi on an emulator Booting an Android emulator using initrd.img Booting an Android emulator using ramdisk.img Debugging Wi-Fi start up processes Summary Chapter 8: Creating Your Own Device on VirtualBox HAL of x86vbox The manifest for x86vbox Creating a new x86vbox device Product definition Makefile of x86vbox Board configuration of x86vbox Common x86 devices Getting the source code and building the x86vbox device Boot up process and device initialization Device initialization before Android start-up HAL initialization during the Android start-up Summary Chapter 9: Booting Up x86vbox Using PXE/NFS Setting up a PXE boot environment Preparing PXE Boot ROM Downloading and building the LAN Boot ROM Fixing up the ROM image Configuring the virtual machine to use the LAN Boot ROM Setting up the PXE boot environment Configuring and testing the PXE boot Setting up the virtual machine Using VirtualBox internal PXE booting with NAT Configuring pxelinux.cfg pxelinux.cfg/default Setting up a serial port for debugging NFS filesystem Preparing the kernel Setting up the NFS server Configuring the PXE boot menu Booting to NFS installation Booting from a hard disk Booting to recovery Summary Chapter 10: Enabling Graphics Introduction to the Android graphics architecture Delving into graphics HAL Loading the Gralloc module Initializing GPU Initializing framebuffer Allocating and releasing the graphic buffer Allocating from framebuffer Allocating from system memory Releasing graphic buffers Rendering framebuffer Graphics HAL of the Android emulator Overview of hardware GLES emulation Initializing GPU0 and FB0 in GLES emulation GPU0 device implementation FB0 device implementation Summary Chapter 11: Enabling VirtualBox-Specific Hardware Interfaces OpenGL ES and graphics hardware initialization Loading OpenGL ES libraries Analyzing the loading process Loading the driver Creating the rendering engine The uvesafb framebuffer driver What is uvesafb? Testing the uvesafb framebuffer driver Initializing uvesafb in x86vbox Integrating VirtualBox Guest Additions Building VirtualBox Guest Additions Integrating vboxsf Integrating vboxvideo Building and testing images with VirtualBox Guest Additions Summary Chapter 12: Introducing Recovery Recovery introduction Android device partitions Analyzing recovery BCB Cache partition Main flow of recovery Retrieving arguments from BCB and cache files Factory data reset OTA update Building recovery for x86vbox Building configuration Changes to x86vbox Changes to recovery Changes to newinstaller Testing recovery Summary Chapter 13: Creating OTA Packages What is inside an OTA package Updater The updater script Edify functions Built-in functions Installation functions Block image functions Device extensions Preparing an OTA package for x86vbox Removing dependencies on /system Hardware initialization in recovery Minimum execution environment in recovery Building and testing Summary Chapter 14: Customizing and Debugging Recovery Debugging and testing native Android applications Debugging with GDB Integration with Eclipse Extending recovery and the updater Extending recovery Extending the updater Extending the Python module Building and testing the extended recovery and updater Supporting the third-party recovery packages Xposed recovery package Opening GApps Summary Index Build, customize, and debug your own Android system About This Book Master Android system-level programming by integrating, customizing, and extending popular open source projects Use Android emulators to explore the true potential of your hardware Master key debugging techniques to create a hassle-free development environment Who This Book Is For This book is for Android system programmers and developers who want to use Android and create indigenous projects with it. You should know the important points about the operating system and the C/C++ programming language. What You Will Learn Set up the Android development environment and organize source code repositories Get acquainted with the Android system architecture Build the Android emulator from the AOSP source tree Find out how to enable WiFi in the Android emulator Debug the boot up process using a customized Ramdisk Port your Android system to a new platform using VirtualBox Find out what recovery is and see how to enable it in the AOSP build Prepare and test OTA packages In Detail Android system programming involves both hardware and software knowledge to work on system level programming. The developers need to use various techniques to debug the different components in the target devices. With all the challenges, you usually have a deep learning curve to master relevant knowledge in this area. This book will not only give you the key knowledge you need to understand Android system programming, but will also prepare you as you get hands-on with projects and gain debugging skills that you can use in your future projects. You will start by exploring the basic setup of AOSP, and building and testing an emulator image. In the first project, you will learn how to customize and extend the Android emulator. Then you'll move on to the real challenge - building your own Android system on VirtualBox. You'll see how to debug the init process, resolve the bootloader issue, and enable various hardware interfaces. When you have a complete system, you will learn how to patch and upgrade it through recovery. Throughout the book, you will get to know useful tips on how to integrate and reuse existing open source projects such as LineageOS (CyanogenMod), Android-x86, Xposed, and GApps in your own system. Style and approach This is an easy-to-follow guide full of hands-on examples and system-level programming tips