refine doc

README.md

@@ -1,11 +1,24 @@
-English | [简体中文](zh/)
+English | [简体中文](README.zh-cn.md)
 
-# ReDroid
+# Table of contents
+- [Overview](#overview)
+- [Getting Started](#getting-started)
+- [Configuration](#configuration)
+- [Native Bridge Support](#native-bridge-support)
+- [GMS Support](#gms-support)
+- [WebRTC Streaming](#webrtc-streaming)
+- [How To Build](#how-to-build)
+- [Troubleshooting](#troubleshooting)
+- [Note](#note)
+- [Contact Me](#contact-me)
+- [License](#license)
+
+## Overview
 **ReDroid** (*Re*mote an*Droid*) is a GPU accelerated AIC (Android In Container) solution. You can boot many
-instances in Linux host (`Docker`, `K8S` etc.). 
+instances in Linux host (`Docker`, `podman`, `k8s` etc.). *ReDroid* supports both `arm64` and `amd64` architectures. 
-*ReDroid* supports both arm64 and amd64 architectures. You can connect to *ReDroid* througth
+*ReDroid* is suitable for Cloud Gaming, VMI (Virtual Mobile Infurstrure), Automation Test and more.
-`scrcpy` or `adb shell`. *ReDroid* is suitable for Cloud Gaming, VDI / VMI (Virtual Mobile Infurstrure), 
+
-Automation Test and more.
+![Screenshot of ReDroid 11](./assets/redroid11.png)
 
 Currently supported:
 - Android 12 (`redroid/redroid:12.0.0-latest`, `redroid/redroid:12.0.0-amd64`, `redroid/redroid:12.0.0-arm64`)
@@ -14,8 +27,6 @@ Currently supported:
 - Android 9 (`redroid/redroid:9.0.0-latest`, `redroid/redroid:9.0.0-amd64`, `redroid/redroid:9.0.0-arm64`)
 - Android 8.1 (`redroid/redroid:8.1.0-latest`, `redroid/redroid:8.1.0-amd64`, `redroid/redroid:8.1.0-arm64`)
 
-**pull first, otherwise your running image may out of date; check image SHA**
-
 Tested Platforms:
 - Ubuntu 16.04 / 18.04 / 20.04 (amd64 / arm64)
 - Amazon Linux 2 (amd64 / arm64)
@@ -27,9 +38,9 @@ Tested Platforms:
 
 \* means need customized kernel
 
-## Quick Start
+## Getting Started
-*ReDroid* runs on modern linux (kernel 4.14+), and require some Android specific modules (`binderfs`, `ashmem` etc.)
+*ReDroid* requires some kernel modules (`binderfs`, `ashmem` etc.).
-check [kernel modules](https://github.com/remote-android/redroid-modules) to install these required kernel modules.
+**Check [kernel modules](https://github.com/remote-android/redroid-modules) to install these modules.**
 
 ```bash
 # start and connect via `scrcpy`
@@ -37,7 +48,7 @@ docker run -itd --rm --memory-swappiness=0 --privileged \
     --pull always \
     -v ~/data:/data \
     -p 5555:5555 \
-    redroid/redroid:9.0.0-latest
+    redroid/redroid:11.0.0-latest
 
 adb connect <IP>:5555
 scrcpy --serial <IP>:5555
@@ -45,11 +56,11 @@ scrcpy --serial <IP>:5555
 ## explains:
 ## --pull always  -- be sure to use the latest image
 ## -v ~/data:/data  -- mount data partition
-## -p 5555:5555 -- 5555 for adb connect, you can run `adb connect <IP>`
+## -p 5555:5555 -- expose adb port, you can connect via `adb connect <HOST-IP>`
 
 ```
 
-## Start Params
+## Configuration
 required params (already added in docker image)
 - qemu=1
 - androidboot.hardware=redroid
@@ -62,6 +73,9 @@ display params
 - redroid.enable_built_in_display=[0|1]
 - redroid.overlayfs=[0|1]
 
+Network:
+- net.eth0.dns1=<IP>
+
 GPU accelerating
 *ReDroid* use mesa3d to accelerate 3D rendering.
 Currently tested paltforms:
@@ -84,53 +98,94 @@ your build. run `docker exec <container> ip r add default via 192.168.232.1 dev
 NOTE: you can override system props prefixed with `qemu.` or `ro.`. for example, you can set `ro.secure=0`, then 
 you can get root adb shell by default.
 
-## Deployment
+## Native Bridge Support
-*ReDroid* support different deploy methods, check [Deploy](./deploy.md) for more details.
+It's possible to run `arm` Apps in `x86` *ReDroid* instance via `libhoudini`, `libndk_translator` or `QEMU translator`.
-- Docker (podman)
-- K8S
-- Package Manager (Planned)
 
-## Native Bridge
+Take `libndk_translator` as an example:
-It's possible to run Arm Apps in x64 *ReDroid* instance with `libhoudini`, `libndk_translator` or `Qemu translator`
 
-Check [Native Bridge](./native_bridge) for more.
+``` bash
+# grab libndk_translator libs from Android 11 Emulator
+find /system \( -name 'libndk_translation*' -o -name '*arm*' -o -name 'ndk_translation*' \) | tar -cf native-bridge.tar -T -
 
-## GMS
+# example structure, be careful the file owner and mode
-It's possible to add GMS (Google Mobile Service) support in *ReDroid* via Google packages or `MicroG`.
 
-Check [GMS](./gms.md) for more.
+system/
+├── bin
+│   ├── arm
+│   └── arm64
+├── etc
+│   ├── binfmt_misc
+│   └── init
+├── lib
+│   ├── arm
+│   └── libnb.so
+└── lib64
+    ├── arm64
+    └── libnb.so
+```
 
-## WebRTC
+```dockerfile
-Compared with `RFB` (VNC), `WebRTC` is more versatile (audio / video / camera / sensors data...). 
+# Dockerfile
-*ReDroid* is planning to implement `WebRTC` protocol. Check [WebRTC](./webrtc.md) for more.
+FROM redroid/redroid:11.0.0-amd64
 
-## Build
+ADD native-bridge.tar /
-Same as AOSP building process. check [AOSP setup](https://source.android.com/setup/build/initializing#installing-required-packages-ubuntu-1404)
 
-Check [ReDroid build](./build.md) for more.
+COPY nativebridge.rc /vendor/etc/init/nativebridge.rc
+```
+
+```
+# nativebridge.rc
+on early-init
+    setprop ro.product.cpu.abilist x86_64,arm64-v8a,x86,armeabi-v7a,armeabi
+    setprop ro.product.cpu.abilist64 x86_64,arm64-v8a
+    setprop ro.product.cpu.abilist32 x86,armeabi-v7a,armeabi
+    setprop ro.dalvik.vm.isa.arm x86
+    setprop ro.dalvik.vm.isa.arm64 x86_64
+    setprop ro.enable.native.bridge.exec 1
+    setprop ro.dalvik.vm.native.bridge libndk_translation.so
+    setprop ro.ndk_translation.version 0.2.2
+```
+
+```bash
+# build docker image
+docker build . -t redroid:11.0.0-amd64-nb
+```
+
+## GMS Support
+It's possible to add GMS (Google Mobile Service) support in *ReDroid* via [Open GApps](https://opengapps.org/) 
+or [MicroG](https://microg.org/).
+
+
+## WebRTC Streaming
+**CALL FOR HELP**
+
+Plan to port `WebRTC` solutions from `cuttlefish`, including frontend (HTML5), backend and many virtual HALs.
+
+## How To Build
+It's Same as AOSP building process. But I suggest to use `docker` to build.
+
+Check [android-builder-docker](./android-builder-docker) for details.
 
 ## Troubleshooting
-- VNC screen hang
+- Container disappeared immediately
-    - try `stop vncserver && start vncserver`
+> make sure the required kernel modules are installed; run `dmesg -T` for detailed logs
-    - or reboot *ReDroid* instance `docker restart ...`
-- icon in launcher gone
-    - if switching between GPU / swiftshader (soft), try to clear launcher data or entire data partition
-- `fuse` setup failed
-    - try `modprobe fuse`
 
-## Contributing
+- Container running, but adb cannot connect (device offline etc.)
-Contributing is always welcome (especially the `HAL` part). check [Contributing](./contributing.md) for more
+> run `docker exec -it <container> sh`, then check `ps -A` and `logcat`
+>
+> try `dmesg -T` if cannot get a container shell
 
-## Workarounds
+
+## Note
 - Kernel 5.7+, need enable `binderfs` / `ashmem`
 - `redroid` require `pid_max` less than 65535, or else may run into problems. Change in host OS, or add `pid_max` separation support in PID namespace
 - SElinux is disabled in *ReDroid*; possible enabled with [selinuxns POC](http://namei.org/presentations/selinux_namespacing_lca2018.pdf)
-- sdcardfs currently not implemented, use `fuse` instead; may need run `modprobe fuse` first in some OS (AmazonLinux2 ?)
+- `sdcardfs` currently not implemented, use `fuse` instead; may need run `modprobe fuse` first in some OS (AmazonLinux2 ?)
 - CGroups errors ignored; some (`stune` for example) not supported in generic linux.
 - `procfs` not fully seperated with host OS; Community use `lxcfs` and some cloud vendor ([TencentOS](https://github.com/Tencent/TencentOS-kernel)) enhanced in their own kernel.
 - vintf verify disabled (since no kernel)
 
-## Contacts
+## Contact Me
 - ziyang.zhou@outlook.com
 - remote-android.slack.com (invite link: https://join.slack.com/t/remote-android/shared_invite/zt-q40byk2o-YHUgWXmNIUC1nweQj0L9gA)
 

README.zh-cn.md

@@ -1,4 +1,4 @@
-[English](..) | 简体中文
+[English](README.md) | 简体中文
 
 # ReDroid
 

android-builder-docker/Dockerfile

@@ -6,7 +6,7 @@ ARG username
 
 # COPY apt.conf /etc/apt/apt.conf
 
-COPY sources.list etc/apt/sources.list
+#COPY sources.list etc/apt/sources.list
 
 # gettext python-mako python3-mako python-enum34 used to build mesa3d
 RUN apt-get update && apt-get install -y git-core gnupg flex bison gperf build-essential zip curl zlib1g-dev gcc-multilib g++-multilib libc6-dev-i386 lib32ncurses5-dev x11proto-core-dev libx11-dev lib32z-dev ccache libgl1-mesa-dev libxml2-utils xsltproc unzip python gettext python-mako python3-mako python-enum34

android-builder-docker/README.md

@@ -0,0 +1,53 @@
+# Build ReDroid with docker
+
+## Sync Code
+ReDroid manifest include several branches:
+- `redroid12-gsi`, `redroid-s-beta-3`
+- `redroid11-gsi`, `redroid-11.0.0`
+- `redroid10-gsi`, `redroid-10.0.0`
+- `redroid-9.0.0`, `redroid9-gsi` (experimental)
+- `redroid-8.1.0`
+
+branch with `-gsi` is recommended (fully treble / VNDK  enforced).
+
+```bash
+# fetch code
+
+mkdir aosp && cd aosp
+repo init -u https://github.com/remote-android/platform_manifests.git -b <BRANCH> --depth=1
+repo sync -c --no-tags
+```
+
+## Build
+```bash
+# create builder docker image
+# adjust apt.conf and source.list if needed
+docker build --build-arg userid=$(id -u) --build-arg groupid=$(id -g) --build-arg username=$(id -un) -t android-build-trusty .
+
+# *inside* builder container
+cd /src
+
+. build/envsetup.sh
+lunch redroid_x86_64-userdebug
+# or lunch redroid_arm64-userdebug
+m
+
+# create redroid docker image in *HOST*
+cd <BUILD-OUT-DIR>
+sudo mount system.img system -o ro
+sudo mount vendor.img vendor -o ro
+sudo tar --xattrs -c vendor -C system --exclude="vendor" . | docker import -c 'ENTRYPOINT ["/init", "qemu=1", "androidboot.hardware=redroid"]' - redroid
+```
+
+## Note
+```bash
+# intent changes for redroid-10 (copyfile hook during repo sync), DO NOT PANIC
+# [Android Clang/LLVM Toolchain](https://github.com/remote-android/platform_manifests/tree/llvm-toolchain-redroid-10.0.0)
+
+project prebuilts/clang/host/linux-x86/         (*** NO BRANCH ***)
+ -m     clang-r353983c/lib64/clang/9.0.3/lib/linux/libclang_rt.scudo_minimal-aarch64-android.a
+ -m     clang-r353983c/lib64/clang/9.0.3/lib/linux/libclang_rt.scudo_minimal-arm-android.a
+ -m     clang-r353983c/lib64/clang/9.0.3/lib/linux/libclang_rt.scudo_minimal-i686-android.a
+ -m     clang-r353983c/lib64/clang/9.0.3/lib/linux/libclang_rt.scudo_minimal-x86_64-android.a
+```
+

build.md

@@ -1,26 +0,0 @@
-# Build ReDroid
-
-## Get Source
-- `repo init -u git@github.com:remote-android/platform_manifests.git -b [BRANCH] --depth=1 && repo sync -c`
-BRANCH could be redroid-8.1.0, redroid-9.0.0, redroid-10.0.0, redroid-11.0.0 etc.
-
-## Build
-suggest build with [docker](./docker)
-
-```bash
-# intent changes for redroid-10.0.0 (copyfile hook during repo sync), DO NOT Panic
-# [Android Clang/LLVM Toolchain](https://android.googlesource.com/toolchain/llvm_android/)
-
-project prebuilts/clang/host/linux-x86/         (*** NO BRANCH ***)
- -m     clang-r353983c/lib64/clang/9.0.3/lib/linux/libclang_rt.scudo_minimal-aarch64-android.a
- -m     clang-r353983c/lib64/clang/9.0.3/lib/linux/libclang_rt.scudo_minimal-arm-android.a
- -m     clang-r353983c/lib64/clang/9.0.3/lib/linux/libclang_rt.scudo_minimal-i686-android.a
- -m     clang-r353983c/lib64/clang/9.0.3/lib/linux/libclang_rt.scudo_minimal-x86_64-android.a
-```
-
-## Run
-1. create rootfs only image (Dev purpose): `cd $BUILD_OUT && sudo tar --xattrs -c -C root . | docker import -c 'ENTRYPOINT ["/init", "qemu=1", "androidboot.hardware=redroid"]' - redroid-dev`
-2. run with rootfs only image: `docker run -v ~/data:/data -itd -p 5555:5555 --rm --memory-swappiness=0 --privileged -v $BUILD_OUT/system:/system -v $BUILD_OUT/vendor:/vendor redroid-dev`
-3. create full image: `cd $BUILD_OUT && sudo tar --xattrs -c system vendor -C root --exclude="system" --exclude="vendor" . | docker import -c 'ENTRYPOINT ["/init", "qemu=1", "androidboot.hardware=redroid"]' - redroid`
-4. run with full image: `docker run -v ~/data:/data -itd -p 5555:5555 --rm --memory-swappiness=0 --privileged redroid`
-

contributing.md

@@ -1,3 +0,0 @@
-# Contributing
-
-TBD

deploy.md

@@ -1,4 +0,0 @@
-# ReDroid Deployment
-
-TBD
-

docker/README.md

@@ -1,13 +0,0 @@
-# Build ReDroid with docker
-
-create build image: `docker build --build-arg userid=$(id -u) --build-arg groupid=$(id -g) --build-arg username=$(id -un) -t android-build-trusty .`
-run build image: `docker run -it --rm -v $ANDROID_BUILD_TOP:/src android-build-trusty`
-
-build *ReDroid*:
-- `cd /src; . build/envsetup.sh`
-- `lunch`
-- `m`
-
-sync build (fix file owner / permission etc.)
-- `export BUILD_OUT=~/redroid_out && cd vendor/redroid && ./sync.sh` (pasword same as `id -un`)
-

gms.md

@@ -1,7 +0,0 @@
-# GMS support in ReDroid
-
-## Open GApps
-[Open GApps](https://opengapps.org/)
-
-## microG
-[microG](https://microg.org/)

native_bridge/Dockerfile

@@ -1,6 +0,0 @@
-ARG BASE_IMAGE
-FROM $BASE_IMAGE
-
-ADD native-bridge.tar /
-
-COPY nativebridge.rc /vendor/etc/init/nativebridge.rc 

native_bridge/README.md

@@ -1,36 +0,0 @@
-# Native Bridge Support in ReDroid
-
-There are several choose to support native bridge (typically for arm) in *ReDroid*
-- `libndk_translation` from Arm
-- `libhoudini` from Intel
-- `QEMU`
-
-Prepare the resource and build the image.
-And run `docker build --build-arg BASE_IMAGE=<Base Image> .`
-
-```bash
-# example structure, be careful the file owner and mode
-
-system/
-├── bin
-│   ├── arm
-│   └── arm64
-├── etc
-│   ├── binfmt_misc
-│   └── init
-├── lib
-│   ├── arm
-│   └── libnb.so
-└── lib64
-    ├── arm64
-    └── libnb.so
-
-# grab libndk_translation from Android 11 Emulator
-find /system \( -name 'libndk_translation*' -o -name '*arm*' -o -name 'ndk_translation*' \) | tar -cf native-bridge.tar -T - 
-
-```
-
-Example: Patch libndk_translation in ReDroid 11
-
-![Screenshot of ReDroid 11 with libndk_translation](./redroid_11_libndk_translation.png)
-

native_bridge/nativebridge.rc

@@ -1,9 +0,0 @@
-on early-init
-    setprop ro.product.cpu.abilist x86_64,arm64-v8a,x86,armeabi-v7a,armeabi
-    setprop ro.product.cpu.abilist64 x86_64,arm64-v8a
-    setprop ro.product.cpu.abilist32 x86,armeabi-v7a,armeabi
-    setprop ro.dalvik.vm.isa.arm x86
-    setprop ro.dalvik.vm.isa.arm64 x86_64
-    setprop ro.enable.native.bridge.exec 1
-    setprop ro.dalvik.vm.native.bridge libndk_translation.so
-    setprop ro.ndk_translation.version 0.2.2

webrtc.md

@@ -1,34 +0,0 @@
-# WebRTC support in ReDroid
-
-TBD
-
-```text
-
-                                               +-------------------------------------------+
-                                               |                 K8S Cluster               |
-                                               |                                           |
-                                               | +-----------+                             |
-                                               | |  Drivers  |                             |
-                   +                           | +-----------+                             |
-                   |                           |                                           |
-                   |                           |                                           |
-                   |                           |                            +-----------+  |
-                   |                           |                 +---------->   Nginx   |  |
-             gRPC  |             gRPC          |                 |          +-----------+  |
-+--------+  WebRTC |  +------+  WebRTC         | +-----------+   |                         |
-| Client <------------>  FW  +-----+------------->   Envoy   +---+                         |
-+--------+         |  +------+     |           | +-----------+   |                         |
-                   |               |           |       |         |          +-----------+  |
-                   |               |           |       |         +---------->  Server   |  |
-                   |               |           |       |                    +-----^-----+  |
-                   |               |           | +-----v-----+                    |        |
-                   |               +------------->   coturn  |                    |        |
-                   |               |           | +-----+-----+                    |        |
-                   |               |           |       |                          |        |
-                   +               |           |       |                          |        |
-                                   |           | +-----v-----+                    |        |
-                                   +------------->  Android  +--------------------+        |
-                                               | +-----------+                             |
-                                               |                                           |
-                                               +-------------------------------------------+
-```