diff --git a/README.md b/README.md index b6128bd..d353a8f 100644 --- a/README.md +++ b/README.md @@ -1,11 +1,24 @@ -English | [简体中文](zh/) +English | [简体中文](README.zh-cn.md) -# ReDroid +# Table of contents +- [Overview](#overview) +- [Getting Started](#getting-started) +- [Configuration](#configuration) +- [Native Bridge Support](#native-bridge-support) +- [GMS Support](#gms-support) +- [WebRTC Streaming](#webrtc-streaming) +- [How To Build](#how-to-build) +- [Troubleshooting](#troubleshooting) +- [Note](#note) +- [Contact Me](#contact-me) +- [License](#license) + +## Overview **ReDroid** (*Re*mote an*Droid*) is a GPU accelerated AIC (Android In Container) solution. You can boot many -instances in Linux host (`Docker`, `K8S` etc.). -*ReDroid* supports both arm64 and amd64 architectures. You can connect to *ReDroid* througth -`scrcpy` or `adb shell`. *ReDroid* is suitable for Cloud Gaming, VDI / VMI (Virtual Mobile Infurstrure), -Automation Test and more. +instances in Linux host (`Docker`, `podman`, `k8s` etc.). *ReDroid* supports both `arm64` and `amd64` architectures. +*ReDroid* is suitable for Cloud Gaming, VMI (Virtual Mobile Infurstrure), Automation Test and more. + +![Screenshot of ReDroid 11](./assets/redroid11.png) Currently supported: - Android 12 (`redroid/redroid:12.0.0-latest`, `redroid/redroid:12.0.0-amd64`, `redroid/redroid:12.0.0-arm64`) @@ -14,8 +27,6 @@ Currently supported: - Android 9 (`redroid/redroid:9.0.0-latest`, `redroid/redroid:9.0.0-amd64`, `redroid/redroid:9.0.0-arm64`) - Android 8.1 (`redroid/redroid:8.1.0-latest`, `redroid/redroid:8.1.0-amd64`, `redroid/redroid:8.1.0-arm64`) -**pull first, otherwise your running image may out of date; check image SHA** - Tested Platforms: - Ubuntu 16.04 / 18.04 / 20.04 (amd64 / arm64) - Amazon Linux 2 (amd64 / arm64) @@ -27,9 +38,9 @@ Tested Platforms: \* means need customized kernel -## Quick Start -*ReDroid* runs on modern linux (kernel 4.14+), and require some Android specific modules (`binderfs`, `ashmem` etc.) -check [kernel modules](https://github.com/remote-android/redroid-modules) to install these required kernel modules. +## Getting Started +*ReDroid* requires some kernel modules (`binderfs`, `ashmem` etc.). +**Check [kernel modules](https://github.com/remote-android/redroid-modules) to install these modules.** ```bash # start and connect via `scrcpy` @@ -37,7 +48,7 @@ docker run -itd --rm --memory-swappiness=0 --privileged \ --pull always \ -v ~/data:/data \ -p 5555:5555 \ - redroid/redroid:9.0.0-latest + redroid/redroid:11.0.0-latest adb connect :5555 scrcpy --serial :5555 @@ -45,11 +56,11 @@ scrcpy --serial :5555 ## explains: ## --pull always -- be sure to use the latest image ## -v ~/data:/data -- mount data partition -## -p 5555:5555 -- 5555 for adb connect, you can run `adb connect ` +## -p 5555:5555 -- expose adb port, you can connect via `adb connect ` ``` -## Start Params +## Configuration required params (already added in docker image) - qemu=1 - androidboot.hardware=redroid @@ -62,6 +73,9 @@ display params - redroid.enable_built_in_display=[0|1] - redroid.overlayfs=[0|1] +Network: +- net.eth0.dns1= + GPU accelerating *ReDroid* use mesa3d to accelerate 3D rendering. Currently tested paltforms: @@ -84,53 +98,94 @@ your build. run `docker exec ip r add default via 192.168.232.1 dev NOTE: you can override system props prefixed with `qemu.` or `ro.`. for example, you can set `ro.secure=0`, then you can get root adb shell by default. -## Deployment -*ReDroid* support different deploy methods, check [Deploy](./deploy.md) for more details. -- Docker (podman) -- K8S -- Package Manager (Planned) +## Native Bridge Support +It's possible to run `arm` Apps in `x86` *ReDroid* instance via `libhoudini`, `libndk_translator` or `QEMU translator`. -## Native Bridge -It's possible to run Arm Apps in x64 *ReDroid* instance with `libhoudini`, `libndk_translator` or `Qemu translator` +Take `libndk_translator` as an example: -Check [Native Bridge](./native_bridge) for more. +``` bash +# grab libndk_translator libs from Android 11 Emulator +find /system \( -name 'libndk_translation*' -o -name '*arm*' -o -name 'ndk_translation*' \) | tar -cf native-bridge.tar -T - -## GMS -It's possible to add GMS (Google Mobile Service) support in *ReDroid* via Google packages or `MicroG`. +# example structure, be careful the file owner and mode -Check [GMS](./gms.md) for more. +system/ +├── bin +│   ├── arm +│   └── arm64 +├── etc +│   ├── binfmt_misc +│   └── init +├── lib +│   ├── arm +│   └── libnb.so +└── lib64 + ├── arm64 + └── libnb.so +``` -## WebRTC -Compared with `RFB` (VNC), `WebRTC` is more versatile (audio / video / camera / sensors data...). -*ReDroid* is planning to implement `WebRTC` protocol. Check [WebRTC](./webrtc.md) for more. +```dockerfile +# Dockerfile +FROM redroid/redroid:11.0.0-amd64 -## Build -Same as AOSP building process. check [AOSP setup](https://source.android.com/setup/build/initializing#installing-required-packages-ubuntu-1404) +ADD native-bridge.tar / -Check [ReDroid build](./build.md) for more. +COPY nativebridge.rc /vendor/etc/init/nativebridge.rc +``` + +``` +# nativebridge.rc +on early-init + setprop ro.product.cpu.abilist x86_64,arm64-v8a,x86,armeabi-v7a,armeabi + setprop ro.product.cpu.abilist64 x86_64,arm64-v8a + setprop ro.product.cpu.abilist32 x86,armeabi-v7a,armeabi + setprop ro.dalvik.vm.isa.arm x86 + setprop ro.dalvik.vm.isa.arm64 x86_64 + setprop ro.enable.native.bridge.exec 1 + setprop ro.dalvik.vm.native.bridge libndk_translation.so + setprop ro.ndk_translation.version 0.2.2 +``` + +```bash +# build docker image +docker build . -t redroid:11.0.0-amd64-nb +``` + +## GMS Support +It's possible to add GMS (Google Mobile Service) support in *ReDroid* via [Open GApps](https://opengapps.org/) +or [MicroG](https://microg.org/). + + +## WebRTC Streaming +**CALL FOR HELP** + +Plan to port `WebRTC` solutions from `cuttlefish`, including frontend (HTML5), backend and many virtual HALs. + +## How To Build +It's Same as AOSP building process. But I suggest to use `docker` to build. + +Check [android-builder-docker](./android-builder-docker) for details. ## Troubleshooting -- VNC screen hang - - try `stop vncserver && start vncserver` - - or reboot *ReDroid* instance `docker restart ...` -- icon in launcher gone - - if switching between GPU / swiftshader (soft), try to clear launcher data or entire data partition -- `fuse` setup failed - - try `modprobe fuse` +- Container disappeared immediately +> make sure the required kernel modules are installed; run `dmesg -T` for detailed logs -## Contributing -Contributing is always welcome (especially the `HAL` part). check [Contributing](./contributing.md) for more +- Container running, but adb cannot connect (device offline etc.) +> run `docker exec -it sh`, then check `ps -A` and `logcat` +> +> try `dmesg -T` if cannot get a container shell -## Workarounds + +## Note - Kernel 5.7+, need enable `binderfs` / `ashmem` - `redroid` require `pid_max` less than 65535, or else may run into problems. Change in host OS, or add `pid_max` separation support in PID namespace - SElinux is disabled in *ReDroid*; possible enabled with [selinuxns POC](http://namei.org/presentations/selinux_namespacing_lca2018.pdf) -- sdcardfs currently not implemented, use `fuse` instead; may need run `modprobe fuse` first in some OS (AmazonLinux2 ?) +- `sdcardfs` currently not implemented, use `fuse` instead; may need run `modprobe fuse` first in some OS (AmazonLinux2 ?) - CGroups errors ignored; some (`stune` for example) not supported in generic linux. - `procfs` not fully seperated with host OS; Community use `lxcfs` and some cloud vendor ([TencentOS](https://github.com/Tencent/TencentOS-kernel)) enhanced in their own kernel. - vintf verify disabled (since no kernel) -## Contacts +## Contact Me - ziyang.zhou@outlook.com - remote-android.slack.com (invite link: https://join.slack.com/t/remote-android/shared_invite/zt-q40byk2o-YHUgWXmNIUC1nweQj0L9gA) diff --git a/zh/README.md b/README.zh-cn.md similarity index 79% rename from zh/README.md rename to README.zh-cn.md index 7b654ea..b3e58ea 100644 --- a/zh/README.md +++ b/README.zh-cn.md @@ -1,4 +1,4 @@ -[English](..) | 简体中文 +[English](README.md) | 简体中文 # ReDroid diff --git a/docker/Dockerfile b/android-builder-docker/Dockerfile similarity index 96% rename from docker/Dockerfile rename to android-builder-docker/Dockerfile index 52dba92..0470e49 100644 --- a/docker/Dockerfile +++ b/android-builder-docker/Dockerfile @@ -6,7 +6,7 @@ ARG username # COPY apt.conf /etc/apt/apt.conf -COPY sources.list etc/apt/sources.list +#COPY sources.list etc/apt/sources.list # gettext python-mako python3-mako python-enum34 used to build mesa3d RUN apt-get update && apt-get install -y git-core gnupg flex bison gperf build-essential zip curl zlib1g-dev gcc-multilib g++-multilib libc6-dev-i386 lib32ncurses5-dev x11proto-core-dev libx11-dev lib32z-dev ccache libgl1-mesa-dev libxml2-utils xsltproc unzip python gettext python-mako python3-mako python-enum34 diff --git a/android-builder-docker/README.md b/android-builder-docker/README.md new file mode 100644 index 0000000..2b30717 --- /dev/null +++ b/android-builder-docker/README.md @@ -0,0 +1,53 @@ +# Build ReDroid with docker + +## Sync Code +ReDroid manifest include several branches: +- `redroid12-gsi`, `redroid-s-beta-3` +- `redroid11-gsi`, `redroid-11.0.0` +- `redroid10-gsi`, `redroid-10.0.0` +- `redroid-9.0.0`, `redroid9-gsi` (experimental) +- `redroid-8.1.0` + +branch with `-gsi` is recommended (fully treble / VNDK enforced). + +```bash +# fetch code + +mkdir aosp && cd aosp +repo init -u https://github.com/remote-android/platform_manifests.git -b --depth=1 +repo sync -c --no-tags +``` + +## Build +```bash +# create builder docker image +# adjust apt.conf and source.list if needed +docker build --build-arg userid=$(id -u) --build-arg groupid=$(id -g) --build-arg username=$(id -un) -t android-build-trusty . + +# *inside* builder container +cd /src + +. build/envsetup.sh +lunch redroid_x86_64-userdebug +# or lunch redroid_arm64-userdebug +m + +# create redroid docker image in *HOST* +cd +sudo mount system.img system -o ro +sudo mount vendor.img vendor -o ro +sudo tar --xattrs -c vendor -C system --exclude="vendor" . | docker import -c 'ENTRYPOINT ["/init", "qemu=1", "androidboot.hardware=redroid"]' - redroid +``` + +## Note +```bash +# intent changes for redroid-10 (copyfile hook during repo sync), DO NOT PANIC +# [Android Clang/LLVM Toolchain](https://github.com/remote-android/platform_manifests/tree/llvm-toolchain-redroid-10.0.0) + +project prebuilts/clang/host/linux-x86/ (*** NO BRANCH ***) + -m clang-r353983c/lib64/clang/9.0.3/lib/linux/libclang_rt.scudo_minimal-aarch64-android.a + -m clang-r353983c/lib64/clang/9.0.3/lib/linux/libclang_rt.scudo_minimal-arm-android.a + -m clang-r353983c/lib64/clang/9.0.3/lib/linux/libclang_rt.scudo_minimal-i686-android.a + -m clang-r353983c/lib64/clang/9.0.3/lib/linux/libclang_rt.scudo_minimal-x86_64-android.a +``` + diff --git a/docker/apt.conf b/android-builder-docker/apt.conf similarity index 100% rename from docker/apt.conf rename to android-builder-docker/apt.conf diff --git a/docker/sources.list b/android-builder-docker/sources.list similarity index 100% rename from docker/sources.list rename to android-builder-docker/sources.list diff --git a/assets/redroid11.png b/assets/redroid11.png new file mode 100644 index 0000000..a26e55c Binary files /dev/null and b/assets/redroid11.png differ diff --git a/build.md b/build.md deleted file mode 100644 index ae0dbbc..0000000 --- a/build.md +++ /dev/null @@ -1,26 +0,0 @@ -# Build ReDroid - -## Get Source -- `repo init -u git@github.com:remote-android/platform_manifests.git -b [BRANCH] --depth=1 && repo sync -c` -BRANCH could be redroid-8.1.0, redroid-9.0.0, redroid-10.0.0, redroid-11.0.0 etc. - -## Build -suggest build with [docker](./docker) - -```bash -# intent changes for redroid-10.0.0 (copyfile hook during repo sync), DO NOT Panic -# [Android Clang/LLVM Toolchain](https://android.googlesource.com/toolchain/llvm_android/) - -project prebuilts/clang/host/linux-x86/ (*** NO BRANCH ***) - -m clang-r353983c/lib64/clang/9.0.3/lib/linux/libclang_rt.scudo_minimal-aarch64-android.a - -m clang-r353983c/lib64/clang/9.0.3/lib/linux/libclang_rt.scudo_minimal-arm-android.a - -m clang-r353983c/lib64/clang/9.0.3/lib/linux/libclang_rt.scudo_minimal-i686-android.a - -m clang-r353983c/lib64/clang/9.0.3/lib/linux/libclang_rt.scudo_minimal-x86_64-android.a -``` - -## Run -1. create rootfs only image (Dev purpose): `cd $BUILD_OUT && sudo tar --xattrs -c -C root . | docker import -c 'ENTRYPOINT ["/init", "qemu=1", "androidboot.hardware=redroid"]' - redroid-dev` -2. run with rootfs only image: `docker run -v ~/data:/data -itd -p 5555:5555 --rm --memory-swappiness=0 --privileged -v $BUILD_OUT/system:/system -v $BUILD_OUT/vendor:/vendor redroid-dev` -3. create full image: `cd $BUILD_OUT && sudo tar --xattrs -c system vendor -C root --exclude="system" --exclude="vendor" . | docker import -c 'ENTRYPOINT ["/init", "qemu=1", "androidboot.hardware=redroid"]' - redroid` -4. run with full image: `docker run -v ~/data:/data -itd -p 5555:5555 --rm --memory-swappiness=0 --privileged redroid` - diff --git a/contributing.md b/contributing.md deleted file mode 100644 index d48c268..0000000 --- a/contributing.md +++ /dev/null @@ -1,3 +0,0 @@ -# Contributing - -TBD diff --git a/deploy.md b/deploy.md deleted file mode 100644 index f3e585a..0000000 --- a/deploy.md +++ /dev/null @@ -1,4 +0,0 @@ -# ReDroid Deployment - -TBD - diff --git a/docker/README.md b/docker/README.md deleted file mode 100644 index dc627f8..0000000 --- a/docker/README.md +++ /dev/null @@ -1,13 +0,0 @@ -# Build ReDroid with docker - -create build image: `docker build --build-arg userid=$(id -u) --build-arg groupid=$(id -g) --build-arg username=$(id -un) -t android-build-trusty .` -run build image: `docker run -it --rm -v $ANDROID_BUILD_TOP:/src android-build-trusty` - -build *ReDroid*: -- `cd /src; . build/envsetup.sh` -- `lunch` -- `m` - -sync build (fix file owner / permission etc.) -- `export BUILD_OUT=~/redroid_out && cd vendor/redroid && ./sync.sh` (pasword same as `id -un`) - diff --git a/gms.md b/gms.md deleted file mode 100644 index 6c076ff..0000000 --- a/gms.md +++ /dev/null @@ -1,7 +0,0 @@ -# GMS support in ReDroid - -## Open GApps -[Open GApps](https://opengapps.org/) - -## microG -[microG](https://microg.org/) diff --git a/native_bridge/Dockerfile b/native_bridge/Dockerfile deleted file mode 100644 index f9f0ad0..0000000 --- a/native_bridge/Dockerfile +++ /dev/null @@ -1,6 +0,0 @@ -ARG BASE_IMAGE -FROM $BASE_IMAGE - -ADD native-bridge.tar / - -COPY nativebridge.rc /vendor/etc/init/nativebridge.rc diff --git a/native_bridge/README.md b/native_bridge/README.md deleted file mode 100644 index 2bee236..0000000 --- a/native_bridge/README.md +++ /dev/null @@ -1,36 +0,0 @@ -# Native Bridge Support in ReDroid - -There are several choose to support native bridge (typically for arm) in *ReDroid* -- `libndk_translation` from Arm -- `libhoudini` from Intel -- `QEMU` - -Prepare the resource and build the image. -And run `docker build --build-arg BASE_IMAGE= .` - -```bash -# example structure, be careful the file owner and mode - -system/ -├── bin -│   ├── arm -│   └── arm64 -├── etc -│   ├── binfmt_misc -│   └── init -├── lib -│   ├── arm -│   └── libnb.so -└── lib64 - ├── arm64 - └── libnb.so - -# grab libndk_translation from Android 11 Emulator -find /system \( -name 'libndk_translation*' -o -name '*arm*' -o -name 'ndk_translation*' \) | tar -cf native-bridge.tar -T - - -``` - -Example: Patch libndk_translation in ReDroid 11 - -![Screenshot of ReDroid 11 with libndk_translation](./redroid_11_libndk_translation.png) - diff --git a/native_bridge/nativebridge.rc b/native_bridge/nativebridge.rc deleted file mode 100644 index 5c5bd0c..0000000 --- a/native_bridge/nativebridge.rc +++ /dev/null @@ -1,9 +0,0 @@ -on early-init - setprop ro.product.cpu.abilist x86_64,arm64-v8a,x86,armeabi-v7a,armeabi - setprop ro.product.cpu.abilist64 x86_64,arm64-v8a - setprop ro.product.cpu.abilist32 x86,armeabi-v7a,armeabi - setprop ro.dalvik.vm.isa.arm x86 - setprop ro.dalvik.vm.isa.arm64 x86_64 - setprop ro.enable.native.bridge.exec 1 - setprop ro.dalvik.vm.native.bridge libndk_translation.so - setprop ro.ndk_translation.version 0.2.2 diff --git a/native_bridge/redroid_11_libndk_translation.png b/native_bridge/redroid_11_libndk_translation.png deleted file mode 100644 index 6ebb44d..0000000 Binary files a/native_bridge/redroid_11_libndk_translation.png and /dev/null differ diff --git a/webrtc.md b/webrtc.md deleted file mode 100644 index d753bc4..0000000 --- a/webrtc.md +++ /dev/null @@ -1,34 +0,0 @@ -# WebRTC support in ReDroid - -TBD - -```text - - +-------------------------------------------+ - | K8S Cluster | - | | - | +-----------+ | - | | Drivers | | - + | +-----------+ | - | | | - | | | - | | +-----------+ | - | | +----------> Nginx | | - gRPC | gRPC | | +-----------+ | -+--------+ WebRTC | +------+ WebRTC | +-----------+ | | -| Client <------------> FW +-----+-------------> Envoy +---+ | -+--------+ | +------+ | | +-----------+ | | - | | | | | +-----------+ | - | | | | +----------> Server | | - | | | | +-----^-----+ | - | | | +-----v-----+ | | - | +-------------> coturn | | | - | | | +-----+-----+ | | - | | | | | | - + | | | | | - | | +-----v-----+ | | - +-------------> Android +--------------------+ | - | +-----------+ | - | | - +-------------------------------------------+ -```