refine doc; add more deploy instructions

README.md

@@ -14,9 +14,9 @@ English | [简体中文](README.zh-cn.md)
 - [License](#license)
 
 ## Overview
-**ReDroid** (*Re*mote an*Droid*) is a GPU accelerated AIC (Android In Container) solution. You can boot many
+*ReDroid* (*Re*mote an*Droid*) is a GPU accelerated AIC (Android In Cloud) solution. You can boot many
-instances in Linux host (`Docker`, `podman`, `k8s` etc.). *ReDroid* supports both `arm64` and `amd64` architectures. 
+instances in Linux host (`Docker`, `podman`, `k8s` etc.). *redroid* supports both `arm64` and `amd64` architectures. 
-*ReDroid* is suitable for Cloud Gaming, VMI (Virtual Mobile Infrastructure), Automation Test and more.
+*ReDroid* is suitable for Cloud Gaming, Virtualise Phones, Automation Test and more.
 
 ![Screenshot of ReDroid 11](./assets/redroid11.png)
 
@@ -29,69 +29,75 @@ Currently supported:
 - Android 9 (`redroid/redroid:9.0.0-latest`, `redroid/redroid:9.0.0-amd64`, `redroid/redroid:9.0.0-arm64`)
 - Android 8.1 (`redroid/redroid:8.1.0-latest`, `redroid/redroid:8.1.0-amd64`, `redroid/redroid:8.1.0-arm64`)
 
-Tested Platforms:
-- Ubuntu 16.04 / 18.04 / 20.04 (amd64 / arm64)
-- Amazon Linux 2 (amd64 / arm64)
-- Alibaba Cloud Linux 2 (amd64)
-- Alibaba Cloud Linux 3 (amd64 / arm64)
-- WSL 2 (Ubuntu) (amd64)
-- CentOS (amd64\*, arm64\*)
-- OpenEuler 20.03 (amd64, arm64\*)
-
-\* customized kernel required
 
 ## Getting Started
-*ReDroid* requires some kernel modules (`binderfs`, `ashmem` etc.).
+*redroid* should capabale running on any linux (with some kernel features enabled).
-**Check [kernel modules](https://github.com/remote-android/redroid-modules) to install these modules.**
+
+Quick start on *Ubuntu 20.04* here; Check [deploy section](deploy/README.md) for other distros.
 
 ```bash
-# start and connect via `scrcpy`
+## install docker https://docs.docker.com/engine/install/#server
-docker run -itd --rm --memory-swappiness=0 --privileged \
+
+## install required kernel modules
+apt install linux-modules-extra-`uname -r`
+modprobe binder_linux devices="binder,hwbinder,vndbinder"
+modprobe ashmem_linux
+
+
+## running redroid
+docker run -itd --rm --privileged \
     --pull always \
     -v ~/data:/data \
     -p 5555:5555 \
     redroid/redroid:11.0.0-latest
 
-adb connect <IP>:5555
+### Explanation:
-scrcpy --serial <IP>:5555
+###   --pull always    -- use latest image
+###   -v ~/data:/data  -- mount data partition
+###   -p 5555:5555     -- expose adb port
 
-## Explanation:
-##   --pull always    -- be sure to use the latest image
-##   -v ~/data:/data  -- mount data partition
-##   -p 5555:5555     -- expose adb port, you can connect via `adb connect <HOST-IP>`
 
+## install adb https://developer.android.com/studio#downloads
+adb connect localhost:5555
+### NOTE: change localhost to IP if running redroid remotely
+
+## view redroid screen
+## install scrcpy https://github.com/Genymobile/scrcpy/blob/master/README.md#get-the-app
+scrcpy -s localhost:5555
+### NOTE: change localhost to IP if running redroid remotely
+###     typically running scrcpy on your local PC
 ```
 
 ## Configuration
-required params (already added in docker image)
-- qemu=1
-- androidboot.hardware=redroid
 
-display params
+```
-- redroid.width=720
+## running redroid with custom settings (custom display for example)
-- redroid.height=1280
+docker run -itd --rm --privileged \
-- redroid.fps=15
+    --pull always \
-- ro.sf.lcd_density=320
+    -v ~/data:/data \
+    -p 5555:5555 \
+    redroid/redroid:11.0.0-latest \
+    redroid.width=1080 \
+    redroid.height=1920 \
+    androidboot.redroid_dpi=480 \
+```
 
-Network:
+| Param | Description | Default |
-- net.eth0.dns1=<IP>
+| --- | --- | --- |
-- net.eth0.proxy.type=[static|pac|none|unassigned]
+| `qemu` | export param with the "ro.kernel." prefix; **NOT** `QEMU-KVM` related | 1 |
-- net.eth0.proxy.host=<IP>
+| `androidboot.hardware` | specify `ro.boot.hardware` prop | redroid |
-- net.eth0.proxy.port=<port>
+| `redroid.width` | display width | 720 |
+| `redroid.height` | display height | 1280 |
+| `redroid.fps` | display FPS | auto-detect |
+| `androidboot.redroid_dpi` | display DPI | 320 |
+| `net.eth0.dns1` | DNS | 8.8.8.8 |
+| `net.eth0.proxy.type` | Proxy type; choose from: `static`, `pac`, `none`, `unassigned` | |
+| `net.eth0.proxy.host` | | |
+| `net.eth0.proxy.port` | | |
+| `redroid.gpu.mode` | choose from: `auto`, `host`, `guest`;<br>`guest`: use software rendering;<br>`host`: use GPU accelerated rendering;<br>`auto`: auto detect | `auto` |
+| `redroid.gpu.node` | | auto-detect |
+| `ro.xxx`| **DEBUG** purpose, allow override `ro.xxx` prop; For example, set `ro.secure=0`, then root adb shell provided by default | |
 
-GPU accelerating
-*ReDroid* use mesa3d to accelerate 3D rendering.
-Currently tested platforms:
-- AMD (arm64, amd64 with `amdgpu` driver)
-- Intel (amd64 with `i915` driver)
-- virtio-gpu (vendor agnostic, arm64 and amd64)
-
-params:
-- redroid.gpu.mode=[auto|host|guest]
-- redroid.gpu.node=[/dev/dri/renderDxxx]
-
-NOTE: you can override system props prefixed with `qemu.` or `ro.`. for example, you can set `ro.secure=0`, then 
-you can get root adb shell by default.
 
 ## Native Bridge Support
 It's possible to run `arm` Apps in `x86` *ReDroid* instance via `libhoudini`, `libndk_translator` or `QEMU translator`.

deploy/README.md

@@ -0,0 +1,84 @@
+# Deploy redroid 
+
+*redroid* should capable to run on any linux environment as long as the 
+required kernel features avaiable (`binderfs` etc.). Fortunately, these 
+kernel features are already included in some linux distributions 
+(maybe need seperate kernel / modules package). *redroid* also provide
+[redroid-modules](https://github.com/remote-android/redroid-modules) repo
+to support some linux distrutions without those kernel features. And as last
+resort, custom kernels are always capable.
+
+
+**madatory kernel features**
+- `binderfs`
+- `ashmem` / `memfd`
+- `ipv6`
+- `ion` / `dma_heap`
+- 4KB page size
+- ...
+
+
+**deployment per distro / platform**
+- [alibaba-cloud-linux](alibaba-cloud-linux.md)
+- [amazon-linux](amazon-linux.md)
+- [arch-linux](arch-linux.md)
+- [centos](centos.md)
+- [debian](debian.md)
+- [kubernetes](kubernetes.md)
+- [ubuntu](ubuntu.md)
+- [wsl](wsl.md)
+
+
+**general deploy redroid instructions**
+```
+## install docker https://docs.docker.com/engine/install/#server
+
+## make sure required kernel features enabled
+
+
+## running redroid
+docker run -itd --rm --privileged \
+    --pull always \
+    -v ~/data11:/data \
+    -p 5555:5555 \
+    --name redroid11 \
+    redroid/redroid:11.0.0-latest
+
+### Explanation:
+###   --pull always    -- use latest image
+###   -v ~/data11:/data  -- mount data partition
+###   -p 5555:5555     -- expose adb port
+
+
+## install adb https://developer.android.com/studio#downloads
+adb connect localhost:5555
+### NOTE: change localhost to IP if running redroid remotely
+
+## view redroid screen
+## install scrcpy https://github.com/Genymobile/scrcpy/blob/master/README.md#get-the-app
+scrcpy -s localhost:5555
+### NOTE: change localhost to IP if running redroid remotely
+###     typically running scrcpy on your local PC
+
+
+## running without `ashmem` (removed since linux 5.18)
+## NOTE: plan to enable `memfd` by default
+docker run -itd --rm --privileged \
+    --pull always \
+    -v ~/data11:/data \
+    -p 5555:5555 \
+    --name redroid11 \
+    redroid/redroid:11.0.0-latest \
+    androidboot.use_memfd=1
+
+
+## running 64bit-only redroid (only redroid12 published)
+## some ARM CPUs only support 64bit mode, or just want to remove legacy 32bit runtime
+docker run -itd --rm --privileged \
+    --pull always \
+    -v ~/data12_64only:/data \
+    -p 5555:5555 \
+    --name redroid12_64only \
+    redroid/redroid:12.0.0_64only-latest \
+
+```

deploy/alibaba-cloud-linux.md

@@ -0,0 +1,14 @@
+# Deploy on Alibaba Cloud Linux
+
+Check [redroid-modules](https://github.com/remote-android/redroid-modules) to install
+required kernel modules.
+
+```
+## running redroid
+docker run -itd --rm --privileged \
+    --pull always \
+    -v ~/data11:/data \
+    -p 5555:5555 \
+    --name redroid11 \
+    redroid/redroid:11.0.0-latest
+```

deploy/amazon-linux.md

@@ -0,0 +1,15 @@
+# Deploy redroid on Amazon Linux
+*5.4* / *4.14* kernel supported currently (*5.10* support still under construction).
+
+Check [redroid-modules](https://github.com/remote-android/redroid-modules) to install
+required kernel modules.
+
+```
+## running redroid
+docker run -itd --rm --privileged \
+    --pull always \
+    -v ~/data11:/data \
+    -p 5555:5555 \
+    --name redroid11 \
+    redroid/redroid:11.0.0-latest
+```

deploy/arch-linux.md

@@ -0,0 +1,15 @@
+# Deploy on Arch Linux
+
+```
+## install zen kernel
+pacman -S linux-zen
+
+## run with memfd enabled
+docker run -itd --rm --privileged \
+    --pull always \
+    -v ~/data11:/data \
+    -p 5555:5555 \
+    --name redroid11 \
+    redroid/redroid:11.0.0-latest \
+    androidboot.use_memfd=1
+```

deploy/centos.md

@@ -0,0 +1,29 @@
+# Deploy redroid on CentOS
+```
+## use custom 5.10 kernel
+
+### enable following kernel features
+
+# codec2 required, can use ION for legacy kernel
+CONFIG_DMABUF_HEAPS=y
+CONFIG_DMABUF_HEAPS_SYSTEM=y
+
+# optional, can use memfd
+CONFIG_STAGING=y
+CONFIG_ASHMEM=y
+
+# binderfs required
+CONFIG_ANDROID=y
+CONFIG_ANDROID_BINDER_IPC=y
+CONFIG_ANDROID_BINDERFS=y
+CONFIG_ANDROID_BINDER_DEVICES="binder,hwbinder,vndbinder"
+
+
+### running redroid
+docker run -itd --rm --privileged \
+    --pull always \
+    -v ~/data11:/data \
+    -p 5555:5555 \
+    --name redroid11 \
+    redroid/redroid:11.0.0-latest
+```

deploy/debian.md

@@ -0,0 +1,21 @@
+# Deploy redroid on Debian
+```
+## Debian 11 (kernel 5.18)
+
+### the included `binder_linux` NOT support `binderfs`
+### so use custom kernel here
+
+### enable binderfs
+CONFIG_ANDROID_BINDERFS=y
+CONFIG_ANDROID_BINDER_DEVICES="binder,hwbinder,vndbinder"
+
+
+### running redroid
+docker run -itd --rm --privileged \
+    --pull always \
+    -v ~/data11:/data \
+    -p 5555:5555 \
+    --name redroid11 \
+    redroid/redroid:11.0.0-latest \
+    androidboot.use_memfd=1
+```

deploy/kubernetes.md

@@ -0,0 +1,7 @@
+# Deploy redroid on K8S
+
+**Make sure required kernel features available, check details on [README.md](README.md)**
+
+```
+kubectl apply -k kubernetes/overlays/<VER>
+```

deploy/kubernetes/base/kustomization.yaml

@@ -0,0 +1,2 @@
+resources:
+- sts.yaml

deploy/kubernetes/base/sts.yaml

@@ -0,0 +1,36 @@
+apiVersion: apps/v1
+kind: StatefulSet
+metadata:
+  name: redroid11
+spec:
+  selector:
+    matchLabels:
+      app: redroid11
+  replicas: 1
+  serviceName: redroid
+  template:
+    metadata:
+      labels:
+        app: redroid11
+    spec:
+      terminationGracePeriodSeconds: 0
+      containers:
+      - name: redroid
+        image: redroid/redroid:11.0.0-latest
+        readinessProbe:
+          exec:
+            command:
+            - /system/bin/sh
+            - -c
+            - test "1" = "`/system/bin/getprop sys.boot_completed`"
+          initialDelaySeconds: 5
+        securityContext:
+          privileged: True
+        stdin: true
+        tty: true
+        volumeMounts:
+        - name: data
+          mountPath: /data
+      volumes:
+      - name: data
+        emptyDir: {}

deploy/kubernetes/overlays/10.0.0/kustomization.yaml

@@ -0,0 +1,23 @@
+bases:
+- ../../base
+
+images:
+- name: redroid/redroid
+  newTag: 10.0.0-latest
+
+patches:
+- target:
+    kind: StatefulSet
+  patch: |-
+    - op: replace
+      path: /metadata/name
+      value: redroid10
+    - op: replace
+      path: /spec/replicas
+      value: 1
+    - op: replace
+      path: /spec/selector/matchLabels/app
+      value: redroid10
+    - op: replace
+      path: /spec/template/metadata/labels/app
+      value: redroid10

deploy/kubernetes/overlays/11.0.0/kustomization.yaml

@@ -0,0 +1,23 @@
+bases:
+- ../../base
+
+images:
+- name: redroid/redroid
+  newTag: 11.0.0-latest
+
+patches:
+- target:
+    kind: StatefulSet
+  patch: |-
+    - op: replace
+      path: /metadata/name
+      value: redroid11
+    - op: replace
+      path: /spec/replicas
+      value: 1
+    - op: replace
+      path: /spec/selector/matchLabels/app
+      value: redroid11
+    - op: replace
+      path: /spec/template/metadata/labels/app
+      value: redroid11

deploy/kubernetes/overlays/12.0.0-64only/kustomization.yaml

@@ -0,0 +1,23 @@
+bases:
+- ../../base
+
+images:
+- name: redroid/redroid
+  newTag: 12.0.0-64only-latest
+
+patches:
+- target:
+    kind: StatefulSet
+  patch: |-
+    - op: replace
+      path: /metadata/name
+      value: redroid12-64only
+    - op: replace
+      path: /spec/replicas
+      value: 1
+    - op: replace
+      path: /spec/selector/matchLabels/app
+      value: redroid12-64only
+    - op: replace
+      path: /spec/template/metadata/labels/app
+      value: redroid12-64only

deploy/kubernetes/overlays/12.0.0/kustomization.yaml

@@ -0,0 +1,23 @@
+bases:
+- ../../base
+
+images:
+- name: redroid/redroid
+  newTag: 12.0.0-latest
+
+patches:
+- target:
+    kind: StatefulSet
+  patch: |-
+    - op: replace
+      path: /metadata/name
+      value: redroid12
+    - op: replace
+      path: /spec/replicas
+      value: 1
+    - op: replace
+      path: /spec/selector/matchLabels/app
+      value: redroid12
+    - op: replace
+      path: /spec/template/metadata/labels/app
+      value: redroid12

deploy/kubernetes/overlays/13.0.0/kustomization.yaml

@@ -0,0 +1,23 @@
+bases:
+- ../../base
+
+images:
+- name: redroid/redroid
+  newTag: 13.0.0-latest
+
+patches:
+- target:
+    kind: StatefulSet
+  patch: |-
+    - op: replace
+      path: /metadata/name
+      value: redroid13
+    - op: replace
+      path: /spec/replicas
+      value: 1
+    - op: replace
+      path: /spec/selector/matchLabels/app
+      value: redroid13
+    - op: replace
+      path: /spec/template/metadata/labels/app
+      value: redroid13

deploy/kubernetes/overlays/8.1.0/kustomization.yaml

@@ -0,0 +1,23 @@
+bases:
+- ../../base
+
+images:
+- name: redroid/redroid
+  newTag: 8.1.0-latest
+
+patches:
+- target:
+    kind: StatefulSet
+  patch: |-
+    - op: replace
+      path: /metadata/name
+      value: redroid8
+    - op: replace
+      path: /spec/replicas
+      value: 1
+    - op: replace
+      path: /spec/selector/matchLabels/app
+      value: redroid8
+    - op: replace
+      path: /spec/template/metadata/labels/app
+      value: redroid8

deploy/kubernetes/overlays/9.0.0/kustomization.yaml

@@ -0,0 +1,23 @@
+bases:
+- ../../base
+
+images:
+- name: redroid/redroid
+  newTag: 9.0.0-latest
+
+patches:
+- target:
+    kind: StatefulSet
+  patch: |-
+    - op: replace
+      path: /metadata/name
+      value: redroid9
+    - op: replace
+      path: /spec/replicas
+      value: 1
+    - op: replace
+      path: /spec/selector/matchLabels/app
+      value: redroid9
+    - op: replace
+      path: /spec/template/metadata/labels/app
+      value: redroid9

deploy/kubernetes/overlays/autogen.sh

@@ -0,0 +1,38 @@
+#!/bin/bash
+
+REPLICAS=1
+
+for ver in 8.1.0 {9..13}.0.0 {12..12}.0.0-64only
+do
+
+    base_ver=`echo $ver | cut -d'.' -f 1`
+    [[ $ver == *"-64only" ]] && base_ver+='-64only'
+
+    [ -d $ver ] || mkdir $ver
+    cat > $ver/kustomization.yaml <<-END
+bases:
+- ../../base
+
+images:
+- name: redroid/redroid
+  newTag: $ver-latest
+
+patches:
+- target:
+    kind: StatefulSet
+  patch: |-
+    - op: replace
+      path: /metadata/name
+      value: redroid$base_ver
+    - op: replace
+      path: /spec/replicas
+      value: $REPLICAS
+    - op: replace
+      path: /spec/selector/matchLabels/app
+      value: redroid$base_ver
+    - op: replace
+      path: /spec/template/metadata/labels/app
+      value: redroid$base_ver
+END
+
+done

deploy/ubuntu.md

@@ -0,0 +1,38 @@
+# Deploy redroid on ubuntu
+```
+# for 5.0+ kernel
+
+## Android required kernel features are packaged in `linux-modules-extra-xxx`
+apt install linux-modules-extra-`uname -r`
+modprobe binder_linux devices="binder,hwbinder,vndbinder"
+modprobe ashmem_linux
+
+## auto load modules
+cat /etc/modules-load.d/redroid.conf
+binder_linux
+ashmem_linux
+
+cat /etc/modprobe.d/redroid.conf
+options binder_linux devices="binder,hwbinder,vndbinder"
+
+
+# for kernel version before 5.0
+
+## option A:
+### upgrade to 5.0+ kernel
+### https://wiki.ubuntu.com/Kernel/LTSEnablementStack
+
+## option B:
+### check https://github.com/remote-android/redroid-modules
+### to install out of tree `binderfs` / `ashmem` kernel modules
+
+
+# running redroid
+docker run -itd --rm --privileged \
+    --pull always \
+    -v ~/data11:/data \
+    -p 5555:5555 \
+    --name redroid11 \
+    redroid/redroid:11.0.0-latest
+
+```

deploy/wsl.md

@@ -0,0 +1,57 @@
+# Deploy redroid on WSL2
+
+```
+## use custom kernel here
+
+## download WSL kernel source from https://github.com/microsoft/WSL2-Linux-Kernel/tags
+wget https://github.com/microsoft/WSL2-Linux-Kernel/archive/refs/tags/linux-msft-wsl-5.10.102.1.tar.gz
+
+tar xf linux-msft-wsl-5.10.102.1.tar.gz
+cd WSL2-Linux-Kernel-linux-msft-wsl-5.10.102.1
+
+cp Microsoft/config-wsl .config
+
+## enable following kernel features
+
+# ipv6 should enabled
+CONFIG_IPV6_ROUTER_PREF=y
+CONFIG_IPV6_ROUTE_INFO=y
+CONFIG_IPV6_MULTIPLE_TABLES=y
+CONFIG_IPV6_SUBTREES=y
+
+# codec2 required, can use ION for legacy kernel
+CONFIG_DMABUF_HEAPS=y
+CONFIG_DMABUF_HEAPS_SYSTEM=y
+
+# optional, can use memfd
+CONFIG_STAGING=y
+CONFIG_ASHMEM=y
+
+# binderfs required
+CONFIG_ANDROID=y
+CONFIG_ANDROID_BINDER_IPC=y
+CONFIG_ANDROID_BINDERFS=y
+CONFIG_ANDROID_BINDER_DEVICES="binder,hwbinder,vndbinder"
+
+
+## build kernel
+make -j`nproc`
+
+## copy `arch/x86_64/boot/bzImage` to some folder in Windows
+
+## follow https://docs.microsoft.com/en-us/windows/wsl/wsl-config#configuration-setting-for-wslconfig
+## to use this new kernel
+cat .wslconfig
+[wsl2]
+kernel=D:\\wsl\\bzImage
+memory=4GB
+
+
+## running redroid
+docker run -itd --rm --privileged \
+    --pull always \
+    -v ~/data11:/data \
+    -p 5555:5555 \
+    --name redroid11 \
+    redroid/redroid:11.0.0-latest
+```