KanTV ("Kan", meaning "watch" in English and "看" in Chinese), an open source project focusing on studying and practicing on-device AI technology in real scenarios (such as performing online-TV playback, realtime transcription, and online-TV recording at the same time) on Android phones:
-
Watch online TV and local media using a customized
. This project is derived from the original
(that project has stopped maintenance since 2021), with many enhancements and new features. Source code of the customized FFmpeg 6.1 can be found in external/ffmpeg according to FFmpeg's license. Source code of all FFmpeg 6.1's dependent libraries can be found in external/ffmpeg-deps .
-
Watch online TV using a customized
. Source code of the customized ExoPlayer 2.15.1 can be found in android/kantvplayer-exo2 .
-
Record online TV to a local file on the phone.
-
2D graphic performance benchmark.
-
AI subtitle (real-time English subtitle for English online-TV (aka OTT TV) via the great & excellent & amazing whisper.cpp ).
-
Well-maintained turn-key / self-contained workbench for AI experts/researchers who focus on high-value on-device AI R&D activities on Android. Some on-device AI R&D activities (AI algorithm validation, AI model validation, and performance benchmark with ASR/LLM/MTMD (multimodal) on Android) can be done via this project easily.
-
Well-maintained turn-key / self-contained workbench for AI beginners to learn on-device AI technology on Android.
-
Built-in AI models are supported and run entirely offline (no Internet required). These supported AI models can be downloaded in the Android APK directly without manual preparation. APK users can compare the real experience of these AI models on the Android phone. Developers can add other AI models manually in the source code KANTVAIModelMgr.java.
Model Type Capability Source ggml-tiny.en-q8_0 ASR speech-to-text whisper.cpp Qwen1.5-1.8B LLM text-only Alibaba Qwen2.5-3B LLM text-only Alibaba Gemma3-4B LLM text + image (MTMD) Google Gemma-4-E2B LLM text-only (default) Google SmolVLM2-256M LLM text + image (realtime-video-recognition) Huggingface Qwen2.5-Omni-3B LLM text + audio (MTMD) Alibaba -
The JZ's ggml-hexagon used in this project is probably the first open-source reference implementation of a specific llama.cpp backend for Qualcomm Hexagon NPU on Android phones. The backend type (Hexagon cDSP vs. generic ggml) is decided at build time, and the DSP-side thread count is automatically clamped based on the target SoC (e.g., 6 threads on Snapdragon 8Elite, 4 threads on Snapdragon 8Gen3). Its PP (prompt processing) and TG (token generation) performance comprehensively surpasses Qualcomm's official implementation on Snapdragon 8Elite (aka 8Gen4); benchmark comparisons can be found here.
- Clone this repository and build locally, see how to build
- Download pre-built Android APK from https://github.com/zhouwg/kantv/releases
- Download pre-built Android APK from Github CI-build: https://github.com/zhouwg/kantv/actions/
- Android 8.0 (2017.08) --- Android 15 (2024.10) and higher versions with ANY mainstream arm64 mobile SoC.
- An Android smartphone equipped with ANY mainstream high-end mobile SoC is highly recommended for the realtime AI-subtitle feature, otherwise unexpected behavior may occur.
- An Android smartphone equipped with one of the below Qualcomm mobile SoCs is required for verifying/running the ggml-hexagon backend on Android phones:
Snapdragon 8 Gen 2
Snapdragon 8 Gen 3
Snapdragon 8 Elite (aka 8 Gen 4)
Snapdragon 8 Elite Gen 5 (aka 8 Gen 5)
Here is a short video to demonstrate realtime AI subtitle by running the great & excellent & amazing whisper.cpp on an Android phone equipped with Qualcomm Snapdragon 8Gen3 mobile SoC - fully offline, on-device.
realtime-subtitle-by-whispercpp-demo-on-xiaomi14-finetune-20240324.mp4
A screenshot to demonstrate realtime-video-recognition via MTMD from llama.cpp + a lightweight multimodal model SmolVLM2-256M from Huggingface on an Android phone equipped with Qualcomm Snapdragon 8Elite mobile SoC - fully offline, on-device.
- How to build
- How to customize tv.xml for personal needs
- How to download supported AI models in the APK
- Authors
- Acknowledgement
- ChangeLog
- Known issues
- ggml-hexagon:history of ggml-hexagon
- ggml-hexagon:high-level data path of ggml-hexagon
Reporting issues on Android phones equipped with mainstream mobile SoCs or submitting PRs to this project is greatly appreciated.
I use GitHub issues for tracking feature requests and issue reports, please see how to submit an issue in this project.
- Inference engine GGML
- ASR engine whisper.cpp
- LLM engine llama.cpp
- CV engine opencv-mobile
- MTMD (multimodal) engine MTMD subsystem in llama.cpp