I wonder what makes them choose rk3588 considering rockchip abandoned the software support as most of software maintainer are the community🤔. Not to mention one of the maintainer that brought fairly recent kernel for rockchip SoC also went away.

Previously I had Orange Pi 5 that happen to have this rk3588 SoC, unfortunately once I discovered that the gpu driver is lacking I went back to x86 based solution (retro emulation station setup).

Maybe this resources can help your local LLM project.

I see you seem fascinated by NPUs, here is probably one of the most recent publications that discuss performance comparison between NPU, GPU, and CPU for Edge computing case using YoloV5 model. And here’s another benchmark provided by Google for its TPU product (Google Coral).

Given the circumstances, it’s unlikely for current generation NPU to fulfill those heavy data tasks like upscaling video or generating video but for something like Edge computing they provide fairly performant solutions at affordable prices (in the case of NPU you can get them on Single Board Computer using Rockchip SoC especially RK3588 one). If you’re using Raspberry Pi 5 or other SBC, you can get Google Coral (M.2 or USB version).

Lastly, as recent as July 28th 2024, Anandtech AI benchmark on the new Ryzen AI 9 HX 370 still using CPU because the software to benchmark the NPU modules isn’t available yet for desktop environment to prevent any bias in result.

Imo NPU for average consumer usage is still niche (compared to something like CUDA/Tensor cores). Just like TPU, NPU is just another hardware acceleration for specific tasks that most of average user can be fulfilled with CPU/GPUs.

Probably they took what most avg user would see. Android? We have custom roms. AI Tools (assuming all of them?) Not even close.

But still find it hilarious that OP even have to post the correction.