Running LLMs on the Indiedroid Nova

We are excited to announce a new guide on how to run large language models (LLMs) on your Indiedroid Nova. This tutorial provides detailed instructions on setting up and executing LLMs using the built-in 6 TOPS NPU (Neural Processing Unit).

Guide Overview

This guide covers everything from cloning the necessary repositories to running RKLLM models.

For detailed instructions, visit our official wiki page.

Step-by-Step Instructions

The guide includes the following steps:

1. Cloning the Repository: Provides the command to clone the necessary GitHub repository.
2. Installing Dependencies: Details the process of navigating the directory and running the installation script.
3. Obtaining Models: Instructions on how to clone preconverted RKLLM models from the "Hugging Face" platform.
4. Running the Model: The final step to execute the model using the appropriate command.

Conclusion

By following this guide, you can effectively run large language models on your Indiedroid Nova. Visit our wiki page for the full tutorial and start exploring the potential of your device today.