{"id":696,"date":"2025-07-08T13:29:28","date_gmt":"2025-07-08T13:29:28","guid":{"rendered":"https:\/\/blog.aetherix.com\/?p=696"},"modified":"2026-03-27T13:12:22","modified_gmt":"2026-03-27T13:12:22","slug":"jetson-generative-ai-jetson-copilot","status":"publish","type":"post","link":"https:\/\/blog.openzeka.com\/en\/jetson-generative-ai-jetson-copilot\/","title":{"rendered":"Jetson Generative AI \u2013 Jetson Copilot"},"content":{"rendered":"

Large Language Models become exponentially more powerful when combined with Retrieval-Augmented Generation (RAG)\u2014Jetson Copilot<\/strong> demonstrates how to run open-source LLMs locally with access to your indexed knowledge base, creating a personalized AI assistant that understands your specific domain and documents in real time<\/strong> on your Jetson device.<\/p>\n

In this article you’ll learn how to set up and run Jetson Copilot, a reference application that combines local LLM inference with RAG capabilities for intelligent document search and question answering.<\/p>\n<\/div>

Requirements<\/h3><\/h2><\/div>\n
\n

 <\/p>\n

 <\/p>\n\n\n\n\n\n\n\n\n\n\n\n
Hardware \/ Software<\/th>\nNotes<\/th>\n<\/tr>\n<\/thead>\n
Jetson AGX Orin (64GB)<\/strong><\/td>\n\n
\n
Recommended for best performance<\/div>\n<\/div>\n<\/td>\n<\/tr>\n
\n
\n
Jetson AGX Orin (32GB)<\/strong><\/div>\n<\/div>\n<\/td>\n
\n
\n
Good performance for most use cases<\/div>\n<\/div>\n<\/td>\n<\/tr>\n
\n
\n
Jetson Orin Nano (8GB)<\/strong><\/div>\n<\/div>\n<\/td>\n
\n
\n
Minimum requirement<\/div>\n<\/div>\n<\/td>\n<\/tr>\n
JetPack 5 (L4T r35.x)\u00a0<\/strong><\/p>\n
\n
JetPack 6 (L4T r36.x)<\/strong><\/div>\n<\/div>\n<\/td>\n
\n
\n
Both versions supported<\/div>\n<\/div>\n<\/td>\n<\/tr>\n
\n
\n
NVMe SSD highly recommended<\/strong><\/div>\n<\/div>\n<\/td>\n
\n
\n
For storage speed and space<\/div>\n<\/div>\n<\/td>\n<\/tr>\n
\n
\n
6GB for jetrag container<\/strong><\/div>\n<\/div>\n<\/td>\n
\n
\n
Container image storage<\/div>\n<\/div>\n<\/td>\n<\/tr>\n
\n
\n
~4GB for default models<\/strong><\/div>\n<\/div>\n<\/td>\n
\n
\n
llama3 and mxbai-embed-large models<\/div>\n<\/div>\n<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n

What is Jetson Copilot?<\/h2><\/div>

Jetson Copilot is a reference application for a local AI assistant that demonstrates:<\/p>\n

    \n
  • Running open-source LLMs<\/strong> (large language models) on device<\/li>\n
  • RAG (retrieval-augmented generation)<\/strong> to let LLM access your locally indexed knowledge<\/li>\n
  • Document indexing<\/strong> and search for personalized knowledge bases<\/li>\n
  • Web-based interface<\/strong> for easy interaction and management<\/li>\n<\/ul>\n<\/div>

    Note:<\/strong> You do not need jetson-containers installed on your system. Jetson Copilot uses the jetrag container image that is managed and built separately.<\/em><\/p>\n<\/div>

    Step-by-Step Setup<\/h2><\/div>

    1.\u00a0 Clone the repository<\/strong><\/h3><\/div>