{"id":1055,"date":"2025-07-31T05:09:53","date_gmt":"2025-07-31T05:09:53","guid":{"rendered":"https:\/\/blog.aetherix.com\/?p=1055"},"modified":"2026-03-27T13:27:00","modified_gmt":"2026-03-27T13:27:00","slug":"jetson-generative-ai-llama-factory","status":"publish","type":"post","link":"https:\/\/blog.openzeka.com\/en\/jetson-generative-ai-llama-factory\/","title":{"rendered":"Jetson Generative AI \u2013 LLaMA Factory"},"content":{"rendered":"

LLaMA Factory provides a unified framework for fine-tuning large language models with an intuitive web interface. This powerful tool brings professional model training capabilities to Jetson devices, enabling you to customize LLMs for your specific use cases with optimized performance for edge deployment.<\/p>\n

In this article, you’ll learn how to run LLaMA Factory on Jetson Orin for efficient LLM fine-tuning and deployment.<\/strong><\/p>\n<\/div>

<\/span><\/div>

Features<\/h4><\/h4><\/div>
    \n
  • Support for multiple LLM architectures including LLaMA, Qwen, ChatGLM, and more<\/li>\n
  • Multiple training stages: Supervised Fine-Tuning, Reward<\/li>\n
  • Modeling, PPO, DPO, KTO, Pre-Training<\/li>\n
  • Three fine-tuning methods: full, freeze, and lora<\/li>\n
  • Gradio-based web UI with Train, Evaluate & Predict, Chat, and<\/li>\n
  • Export tabs<\/li>\n
  • Built-in dataset support with preview functionality<\/li>\n
  • Integrated chat interface for testing models<\/li>\n
  • Real-time training loss visualization<\/li>\n
  • Advanced configurations for quantization, LoRA, RLHF, and more<\/li>\n
  • Model evaluation with customizable generation parameters<\/li>\n<\/ul>\n<\/div>

    Requirements<\/h4><\/h4><\/div>\n
    \n

     <\/p>\n\n\n\n\n\n\n\n\n\n
    \n
    \n
    Hardware \/ Software<\/div>\n<\/div>\n<\/th>\n
    		\n
    \n
    Notes<\/div>\n<\/div>\n<\/th>\n<\/tr>\n<\/thead>\n
    \n
    \n
    Jetson AGX Orin with \u2265 32 GB RAM<\/strong><\/div>\n<\/div>\n<\/td>\n
    		64GB recommended for larger models<\/td>\n<\/tr>\n
    \n
    \n
    JetPack 6.0+<\/strong><\/div>\n<\/div>\n<\/td>\n
    		For CUDA 12.x support<\/td>\n<\/tr>\n
    \n
    \n
    NVMe SSD<\/strong><\/div>\n<\/div>\n<\/td>\n
    		Essential for model storage and caching<\/td>\n<\/tr>\n
    \n
    \n
    Hugging Face token<\/strong><\/div>\n<\/div>\n<\/td>\n
    		Required for accessing gated models<\/td>\n<\/tr>\n
    \n
    \n
    ~50 GB free storage<\/strong><\/div>\n<\/div>\n<\/td>\n
    		For models and training checkpoints<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<\/div>\n

    Step-by-Step Setup<\/h4><\/h4><\/div>

    1. Create necessary directories<\/h4><\/h4><\/div>