{"id":1372,"date":"2025-09-17T13:28:53","date_gmt":"2025-09-17T13:28:53","guid":{"rendered":"https:\/\/blog.aetherix.com\/?p=1372"},"modified":"2026-03-27T13:38:57","modified_gmt":"2026-03-27T13:38:57","slug":"hammerbench-agx-thors-power-meets-ollama","status":"publish","type":"post","link":"https:\/\/blog.openzeka.com\/en\/hammerbench-agx-thors-power-meets-ollama\/","title":{"rendered":"HammerBench : AGX Thor\u2019s Power Meets Ollama"},"content":{"rendered":"

What is an LLM benchmark and why is it important?<\/h3><\/div>

LLM benchmarks<\/strong> are standardized tests designed to measure how fast, efficient, and accurate large language models (LLMs) perform across different hardware and environments. These tests evaluate metrics such as latency, throughput, and sometimes accuracy to provide an objective view of performance.<\/p>\n

As LLMs continue to grow larger and more complex, choosing the right hardware to run them on becomes a critical decision. Benchmark results are essential to understand which device or infrastructure delivers better performance, to balance cost and efficiency, and to identify the most suitable solution for real-world use cases. In short, LLM benchmarks give both researchers and developers a clear roadmap of how models perform in practice.<\/p>\n

To showcase the performance of Jetson AGX Thor<\/strong>, we are sharing our results and performance charts with you. At the same time, you can also run benchmarks across different GPU types<\/strong> to compare and validate performance for your own workloads. If you want to measure the performance metrics of your own devices and test your models under real-world conditions, get in touch with us. With our solution, your measurements turn into more than just numbers \u2014 they become actionable insights that drive strategic decisions.<\/p>\n<\/div>

How to use HammerBench ?<\/h3><\/div>

\ud83d\udda5\ufe0f What the App Does<\/strong><\/p>\n

This is a Streamlit-based LLM Benchmark Tool interface designed to evaluate large language models (LLMs) on NVIDIA Jetson AGX Thor hardware using Ollama as the backend.<\/p>\n

\u2699\ufe0f Configuration (Left Sidebar)<\/strong><\/p>\n

    \n
  • GPU Information:<\/li>\n
  • Detects if the device is a Jetson (in this case, a Jetson AGX Thor Developer Kit).<\/li>\n
  • Shows details about the GPU (NVIDIA Jetson AGX Thor) and available memory (125,772 MB \u2248 122.8 GB).<\/li>\n<\/ul>\n

    Use Only GPU:<\/strong><\/p>\n

    A checkbox option that allows restricting benchmarks to GPU-only execution.<\/p>\n

    \ud83d\udcca Main Panel<\/strong><\/p>\n

    Title:<\/strong> LLM Benchmark Tool with description: Benchmark LLM models using Ollama with real-time progress tracking.<\/strong><\/em><\/p>\n

    Models Compatible with GPU memory (VRAM) requirements:<\/strong><\/p>\n

      \n
    • Displays a table of available models (llama3.2.1b, gemma3.4b, qwen3.14b, gpt-oss20b, etc.)<\/li>\n
    • Shows how much memory (VRAM in GB) each model requires.<\/li>\n
    • Marks them with \u2705 if they are runnable on the detected GPU.<\/li>\n<\/ul>\n

      Select Models to Benchmark:<\/strong><\/p>\n

        \n
      • Lists the same models with checkboxes so the user can pick which ones to run benchmarks on.<\/li>\n
      • Each option shows the memory requirement for clarity (e.g., gemma3.27b (17 GB), gpt-oss-120b (65 GB)).<\/li>\n<\/ul>\n

        \ud83d\ude80 Purpose<\/strong><\/p>\n

        The tool helps developers and researchers:<\/p>\n

          \n
        • See which LLMs are compatible with their GPU memory.<\/li>\n
        • Select multiple models and run benchmarks to measure performance (latency, throughput, GPU utilization).<\/li>\n
        • Use the results to compare models and make better deployment or scaling decisions.<\/li>\n<\/ul>\n<\/div>