About this Video

• Video Title: 最新Mac Studio M3 Ultra (512GB)でDeepSeek R1（671B）をローカル実行！その性能とは？
• Channel: テックナビ
• Speakers: トール
• Duration: 30:41

Introduction

This video tests the performance of DeepSeek R1 (671B parameter) large language model (LLM) running locally on a new Mac Studio with an M3 Ultra chip (512GB unified memory). The speaker compares its performance to other LLMs of varying sizes and explores the memory bandwidth and other specifications of the M3 Ultra.

Key Takeaways

• The M3 Ultra's high memory bandwidth (significantly exceeding that of NVIDIA's RTX 5000 series) is a major advantage for running large LLMs.
• The 512GB unified memory allows for running LLMs that would be impossible on GPUs with smaller VRAM (e.g., RTX 5090 with 32GB).
• The video benchmarks DeepSeek R1 at different parameter sizes (1.5B, 7B, 14B, 32B, 70B, and 671B), showing a decrease in tokens per second as the model size increases. However, even the 671B parameter model (4-bit quantized version) runs locally, albeit slower than smaller models.
• The performance of DeepSeek R1 in MLX format (optimized for Apple silicon) is compared against its GGUF format (not optimized for Apple silicon). The MLX version shows significantly better performance.
• The video demonstrates the system resource usage (CPU, GPU, Memory) during LLM execution, showing a high GPU utilization, especially for larger models.