✅ Verified on M1 Max

Train AI Agents
on Your Mac

Teach AI models to get better at tasks through practice. Reinforcement learning that runs on Apple Silicon — no expensive GPU servers required.

GitHub → Quick Start 🧪 Test Results

What is this?

ART-MLX trains AI models to improve at specific tasks through trial and error — like how you get better at video games by playing more.

🎯 The Problem

ChatGPT and Claude are general-purpose. They're okay at everything but great at nothing specific. What if you want an AI that's really good at your specific task — answering your emails, navigating your app, playing your game?

💡 The Solution

Take a smaller model, have it practice your task thousands of times, reward it when it does well, and update its weights. This is called reinforcement learning (RL).

The catch (before ART-MLX)

The tools for this (like OpenPipe's ART) only run on NVIDIA GPUs. If you have a Mac, your options were:

Option Cost Friction
Rent cloud GPU (RunPod, Lambda) $0.50-3/hour Upload code, wait, download
Use W&B ServerlessBackend $15-200/run Network latency, API limits
ART-MLX (this) $0 Run locally in seconds

How it works

GRPO (Group Relative Policy Optimization) — the same algorithm used to train DeepSeek-R1's reasoning abilities.

┌─────────────────────────────────────────────────────────────────┐ │ THE TRAINING LOOP │ └─────────────────────────────────────────────────────────────────┘ 1. PRACTICE 2. EVALUATE 3. LEARN ─────────── ────────── ───────── Agent tries Did it work? Update weights the task ✓ Win → +1 Good actions "Play this ✗ Lose → -1 get more likely game move" ~ Draw → 0 Bad actions (multiple get less likely attempts) Compare to group average → Repeat!

What you're actually training

Base Model

A pre-trained LLM (e.g., Qwen 0.5B-7B). This stays frozen — we don't change it.

LoRA Adapters

Small trainable layers (~1.5M params) that modify the model's behavior. This is what we update.

Output

A set of adapter weights you can merge into the base model or share on Hugging Face.

Training data

You don't need a dataset. The model generates its own training data by trying the task. You just define:

  • The task: What should the agent try to do?
  • The reward: How do you score success/failure?

What can you build?

Any task where you can define success/failure and run many attempts.

📧 Email Agent

Train a model to draft replies that match your style. Reward = human approval rating.

python examples/industry/email_support_agent.py

🗏️ SQL Assistant

Natural language to SQL that actually works. Reward = query correctness + performance.

python examples/industry/sql_assistant.py

👀 Code Review

PR reviews that identify real issues. Reward = developer acceptance rate.

python examples/industry/code_review_agent.py

🎮 Game Bots

Tic Tac Toe (simple) and 2048 (complex). Reward = win/lose or score.

python examples/mlx_tictactoe.py

Output artifacts

After training, you get LoRA adapter weights. You can:

  • Upload to Hugging Face: python -m art.mlx.export --checkpoint ./model --repo user/name --push
  • Merge into base model for a single fine-tuned model
  • Convert for Ollama/llama.cpp using standard GGUF conversion
  • Keep separate and load dynamically (hot-swap behaviors)

Quick Start

Get training in 3 commands. Requires Apple Silicon Mac (M1/M2/M3/M4). 

# Clone the repo
git clone https://github.com/menonpg/art-mlx
cd art-mlx

# Install with MLX dependencies
pip install -e ".[mlx]"

# Run the Tic Tac Toe training example
python examples/mlx_tictactoe.py

What happens

  1. Downloads Qwen 0.5B 4-bit model (~300MB first time)
  2. Applies LoRA adapters (1.47M trainable params)
  3. Plays 30 evaluation games → shows initial win rate
  4. Trains for 15 steps (90 games total) with GRPO
  5. Plays 30 more games → shows final win rate

Expected result: ~53% → ~57% win rate in ~90 seconds. See full test results →

Status & Roadmap

✓
MLX Backend — Drop-in replacement for CUDA backend
✓
GRPO Algorithm — Policy gradient with group-relative advantages
✓
LoRA Training — QLoRA with frozen quantized base
✓
Gradient Updates — Real backprop via mx.value_and_grad
✓
Tic Tac Toe Example — End-to-end training verified on M1 Max
✓
Checkpoint Save/Load — Persist training progress
✓
2048 Game Example — More complex task with longer trajectories
✓
Industry Examples — Email support, SQL assistant, code review agents
✓
HuggingFace Export — python -m art.mlx.export --push
○
Benchmarks — Performance comparison vs ServerlessBackend
○
PR to OpenPipe — Contribute MLX backend upstream

About

ART-MLX is an open-source project from The Menon Lab. We build practical AI tools that work on hardware you already own.

GitHub · Blog · The Menon Lab