ART-MLX
Verified Test Report

Practical Business Use-Cases
trained on a MacBook

Three real-world GRPO examples run end-to-end on Apple Silicon — a natural-language-to-SQL assistant, a customer-support email agent, and a code-review agent. Each teaches a small model from a page of hand-written examples plus a grading rubric — on your Mac, with no cloud GPU, database, or API keys. Jump to “what the training data actually is” below.

✓ SQL assistant ✓ Email support agent ✓ Code-review agent MLX 0.32.0 Qwen2.5-0.5B-4bit Apple M1 GPU

Tested 2026-07-18 · Python 3.13 · mlx-lm 0.31.3 · ~1.47M trainable LoRA params each

01What these examples are

Each script follows the same pattern: take a page of hand-written example tasks, score the model's attempts with a built-in rubric, run 8 rounds of GRPO training (reinforcement learning) that nudge trainable LoRA adapters toward higher-scoring answers, then re-score. They are self-contained and run offline — the next section explains exactly what that example data and rubric are.

Example 1
SQL Assistant
Data Analytics / BI
Example 2
Email Support Agent
Customer Service
Example 3
Code-Review Agent
Developer Tools

02What the “training data” actually is (the key idea)

There is no giant dataset and no database. Everything the model learns from is written in plain text directly inside each script — about a page of examples. Teaching the model works exactly like coaching a new hire with a worked-examples sheet and a grading rubric. Two ingredients:

Ingredient 1
Example tasks

A short, hand-written list of realistic tasks (6 questions / 6 tickets / 5 code diffs). Literally a Python list at the top of the file. In production you paste in your own real ones.

Ingredient 2
A grader (the rubric)

A set of plain rules that award points for what a good answer looks like — the “answer key.” In production you swap it for a real outcome (did the query return the right rows? did the customer come back happy?).

During training, for each example task the model writes several attempts, the grader scores each one, and training nudges the model toward the higher-scoring attempts. Repeat 8 times. That’s the whole loop.

Exactly what data each example ships with

ExampleThe example tasks (the “data”)How it was createdWhat the grader rewards
SQL Assistant 6 English questions about a 4-table e-commerce database (each with a reference SQL answer) Hand-written synthetic, in the SAMPLE_QUESTIONS list Uses SELECT, the correct tables, the right keywords (SUM/GROUP BY/LIMIT); penalizes SELECT *
Email Support 6 support tickets (login, billing, feature request, integration, cancellation, praise) tagged with category + urgency Hand-written synthetic, in the SAMPLE_TICKETS list Empathy, a concrete next step, greeting/closing, right length; category-aware bonuses
Code Review 5 code diffs, each with a planted bug (hardcoded password, SQL injection, exposed secret, sleep-in-loop, mutable default) Hand-written synthetic, in the SAMPLE_DIFFS list Naming the real issue, being actionable, explaining why, matching severity

How a non-technical person actually does this

  1. Collect ~5–10 real examples of the task. Real tickets from your inbox, real questions your team asks, real pull requests. Pure copy-paste — no code.
  2. Decide what a good answer looks like. Either jot down simple rules (“must apologize, must give a next step, keep under 6 sentences”) or have a colleague rate a handful of answers 1–5. This rubric is the most important part.
  3. Hand both to the script. Paste your examples into the list at the top of the file (or give them to a developer for ~15 minutes to wire in).
  4. Run one command. The model writes attempts, grades itself against your rubric, and practices — all on your Mac, offline.
  5. Graduate to real feedback. When ready, replace the rule-based grader with a real signal: actually run the SQL and check the rows, track whether the customer replied happy, or track whether the developer accepted the review.
The one-sentence version: you teach the model by giving it a few example tasks and a rubric that scores answers — then it practices against the rubric until it scores higher. You are grading homework, and the model is the student.

03Results summary

All three trained through all 8 steps with real gradient updates. "Best" is the top single-attempt score reached during training; the heuristic scorer runs 0.0–1.0.

ExampleBusiness taskBeforeBest in trainingStep timeStatus
sql_assistant.pyNatural language → SQL0.900.90~6–11s✅ ran
email_support_agent.pySupport email drafting0.850.85~12–18s✅ ran
code_review_agent.pyPR code review0.801.00~18–20s✅ ran
Honest caveat. These runs prove the training pipeline works end-to-end on Apple Silicon — not that the model's quality reliably improves. With a tiny 0.5 B model, a keyword-based reward, and only 8 steps, final outputs sometimes degrade (e.g. repetition loops). Real deployments need a larger model, a genuine reward signal (execution results / human feedback), and many more steps.

04SQL Assistant — examples/industry/sql_assistant.py

Use case: let a non-technical analyst ask a question in plain English and get the SQL for a BI tool. Scored on structure (SELECT, JOINs, GROUP BY, LIMIT) and table usage.

TRAINING DATA — 6 English questions about this e-commerce schema (in SAMPLE_QUESTIONS):
  tables: customers, orders, order_items, products

One example item:
  question:          "What are the top 5 customers by total spend?"
  expected_tables:   customers, orders
  expected_keywords: SUM, GROUP BY, ORDER BY, LIMIT
  reference_sql:     SELECT c.name, SUM(o.total_amount) AS total_spend
                     FROM customers c JOIN orders o ON c.id = o.customer_id
                     GROUP BY c.id, c.name ORDER BY total_spend DESC LIMIT 5

Grader: +0.2 has SELECT | +0.1 each correct table | +0.1 each keyword | -0.1 SELECT *
BEFORE TRAINING
Question: What are the top 5 customers by total spend?
Score: 0.90  (+has_select, +uses_customers, +uses_orders, +has_sum,
                +has_group by, +has_order by, +has_limit, +proper_join)

TRAINING
Step 1/8: score=0.56 best=0.70 (5.9s)
Step 2/8: score=0.78 best=0.85 (5.3s)
Step 3/8: score=0.47 best=0.90 (11.2s)
Step 4/8: score=0.62 best=0.90 (9.0s)
Step 5/8: score=0.59 best=0.70 (9.7s)
Step 6/8: score=0.56 best=0.90 (10.5s)
Step 7/8: score=0.76 best=0.85 (7.5s)
Step 8/8: score=0.73 best=0.90 (6.5s)

AFTER TRAINING
Score: 0.90  (query became more garbled but hit the same keyword score)

05Email Support Agent — examples/industry/email_support_agent.py

Use case: draft a fast, consistent first-response email so support agents reply quicker. Scored on empathy, a concrete action, greeting/closing, and length.

TRAINING DATA — 6 real-style support tickets (in SAMPLE_TICKETS):
  login issue (high) | billing question (med) | feature request (low)
  integration error (high) | cancellation (high) | praise (low)

One example item:
  subject:  "Can't login to my account"
  body:     "...trying to login for the past hour... really frustrating, I have a deadline today."
  category: account_access    urgency: high

Grader: +empathy words | +a concrete next step | +greeting/closing | right length
         + category-aware bonuses (churn → retention offer, billing → mentions charge/refund,
         high urgency → acts immediately)
BEFORE TRAINING
Ticket: Can't login to my account
Score: 0.85  (+empathy, +action, +greeting, +closing)

TRAINING
Step 1/8: score=0.69 best=0.75 loss= 0.0021 (13.4s)
Step 2/8: score=0.74 best=0.80 loss= 0.0041 (12.4s)
Step 3/8: score=0.73 best=0.85 loss=-0.0051 (13.2s)
Step 4/8: score=0.61 best=0.85 loss= 0.0246 (15.8s)
Step 5/8: score=0.64 best=0.70 loss= 0.0069 (18.2s)
Step 6/8: score=0.69 best=0.80 loss= 0.0062 (18.0s)
Step 7/8: score=0.71 best=0.80 loss= 0.0009 (18.3s)
Step 8/8: score=0.69 best=0.70 loss= 0.0032 (18.5s)

AFTER TRAINING
Score: 0.70  (-too_long — model fell into an "I apologize" repetition loop)

06Code-Review Agent — examples/industry/code_review_agent.py

Use case: an automated first-pass reviewer that catches security bugs before a human looks. Scored on catching the planted issue, being actionable, and matching severity. This example hit a perfect 1.00 at step 2.

TRAINING DATA — 5 code diffs, each with a planted bug (in SAMPLE_DIFFS):
  auth.py (hardcoded password) | api.py (SQL injection) | utils.py (sleep in loop)
  config.py (exposed AWS secret) | models.py (mutable default)

One example item (api.py):
  -    return db.query(User).filter(User.id == user_id).first()
  +    query = f"SELECT * FROM users WHERE id = {user_id}"      <- SQL injection
  +    return db.execute(query)
  known_issues: sql_injection, raw_query    severity: critical

Grader: +0.15 each named issue | +actionable suggestion | +explains WHY | +matches severity
         -penalizes nitpicking (formatting/whitespace) on critical bugs
BEFORE TRAINING
File: api.py  (diff introduces an f-string SQL query = injection risk)
Score: 0.80  (+identified_sql_injection, +identified_raw_query,
                +actionable, +severity_match)

TRAINING
Step 1/8: score=0.64 best=0.90 (18.2s)
Step 2/8: score=0.83 best=1.00 (19.2s)   <- caught the SQL injection perfectly
Step 3/8: score=0.61 best=0.90 (17.6s)
Step 4/8: score=0.63 best=0.95 (19.0s)
Step 5/8: score=0.69 best=0.90 (19.8s)
Step 6/8: score=0.46 best=0.70 (18.9s)
Step 7/8: score=0.41 best=0.65 (18.6s)
Step 8/8: score=0.47 best=0.70 (18.3s)

AFTER TRAINING
Score: 0.70  (+identified_sql_injection, +identified_raw_query, +severity_match)

07Reproduce it

How to make these actually improve. Swap the heuristic score_*() for a real reward (run the SQL and compare rows; measure ticket resolution; track whether devs accept the review), use a bigger base model, and train for far more steps. The plumbing shown here stays the same.