Three real-world GRPO examples run end-to-end on Apple Silicon — a natural-language-to-SQL assistant, a customer-support email agent, and a code-review agent. Each teaches a small model from a page of hand-written examples plus a grading rubric — on your Mac, with no cloud GPU, database, or API keys. Jump to “what the training data actually is” below.
Each script follows the same pattern: take a page of hand-written example tasks, score the model's attempts with a built-in rubric, run 8 rounds of GRPO training (reinforcement learning) that nudge trainable LoRA adapters toward higher-scoring answers, then re-score. They are self-contained and run offline — the next section explains exactly what that example data and rubric are.
There is no giant dataset and no database. Everything the model learns from is written in plain text directly inside each script — about a page of examples. Teaching the model works exactly like coaching a new hire with a worked-examples sheet and a grading rubric. Two ingredients:
A short, hand-written list of realistic tasks (6 questions / 6 tickets / 5 code diffs). Literally a Python list at the top of the file. In production you paste in your own real ones.
A set of plain rules that award points for what a good answer looks like — the “answer key.” In production you swap it for a real outcome (did the query return the right rows? did the customer come back happy?).
During training, for each example task the model writes several attempts, the grader scores each one, and training nudges the model toward the higher-scoring attempts. Repeat 8 times. That’s the whole loop.
| Example | The example tasks (the “data”) | How it was created | What the grader rewards |
|---|---|---|---|
| SQL Assistant | 6 English questions about a 4-table e-commerce database (each with a reference SQL answer) | Hand-written synthetic, in the SAMPLE_QUESTIONS list |
Uses SELECT, the correct tables, the right keywords (SUM/GROUP BY/LIMIT); penalizes SELECT * |
| Email Support | 6 support tickets (login, billing, feature request, integration, cancellation, praise) tagged with category + urgency | Hand-written synthetic, in the SAMPLE_TICKETS list |
Empathy, a concrete next step, greeting/closing, right length; category-aware bonuses |
| Code Review | 5 code diffs, each with a planted bug (hardcoded password, SQL injection, exposed secret, sleep-in-loop, mutable default) | Hand-written synthetic, in the SAMPLE_DIFFS list |
Naming the real issue, being actionable, explaining why, matching severity |
All three trained through all 8 steps with real gradient updates. "Best" is the top single-attempt score reached during training; the heuristic scorer runs 0.0–1.0.
| Example | Business task | Before | Best in training | Step time | Status |
|---|---|---|---|---|---|
sql_assistant.py | Natural language → SQL | 0.90 | 0.90 | ~6–11s | ✅ ran |
email_support_agent.py | Support email drafting | 0.85 | 0.85 | ~12–18s | ✅ ran |
code_review_agent.py | PR code review | 0.80 | 1.00 | ~18–20s | ✅ ran |
examples/industry/sql_assistant.pyUse case: let a non-technical analyst ask a question in plain English and get the SQL for a BI tool. Scored on structure (SELECT, JOINs, GROUP BY, LIMIT) and table usage.
TRAINING DATA — 6 English questions about this e-commerce schema (in SAMPLE_QUESTIONS): tables: customers, orders, order_items, products One example item: question: "What are the top 5 customers by total spend?" expected_tables: customers, orders expected_keywords: SUM, GROUP BY, ORDER BY, LIMIT reference_sql: SELECT c.name, SUM(o.total_amount) AS total_spend FROM customers c JOIN orders o ON c.id = o.customer_id GROUP BY c.id, c.name ORDER BY total_spend DESC LIMIT 5 Grader: +0.2 has SELECT | +0.1 each correct table | +0.1 each keyword | -0.1 SELECT *
BEFORE TRAINING Question: What are the top 5 customers by total spend? Score: 0.90 (+has_select, +uses_customers, +uses_orders, +has_sum, +has_group by, +has_order by, +has_limit, +proper_join) TRAINING Step 1/8: score=0.56 best=0.70 (5.9s) Step 2/8: score=0.78 best=0.85 (5.3s) Step 3/8: score=0.47 best=0.90 (11.2s) Step 4/8: score=0.62 best=0.90 (9.0s) Step 5/8: score=0.59 best=0.70 (9.7s) Step 6/8: score=0.56 best=0.90 (10.5s) Step 7/8: score=0.76 best=0.85 (7.5s) Step 8/8: score=0.73 best=0.90 (6.5s) AFTER TRAINING Score: 0.90 (query became more garbled but hit the same keyword score)
examples/industry/email_support_agent.pyUse case: draft a fast, consistent first-response email so support agents reply quicker. Scored on empathy, a concrete action, greeting/closing, and length.
TRAINING DATA — 6 real-style support tickets (in SAMPLE_TICKETS): login issue (high) | billing question (med) | feature request (low) integration error (high) | cancellation (high) | praise (low) One example item: subject: "Can't login to my account" body: "...trying to login for the past hour... really frustrating, I have a deadline today." category: account_access urgency: high Grader: +empathy words | +a concrete next step | +greeting/closing | right length + category-aware bonuses (churn → retention offer, billing → mentions charge/refund, high urgency → acts immediately)
BEFORE TRAINING Ticket: Can't login to my account Score: 0.85 (+empathy, +action, +greeting, +closing) TRAINING Step 1/8: score=0.69 best=0.75 loss= 0.0021 (13.4s) Step 2/8: score=0.74 best=0.80 loss= 0.0041 (12.4s) Step 3/8: score=0.73 best=0.85 loss=-0.0051 (13.2s) Step 4/8: score=0.61 best=0.85 loss= 0.0246 (15.8s) Step 5/8: score=0.64 best=0.70 loss= 0.0069 (18.2s) Step 6/8: score=0.69 best=0.80 loss= 0.0062 (18.0s) Step 7/8: score=0.71 best=0.80 loss= 0.0009 (18.3s) Step 8/8: score=0.69 best=0.70 loss= 0.0032 (18.5s) AFTER TRAINING Score: 0.70 (-too_long — model fell into an "I apologize" repetition loop)
examples/industry/code_review_agent.pyUse case: an automated first-pass reviewer that catches security bugs before a human looks. Scored on catching the planted issue, being actionable, and matching severity. This example hit a perfect 1.00 at step 2.
TRAINING DATA — 5 code diffs, each with a planted bug (in SAMPLE_DIFFS): auth.py (hardcoded password) | api.py (SQL injection) | utils.py (sleep in loop) config.py (exposed AWS secret) | models.py (mutable default) One example item (api.py): - return db.query(User).filter(User.id == user_id).first() + query = f"SELECT * FROM users WHERE id = {user_id}" <- SQL injection + return db.execute(query) known_issues: sql_injection, raw_query severity: critical Grader: +0.15 each named issue | +actionable suggestion | +explains WHY | +matches severity -penalizes nitpicking (formatting/whitespace) on critical bugs
BEFORE TRAINING File: api.py (diff introduces an f-string SQL query = injection risk) Score: 0.80 (+identified_sql_injection, +identified_raw_query, +actionable, +severity_match) TRAINING Step 1/8: score=0.64 best=0.90 (18.2s) Step 2/8: score=0.83 best=1.00 (19.2s) <- caught the SQL injection perfectly Step 3/8: score=0.61 best=0.90 (17.6s) Step 4/8: score=0.63 best=0.95 (19.0s) Step 5/8: score=0.69 best=0.90 (19.8s) Step 6/8: score=0.46 best=0.70 (18.9s) Step 7/8: score=0.41 best=0.65 (18.6s) Step 8/8: score=0.47 best=0.70 (18.3s) AFTER TRAINING Score: 0.70 (+identified_sql_injection, +identified_raw_query, +severity_match)
pip install -e ".[mlx]"python examples/industry/sql_assistant.pypython examples/industry/email_support_agent.pypython examples/industry/code_review_agent.pycaffeinate -i python -u ... so sleep can't stall the GPU.score_*() for a real reward
(run the SQL and compare rows; measure ticket resolution; track whether devs accept the review), use a
bigger base model, and train for far more steps. The plumbing shown here stays the same.