The Complete Unsloth Guide

Part one of three. A twenty-five-minute review of the Beginner Detailed track, then the questions interviewers ask about Unsloth and fine-tuning. Mid-level reviews this plus its own material; Senior reviews all three.

The thirty-second answer

Unsloth is an open-source toolkit that makes fine-tuning and local inference of open language models faster and lighter on GPU memory. It rewrites the slow parts of the Hugging Face training stack with custom kernels and smarter memory handling, and it ships as a Python library, a browser-based Studio and a desktop app. You typically load a 4-bit base model, train a small LoRA adapter on your own examples, and export the result as an adapter, a merged model or a GGUF file.

Then the line that shows you have used it: "it sits on top of transformers, TRL and PEFT rather than replacing them, so I import it first, and the thing most likely to go wrong afterwards is a chat template mismatch, not the training itself."

The model

Three installs to name without hesitating: Desktop (native app), Studio (web UI launched with unsloth studio) and Core (the unsloth Python package). Say that Core is the mature part and that Studio and Desktop were still labelled beta when you last checked.

Commands and code to have ready

Know these cold, because interviewers sometimes ask you to write them on a whiteboard.

curl -fsSL https://unsloth.ai/install.sh | sh     # Studio installer
unsloth studio -p 8888                             # launch Studio on loopback
unsloth studio stop
unsloth run --model unsloth/gemma-3-270m-it-GGUF:Q4_K_M
unsloth studio verify-install
uv venv unsloth_env --python 3.13
uv pip install unsloth --torch-backend=auto        # Core install

from unsloth import FastLanguageModel      # first, before transformers/trl/peft
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/gemma-3-270m-it", max_seq_length=2048, load_in_4bit=True)
model = FastLanguageModel.get_peft_model(
    model, r=16, lora_alpha=16,
    target_modules=["q_proj","k_proj","v_proj","o_proj","gate_proj","up_proj","down_proj"],
    use_gradient_checkpointing="unsloth")
# SFTTrainer(...).train(), then:
FastLanguageModel.for_inference(model)
model.save_pretrained_gguf("gguf_model", tokenizer, quantization_method="q4_k_m")

Notes to say aloud while writing it. Only one of load_in_4bit, load_in_8bit, load_in_16bit and full_finetuning can be true. The starting learning rate is 2e-4 for LoRA and QLoRA. use_gradient_checkpointing = "unsloth" is the memory-saving variant. After training you call for_inference for fast generation.

The classic traps and why they happen

Importing in the wrong order. Unsloth works by patching transformers, TRL and PEFT when it is imported. If you import trl first, the patches cannot apply, so you lose the speed and memory savings and may see errors. The cause is mechanical, the fix is one line, and it is a favourite warm-up question.

Template mismatch. A model trained with one chat template and run with another, or with a different end-of-sequence token, produces rambling or gibberish output in Ollama, vLLM or llama.cpp even though it looked perfect in the notebook. People assume the training failed and retrain. The real fix is to use the same template at inference, and to check for a doubled beginning-of-sequence token.

Out of memory. The budget has four spenders: weights, optimizer state, activations and batch. The answer in order of effect is a smaller model, QLoRA instead of LoRA, a shorter context and a smaller per-device batch with more gradient accumulation.

Overfitting. A training loss near zero on a small dataset means the model memorised it. The docs describe roughly 0.5 to 1.0 as a healthy loss for this kind of training. The fix is better, more varied data and fewer epochs, not more steps.

Training a GGUF. GGUF files are inference-only. You train a normal model and convert afterwards. People waste an hour trying to load a GGUF into the trainer.

Using the wrong update command. The official guidance is to update Studio by re-running the install script, and not to use unsloth studio update. Quoting that shows you read the changelog.

Exposing Studio carelessly. Studio binds to 127.0.0.1 by default. Starting it with -H 0.0.0.0 makes it reachable from the network over plain HTTP, and its tools can run code as your user. Mention this even at a beginner interview; it signals good instincts.

Common interview questions

What problem does Unsloth solve, and how? Fine-tuning is memory-hungry because you store weights, gradients, optimizer state and activations. Unsloth cuts the cost with hand-written Triton kernels, a manual backward pass, memory-efficient gradient checkpointing and pre-quantized models, and its documentation claims about 2x faster training with about 70% less VRAM and no accuracy loss. That makes fine-tuning feasible on a free Colab T4 or a consumer card. A strong answer also names the trade-off: broader support lives in the plain Hugging Face stack, and Unsloth is stricter about versions.

What is the difference between LoRA and QLoRA? Both train a small adapter on top of frozen weights. In LoRA the frozen base stays in 16-bit; in QLoRA the base is loaded in 4-bit, which cuts memory sharply and costs a little precision in the frozen part. For an 8B model the docs list roughly 6 GB minimum for QLoRA against 22 GB for LoRA. The recommendation for beginners is QLoRA.

What is a good starting configuration? QLoRA with a recent instruction-tuned model, rank 16, alpha 16, all seven target modules, learning rate 2e-4, one to three epochs, per-device batch of 2 with accumulation of 4, and the 8-bit AdamW optimizer. Say that you would change one setting at a time.

What does r do, and how do you choose it? It is the rank of the adapter matrices, so it sets capacity. Larger ranks can learn more but use more memory and overfit more easily. Start at 16, try 32 if the model cannot learn the task, and keep alpha equal to the rank or twice it.

Why do we use gradient accumulation? To get a large effective batch without holding all the examples in memory at once. The trainer sums gradients across several small batches and then updates once. Per-device batch times accumulation steps gives the effective batch. Unsloth fixed an earlier accumulation bug, so different combinations with the same product give matching loss curves.

How do you know training worked? Not from loss alone. Loss should trend down and not hit zero. Then you compare the model's answers on a fixed set of prompts, including prompts not in the training data, against a baseline you saved before training. A model that fits the training set but fails on new prompts is overfitted.

What are the three ways to save a model? A LoRA adapter, which is small and needs the base model; a merged 16-bit model, which is a normal standalone model for vLLM or Transformers; and a GGUF file for llama.cpp, Ollama and Studio. Say that merging into 4-bit is discouraged because it loses accuracy.

What is GGUF and what does Q4_K_M mean? GGUF is a single-file format for running quantized models with llama.cpp-based tools. Q4_K_M is a 4-bit quantization with a medium-quality scheme, the usual balance of size and fidelity, and the default for exports from recent Studio releases. A name starting with UD is an Unsloth Dynamic quantization that keeps sensitive layers at higher precision.

What are the three ways to install Unsloth, and when would you pick each? Desktop for the easiest start with no terminal, Studio for a web interface that can run on a remote machine, Core for notebooks and scripts that you will automate or run in a pipeline. They can coexist.

What hardware do you need? For training, an NVIDIA GPU with CUDA capability 7.0 or above, such as a T4 or an RTX 20 series card or newer. AMD and Intel have their own guides. Mac users should use Studio or Desktop, since Core support for Apple Silicon was described as in the works. For a rough size budget, the docs say about 6 GB for an 8B model with QLoRA.

Why did your fine-tune behave differently in Ollama? First suspect the chat template, then the end-of-sequence token, then whether the exported file was built from the same model you tested. Unsloth's notebook exports create the Ollama Modelfile with the template for you, so I would compare it with the one used in training.

Can you fine-tune to add knowledge? You can, but fine-tuning is mainly good at changing behaviour, tone and format. For facts that must be exact or that change, retrieval is the more reliable approach. A good answer says the two are often combined.

How do you keep a run reproducible? Pin both unsloth and unsloth_zoo versions together, set the random seed, record the model name and dataset version, and keep the configuration in a file. Unsloth releases most weeks, so unpinned runs drift.

How should a new user handle secrets such as a Hugging Face token? Through environment variables or the login command, never pasted into a notebook that will be shared. Tokens are needed for gated models and for pushing to the Hub.

What a strong answer covers that a weak one misses

A weak answer lists features: "it is faster and uses less memory." A strong answer explains the mechanism of the saving in one sentence, names the trade-off, and ties it to a number from the docs.

A weak answer says training "worked" because the loss went down. A strong answer describes a baseline, held-out prompts and the danger of overfitting.

A weak answer treats export as a single step. A strong answer chooses the format for the destination: adapter for experiments, merged 16-bit for a server, GGUF for local use.

A weak answer ignores the environment. A strong answer mentions version pinning, Python 3.11 to 3.13 for Studio, and the fact that the project changes weekly, so tutorials from a year ago may be wrong.

A weak answer never mentions risk. A strong answer notes data privacy for customer text, the loopback default for Studio, and the licence of any model or dataset used. In the Gulf and Egypt, where employers often have residency rules for customer data, that last habit matters in real work too.

A short worked example to narrate

Interviewers sometimes say, "Walk me through fine-tuning a small model for a support bot." A tight answer sounds like this.

"I would start from a small instruction-tuned model such as a 270M or a 3B model, depending on my GPU memory. I would save five prompts and the base model's answers as a baseline. Then I would write or collect a few hundred clean examples in the exact style I want, convert them to the model's chat template, and read twenty formatted rows by eye. I would load the model in 4-bit, attach a rank-16 LoRA adapter on all seven target modules, and train for one to three epochs at a learning rate of 2e-4 with an effective batch of 8. I would watch the loss for a steady decline that does not hit zero, then test the same five prompts plus new ones. If the results are good I would export a Q4_K_M GGUF, load it in Ollama, and repeat the test there to catch template problems. I would record versions and settings in a README."

Notice what that answer contains: a decision about hardware, a baseline, data quality, specific numbers, a check for overfitting, an export choice, and reproducibility. Nothing in it requires a long list of features. When you practise, time yourself at about ninety seconds.

Questions to ask back

At the end of an interview you can ask your own questions, and good ones show experience. Ask what GPU the team trains on and how they handle version pinning given weekly releases. Ask whether they fine-tune or use retrieval for domain knowledge, and how they evaluate a fine-tuned model before shipping it. Ask where models and datasets are stored and who can see them. These questions show that you think about operations and not only about getting a notebook to run.

Phrases that help, and phrases to avoid

Phrases that land well: "I would measure against a baseline", "the template has to match at inference", "QLoRA first, then scale up", "I pin the versions together" and "I would check the data before the hyperparameters".

Phrases to avoid: "just train for more epochs" (it invites overfitting), "Unsloth is a model" (it is a toolkit), "I would use the newest settings from a blog post" (many are stale), and "I can fine-tune it to know our whole wiki" (retrieval is usually better for that).

If you do not know something, say so and describe how you would find out. For example, if asked about a flag you have not used, say that you would check the installed command's help and the changelog, because the tool changes weekly. Interviewers value that more than a confident wrong answer.

More model answers

What is an epoch, a step and the loss? An epoch is one pass over the whole dataset. A step is one weight update, which processes one effective batch. Loss measures how wrong the model's predictions on the training text were, and training tries to reduce it. Mention that with gradient accumulation, one step covers several small batches.

Why prefer the unsloth/ model uploads? They are tested with Unsloth and often include fixes to tokenizers and chat templates that original uploads lack. Names ending in unsloth-bnb-4bit are Unsloth Dynamic 4-bit builds that leave some layers unquantized, which costs slightly more memory and gives higher accuracy than plain bnb-4bit.

What does trust_remote_code do, and when is it safe? It lets a model repository run its own Python code on your machine. Enable it only for sources you trust, and leave it at the default of false otherwise.

What would you check if installation fails? Python version first, because Studio needs 3.11 to 3.13. Then the GPU driver with nvidia-smi, then build tools such as cmake and git, then unsloth studio verify-install. If PyTorch cannot see the GPU, the cause is usually a CPU-only PyTorch build, which --torch-backend=auto helps avoid.

How is Studio different from calling the Python API? Studio wraps the same training code in a web interface and adds chat, dataset upload, a Data Recipes workflow and an OpenAI-compatible server. The Python API is what you automate, version-control and run in a pipeline. A good habit is to prototype in Studio and move to code once the settings are decided.

Sixty-second self-test

Answer aloud, without notes, in about a minute.

1. What are the three ways to install Unsloth?
2. What must you import first, and why?
3. What is the difference between LoRA and QLoRA, and which would you start with?
4. Name the four numbers in a beginner configuration: learning rate, rank, alpha and effective batch size.
5. Why can you not train a GGUF file?
6. What are the three save formats, and where does each one run?
7. Your model gives gibberish in Ollama but was fine in the notebook. What do you check first?
8. Training loss falls to nearly zero in the first few steps on 30 examples. What happened?
9. How should you update Studio?
10. Why is -H 0.0.0.0 something to think twice about?

If you could answer eight of the ten without hesitation, you are ready for a beginner-level conversation about Unsloth. If you missed any, reread the matching section of the Beginner Detailed track and answer again tomorrow.

Part one of three. These are the mistakes beginners actually make with Unsloth, each with the symptom you will see, the cause and the fix, followed by habits worth adopting on day one and a short checklist. Skim the symptoms now; come back when one of them happens to you.

The mistakes beginners actually make

Importing libraries in the wrong order

Symptom. Training is slower than advertised, memory use is higher than expected, or you see odd errors from deep inside transformers or trl. The banner "Unsloth: Will patch your computer to enable 2x faster free finetuning" never appeared.

Cause. Unsloth speeds things up by patching transformers, trl and peft at import time. If one of them is imported first, the patches cannot take effect.

Fix. Make from unsloth import FastLanguageModel the first import in the file or the first cell of the notebook, and restart the kernel if you changed the order.

from unsloth import FastLanguageModel   # first
from trl import SFTTrainer, SFTConfig   # then everything else
from datasets import load_dataset

Picking a model that cannot fit

Symptom. An out-of-memory message in the first minute, often with the words "CUDA out of memory" and a count of gigabytes you tried to allocate.

Cause. The model was too big for the card, or the context length and batch size were too generous. The documentation gives minimums such as about 6 GB for an 8B model with QLoRA and 22 GB with 16-bit LoRA, and those minimums leave no margin.

Fix. Work down this list, one change at a time: choose a smaller model, switch to QLoRA (load_in_4bit = True), shorten max_seq_length, lower per_device_train_batch_size to 1 or 2 and raise gradient_accumulation_steps to compensate. Restart the Python session between attempts, because a failed run can leave memory allocated.

Trying to train a GGUF file

Symptom. An error when loading a model whose name ends in -GGUF, or Studio refusing to offer it for training.

Cause. GGUF is an inference format for llama.cpp, Ollama and Studio chat. It is not a trainable checkpoint.

Fix. Train from the normal model (for example unsloth/gemma-3-270m-it, or an unsloth-bnb-4bit variant) and convert to GGUF at the end with save_pretrained_gguf or the Export step.

Turning on two precision modes at once

Symptom. An error or strange behaviour right after from_pretrained.

Cause. Only one of load_in_4bit, load_in_8bit, load_in_16bit and full_finetuning may be true at a time. Copying a snippet and flipping one flag without turning off the other leaves two on.

Fix. Decide on one mode. For QLoRA set load_in_4bit = True and the rest False.

Copying a learning rate from the wrong kind of tutorial

Symptom. The loss explodes, stays flat or turns to nan.

Cause. The recommended learning rate for LoRA and QLoRA supervised fine-tuning is 2e-4. Reinforcement-learning tutorials use something near 5e-6, and full fine-tuning uses far smaller values than adapters. A number from the wrong recipe is either too aggressive or too timid.

Fix. Start with 2e-4, change one thing at a time, and keep a note of each run's settings and loss curve so you can compare.

Forgetting the chat template

Symptom. The model is perfect in your notebook, then rambles, repeats itself or never stops in Ollama or vLLM. Answers sometimes begin with strange marker text.

Cause. The template, the end-of-sequence token or the beginning-of-sequence token at inference differs from training. Each model family has its own format, and a tool that applies a default template will not match yours.

Fix. Apply the same template in both places. In Unsloth that means get_chat_template(tokenizer, chat_template = "gemma-3") (or the right name for your family) before formatting data, and apply_chat_template(..., add_generation_prompt = True) for generation. When exporting to Ollama through an Unsloth notebook, let it generate the Modelfile. Print the formatted prompt in both places and compare them character by character.

Training on data you never read

Symptom. The fine-tuned model repeats junk, copies a stray header into every answer or answers in two different styles.

Cause. The dataset contained duplicates, leftover HTML, truncated answers or inconsistent formats. The model faithfully learned all of it.

Fix. Before training, print and read at least twenty random formatted rows. Remove duplicates and contradictions. Make answers consistent in length and tone. If the output looks wrong, fix the data before touching any hyperparameter.

Using too few examples and too many epochs

Symptom. Training loss drops almost to zero within a few steps. The model parrots your examples but fails on any new question.

Cause. With a tiny dataset and several epochs, the adapter memorises the examples, which is overfitting. The documentation treats a loss near zero as a warning sign and suggests roughly 0.5 to 1.0 as the usual healthy range.

Fix. Add more varied examples, reduce the epochs to between one and three, and test with prompts that are not in the training data.

Judging success by loss alone

Symptom. The loss curve looks lovely and the model is still not useful.

Cause. Loss measures predictions on the training text, not usefulness on your task.

Fix. Before training, save five prompts and the base model's answers as a baseline. After training, ask the same five plus five new ones. Compare by reading. This is the cheapest evaluation there is, and it catches most problems.

Saving the wrong thing

Symptom. You find a folder of a few megabytes where you expected a full model, or a folder of many gigabytes where you wanted something small. Or you cannot load the saved model in another tool.

Cause. There are three save formats. save_pretrained writes only the LoRA adapter, which needs the base model. save_pretrained_merged with merged_16bit writes a full standalone model. save_pretrained_gguf writes a GGUF file for llama.cpp tools.

Fix. Choose by destination: adapter for experiments, merged 16-bit for servers such as vLLM, GGUF for Ollama and Studio chat. Avoid merged_4bit, which the docs discourage because it loses accuracy.

Running out of memory while saving

Symptom. Training finished, and then the notebook crashes at the save step.

Cause. Merging the adapter into the base weights briefly needs a lot of GPU memory.

Fix. Lower the peak with maximum_memory_usage = 0.5 (the default is 0.75) in the save call, and save the adapter first so you do not lose the training.

model.save_pretrained("lora_model")          # safe, small, do this first
tokenizer.save_pretrained("lora_model")
model.save_pretrained_gguf("gguf_model", tokenizer,
                           quantization_method = "q4_k_m")

Updating Studio the old way

Symptom. After running unsloth studio update the app does not receive the latest changes, or a blog post's instructions no longer match your screen.

Cause. The changelog says not to use unsloth studio update any more, because packaging will not get the latest updates.

Fix. Re-run the install one-liner for your system (curl -fsSL https://unsloth.ai/install.sh | sh, or the PowerShell equivalent). For Desktop, use Check for updates in Settings. For Core, upgrade both packages together: pip install --upgrade unsloth unsloth_zoo.

Upgrading one library and breaking the others

Symptom. After pip install --upgrade transformers or trl, Unsloth raises errors about incompatible versions or models load with warnings that weights were not initialised.

Cause. Unsloth pins version ranges for transformers, trl, peft, datasets and others, and it patches their internals. Installing one of them independently can push it outside the range Unsloth was tested against.

Fix. Let Unsloth choose the versions. Upgrade unsloth and unsloth_zoo together, or create a fresh virtual environment and reinstall. When you find a combination that works, record the versions.

Putting Unsloth in the same environment as everything else

Symptom. Installing Unsloth breaks an unrelated project, or the reverse.

Cause. Unsloth carries tight dependency requirements, including PyTorch, bitsandbytes and xformers. A shared global Python environment cannot satisfy every project.

Fix. Create a dedicated environment, as in the guide: uv venv unsloth_env --python 3.13, then activate it. Use a separate environment per project when versions differ.

Exposing Studio to the network by habit

Symptom. None, which is the danger. Everything works, and a service with powerful tools is reachable by other machines.

Cause. Tutorials often launch Studio with -H 0.0.0.0 so that a phone or another computer can reach it. That makes it listen on every network interface over plain HTTP. Studio's server-side tools can run code as your operating-system user.

Fix. Keep the default, which listens on 127.0.0.1 only. If you need remote access, read the security guidance first, use the password gate, and prefer the --secure mode, which gives a protected HTTPS address. Never expose a Docker container's ports on a cloud machine without restricting them.

Leaving secrets in notebooks

Symptom. A Hugging Face token appears in a notebook you share or commit.

Cause. Code snippets show token = "hf_..." inline, and it is easy to paste the real value.

Fix. Put tokens in environment variables such as HF_TOKEN, or log in with the Hugging Face command-line tool. If a token leaks, revoke it from your Hugging Face account settings immediately and create a new one.

Expecting to fix a knowledge gap with fine-tuning

Symptom. The model has the right style but invents policies, prices or dates.

Cause. Fine-tuning on a small set mostly teaches format and habits. It does not reliably store facts, and it can make a model more confident while remaining wrong.

Fix. For facts that must be exact, give the model the documents at question time through retrieval, and use fine-tuning for how the answer is written.

Forgetting to switch to inference mode

Symptom. Generation is slower than it should be after training.

Cause. The model is still set up for training.

Fix. Call FastLanguageModel.for_inference(model) before generate. The docs describe it as making generation about twice as fast.

Expecting the first epoch to be fast

Symptom. The first few minutes are slow, and you assume the speed claims were wrong.

Cause. Unsloth uses torch.compile, and the compiler warms up for several minutes on the first run.

Fix. Judge speed after the warm-up. If you are debugging a strange failure instead, UNSLOTH_COMPILE_DISABLE=1 skips compilation, though it must be set before importing Unsloth.

Habits worth adopting on day one

Keep a run log. Write down, for every run, the date, the Unsloth and unsloth_zoo versions, the model name, the dataset version, the key settings and what you saw. A plain text file is enough. You will return to a run weeks later and need to know what produced it. Because the library releases most weeks, the version line is not decoration.

Pin versions once something works. When a training run succeeds, freeze the versions: pip install unsloth==2026.9.12 unsloth_zoo==2026.9.8 was the current pair when this guide was checked. Pinning protects you from a surprise change during a project, and you can upgrade deliberately when you have time to retest.

Start tiny, then scale. Prove your whole pipeline on a 270M-parameter model with 60 steps. Only then move to a 3B or 8B model. The pipeline errors you hit will be identical, and finding them costs minutes instead of hours.

Save a baseline. Capture the untouched model's answers to a fixed set of prompts before you train anything. It turns a vague feeling of "better" into a comparison you can show someone.

Change one thing at a time. If you alter the learning rate, rank and dataset together and the result improves, you have learned nothing. Make a single change per run and note it.

Read your data. Twenty random rows, printed after formatting, every time the dataset changes.

Save the adapter early. The adapter is small. Save it the moment training ends, before attempting the heavier merge or GGUF conversion, so a crash cannot cost you the run.

Test in the tool where the model will live. Load the exported file in Ollama or Studio and ask your baseline prompts there. The notebook is not the destination.

Mind your data. Check the licence of every model and dataset. Remove personal information before training, and do not upload customer records to public repositories or shared notebooks. If your employer is in the Gulf or Egypt and must keep customer data inside a region, train on infrastructure that satisfies that rule, whether that is your own machine or a cloud region you are allowed to use.

Read the changelog before you copy a command. Unsloth changes quickly, and the changelog records deprecations such as the Studio update command. A blog post that is six months old is a hint, not an instruction.

Learn the neighbours. Unsloth builds on other tools, and understanding them makes errors much less mysterious. Read the PEFT and TRL guide to see what the adapter and trainer objects do, the Hugging Face guide for models, datasets and tokens, and the Ollama guide for running your result.

Reading the first training run like a pro

A first run produces a lot of text, and it helps to know which parts matter. The opening banner confirms the patch, names your GPU and shows available memory. Below it, the trainer prints a summary: number of examples, number of epochs, batch size per device, gradient accumulation steps, total optimisation steps and trainable parameters. Check the last figure. With LoRA the trainable parameters should be a small percentage of the total, around one percent for typical settings. If it says that nearly all parameters are trainable, you may have switched on full fine-tuning by accident.

Then comes the table of steps and losses. The first few values are noisy because the learning rate is still warming up. A healthy run shows a downward trend that flattens. Spikes up to a very large number mean the learning rate is too high. A value of nan means the numbers overflowed, which is also usually a learning-rate or precision problem. A loss that does not move at all can mean that the learning rate is too small, that the dataset column is empty, or that the model never saw your examples because the text field was wrong. Printing a formatted example catches the last one.

Finally, look at timing. A tiny model should take seconds per step on a modern GPU. If it takes much longer, check whether the GPU is actually in use with nvidia-smi in another terminal while training. If memory use is near zero, the run is on CPU, which is the signature of a CPU-only PyTorch.

A short checklist

Before you start a run:

• I know my GPU and its memory, and the model size fits with a margin.
• from unsloth import ... is my first import.
• I picked one precision mode: QLoRA with load_in_4bit = True for my first run.
• The model is not a GGUF.
• I saved a baseline of five prompts and answers.
• I printed and read twenty formatted rows of data.
• The chat template matches my model family.
• My settings are the documented starting values: learning rate 2e-4, rank 16, alpha 16, batch 2 with accumulation 4.
• Tokens are in environment variables, not in the file.

After the run:

• The loss trended down and did not fall to zero.
• I compared answers on the baseline and on new prompts.
• I saved the adapter first, then the format I need.
• I tested the export in the tool where it will run.
• I wrote down the versions, the model, the dataset and the settings.
• I did not expose Studio beyond 127.0.0.1 without reading the security notes.

Keep this checklist next to your notes. Run through it before every run for the first month, and the list of common mistakes above will mostly stop applying to you.

This is part two of three. It assumes you have already fine-tuned a small model with Unsloth, either in a notebook or in Studio, and now need that work to be repeatable, evaluated, exported correctly and safe to run on a shared machine. The guide is written against Unsloth Core 2026.9.12 with unsloth_zoo 2026.9.8, and Studio v0.1.900-beta.

Where this picks up

Each section starts with the problem it solves and ends with a Try it you can run on a real project.

What the import actually does

Beginner gave you the rule: put from unsloth import ... before anything from transformers, trl or peft. Knowing why lets you predict a whole class of bugs instead of memorising the rule.

Unsloth is not a separate training framework. It is a layer of patches over the Hugging Face stack. When you import it, it rewrites parts of the model classes, the attention and MLP forward passes, the cross-entropy loss, and the trainer internals with its own hand-written Triton kernels and manually derived backward passes. The official claim is up to twice the speed with about 70% less VRAM and no accuracy loss, and the mechanism is a mix of fused kernels, smarter gradient checkpointing and dynamic quantisation rather than any approximation of the maths. Because everything else is still TRL, PEFT and datasets, your trainer code, your callbacks and your Hugging Face tokens work unchanged.

The banner is your confirmation. If you run a script and do not see the line "Unsloth: Will patch your computer to enable 2x faster free finetuning", the patches did not apply, and you are running stock Hugging Face code that is slower and hungrier. The usual cause is importing trl or transformers first. Patching works by replacing attributes on classes, so a class that was already imported and instantiated elsewhere may keep the old behaviour.

The second banner block is just as useful. It names the Unsloth version, the model family it patched, the detected GPU and its memory, the torch and CUDA versions, whether bfloat16 is available, and which attention backend is active. Copy it into every bug report and every run log. When a colleague says "it works on my machine", this block is how you find out they are on a different torch.

Two further pieces are worth understanding. First, use_gradient_checkpointing = "unsloth" is not the standard Hugging Face checkpointing. It offloads activations to system RAM asynchronously, which the docs describe as using about 30% less VRAM and enabling much longer context. The cost is host memory and a little PCIe traffic, which matters on a box where several jobs share RAM. Second, on first use the auto-compiler builds fused versions of parts of the model with torch.compile. The docs note a warm-up of around five minutes. If you benchmark a ten-step run you will conclude Unsloth is slow, so always measure after warm-up.

Versions, pinning and the dependency envelope

Unsloth releases roughly weekly. The PyPI version uses calendar versioning: 2026.9.12 means the twelfth release of September 2026. There are two packages that must move together, unsloth and unsloth_zoo, and a separate versioning line for Studio and Desktop, which are at v0.1.900-beta. Mixing these up is the most common reason an upgrade "does nothing".

The reproducible form pins both:

uv venv unsloth_env --python 3.13
source unsloth_env/bin/activate
uv pip install "unsloth==2026.9.12" "unsloth_zoo==2026.9.8" --torch-backend=auto

The reason to pin is the dependency envelope. In 2026.9.12 the package metadata bounds torch below 2.13, transformers at or below 5.5.0 with a list of individual versions excluded, TRL at or below 0.24.0, and datasets below 4.4.0 with some 4.0 and 4.1 releases excluded. Those bounds exist because Unsloth patches internals that other libraries rewrite freely. Upgrading TRL on its own to get a new trainer feature can therefore break the patches. Let the resolver pick compatible versions of everything, and if you must override one, do it with --no-deps in a scratch environment first.

Two behaviours surprise people in production. First, unsloth_zoo can update itself. For a reproducible image or a nightly job, set UNSLOTH_DISABLE_AUTO_UPDATES=1, otherwise two runs a week apart can differ without any change to your lockfile. Second, Python requirements differ by product: the package metadata allows a wide range, but the docs ask for Python 3.11 to 3.13 for Studio.

To update Core, use pip install --upgrade unsloth unsloth_zoo. To update without touching dependencies, the docs give pip install --upgrade --force-reinstall --no-cache-dir --no-deps unsloth unsloth_zoo. To update Studio, re-run the install one-liner. The changelog is explicit that you should not use unsloth studio update, because the packaging it relies on will not get the latest updates.

Licensing also splits. The core package is Apache-2.0, while Studio components such as the UI and the CLI sources under unsloth_cli carry AGPL-3.0 headers. If your company embeds or modifies Studio, have someone read the AGPL terms. Using Core in an internal training pipeline is the common, simple case.

Choosing a method from a memory budget

Beginner picked QLoRA because the docs recommend starting there. At work you need to justify the choice, and the justification is a VRAM budget plus a quality requirement.

There are three modes. QLoRA loads the base model in 4-bit with bitsandbytes and trains LoRA adapters on top. LoRA keeps the base model in 16-bit and trains the same adapters. Full fine-tuning trains every weight. In the loader these correspond to load_in_4bit=True, load_in_16bit=True and full_finetuning=True, and only one of the flags (including load_in_8bit) may be true at once.

The documentation gives absolute minimums for fitting a model. For QLoRA: 3.5 GB for a 3B model, 5 GB for 7B, 6 GB for 8B, 8.5 GB for 14B, 22 GB for 27B, 26 GB for 32B and 41 GB for 70B. For 16-bit LoRA the same models need 8, 19, 22, 33, 64, 76 and 164 GB. These are floors, not recommendations: they assume short sequences and small batches. Treat them as the point below which the job will not start, and add headroom for context length and evaluation.

QLoRA's saving is large, over 75% less VRAM, and the accuracy cost is usually small, which is why it is the default. Use 16-bit LoRA when you have the memory and want to remove quantisation as a variable, for instance when you are chasing a quality gap and need to know whether 4-bit is responsible. Use full fine-tuning rarely: it needs the most memory, produces a full-size checkpoint per run, and is easier to overfit.

There is a further choice inside 4-bit. Repositories ending in unsloth-bnb-4bit are Unsloth Dynamic 4-bit: selected layers are left unquantised, so they use slightly more VRAM than a plain bnb-4bit repository and give noticeably better accuracy. Prefer these when they exist. A repository with no suffix is the original 16-bit weights, sometimes with tokenizer or chat-template fixes applied. By default Unsloth redirects some model names to its own uploads; pass use_exact_model_name=True if you need exactly the repository you named, for example to audit a specific upstream revision.

Sequence length is the other memory lever, and attention cost grows faster than linearly with it. Raising max_seq_length from 2048 to 8192 can matter more than switching from 4-bit to 16-bit. Measure your real token lengths first and set the cap at a high percentile, not at the model's maximum.

Data, chat templates and response-only loss

Most failed fine-tunes are data failures, and the single most common specific failure is a mismatch between the template used in training and the one used at inference. The docs call out wrong templates, EOS tokens or BOS tokens as the leading cause of a model that looks good in Unsloth and produces gibberish or endless output in Ollama, vLLM or llama.cpp.

A chat template turns a list of role-tagged messages into the exact string the model was trained on, including special tokens. Unsloth gives you get_chat_template to attach a known template to the tokenizer:

from unsloth.chat_templates import get_chat_template, standardize_sharegpt

tokenizer = get_chat_template(tokenizer, chat_template="gemma-3")
dataset = standardize_sharegpt(dataset)  # from/value -> role/content

def formatting_prompts_func(examples):
    texts = [
        tokenizer.apply_chat_template(c, tokenize=False, add_generation_prompt=False)
        for c in examples["conversations"]
    ]
    return {"text": texts}

dataset = dataset.map(formatting_prompts_func, batched=True)

Built-in names include chatml, llama, mistral, zephyr, alpaca, vicuna and unsloth, and a custom template is passed as a tuple of the template string and the EOS token. Use the template that matches the model family you load. If you fine-tune an instruct model, keep its own template. Inventing a new one makes the model relearn formatting from your small dataset, and then you must ship that template everywhere the model runs.

The next lever is response-only loss. By default the loss is computed on every token, including the user's turn. Usually you want the model to learn what to say, not to imitate the questions. train_on_responses_only masks the instruction tokens:

from unsloth.chat_templates import train_on_responses_only

trainer = train_on_responses_only(
    trainer,
    instruction_part="<start_of_turn>user\n",
    response_part="<start_of_turn>model\n",
)

Those two strings are for Gemma 2 and 3. For Llama 3.x they are <|start_header_id|>user<|end_header_id|>\n\n and <|start_header_id|>assistant<|end_header_id|>\n\n. They must match the rendered template exactly, including newlines. When they do not, you get the error "All labels in your dataset are -100. Training losses will be all 0." That message is the masking removing every token because it never found the response marker. Print one rendered example and copy the markers from it rather than from memory.

Dataset hygiene matters as much as formatting. Remove duplicates, because repeated rows are memorised quickly. Check that the final assistant turn is present and complete, since truncated examples teach the model to stop mid-sentence. Keep the system prompt consistent between training and serving. And keep a held-out split before you start, as the next sections need one.

Hyperparameters, and which ones matter

Unsloth's defaults are good enough that many people never change them, which is why the common failure is changing five things at once. Work through them in order of leverage.

Data quality and size beat every other knob. Beyond that, the highest-impact settings are the learning rate, the number of epochs and the effective batch size.

The recommended learning rate is 2e-4 for LoRA and QLoRA, and a much smaller 5e-6 for reinforcement learning methods such as DPO and GRPO. Epochs should be one to three. More epochs on a small dataset push the loss toward zero, and the docs flag a loss near zero as a sign of overfitting. A healthy training loss is often roughly 0.5 to 1.0, though this depends on the task. Weight decay of 0.01, a warmup of about 5 to 10% of steps, and a linear or cosine scheduler are the standard companions.

Effective batch size is per_device_train_batch_size × gradient_accumulation_steps × number of GPUs. Unsloth fixed a gradient-accumulation bug, so batch 2 with accumulation 8 and batch 16 with accumulation 1 now produce matching loss curves. In practice that means you choose the per-device batch to fit memory, and then choose accumulation to reach the effective batch you want. If you hit an out-of-memory error, lower the per-device batch to 1 to 3 and raise accumulation, rather than lowering the effective batch and changing the training dynamics.

LoRA rank and alpha control adapter capacity. The docs recommend r of 16 or 32, and lora_alpha equal to r or twice r. Larger ranks add parameters and memory but rarely fix a data problem. Target all seven projection modules, the attention ones (q_proj, k_proj, v_proj, o_proj) and the MLP ones (gate_proj, up_proj, down_proj), since adapting only attention usually underperforms. lora_dropout=0 and bias="none" are the optimised settings in Unsloth's kernels. Variants exist, including use_rslora and use_dora in the CLI config, but treat them as experiments you run once the basics are sound.

A note on defaults that differ between surfaces. The notebooks use r=16 and lora_alpha=16. The unsloth train CLI config defaults to lora_r: 64 with lora_alpha: 16, which changes the effective scaling. If you move a recipe from a notebook to the CLI, write the numbers explicitly instead of relying on defaults.

Packing concatenates short examples to fill the sequence length, which raises throughput on datasets of short rows. It is off by default in the CLI config. It can change what the model sees in each sequence, so compare evaluation loss with and without it before adopting it. The 2026-07-07 changelog notes that GRPO now supports sequence packing by default, which is a separate matter from SFT packing.

Change one variable per run, name runs so the change is obvious, and let evaluation, not the training loss, decide.

Evaluation, early stopping and resumable runs

A training loss that goes down tells you the model is fitting the data, not that it is getting better. At work you need a held-out signal and the ability to survive an interruption.

Split before formatting, and always shuffle:

split = dataset.train_test_split(test_size=0.01, shuffle=True, seed=3407)
train_ds, eval_ds = split["train"], split["test"]

Then enable evaluation in the trainer configuration:

from trl import SFTConfig

args = SFTConfig(
    per_device_train_batch_size=2,
    gradient_accumulation_steps=4,
    per_device_eval_batch_size=2,
    eval_accumulation_steps=4,
    eval_strategy="steps",
    eval_steps=50,
    save_strategy="steps",
    save_steps=50,
    load_best_model_at_end=True,
    metric_for_best_model="eval_loss",
    greater_is_better=False,
    fp16_full_eval=True,
    output_dir="outputs",
)

Evaluation memory is a classic trap. The eval batch runs through the same model, and a large eval set or batch can run out of memory even though training was fine. Keep per_device_eval_batch_size small, use fp16_full_eval=True, and use a modest eval split. The docs note that full-precision evaluation flags have been on by default since June 2025, so be explicit if you want to control it.

Early stopping uses the standard Hugging Face callback, and it requires the save and metric settings above so that there is a "best" to return to:

from transformers import EarlyStoppingCallback

trainer.add_callback(EarlyStoppingCallback(
    early_stopping_patience=3,
    early_stopping_threshold=0.0,
))

A rising evaluation loss while training loss keeps falling is overfitting. Stop, lower the epochs or learning rate, or add data.

Resuming matters because GPU jobs get preempted. With save_strategy="steps" and a sensible save_steps, you can continue with trainer.train(resume_from_checkpoint=True). The checkpoint holds the adapter, optimiser and scheduler state, so the resumed run continues the same curve. Put output_dir on storage that survives the container, a mounted volume or a network disk, or the checkpoint dies with the machine. Studio reflects the same idea: a run is a session, and a run holds checkpoints. As of v0.1.900 you can export checkpoints even from crashed or cancelled runs.

For tracking, set report_to="wandb", along with the WANDB_PROJECT environment variable, and optionally WANDB_LOG_MODEL="checkpoint" to upload checkpoints as artefacts. See the Weights & Biases guide or the MLflow guide for experiment tracking in more depth.

Exporting without breaking the model

Training produces an adapter. Production needs a format a server can load. The export step is where quality gets lost, so be deliberate.

In Core the methods are:

# Merge adapters into 16-bit weights for vLLM, SGLang or Hugging Face
model.save_pretrained_merged("out", tokenizer, save_method="merged_16bit")

# Adapters only (small, requires the base model at load time)
model.save_pretrained_merged("out", tokenizer, save_method="lora")

# GGUF for llama.cpp and Ollama
model.save_pretrained_gguf("gguf_out", tokenizer, quantization_method="q4_k_m")

# Push straight to the Hub
model.push_to_hub_merged("org/model", tokenizer, save_method="merged_16bit", token="hf_...")
model.push_to_hub_gguf("org/model-gguf", tokenizer, quantization_method="q4_k_m")

Think of exports as three families. Adapter-only is smallest and keeps the base model separate, which suits serving many fine-tunes over one base model with LoRA hot-swapping in vLLM. Merged 16-bit bakes the adapter into full weights, giving a self-contained model any engine can load. GGUF is the llama.cpp format, quantised for CPU and consumer GPU serving. merged_4bit exists but is discouraged, and the docs say you need merged_4bit_forced to force it, because merging into a quantised base loses precision.

The GGUF quantization_method accepts many values, among them f16, q8_0, q5_k_m, q4_k_m, q6_k and the lower q3_* and q2_k variants. Start with q4_k_m as the balance of size and quality, move to q5_k_m or q8_0 if evaluation shows degradation, and always re-evaluate the quantised file, not just the 16-bit model. In Studio the default export changed in v0.1.812 to Q4_K_M, previously F16 or BF16, so a team script that assumed the old default now produces smaller, lossier files.

The CLI form is unsloth export <checkpoint> <output_dir> with --format of merged-16bit (the default), merged-4bit, gguf or lora, and --quantization of q4_k_m (default), q5_k_m, q8_0 or f16. It can also push with --push-to-hub --repo-id <id>.

Memory during export is a separate trap. Merging peaks GPU memory, and a crash while saving is a known failure; the docs suggest maximum_memory_usage=0.5 against a default of 0.75. In Colab, saving may default to .bin, and passing safe_serialization=None gives safetensors.

The hand-offs are short. For Ollama, the notebooks export a GGUF and create the Modelfile, including the chat template, for you, after which you use ollama create and ollama run; see the Ollama guide and llama.cpp guide. For vLLM, save merged 16-bit and run vllm serve ./finetuned_model; see the vLLM guide. Unsloth's own docs point enterprise or multi-user serving at vLLM.

Running training outside a notebook

A notebook is a good place to explore and a poor place to run production training. You want a job with a config file, fixed versions, durable output and clean exit codes.

Unsloth gives you two routes. The first is plain Core: a Python script with your FastLanguageModel and TRL code, run by whatever scheduler you already use. It is a library, not a service, so it fits a Kubernetes Job, a Slurm batch, a SageMaker training job or a CI runner with a GPU. The second is the installed unsloth train command, which takes a YAML or JSON config:

unsloth train --config train.yaml --dry-run
unsloth train --config train.yaml

The --dry-run flag validates the configuration without training, which belongs in your CI. The config schema rejects unknown keys, so a misspelled field fails immediately instead of being silently ignored:

model: unsloth/gemma-3-270m-it
data:
  dataset: yahma/alpaca-cleaned
  format_type: auto
training:
  training_type: lora
  max_seq_length: 2048
  load_in_4bit: true
  output_dir: ./outputs
  num_epochs: 1
  learning_rate: 2e-4
  batch_size: 2
  gradient_accumulation_steps: 4
  save_steps: 50
  random_seed: 3407
lora:
  lora_r: 16
  lora_alpha: 16
logging:
  enable_wandb: true
  wandb_project: unsloth-training

Other flags mirror the config fields in kebab-case and override the file, so --learning-rate 1e-4 works for a sweep. Secrets come from --hf-token or the HF_TOKEN environment variable and --wandb-token or WANDB_API_KEY. Prefer environment variables injected by your scheduler's secret store over command-line values that end up in process listings and shell history. One caution from the docs: the YAML that Studio saves is similar to the CLI config but not guaranteed to be identical, since Studio shows num_train_epochs and per_device_train_batch_size where the CLI schema uses num_epochs and batch_size. Validate with --dry-run rather than assuming.

Use unsloth list-checkpoints --outputs-dir ./outputs to see what a run produced.

For containers, the official NVIDIA image unsloth/unsloth needs a driver of 570.26 or newer and the NVIDIA Container Toolkit; an AMD image unsloth/unsloth-rocm also exists. A typical run:

docker run -d --name unsloth --gpus all --ipc=host \
  --ulimit memlock=-1 --ulimit stack=67108864 \
  -p 8000:8000 -p 8888:8888 \
  -e JUPYTER_PASSWORD="change-me" \
  -v "$PWD":/workspace/host \
  -v "$HOME/.cache/huggingface":/workspace/.cache/huggingface \
  -v unsloth-studio:/opt/unsloth-studio \
  unsloth/unsloth

Inside the container Studio listens on port 8000 and JupyterLab on 8888. Keep /opt/unsloth-studio on a named volume, because Studio needs symlinks there and host folders on Windows or macOS can stop the container from starting. The container runs as root, so files in a mounted project folder become root-owned; use unsloth/unsloth:core with --user if that matters. For general container practice see the Docker guide. Mount the Hugging Face cache so models are not re-downloaded on every run, and pin an image tag or digest in production rather than using the rolling :latest.

Multi-GPU and scaling a job

When one GPU is not enough, there are two different problems: you may need more throughput, or you may need more memory for a model that does not fit.

For throughput, data parallelism is the answer. Unsloth uses Distributed Data Parallel, launched with accelerate launch train.py or torchrun --nproc_per_node N train.py, and the docs describe it as turning on automatically when more than one GPU is present, with throughput scaling roughly linearly. Remember the effective batch size now multiplies by the number of GPUs, so if you keep the per-device batch and accumulation the same, your global batch grows and you may want to lower the learning rate or accumulation to compensate.

For memory, you can split a single model across devices by passing device_map="balanced" to from_pretrained, which is how a model such as Llama 70B is loaded across several cards. FSDP and DeepSpeed work through Accelerate; the documentation still describes fully official multi-GPU support as coming soon even though Studio says multi-GPU works automatically, so test your exact configuration before committing a schedule to it. For DeepSpeed itself, see the DeepSpeed guide.

A caution about the DDP page in the docs: it uses the repository script unsloth-cli.py with snake_case flags, which is an older argparse script and not the installed unsloth console command. Do not mix the two syntaxes in one runbook.

Studio, by contrast, is a single-host product. It has no documented horizontal scaling or high availability. Multi-node training is possible through torchrun but is not a documented Studio feature, so for anything multi-node, build on Core.

On the inference side, llama.cpp concurrency uses slots, controlled by --parallel, and tensor parallelism for GGUF is available in Studio. When many users need low-latency serving, export merged weights and use a dedicated engine rather than Studio.

Studio as a service: API, keys and exposure

Studio is more than a browser tool. It runs a FastAPI server that speaks OpenAI-compatible and Anthropic-compatible protocols, which makes it a convenient local gateway for scripts and coding agents. It also runs code on your machine, which is why this section has as many warnings as features.

Launch it with unsloth studio -p 8888. The code default is to bind 127.0.0.1, loopback only, and on first launch it redirects you to /change-password to set the admin password. The documentation often shows -H 0.0.0.0, which exposes the server on the network over plain HTTP; keep that in mind when you copy an example.

For programmatic use, create an API key under Settings and API. Keys begin with sk-unsloth-, are stored only as a hash, are shown once, can have an expiry, and a revoked key returns 401. The endpoints share the Studio port:

curl http://127.0.0.1:8888/v1/chat/completions \
  -H "Authorization: Bearer $UNSLOTH_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"unsloth/gemma-3-270m-it","messages":[{"role":"user","content":"Say hello"}]}'

/v1/responses, /v1/messages (the Anthropic format) and GET /v1/models are also available. If you are using a model that is not loaded, load it first.

Tools are the sharp edge. Studio can run server-side tools named python, web_search and terminal, and they execute as your operating-system user. The documentation states that anyone who can reach the server with the API key can run code on the machine. The default policy helps: tools are on for loopback and off for any non-loopback bind, and a request body cannot override that policy. If you deliberately enable tools on 0.0.0.0 you are asked to confirm, and the -y flag skips the confirmation. When you expose Studio at all, pass --disable-tools.

The exposure modes form a ladder. Plain unsloth studio is this machine only. -H 0.0.0.0 is the LAN, unencrypted. Adding --cloudflare also gives a public URL and is the least private. --secure gives a public HTTPS URL only, with a loopback bind, and it fails closed: if the tunnel cannot be established it refuses to fall back to an open port. Quick-tunnel URLs are random and change every start, so treat the URL as a secret and set stream: false in requests, since server-sent events do not survive the quick tunnel. For passwords, use --password - to read from standard input, never a literal value that shows up in ps and history. Behind your own reverse proxy, set UNSLOTH_STUDIO_TRUST_FORWARDED=1.

Flag order matters: options go after the subcommand, as in unsloth studio run --secure. Putting --secure before run exits with code 2.

Multi-user accounts arrived in September 2026, with isolation between accounts, but this is one host running tools as one OS user on shared GPUs. It is not a hardened multi-tenant platform. For real multi-tenant serving put vLLM behind your own gateway and authentication.

Pointing agents and tools at your model

Once you have a fine-tuned model running locally, a common next step is using it from a coding agent or an application. Unsloth ships a launcher for this: unsloth start <agent>, where the agent is one of claude, codex, opencode, hermes, openclaw, pi or dsh. The old name unsloth connect is a hidden, deprecated alias, so update old scripts.

The command finds a running Studio server or starts a temporary one when you pass --model, and then sets up the environment the agent needs. Useful options include --api-key (or the UNSLOTH_API_KEY environment variable), --gguf-variant, --context-length, --persist and --launch. To reach a remote server, set UNSLOTH_STUDIO_URL and UNSLOTH_API_KEY. Two details are worth remembering. Codex needs a GGUF model served through llama-server, and several agents need --persist on every launch if you want sessions to resume. Also note that --no-launch prints the environment and the command, which contain credentials, so do not paste that output into a ticket or a log.

If Claude Code still shows the default Anthropic model, the usual cause is that the environment variables were not exported in the same shell. Check ANTHROPIC_BASE_URL, ANTHROPIC_AUTH_TOKEN and ANTHROPIC_MODEL, then use /model inside the agent. "No Unsloth server was found" simply means Studio is not running.

For ordinary applications you do not need the launcher. Point any OpenAI-compatible client at http://127.0.0.1:8888/v1 with a Studio key. For a team service, though, remember the architecture: a single Studio process serving four llama.cpp slots by default is a desk tool. If the application serves customers, export the model and put it behind vLLM or another production engine, and keep Studio for experimentation and quick checks. The same reasoning applies to LiteLLM-style gateways: they can front either.

Reinforcement learning with GRPO, the working basics

Supervised fine-tuning teaches imitation. When you can score an answer automatically, for instance a maths result you can check or output that must parse as JSON, reinforcement learning with verifiable rewards can push beyond the data. Unsloth supports TRL's GRPOTrainer. This section covers the working basics; the guide for senior readers goes deeper.

GRPO, Group Relative Policy Optimisation, drops the separate value model and reward model that PPO needs. For each prompt it samples a group of completions, scores them with your reward functions, and uses each completion's score relative to the group as its advantage. That is why the setup is lighter: you supply Python functions that return numbers.

Loading uses fast_inference=True, which brings up a vLLM engine that shares GPU memory with the training model so rollouts are fast. vLLM must be installed, it works on Linux and WSL but not native Windows, and you must set fast_inference=False for models vLLM does not support. For QLoRA the docs give a rule of thumb that the parameter count in billions is about the VRAM needed in GB.

The knobs you will touch in GRPOConfig:

• num_generations is the group size. The effective batch size must be divisible by it, and the number of unique prompts, effective batch divided by num_generations, should be above 2.
• learning_rate is far smaller than SFT, with 5e-6 the documented recommendation.
• loss_type defaults to "dapo". The older "grpo" form is length-biased and not recommended, and other options include "dr_grpo", "bnpo" and "gspo".
• beta, the KL penalty, defaults to 0.0.
• mask_truncated_completions is off by default and the docs recommend leaving it off, because enabling it can make the KL term NaN.

Reward functions are passed as reward_funcs=[...], and the habit that pays off is several small functions rather than one big one: a format check, a correctness check, a length guard. Watch each reward's mean separately. A reward that never moves gives no gradient, and one that saturates immediately is too easy.

Expect RL to be slower to show progress than SFT, so judge by trends over many steps rather than the first few. Always evaluate against a held-out set of prompts the policy never trained on, and read sample outputs, because reward hacking, where the model finds a shortcut that scores well and is useless, is visible mainly by reading.

Environment flags and debugging beyond the basics

When something goes wrong, the question is whether the fault is in your data, in Unsloth's patches, or in the environment. The tools below separate those quickly.

The most useful diagnostic is to switch the optimisations off and see whether the bug survives. Setting UNSLOTH_COMPILE_DISABLE=1 turns off the auto-compiler, which the docs recommend for debugging a bad fine-tune, and UNSLOTH_DISABLE_FAST_GENERATION=1 disables the fast generation path. Both must be set before importing Unsloth, so set them in the shell or at the very top of the script. A "CUDA error: device-side assert triggered" is often resolved or at least clarified this way. If the problem disappears with compilation off, you have a precise bug report. UNSLOTH_ENABLE_LOGGING=1 shows compiler logging, and UNSLOTH_FORCE_FLOAT32=1 forces fp32 on GPUs that only support fp16, which the docs mention for Gemma 3. UNSLOTH_RETURN_LOGITS=1 makes the model return logits, useful for custom evaluation.

Here are the failures you will meet most, with their real causes:

Take the Gemma 3n row seriously: the docs mark that warning as critical because outputs are wrong rather than merely slow.

Beyond flags, use ordinary tools. nvidia-smi tells you real memory use. python -m bitsandbytes and python -m xformers.info check those dependencies. unsloth studio verify-install returns exit code 0 when the Studio install is complete, which makes it a good readiness probe. For Studio itself, the Settings page has an API monitor that shows requests, token counts, time to first token and errors.

When filing an issue, include the banner block, the pinned versions, the smallest script that reproduces, and whether the bug survives UNSLOTH_COMPILE_DISABLE=1.

Performance and cost, measured

Unsloth's headline numbers are claims about its kernels, and your job's speed depends on your data and hardware. Measure instead of assuming.

Start with where time goes. Throughput is tokens per second, not steps per second, so change in sequence length or packing alters step time without changing efficiency. Log tokens processed per second, and compare after the compile warm-up. Look at GPU utilisation: if it sits well below full, the bottleneck may be data loading or tokenisation, so check dataset_num_proc and whether you are re-tokenising each run. Cache the processed dataset to disk.

The big memory levers, in order: quantisation level, sequence length, per-device batch size, gradient checkpointing mode and LoRA rank. Spend them in that order. If you have headroom, raise the per-device batch before anything else, since larger batches use the GPU better and reduce accumulation overhead.

Cost follows from this. Unsloth itself is free: the core is Apache-2.0 and there is no paid tier in the docs. The money is GPU hours. A job that fits on a 24 GB card instead of an 80 GB card, or finishes in half the time, is cheaper, and QLoRA often makes the first possible. For teams in the Gulf and Egypt, cloud GPU capacity and data-residency rules both vary by region. If your training data cannot leave a jurisdiction, training on your own hardware or a regional provider with Core is straightforward, since nothing in the training path requires an external service beyond downloading model weights. Studio's public-reachability check contacts external sites on a wildcard bind and can be disabled with UNSLOTH_STUDIO_DISABLE_PUBLIC_CHECK=1, which matters in air-gapped or privacy-sensitive networks.

One caution on comparisons: when you compare a recipe against a plain TRL baseline, hold data, sequence length, batch and learning rate fixed, or you measure your changes instead of the tool.

Putting it all together

Here is one pipeline that uses the pieces above, designed to be run by a scheduler and not by hand.

1. Pin the environment. A container built from a pinned unsloth/unsloth tag, or a virtual environment from a constraints file with unsloth==2026.9.12 and unsloth_zoo==2026.9.8, plus UNSLOTH_DISABLE_AUTO_UPDATES=1.
2. Prepare data once. Deduplicate, apply the model's chat template, create a shuffled held-out split with a fixed seed, and store the processed dataset with a content hash.
3. Validate the config. Run unsloth train --config train.yaml --dry-run in CI so typos fail the pull request, not the GPU job.
4. Train with evaluation. QLoRA on a Dynamic 4-bit repository, response-only loss, evaluation every 50 steps, load_best_model_at_end, early stopping with patience 3, checkpoints on durable storage, and metrics sent to Weights & Biases or MLflow.
5. Export two ways. A merged 16-bit model for vLLM and a q4_k_m GGUF for local use, each with the identical template.
6. Evaluate the artefact. Serve each export with the engine that will run it, and run a fixed prompt set plus task metrics. Fail the pipeline if the quantised model drops below the threshold you chose.
7. Record everything. Write the banner, the versions, the config and the evaluation numbers next to the model, so any model can be reproduced or rolled back.
8. Share carefully. Publish to your registry with a tag that includes the date and the Unsloth version. If you use Studio for review, keep it on loopback or behind --secure with tools disabled.

The order encodes lessons from earlier sections: validate cheap things before expensive ones, evaluate the file you will ship, and keep a record that makes a result trustworthy next month. For the broader training toolchain, compare this with the PEFT and TRL guide, which covers the libraries Unsloth wraps.

Where you are now

You can predict what the Unsloth import changes and notice when it did not apply. You pin unsloth and unsloth_zoo together and understand the dependency bounds that make independent upgrades risky. You choose between QLoRA, LoRA and full fine-tuning from a VRAM budget and a quality question rather than habit. You format data with the right chat template, mask the loss to responses, and recognise the all-minus-100 error. You change one hyperparameter at a time, reason about effective batch size, and judge runs by held-out evaluation. You checkpoint, resume, export to merged, adapter and GGUF forms, and test the exported file in the engine that will serve it. You can run training from a validated YAML config and a pinned container, scale to several GPUs with DDP, and use Studio's API without exposing a remote shell. You have met GRPO and know the first knobs to watch.

The senior level takes this from a skill to a platform: failure modes of Studio and Core at scale, supply-chain and trust questions around model code and tools, multi-tenant cost and governance, advanced RL and quantisation-aware training, and the cases where another tool is the better choice.

Sources

• Unsloth documentation
• Unsloth changelog
• Install overview
• pip install guide
• Docker install guide
• Unsloth requirements
• Fine-tuning LLMs guide
• LoRA hyperparameters guide
• Datasets guide
• Chat templates
• Finetuning from the last checkpoint
• Saving to GGUF
• Saving to Ollama
• vLLM deployment guide
• Multi-GPU training
• DDP guide
• Environment flags
• Troubleshooting and FAQs
• Reinforcement learning guide
• Unsloth Studio
• Studio API
• Unsloth start integration
• Secure remote access with Cloudflare
• unsloth on PyPI
• unsloth on GitHub

Part two of three. At mid level an interviewer stops asking what Unsloth is and starts asking what you do when a fine-tune misbehaves, how you would choose between options, and whether you understand the system well enough to predict its behaviour. This file is organised as scenarios, trade-offs and the difference between strong and weak answers. Read each question, answer it aloud, then compare.

Where this picks up

The ninety-second version of the whole tool

Unsloth is an open-source layer over the Hugging Face training stack. When you import it before transformers, trl and peft, it patches model classes with hand-written Triton kernels and manual backward passes, so training runs faster and uses much less VRAM while you keep using TRL trainers, PEFT adapters and datasets. It ships three ways: Core, the Python library; Studio, a browser UI with a FastAPI server; and Desktop, a native app around the same backend. Core is the mature, scriptable part, and Studio and Desktop are still labelled beta.

Have these facts ready, because they separate people who have used it from people who have read about it:

• PyPI versions are calendar-based, such as 2026.9.12, and unsloth and unsloth_zoo are versioned and pinned together. Studio has its own version line.
• Recommended starting values are learning rate 2e-4 for LoRA and QLoRA, 5e-6 for RL, r of 16 or 32, alpha equal to r or twice r, one to three epochs.
• Repos ending in unsloth-bnb-4bit are Dynamic 4-bit, with selected layers left unquantised.
• The import-order rule exists because the patches apply at import time.

Scenario questions

The model is great in the notebook and gibberish in Ollama

Strong answer. The most likely cause is a mismatch between the template used in training and the one used at inference: a different chat template, a different EOS token, or a doubled BOS token. I would not touch hyperparameters yet. I would take one prompt, render it through the training tokenizer, render it the way the serving stack does, and compare the token IDs. If the Modelfile was produced by the Unsloth notebook it already carries the template, so I would check whether someone replaced it. Then I would check the quantisation: I would run the same prompts against the 16-bit merged model and the quantised file to see whether the damage comes from the template or from the quantisation level.

What a weak answer misses. It jumps to "train longer" or "lower the learning rate", treating a formatting fault as a quality fault. It also never separates template from quantisation, so the investigation cannot converge.

Training loss is all zeros

Strong answer. I would look for the message "All labels in your dataset are -100". That means train_on_responses_only masked every token because its instruction_part and response_part strings did not match the rendered template, often by one newline. I would print a formatted example, copy the exact markers from it, and rerun. I would also confirm the family: Gemma markers differ from Llama 3 markers.

Weak answer. "The model has learned perfectly." A loss of exactly zero from step one is a masking bug, not success. Even a loss that approaches zero over time is a warning sign of overfitting, since a healthy range is often around 0.5 to 1.0.

A job runs out of memory halfway through

Strong answer. First find when: at start, during training steps, during evaluation, or during saving. Each has a different fix. At start, I would reduce sequence length, drop to QLoRA, or check that Unsloth patching applied. During training, I would lower the per-device batch to 1 to 3 and raise gradient accumulation, which keeps the effective batch the same. During evaluation, I would shrink the eval batch and set fp16_full_eval=True, because evaluation can fail even when training is fine. During saving, I would lower maximum_memory_usage from its default of 0.75, for example to 0.5, since merging has its own peak.

Weak answer. "Use a bigger GPU." It may be right eventually, but it skips the diagnosis, and the cheapest fix is usually a setting.

Two runs from the same config give different results a week apart

Strong answer. The likely culprits are unpinned versions and unsloth_zoo updating itself. I would compare the banner blocks from both runs for Unsloth, torch and transformers versions, then pin unsloth and unsloth_zoo together and set UNSLOTH_DISABLE_AUTO_UPDATES=1. I would also check that the dataset split used a fixed seed and that the dataset itself had not changed upstream. Only after eliminating those would I consider real nondeterminism in GPU kernels.

Weak answer. "Training is random." Seeds, versions and data explain most differences, and a candidate who cannot list those has not run reproducible jobs.

A teammate wants to run Studio on a shared server so everyone can use it

Strong answer. I would ask what they need. If it is shared chat access to a local model, a loopback bind with an SSH tunnel is the simplest. If they need wider access, I would use --secure, which gives an HTTPS URL with a loopback bind and fails closed if the tunnel cannot start, and I would pass --disable-tools, because server-side tools such as python and terminal run as the OS user. I would set a real admin password, create per-person API keys with expiry, and explain that Studio is a single host with tools running as one user, so it is not a multi-tenant platform. If many users need a production endpoint, I would export the model and serve it with vLLM behind our own gateway and authentication.

Weak answer. "Launch it with -H 0.0.0.0." That is plain HTTP on the network, and it ignores the tool risk entirely.

Evaluation loss goes up while training loss goes down

Strong answer. This is overfitting. The model is memorising the training set. I would stop at the best checkpoint, which is what load_best_model_at_end with metric_for_best_model="eval_loss" and an early-stopping callback provide. For the next run I would reduce epochs or learning rate, check for duplicate rows between training and evaluation sets, add data, or lower the rank. I would also check the evaluation split was shuffled, because an unshuffled split from sorted data can look misleadingly different.

Weak answer. Ignores the evaluation curve, or concludes that the model is "still learning" because training loss falls.

The resumed run starts from scratch

Strong answer. I would check three things. Was save_strategy="steps" set with a sensible save_steps so checkpoints existed? Was output_dir on storage that survived the container restart? And was the call trainer.train(resume_from_checkpoint=True)? A container writing to its own writable layer loses checkpoints on restart, so a mounted volume is the usual fix.

Weak answer. Re-runs training from step zero and accepts the cost.

After exporting, the GGUF is much worse than the notebook model

Strong answer. I would isolate the variable. Evaluate the merged 16-bit export through the same serving path. If it is fine, the quantisation level is the problem, and I would try q5_k_m or q8_0 instead of q4_k_m. If it is also bad, the issue is in the merge or the template. I would also check whether someone relied on an old default: in Studio the default GGUF changed to Q4_K_M, previously F16 or BF16.

Weak answer. Blames GGUF in general, without comparing against the unquantised export.

A new Unsloth release breaks the pipeline

Strong answer. Roll back by pinning the last good unsloth and unsloth_zoo pair, then read the changelog for the intervening releases. The dependency bounds are tight: transformers and TRL have upper limits, so an independent upgrade of TRL is a common cause. I would reproduce in a scratch environment, bisect by version, and only then decide whether to adapt or wait. Going forward, upgrades go through a canary job that runs a small fixed evaluation.

Weak answer. "Upgrade everything to latest." That is how the break happened.

Trade-off questions

QLoRA, LoRA or full fine-tuning?

Strong answer. It is a memory-versus-fidelity trade. QLoRA uses over 75% less VRAM and is the sensible default; the documented floor for a 14B model is about 8.5 GB in QLoRA against about 33 GB in 16-bit LoRA. I choose 16-bit LoRA when I want quantisation removed as a variable or have memory to spare. I choose full fine-tuning rarely, for a large dataset with a big domain shift and a budget, because it needs the most memory, produces full-size checkpoints, and overfits more easily. I would also say that I would try to prove the need with evaluation first: if QLoRA reaches the quality target, the heavier mode buys nothing.

What it covers that a weak answer misses. Numbers, the evaluation condition for escalating, and the downstream cost of checkpoint size.

Merged 16-bit, adapter-only, or GGUF?

Strong answer. Adapter-only is the smallest and suits serving many fine-tunes over one base model with LoRA hot-swapping, at the cost of needing the base model at load time. Merged 16-bit is self-contained and is what vLLM, SGLang and Hugging Face load. GGUF targets llama.cpp and Ollama for CPU and consumer GPUs, and it is quantised, so it needs its own evaluation. I would often keep the adapter as the source of truth and generate the others from it. I would avoid merged 4-bit, which the docs discourage because merging into a quantised base loses precision.

Weak answer. Names the formats but cannot say which serving engine each fits.

Notebook, unsloth train, or a custom script?

Strong answer. Notebooks are for exploration. For repeatable work I prefer a config-driven job. unsloth train takes a YAML config whose schema rejects unknown keys and supports --dry-run, which is excellent for CI, and flags override file values for sweeps. A custom Python script is better when I need custom reward functions, a custom callback, or logic the config does not express, such as GRPO. I would not assume that a YAML exported from Studio is identical to the CLI schema, because the docs state they are similar but not guaranteed identical.

Weak answer. Defaults to notebooks for everything, or cannot name the validation advantage.

Studio or Core for the team's pipeline?

Strong answer. Core, for production training, because it is a library that fits any scheduler, whereas Studio is a single-node application with a SQLite store and no documented horizontal scaling. Studio is excellent for exploring models, quick runs, chatting with an export, and Data Recipes. A team can use both: Studio for discovery, Core for the pipeline that ships models.

Studio's built-in serving or vLLM?

Strong answer. Studio serves GGUF through llama-server, with a configurable number of slots, which is fine for a desk or a small group. For multi-user production, the Unsloth docs themselves point to vLLM for enterprise and multi-user inference. vLLM gives continuous batching and a production serving surface, so I would export merged 16-bit and run vllm serve.

One GPU with tight memory, or more GPUs?

Strong answer. I would exhaust single-GPU efficiency first: QLoRA, use_gradient_checkpointing="unsloth", a sensible sequence length and per-device batch. For throughput, DDP with torchrun or accelerate launch scales roughly linearly, and I would adjust accumulation because the effective batch grows with GPU count. For a model that does not fit at all, device_map="balanced" splits it across cards. I would also test my exact multi-GPU setup rather than trusting the docs, since official multi-GPU is described as coming soon.

SFT or GRPO?

Strong answer. SFT when I have good examples of the desired output. GRPO when I can verify answers automatically and want the model to improve beyond imitation. GRPO drops the value and reward models of PPO and uses group-relative scores from my reward functions. It needs vLLM for fast rollouts, so Linux or WSL, and a low learning rate around 5e-6. I would also note the cost: more moving parts, slower visible progress, and the risk of reward hacking, so I would read sample outputs.

What strong answers have in common

Strong answers share four traits. They start with diagnosis: find out when and where the problem occurs before proposing a fix. They name the mechanism: not "use QLoRA" but "QLoRA keeps the frozen base in 4-bit and trains small adapters, which is why memory drops". They quantify: a floor of 8.5 GB versus 33 GB for a 14B model is more convincing than "it saves a lot". And they close the loop with evidence: an evaluation set, a comparison of exports, a pinned version.

Weak answers also share traits. They treat the training loss as the goal, change several variables at once, forget that training and serving are separate systems, and assume the defaults are the same on every surface. A frequent specific miss is the default difference between the notebook and the CLI: notebooks use r=16 and lora_alpha=16, while the CLI config defaults to lora_r: 64 with lora_alpha: 16.

Commands and snippets to have ready

Be able to write these from memory, because they come up on whiteboards and in live exercises.

from unsloth import FastLanguageModel   # first

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/gemma-3-270m-it",
    max_seq_length=2048,
    load_in_4bit=True,
)
model = FastLanguageModel.get_peft_model(
    model,
    r=16,
    lora_alpha=16,
    target_modules=["q_proj","k_proj","v_proj","o_proj","gate_proj","up_proj","down_proj"],
    lora_dropout=0,
    bias="none",
    use_gradient_checkpointing="unsloth",
    random_state=3407,
)

model.save_pretrained_merged("out", tokenizer, save_method="merged_16bit")
model.save_pretrained_gguf("gguf_out", tokenizer, quantization_method="q4_k_m")

unsloth train --config train.yaml --dry-run
unsloth export ./outputs/checkpoint-100 ./export --format gguf --quantization q4_k_m
unsloth studio -p 8888
unsloth studio stop

Questions about running it safely

What are the risks of exposing Studio?

Server-side tools run as your OS user, so anyone who can reach the server with an API key can run code on the machine. The defaults reduce the risk: tools are on for loopback and off for other binds, and a request cannot override that. -H 0.0.0.0 is unencrypted HTTP on the LAN, --cloudflare adds a public URL, and --secure serves HTTPS only and fails closed. I would use --disable-tools whenever exposing it, supply the password through stdin with --password - instead of the command line, and treat quick-tunnel URLs as secrets because they are random but not authenticated by themselves. Also note that Studio's tool sandbox is a preview and network access remains unrestricted.

How do you handle secrets in a training job?

Use environment variables from a secret store: HF_TOKEN for gated models and push-to-hub, WANDB_API_KEY for tracking. In Docker, -e values are visible through docker inspect, so use restricted env files or the orchestrator's secrets. Do not log the output of unsloth start --no-launch, since it prints credentials. Keep trust_remote_code off unless the repository is trusted, as it executes code from the model repo.

Follow-up questions interviewers like

How does Unsloth relate to PEFT and TRL?

It builds on them. PEFT provides the LoRA machinery, TRL provides trainers such as SFTTrainer, GRPOTrainer and DPOTrainer, and Unsloth patches and accelerates them. That is why your existing TRL code mostly works, and why upgrading TRL independently can break things. Read the PEFT and TRL guide alongside this one.

How would you evaluate a fine-tuned model?

Hold out data before training, track eval_loss during training, and then evaluate the served artefact on task metrics and a fixed prompt set. Compare against the base model with the same prompts. For generation tasks, loss is a weak proxy, so I add task-specific checks or an LLM-as-judge with a rubric, and I read samples.

Sixty-second self-test

Answer each aloud without scrolling up.

1. Why must the Unsloth import come first, and how do you confirm patching happened?
2. Give the VRAM floor logic for choosing QLoRA over 16-bit LoRA for a 14B model.
3. What does train_on_responses_only do, and what error appears when its strings are wrong?
4. How do you compute effective batch size, and what do you change on an out-of-memory error?
5. Which settings make early stopping work?
6. Name the three export families and one serving engine for each.
7. Which export default changed in Studio v0.1.812, and why does it matter?
8. What is the tool default for loopback versus non-loopback binds?
9. What does unsloth train --dry-run buy you?
10. Which two packages do you pin together, and which environment variable stops one of them updating itself?

A note on honesty in answers

Unsloth changes weekly, so it is a strong move to say "the docs describe this as of the version I used, and I would check the changelog". Do not invent flag names, and state what you measured versus what you read. Recommending unsloth studio update, which the changelog says not to use, reveals knowledge from old tutorials.

Finally, connect the tool to the rest of the stack when you can: tracking with Weights & Biases, serving with vLLM, local runtimes with Ollama and llama.cpp. Interviewers hire people who see a pipeline, not a single library.

Part two of three. At this level your fine-tunes run, so the problems change character. They are no longer "it will not install" but "it trained and the answers are wrong", "it worked yesterday", and "it fits on my card but not in the pipeline". Start with the error table, then the practices in order of leverage, then the practice cards and recipes you can paste into a runbook.

Where this picks up

Common errors at this level

Cumulative: the beginner errors still apply. These are the ones that appear once things mostly work.

Three of these deserve a closer look because they cost the most time.

The silent patch failure. Nothing errors when the import order is wrong. The job runs, slowly, and runs out of memory at a batch size that worked before. The check is cheap: the first lines of output should include the Unsloth patching banner and the version block naming the GPU. Make your training script assert the order by importing Unsloth in a single place that every other module imports through.

Template drift. A model trained with one template and served with another is the most common cause of confident-looking nonsense. The fix is not more training. The fix is a test that runs identical prompts through the training tokenizer and the serving stack and compares the token IDs.

Evaluation memory. Evaluation runs inside the same process with leftover memory from training. Teams get through hours of training and die at the first evaluation step. Test evaluation in the first ten steps by setting eval_steps small in a smoke run.

The practices that pay off most, in order of leverage

1. Pin unsloth and unsloth_zoo together, and record the banner

Unsloth releases weekly with calendar versions such as 2026.9.12. Pin the pair, set UNSLOTH_DISABLE_AUTO_UPDATES=1 in images, and save the version banner with every run. This single habit makes most "it worked yesterday" problems solvable in minutes instead of days. When you must upgrade, do it as its own change, and rerun the same small evaluation before and after.

2. Hold out data and let evaluation decide

Before any tuning, split the data with train_test_split(test_size=0.01, shuffle=True, seed=3407). Enable eval_strategy="steps", save_strategy="steps", load_best_model_at_end=True and metric_for_best_model="eval_loss", and add the early-stopping callback. Training loss alone tells you only that the model fits the data. Held-out loss, and then task-level checks, tell you whether it improved.

3. Test the artefact you will ship

Export, serve with the real engine and the real template, then run a fixed prompt set and your task metric. Compare the merged 16-bit and the quantised GGUF. This catches template drift, quantisation damage and merge problems at once, and it is the habit that most separates reliable teams from lucky ones.

4. Change one thing at a time

Learning rate, rank, epochs, packing, sequence length and data all interact. Change one, rerun, compare held-out loss. Name runs after the change. A run ledger with versions, model, method, sequence length, effective batch, tokens per second, peak memory and evaluation loss pays for itself the first time a result needs defending.

5. Spend memory in the right order

When memory is tight, adjust in this order: quantisation level, sequence length, per-device batch size, gradient checkpointing and then rank. Lower the per-device batch to 1 to 3 and raise gradient accumulation rather than shrinking the effective batch. Remember that evaluation and saving have their own peaks.

6. Make training a job, not a session

Use a config file, ideally validated with unsloth train --config train.yaml --dry-run in CI. Write checkpoints to durable storage, inject tokens from a secret store, and exit non-zero on failure. A notebook is for exploring and a job is for repeating.

7. Match template, EOS and system prompt everywhere

Use the model's own chat template. Keep the system prompt identical between training and serving, and use train_on_responses_only with the correct markers for the model family. If you serve through Ollama, remember the notebooks write the Modelfile with the template; do not hand-edit it without a reason.

8. Treat Studio like a server with a shell

Keep loopback binds by default. When sharing, use --secure and --disable-tools, supply passwords through stdin with --password -, and give each person a key with an expiry. Server-side tools run as your OS user.

9. Read the changelog weekly while you depend on the tool

Defaults move. The GRPO loss_type default is dapo, the default Studio GGUF quantisation is now Q4_K_M, and full-evaluation precision defaults changed in 2025. Reading the changelog for ten minutes weekly beats debugging a behaviour change for a day.

Practice cards

Each card is a self-contained exercise of fifteen to thirty minutes. Do them on a small model such as unsloth/gemma-3-270m-it so they are cheap.

Operational recipes

Pin and reproduce an environment

uv venv unsloth_env --python 3.13
source unsloth_env/bin/activate
uv pip install "unsloth==2026.9.12" "unsloth_zoo==2026.9.8" --torch-backend=auto
export UNSLOTH_DISABLE_AUTO_UPDATES=1
pip freeze | grep -iE "unsloth|trl|peft|transformers|torch|datasets|bitsandbytes" > constraints.txt

Commit constraints.txt next to the training code, and install from it in CI and in the image.

Debug a suspect fine-tune by turning optimisations off

export UNSLOTH_COMPILE_DISABLE=1
export UNSLOTH_DISABLE_FAST_GENERATION=1
python train.py 2>&1 | tee run-debug.log

Set these before the script imports Unsloth. If the bug disappears, you have a precise report for the maintainers, and if it persists, the cause is probably in your data or template. Remove the flags afterwards: they cost speed.

Check a containerised Unsloth for GPU and logs

docker exec unsloth nvidia-smi
docker logs unsloth | grep -i "container ready"
docker volume ls | grep unsloth-studio

The first command proves the container can see the GPU. If it cannot, the NVIDIA Container Toolkit is missing or --gpus all was left off, and for AMD a permission error on /dev/kfd means the --group-add ids are missing. The second finds the generated passwords, which Studio prints after the line "Unsloth container ready". The third confirms Studio's data sits on a named volume, which it must, because Studio needs symlinks under /opt/unsloth-studio that host folders on Windows or macOS can prevent. Deleting the container keeps volumes, while removing the unsloth-studio volume wipes Studio data, so back up anything you care about first. Pin an image tag or digest rather than relying on :latest, and mount the Hugging Face cache so weights are not downloaded again. For general container habits see the Docker guide.

Export a checkpoint from the CLI

unsloth list-checkpoints --outputs-dir ./outputs
unsloth export ./outputs/checkpoint-100 ./export-gguf --format gguf --quantization q4_k_m
unsloth export ./outputs/checkpoint-100 ./export-16bit --format merged-16bit

The first command shows what runs and checkpoints exist. The next two produce a GGUF and a merged 16-bit model from the same checkpoint, ready for Ollama or llama.cpp and for vLLM respectively. Add --push-to-hub --repo-id org/name to publish.

Serve a merged export with vLLM

vllm serve /absolute/path/to/export-16bit

Use an absolute path if a relative one is not found. For multi-user traffic this is the route Unsloth's own documentation recommends over Studio. See the vLLM guide for production settings.

Compare two exports on one prompt set

import json, requests

prompts = [json.loads(l)["prompt"] for l in open("eval_prompts.jsonl")]
for name, url in [("merged16", "http://localhost:8000/v1/chat/completions"),
                  ("gguf", "http://localhost:11434/v1/chat/completions")]:
    for p in prompts:
        r = requests.post(url, json={
            "model": "finetuned",
            "messages": [{"role": "user", "content": p}],
            "temperature": 0,
        }, timeout=120)
        print(name, "|", p[:40], "|", r.json()["choices"][0]["message"]["content"][:80])

Use temperature zero so differences come from the model files, not sampling. Adjust the model names to whatever your two servers expect.

Expose Studio safely for a short demo

unsloth studio run --secure --disable-tools

This serves a public HTTPS URL with a loopback bind, forces the password gate, and fails closed if the tunnel cannot be established. The URL is random and changes on every start, so share it only with the people who need it, and stop the server when the demo ends with unsloth studio stop. Set stream: false in client requests, because streaming events do not survive the quick tunnel.

Reset a forgotten Studio password

unsloth studio reset-password

In Docker, run docker exec unsloth unsloth studio reset-password --username unsloth. In the container case this signs out existing sessions and revokes API keys, so tell the people who use them.

Start over on a broken llama.cpp build

If Studio reports that a build failed, remove ~/.unsloth/llama.cpp and re-run setup. A llama-server build failure is non-fatal for training, but GGUF inference is unavailable until it succeeds. Check that cmake and git are installed, and that on a GPU machine the NVIDIA driver and CUDA toolkit are present. To check the whole install, run unsloth studio verify-install, which returns exit code 0 when complete.

When to escalate and when to stop

Know when a problem is not yours to solve with settings. If the bug survives UNSLOTH_COMPILE_DISABLE=1, survives a fresh environment from your constraints file, and reproduces on a tiny dataset, it is worth a report to the maintainers with the banner block, the versions and the smallest script. If it disappears with compilation off, say so in the report, because that narrows the search enormously. If the model is simply not good enough after clean evaluation, the cause is almost always data: more varied examples, better labels or a cleaner split will beat another week of hyperparameter sweeps.

Habits worth building

• Write the data-format check before the training script. Render one example, read it, and keep it in the repository as a test fixture.
• Keep a short checklist in the repository for each model: base model, template, sequence length, method, effective batch, evaluation result, export format, serving engine and the pinned versions.
• Prefer Dynamic 4-bit repositories ending in unsloth-bnb-4bit when they exist, and record which repository revision you used.
• Keep trust_remote_code off unless you have reviewed the repository.
• Store secrets in the scheduler, never in notebooks or YAML files in version control.
• Watch the cost per experiment as well as the quality: tokens per second and GPU hours belong in the ledger beside evaluation loss.
• When a result is surprising in either direction, assume a bug before a discovery, and look for leakage between training and evaluation data first.

Mid-level checklist

Before you call a fine-tune done, confirm each of these:

• The banner shows patching applied, and the versions are saved with the run.
• There was a shuffled held-out split with a fixed seed.
• The loss is computed on responses only, with markers copied from a rendered example.
• Evaluation loss, not training loss, chose the checkpoint.
• Checkpoints went to durable storage and resume was tested once.
• The export was served by the real engine with the real template and compared against the unquantised model.
• Secrets came from the environment, and any Studio server is loopback, or secure with tools disabled.
• Another engineer can reproduce the run from the pinned versions and the config file.

For tracking experiments across all of these runs, continue with the Weights & Biases or MLflow guides, and for the libraries underneath Unsloth see PEFT and TRL.

This is part three of three. The earlier levels taught you to fine-tune a model and to run training properly. This one is about being the person who owns Unsloth as a platform: the one who decides which version everyone uses, who is paged when a training job corrupts an export, who answers the security review, and who explains to leadership why the GPU bill is what it is.

Where this picks up

By now you can train a LoRA adapter with FastLanguageModel, shape a dataset with a chat template, mask the user turns with train_on_responses_only, export to GGUF or merged 16-bit, and run a GRPO job. Those skills do not change. What changes is that every one of them now has a second owner's question attached.

I will start with the architecture, because every failure mode in this guide follows from where the boundaries between the pieces sit.

Three products, one codebase

The most important fact for a platform owner is that Unsloth is no longer one thing. The install page describes three distinct ways to install it, and they have different maturity, different licences, different update mechanisms and different blast radii.

Unsloth Core is the Python package, pip install unsloth, plus the companion unsloth_zoo. It is a library, not a service. It runs inside whatever process you start: a Kubernetes Job, a Slurm allocation, a notebook, a CI runner. It wraps the Hugging Face ecosystem rather than replacing it, patching transformers, TRL and PEFT classes at import time and substituting hand-written Triton kernels. This is the mature part, and it is the part a production training pipeline should be built on.

Unsloth Studio is a web application: a FastAPI and uvicorn backend, a React frontend, a SQLite database called studio.db, and worker subprocesses for training. For inference it manages a llama-server process from llama.cpp for GGUF models, uses transformers for safetensors models and MLX on a Mac. It is a single-node server. It was still labelled Beta at the time of writing, at release v0.1.900-beta.

Unsloth Desktop is a native Tauri application wrapping the same backend. It is the easiest thing to hand to an individual, and the least interesting for a platform owner, because it is a per-user install that updates itself.

Where state lives matters for backup and for clean-up. Studio's home directory is ~/.unsloth/studio, relocatable with UNSLOTH_STUDIO_HOME. It holds an auth/ directory, studio.db, a cache and a llama.cpp build; the llama.cpp build itself sits in ~/.unsloth/llama.cpp. Models live in the Hugging Face cache, which you can move with HF_HUB_CACHE or HF_HOME. A fact that surprises people: the uninstall script never deletes the Hugging Face model cache, but deleting ~/.unsloth removes every chat, checkpoint and export irreversibly.

The practical consequence of this split is a recommendation you will defend repeatedly. Use Core, in a container, driven by a config you version, for anything that must be reproducible. Use Studio where a human wants an interface: exploration, quick evaluation, demonstrations, and data recipes. Do not build a business process on a Beta web UI's internal HTTP routes, because the docs themselves warn that Studio's API reference may lag behind the code.

The dependency envelope and version pinning

Unsloth uses calendar versions of the form YYYY.M.N, where N counts releases within the month. September 2026 went from 2026.9.5 to 2026.9.12, which is more than one release every two days. At that cadence "latest" is not a version, it is a moving target, and an unpinned install is a different system every time it is built.

The second thing to understand is how tightly Unsloth bounds the libraries it patches. At 2026.9.12 the package requires, among others, torch>=2.4.0,<2.13.0, transformers>=4.51.3,<=5.5.0 with many individual releases excluded (5.0.0 and 5.1.0 among them), trl>=0.18.2,<=0.24.0 with 0.19.0 excluded, peft>=0.18.0, datasets>=3.4.1,<4.4.0 and bitsandbytes>=0.45.5 with 0.46.0 and 0.48.0 excluded. Those exclusions are not decoration. Each represents a version that broke the patches, and the lists change from release to release.

The consequence for a platform: you cannot independently upgrade TRL or transformers in an image that contains Unsloth and expect it to work. Either let Unsloth's resolver choose, or, if you must override with --no-deps, treat that as an experiment that requires its own validation run. A team that pins trl to a new release for a feature it wants will be the first to find an upper bound.

Pin the two Unsloth packages together, and pin them exactly:

unsloth==2026.9.12
unsloth_zoo==2026.9.8

Generate a full lock file from a known-good environment (with uv pip compile or pip freeze) and build the image from the lock, so that transitive dependencies are frozen too. Then deal with the one component that can change under you at runtime. unsloth_zoo can update itself, which is convenient on a laptop and a hazard in a production image. Set UNSLOTH_DISABLE_AUTO_UPDATES=1 in any image you intend to reproduce.

Python is a smaller trap but a real one. The PyPI metadata allows >=3.9,<3.15, while the docs require 3.11 to 3.13 for Studio and its training. Standardise on one version, and the docs recommend 3.13 for uv environments. Pick based on what your CUDA wheels and your other platform tooling support, and write it down.

Reproducible training environments

A training job on a platform is a container image plus a config plus a data reference plus a seed. If any of those is implicit, you cannot reproduce a run, and "which adapter was trained how" becomes archaeology. This section is about making each one explicit.

Unsloth publishes official images: unsloth/unsloth for NVIDIA, unsloth/unsloth-rocm for AMD (built against ROCm 7.2), and unsloth/unsloth:core for a non-root variant used with --user. They are on a rolling :latest tag, which is the same problem as an unpinned pip install. Pin a tag or, better, a digest in production, and consider building your own image from a base you control, installing the pinned requirements from above. The cross-reference here is the Docker guide, whose senior section covers digest pinning, scanning and signing; all of it applies unchanged to a training image.

If you do run the official image, know its conventions. The NVIDIA image needs a driver at 570.26 or newer and the NVIDIA Container Toolkit. Inside the container Studio listens on port 8000 and JupyterLab on 8888. The container runs as root, and published ports listen on all interfaces. Studio data should be on a named volume mounted at /opt/unsloth-studio, not a host folder, because Studio needs symlinks there and bind mounts on macOS or Windows can stop the container from starting.

For batch training, bypass Studio completely. The installed CLI has a unsloth train command that reads a YAML or JSON config. Its schema is validated with Pydantic with extra="forbid", so a misspelled key is an error rather than a silently ignored setting. That is a gift to a platform owner: you get config validation for free.

model: unsloth/gemma-3-270m-it
data:
  dataset: yahma/alpaca-cleaned
  format_type: alpaca
training:
  training_type: lora
  max_seq_length: 2048
  load_in_4bit: true
  output_dir: ./outputs
  num_epochs: 1
  learning_rate: 2e-4
  batch_size: 2
  gradient_accumulation_steps: 4
  save_steps: 50
  random_seed: 3407
  gradient_checkpointing: unsloth
lora:
  lora_r: 16
  lora_alpha: 16
logging:
  enable_wandb: true
  wandb_project: team-adapters

unsloth train --config train.yaml --dry-run
unsloth train --config train.yaml

Three cautions. First, the CLI defaults differ from the notebooks: the CLI schema defaults lora_r to 64 with lora_alpha 16, where the notebooks use 16 and 16. Always set both explicitly so that the config, not a default, defines the run. Second, the YAML files that Studio saves are similar to this schema but not guaranteed identical: Studio's docs show num_train_epochs and per_device_train_batch_size where the CLI uses num_epochs and batch_size. Do not assume a file round-trips. Run --dry-run as the validation step. Third, secrets go in as HF_TOKEN and WANDB_API_KEY environment variables, which the CLI reads, rather than as keys in a committed file.

If your team writes Python rather than YAML, the equivalent discipline applies: a single entry-point script, all hyperparameters from a config object, random_state and seed fixed (the docs use 3407), and the import order rule (from unsloth import ... before transformers, trl and peft) enforced by a lint or a template. Violating import order produces patches that silently do not apply, which is a slow, quiet performance regression rather than a crash.

Scaling: which component breaks first

Ask "how does this scale" about each part separately, because the answers are very different. For the full tool-by-tool view, see the neighbouring PEFT and TRL guide and the DeepSpeed guide for the alternatives.

Training throughput scales vertically and across GPUs on one node. More VRAM lets you use a larger model or a longer context. With more than one GPU, distributed data parallel turns on automatically and throughput scales roughly linearly, launched with accelerate launch train.py or torchrun --nproc_per_node N train.py. For a model too big for one card, device_map="balanced" in from_pretrained splits it across GPUs. FSDP and DeepSpeed work through Accelerate. One honest caveat: the docs have described "official" multi-GPU support as coming soon while the Studio docs say multi-GPU works automatically, so test your own configuration before promising anything to a team. Multi-node via torchrun is possible in principle, but it is not a documented, supported Studio feature, so I would not make it a platform commitment.

Memory is the first wall. The documented minimum VRAM figures make the planning easy. QLoRA on a 7B model needs about 5 GB, on a 14B about 8.5 GB, on a 32B about 26 GB and on a 70B about 41 GB. Plain 16-bit LoRA needs roughly four times as much: 19 GB for 7B, 76 GB for 32B and 164 GB for 70B. These are minimums, not recommendations. Real jobs add headroom for context length, batch size and evaluation. For GRPO the rule of thumb is that, with QLoRA, the parameter count in billions equals the VRAM in GB, and the vLLM engine shared through fast_inference is a significant extra consumer.

Studio is the thing that does not scale out. It is one host, one SQLite database, one OS user running training workers and tools. There is no documented horizontal scaling or high availability. If ten teams start training in the same Studio, they queue for the same GPUs and fight for the same disk. That is the single most important scaling statement in this guide.

Serving is the second place you outgrow Unsloth. llama-server handles concurrency through slots, four by default through --parallel, and MLX gained batched serving in v0.1.900. That is adequate for individuals, small teams and evaluation. The docs are direct about the line: for enterprise or multi-user inference of FP8 or AWQ models, use vLLM. The standard path is to export merged 16-bit and run vllm serve, with LoRA hot-swapping documented for adapters. The vLLM guide covers that side.

Disk and downloads are the quiet bottlenecks. Every base model, every checkpoint and every export is gigabytes. A shared Hugging Face cache on fast storage, set through HF_HUB_CACHE, avoids fifty copies of the same weights. If downloads stall at 90 to 95 percent, set UNSLOTH_STABLE_DOWNLOADS=1 before importing for synchronous downloads. Also remember torch.compile warm-up takes around five minutes at the start of a run, so do not benchmark the first minutes of a job, and do not kill a job during that window.

Mixture-of-experts models deserve a mention. Training kernels for them (torch._grouped_mm or an Unsloth Triton backend, with split LoRA) are on by default, selectable with UNSLOTH_MOE_BACKEND set to grouped_mm, unsloth_triton or native_torch, and the Triton backend autotunes once for about two minutes. If a MoE job is slow only on its first run, that is the explanation.

The trust model: who can do what to the machine

This is the section a security reviewer will read first, and it is where a platform owner can do the most harm by accident. The key idea is simple and uncomfortable: Studio can run code, and the code runs as the OS user that started Studio.

Studio exposes server-side tools, namely a Python interpreter, web search and a terminal, that a model can call during a chat. The docs state it plainly: anyone who can reach the server with an API key can run code on that machine. That is a design feature of an agent-capable local workbench, and a hazard the moment the port leaves the machine.

Unsloth's defaults are sensible. The server binds to loopback, 127.0.0.1:8888. An admin password must be set on first launch, when you are redirected to /change-password. Tools are on by default for loopback and off for any non-loopback bind, a process-level override that request bodies cannot bypass. Passing --enable-tools on 0.0.0.0 asks for a y or N confirmation, which -y skips. Your job is to keep those defaults from being worn away by convenience.

The exposure modes form a ladder, from most private to least:

--secure deserves a closer look because it is the mode you are most likely to recommend. It binds to loopback and publishes through a Cloudflare tunnel, and if the tunnel cannot be established it refuses with A secure Cloudflare link is not allowed, use --no-secure which provides a 0.0.0.0 link and exits with code 1. That is the right behaviour, failing closed, and the error message is a trap for the unwary: the suggested fix, --no-secure, exposes a raw port. The correct response is to fix outbound network access to the tunnel. Quick-tunnel URLs are random, change on every start and cannot be pinned, so the URL itself is a secret and a poor basis for a service other people bookmark. Over a quick tunnel, set stream: false in API calls, because server-sent events do not survive it.

Several smaller behaviours matter to a reviewer. Authentication is rate-limited and proxy-aware. If you put your own reverse proxy in front, set UNSLOTH_STUDIO_TRUST_FORWARDED=1 so that client addresses are read correctly; behind the Cloudflare tunnel, CF-Connecting-IP is honoured automatically. On a wildcard bind, Studio contacts ifconfig.me and check-host.net to test public reachability; set UNSLOTH_STUDIO_DISABLE_PUBLIC_CHECK=1 in air-gapped or privacy-sensitive environments. stdio MCP servers are auto-revoked while a tunnel is live, and UNSLOTH_STUDIO_ALLOW_STDIO_MCP can override that in either direction. Remote and LAN access, introduced as a preview in August 2026, is disabled by default and its control routes require a UI session, so an API key gets a 403 with Remote access requires a UI session.

There is a preview OS-level sandbox for tools on Linux and macOS, and the release notes say, in effect, that network access remains unrestricted. File access outside the sandbox asks for approval, and a "Bypass Permissions" mode disables those safeguards. Treat the sandbox as defence in depth, not as the boundary. The boundary is the host, the OS user and the network path.

Secrets and the supply chain

Fine-tuning touches more credentials than people expect. A typical run uses a Hugging Face token for gated models and for pushing results, a Weights and Biases key, possibly cloud storage credentials for data, and, if Studio is involved, an Unsloth API key and an admin password. The goal is that none of them live in a place that outlives the job.

The documented conventions are good ones. HF_TOKEN is token= in the Core API and --hf-token or the HF_TOKEN environment variable in the CLI. WANDB_API_KEY is read from the environment or passed as --wandb-token. Clients such as unsloth start read UNSLOTH_API_KEY and UNSLOTH_STUDIO_URL. Studio API keys start with sk-unsloth-, are shown once, are stored only as a hash, can have an expiry and can be revoked. That last property is worth using: issue short-lived keys per consumer instead of one permanent key.

Some details you will otherwise learn the hard way. A literal --password VALUE is visible in ps output and shell history, so use --password - to read from standard input. Environment variables passed with docker run -e are visible through docker inspect, so prefer a secret store or a restricted env file. In Kubernetes or a secrets manager such as Vault, inject at runtime and scope tokens to least privilege: a read-only Hugging Face token for downloading, and a separate write token only for the publishing step. unsloth start --no-launch prints the environment and command including credentials, so do not capture its output in logs. A password reset with unsloth studio reset-password --username unsloth in the container generates a new password, signs out existing sessions and revokes API keys, so plan for that in a rotation procedure.

On the supply chain, three separate risks exist. The first is the package itself. Pin exact versions and install from PyPI or from the official images, then verify what you built. The project's changelog states that it was not affected by the LiteLLM compromise of versions 1.82.7 and 1.82.8 and that the Data Designer component has since removed LiteLLM. I cannot independently audit that claim, but it is a useful illustration of why a dependency lock and an SBOM matter: dependency incidents arrive from transitive packages. Scan your images with a tool such as Trivy and gate on fixable high-severity findings.

The second is model weights and model code. Setting trust_remote_code=True executes Python from the model repository, so enable it only for repositories you trust, and prefer pinning a model revision. Pickle-based formats are a classic vector, and Unsloth's July 2026 hardening removed the torch.load fallback on training_args.bin so that an untrusted pickle in a checkpoint cannot execute. Hugging Face virus-scan and dangerous-file detection were also added. Even so, a checkpoint from outside your organisation is untrusted input.

The third is the redirect behaviour. Unsloth automatically redirects some model names to its own repositories, which is usually what you want, but a platform that must know exactly which weights were used should set use_exact_model_name=True in from_pretrained and record the resolved repository and revision with the run.

Multi-tenancy: what Studio does and does not give you

On 17 September 2026 Studio gained multi-user accounts. Under Settings and Accounts, an owner creates one-time setup codes, each person sets an individual password, work is isolated between accounts, and accounts share a loaded model when the settings match. The owner can also allow other accounts to connect to local model servers. This is a real feature and a genuine improvement over a shared admin password.

It is not a hardened multi-tenant platform, and it is important to say so clearly. There is one host. Training workers and tools run as one OS user. GPUs are shared between accounts, so one tenant's out-of-memory error is another tenant's failed job. Disk, the Hugging Face cache and llama.cpp slots are shared. Account isolation is a software boundary on top of a single process tree, and tools execute with the host user's permissions. If tenant A can run the Python tool, tenant A can read what the OS user can read, regardless of how the UI separates chat histories. Treat the accounts feature as a convenience for a trusted group, such as a research team that shares a workstation, not as a security boundary between mutually distrusting groups.

For genuine isolation, move the boundary out of Studio. The patterns, in increasing strength:

1. One Studio per team on separate hosts or containers. Each has its own volume, its own GPUs and its own credentials. Simple and effective. The cost is idle GPUs.
2. Batch jobs instead of interactive Studio. Each team submits unsloth train jobs to a scheduler, such as Kubernetes (Kubernetes guide) with GPU limits and namespaces. The scheduler, not Studio, enforces quotas and isolation.
3. A separate serving tier. Models are exported once and served through vLLM behind your own gateway with your own authentication and rate limits, for example an LLM gateway such as LiteLLM. Tenants never touch Studio for inference.

Whichever you choose, define what a tenant owns: a namespace or project, a GPU quota, an artifact prefix in object storage, a model registry namespace, and a cost label. If those five are not defined, there is no tenancy, only shared access.

Cost: where the bill actually comes from

Unsloth itself costs nothing, and there is no paid tier in the documentation, so "cost" means everything around it. The honest cost model has four lines.

GPU hours dominate. Unsloth's value claim is training two times faster with 70 percent less VRAM, with QLoRA using over 75 percent less memory than plain 16-bit. Treat those as the project's own claims and measure your own. The meaningful effect for a platform is that a job that needed a large GPU may fit on a smaller one, and a card that was too small becomes usable. The saving is real when you act on it, by right-sizing instances, and imaginary when everyone keeps requesting the largest GPU out of habit. Free Colab and Kaggle T4 GPUs are enough for small models, which makes onboarding cheap.

Idle capacity is the biggest hidden cost of a shared Studio host. A GPU server sitting on a desk, or an always-on cloud instance with a Studio running, costs the same whether anyone is training. Prefer ephemeral job-based training with scale-to-zero node pools, and schedule shutdown for interactive hosts.

Storage and egress are the next. A 70B model's merged 16-bit export is hundreds of gigabytes. Checkpoints at save_steps=50 over a long run multiply that. Apply a retention policy: keep the final adapter and the evaluation record forever, keep intermediate checkpoints for days. Egress from downloading base models repeatedly can be reduced with a shared cache or a regional mirror. Where you operate in the Gulf or Egypt, the nearest regions and data-residency rules for training data matter, so check where your training data may legally be processed before choosing a cloud region; that is a policy decision, not an Unsloth setting.

Engineering time is the cost nobody budgets. At roughly weekly releases, someone has to own upgrades, read changelogs and triage breakage. That is a real fraction of a person, and it is the correct comparison against a managed fine-tuning service. When leadership asks why not buy a managed service, the answer is a calculation: your GPU hours saved against the maintenance load and the data-control benefit.

Attribute cost by label. Put the team, project and run identifier into your job metadata and your Weights and Biases or MLflow run, so that a bill can be split without detective work.

Upgrades and migrations

A project that releases this often needs an upgrade process designed for it, otherwise you either never upgrade, and accumulate security and compatibility debt, or upgrade constantly, and accumulate outages.

A cadence, not a reaction. Choose one: for example, evaluate a new release every two to four weeks, promote it only when a fixed regression suite passes, and keep the previous known-good image available for rollback. Security fixes can jump the queue.

A regression suite that catches what matters. For fine-tuning platforms, "the import works" is not a test. The suite should include a short deterministic training run on a tiny model, compared against a stored loss curve with a tolerance; a merge and export round trip; a chat template check that formats a fixed conversation and compares the exact string, EOS token and BOS token; and a load of the exported model in the serving runtime with a smoke prompt. The gradient accumulation fix is a good reason for the loss comparison: the docs say batch two with accumulation eight and batch sixteen with accumulation one now match, so a run that diverges from its stored curve after an upgrade is a signal worth investigating rather than ignoring.

Different channels, different mechanisms. Core upgrades are pip install --upgrade unsloth unsloth_zoo, but for a platform you rebuild the image with new pins instead. Studio updates by re-running the install one-liner; the official changelog says not to use unsloth studio update, because packaging will not get the latest updates, even though the subcommand still exists. Desktop updates itself through Settings, General, Check for updates, except the Linux .deb, which you install over the old package manually. Docker images are updated by pulling and recreating the container, with data persisting on the named volumes. Each channel needs its own line in the runbook.

Known migration traps from 2026. The default GGUF quantization in Studio changed to Q4_K_M; if a downstream process expected the F16 or BF16 output, it now gets a smaller, lossier file. Dynamic v3.0 GGUFs arrived in August, so quantization names in a model reference may change. The Docker image was reworked on 17 September, with Studio data moving to a named volume and passwords now generated: anyone who automated against the old image layout needs to re-check it. A migration from the legacy extras such as unsloth[cu124-torch250] to uv pip install unsloth --torch-backend=auto is the right direction, since the extras path is documented as legacy. And documentation drifts: the DDP guide refers to python unsloth-cli.py with snake_case flags, an older script rather than the installed unsloth command. When docs and installed behaviour disagree, run --help on the installed version.

Rollback. Keep the previous image tag, and keep adapters tagged with the Unsloth version that produced them. Adapters are usually portable across versions, but a merged export bakes in a chat template and tokenizer files, which is where regressions hide.

Governance: contracts, evaluation gates and lineage

Governance for a fine-tuning platform is not paperwork. It is a small number of contracts that stop one team's shortcut becoming another team's incident.

The export contract. The most common cross-team failure is a model that is good in the training notebook and gibberish, endless or repetitive in the serving runtime. The documented cause is a different chat template, EOS token or BOS token at inference than at training. Make the contract explicit. A published model must ship with its tokenizer and chat template, the template name used in training (for example the get_chat_template name), the exact instruction_part and response_part strings if responses-only training was used, the stop tokens, the sampling settings and a golden prompt with its expected output. The serving team runs the golden prompt in their runtime before accepting the artifact. This turns the most frequent incident into an automated acceptance test.

The evaluation gate. Training loss is not quality. A loss near zero often means overfitting, and a healthy loss of around 0.5 to 1.0 says nothing about task performance. Require a held-out split, shuffled with a fixed seed, an evaluation set that the training team does not tune on, and a comparison to the base model on the same prompts. For regression detection and LLM evaluation tooling, see the Ragas guide and Promptfoo guide. Early stopping on evaluation loss is a useful default but not a quality gate.

Data governance. Decide what may be in a training set before the first run. Personal data, customer content and licensed text all need a rule, and the rule should be machine-checkable where possible. Studio's Data Recipes, which generate synthetic data through NVIDIA NeMo Data Designer, produce data whose provenance still needs recording. Where regional regulations apply to your employers' data, record where training occurred and where the artifacts are stored.

Lineage. Every artifact answers: which base model and revision, which dataset revision, which config, which Unsloth and unsloth_zoo versions, which image digest, which seed, who launched it, and which evaluation passed. An experiment tracker such as Weights and Biases or MLflow holds most of this if you make logging mandatory in the config (enable_wandb in the CLI schema, report_to="wandb" in Core). Name artifacts with run identifiers, not with "final" or "v2".

Licences. Base models carry licences that may restrict commercial use, redistribution or the use of outputs to train other models. Record the base model licence with the artifact. Separately from that, remember the AGPL note about Studio.

Incident playbooks

Each playbook below is written as symptom, first checks, fix and prevention, using the exact messages from the docs where they exist. Work from the symptom, not from a theory.

Playbook 1: good in training, gibberish in serving. Symptom: outputs that are fluent in the notebook are repeated, endless or nonsensical in Ollama, vLLM or llama.cpp. Cause: a mismatch of template, EOS or BOS token, sometimes a doubled BOS. First checks: render the same conversation with tokenizer.apply_chat_template in training and print the string the serving runtime builds; compare line by line. Fix: align the template and stop tokens, and re-export if the template was baked into the model files. Prevention: the export contract and golden prompt.

Playbook 2: all training loss is zero. Symptom: All labels in your dataset are -100. Training losses will be all 0. Cause: the instruction_part and response_part passed to train_on_responses_only do not match the model's template, so every token is masked. Fix: use the exact strings for the model family. For Gemma they are <start_of_turn>user\n and <start_of_turn>model\n. For Llama 3.x they use the header tokens. Prevention: print one tokenised, masked example in the pipeline and assert that some labels are not -100.

Playbook 3: CUDA out of memory. During training or evaluation: lower the batch size to between one and three and raise gradient accumulation instead, keep fp16_full_eval or bf16_full_eval on, and shrink the evaluation set. During export: crash or out-of-memory when merging or writing GGUF is a peak memory problem, so pass maximum_memory_usage=0.5 (the default is 0.75) to the save call. Prevention: size from the documented table with headroom.

Playbook 4: a crash with a device-side assert. RuntimeError: CUDA error: device-side assert triggered is often a compiler or fast-generation issue. Set UNSLOTH_COMPILE_DISABLE=1 and UNSLOTH_DISABLE_FAST_GENERATION=1 before importing Unsloth, restart the process and file a bug with the version information. These flags are also the right first move whenever a fine-tune comes out badly and you suspect the compiler, because disabling them separates an Unsloth patch problem from a data or configuration problem. UNSLOTH_RETURN_LOGITS=1 helps when you need logits for debugging.

Playbook 5: the dependency mismatch. A warning such as Some weights of Gemma3nForConditionalGeneration were not initialized from the model checkpoint is flagged by the docs as critical, because outputs are wrong. Cause: version mismatch. Fix: force-reinstall unsloth and unsloth_zoo, then transformers and timm with --no-deps. Prevention: a locked image, so this cannot happen at runtime.

Playbook 6: Studio unreachable or stopped. Several distinct messages. Error: Unsloth Studio is already running on port 8888 means run unsloth studio stop or choose another port. In Docker, Studio stopping about an hour after start is the bootstrap timeout expiring because the generated password was not changed; restart the container and change the password. Lost connection to the model server means the llama.cpp server crashed or the model tab closed; reload the model. Tunnel messages such as cloudflared is unavailable or did not register a connection point at egress filtering, so look at the network path before the application.

Playbook 7: suspected exposure. If Studio with tools may have been reachable by an untrusted party: stop it with unsloth studio stop, treat the host as compromised, rotate every credential the OS user could read, revoke API keys, reset the admin password, preserve logs and the studio.db file for investigation, and rebuild the host from the image rather than cleaning it. Prevention is the trust-model section.

Playbook 8: GRPO instability. Check the defaults you may have inherited: loss_type is "dapo", beta is 0.0, and mask_truncated_completions should stay off because the docs warn it can make the KL term NaN. Check that the effective batch size is divisible by num_generations and that unique prompts per step is above two. Use a learning rate near 5e-6 for RL rather than the 2e-4 that suits LoRA SFT, and remember vLLM is not available on native Windows, so GRPO there fails.

Where Unsloth stops

A senior engineer's credibility depends on naming a tool's limits before somebody discovers them. Unsloth is excellent at a specific job: efficient single-node fine-tuning and RL of open models, with a convenient local inference workbench. Past that, choose something else.

High-throughput multi-user serving. Use vLLM. The docs say so themselves. See the vLLM guide, and for Kubernetes-native serving the neighbouring guides in this catalogue.

Large-scale multi-node pretraining or very large fine-tunes. When a model needs sharding across many nodes, with elastic scheduling and mature fault tolerance, a framework built for that is the right fit, such as DeepSpeed or Axolotl on a proper cluster. Unsloth's multi-node story is not a documented feature, so do not rely on it.

A multi-tenant, audited, HA platform. Studio is one host with one OS user and no documented high availability. If you need role-based access, audit trails, quotas and failover, build that around batch jobs and a gateway, or buy it.

Custom architectures and exotic training objectives. Unsloth supports a very large and growing list of models, but its speed comes from hand-written kernels and patches for specific architectures. If you are researching a novel architecture or modifying the forward pass, the patches may fight you, and plain PEFT and TRL are easier to reason about.

Production inference on a Mac or for edge use. Studio supports MLX and GGUF inference, and Core's support for Apple Silicon is described as in the works. For a stable on-device runtime, take the exported GGUF to llama.cpp or Ollama directly.

Regulated change control. An open source project with near-daily releases and a Beta UI is not compatible with a change process that demands quarterly releases and vendor attestations, unless you absorb the difference by pinning and validating releases yourself.

None of this says Unsloth is the wrong choice. It is the right choice for the fine-tuning step, and it should not also be your serving platform, your scheduler and your identity system.

Running Unsloth as a platform for other teams

Putting the previous sections together, here is the shape of a platform I would be comfortable owning.

The paved road. One documented way to fine-tune: a base image with pinned Unsloth packages, a config template for unsloth train, a standard evaluation step, and an export step that produces a model card with the export contract. Teams that stay on the paved road get support; teams that leave it own their problems. Most platform failures are a lack of a default, not a lack of options.

Self-service with guardrails. Teams submit jobs, not tickets. A job specifies a base model from an approved list, a dataset reference, a config and a GPU class. The platform validates the config (the Pydantic schema helps), checks quotas, runs the job in an isolated container with scoped credentials, and records lineage automatically. Interactive Studio, if offered, is per-team and on loopback or behind your own authentication, with tools off unless a team asks and accepts the risk.

Observability for the platform itself. Studio provides a training status panel, GPU monitor, charts for loss, gradient norm, learning rate and utilisation, an SSE stream at /api/train/stream and an API monitor with token counts, time to first token and throughput. For Core jobs, rely on Trainer logging to Weights and Biases or TensorBoard, plus standard GPU metrics scraped into Prometheus. The platform signals to watch are queue wait time, GPU utilisation per job, job failure rate by cause, time from request to evaluated artifact, and the age of the image in use against the current release.

CI/CD for the pipeline. Treat a training pipeline like a service. Changes to configs and the image go through review and the regression suite in GitHub Actions or your CI, and image builds are signed and scanned. Infrastructure for GPU pools belongs in code, for example with Terraform.

Support and communication. Publish a short support matrix: supported Unsloth version, supported Python version, supported GPUs, supported model families, supported exports and the serving runtimes you have verified. Publish a deprecation policy. Say plainly that Studio is Beta.

Ownership. Name an owner for the image, an owner for the evaluation suite and an owner for the serving contract. A platform where everyone owns the export format is one where nobody does.

The review checklist

Use this for a design document or a pull request that touches the training platform.

• Are the unsloth and unsloth_zoo versions pinned exactly, with a lock file for the rest, and is UNSLOTH_DISABLE_AUTO_UPDATES=1 set?
• Is the image pinned by digest, scanned, and built from a documented base?
• Is the Python version within the supported 3.11 to 3.13 range?
• Does the config set lora_r, lora_alpha, seed, learning rate and gradient checkpointing explicitly, and does it pass --dry-run?
• Does from unsloth import ... come before transformers, trl and peft?
• Is Studio bound to loopback, or, if not, are tools disabled, the bootstrap password changed and the exposure justified?
• Is --secure used rather than --no-secure, and is the tunnel URL treated as a secret?
• Are HF_TOKEN, WANDB_API_KEY and sk-unsloth- keys injected at runtime, scoped, and absent from logs and compose files?
• Is trust_remote_code false unless a reviewer approved the repository?
• Does each artifact carry the export contract: template, stop tokens, responses-only strings and a golden prompt?
• Is there a held-out evaluation compared against the base model?
• Is VRAM sized from the documented table with headroom, and is peak memory observed?
• Is there a retention policy for checkpoints and exports?
• Is there a rollback image and a regression suite for upgrades?
• Does the document say when this should move to vLLM or another platform?

The complete picture

Unsloth is a fast, memory-efficient way to fine-tune open models, and at senior level the interesting questions are all around it. Core is a library you embed in a reproducible container job. Studio is a single-node Beta workbench with code-execution tools, to be bound to loopback or tunnelled with --secure, never casually exposed. Versions move almost daily and are bounded tightly against Torch, transformers and TRL, so you pin them as a set. The platform's job is to make the right path the easy path: a validated config, a pinned image, an evaluation gate, an export contract and a lineage record, with serving handed off to a runtime built for it.

Where the series leaves you

You started by running a notebook, then learned to shape data, templates and hyperparameters, and now you can reason about the platform: its trust boundaries, its scaling limits, its upgrade risk and its place among neighbouring tools. The next steps are practical. Build the regression suite and the export contract for your own team. Run a tabletop exercise of the exposure playbook. Compare Unsloth's serving against vLLM on your own workload, and read the PEFT and TRL guide to understand the layers Unsloth patches, because understanding what it wraps is the best protection when a release breaks. Then keep reading the changelog, because with this project that is part of the job.

Sources

• Unsloth documentation
• Unsloth changelog
• Installing Unsloth
• Unsloth with Docker
• Unsloth requirements and VRAM
• Unsloth Studio
• Secure remote access with Cloudflare
• LAN access
• Unsloth API
• Environment flags
• Multi-GPU training
• Troubleshooting and FAQs
• Inference and deployment
• Serving with vLLM
• Reinforcement learning guide
• Unsloth on GitHub
• unsloth on PyPI
• unsloth-zoo on PyPI

Part three of three. This is interview preparation for someone who owns Unsloth as a platform: system-design prompts, incident questions, and the arguments you make to leadership. Read the opening review quickly, then spend your time on the prompts, because senior interviews reward reasoning out loud more than recall.

The thirty-second answer

Unsloth is an open-source toolkit that makes fine-tuning and reinforcement learning of open models faster and lighter on GPU memory, using hand-written Triton kernels and patches to the Hugging Face stack. It ships as Core (a Python library you embed in jobs), Studio (a single-node Beta web UI with local inference and code-running tools) and Desktop. As a platform owner I treat Core as the production path in a pinned container, Studio as an interactive workbench that must never be casually exposed, and vLLM as the serving tier.

The facts to have ready

Interviewers at this level probe whether your knowledge is current. These are the points that most often separate a fresh answer from a stale one.

• Three products. Core, Studio and Desktop have different maturity, update paths and licences. Core is Apache-2.0; the Studio UI and CLI source are AGPL-3.0.
• Versioning. Calendar versions, 2026.9.12 for unsloth and 2026.9.8 for unsloth_zoo at the time of writing, released many times a month. Pin both together.
• Tight bounds. Unsloth caps transformers, trl and torch, and excludes specific releases. You do not upgrade TRL on its own.
• Studio is single-node. One host, SQLite, worker subprocesses, and tools that run as the OS user. No documented high availability or horizontal scale.
• Security defaults. Loopback bind, admin password on first launch, tools on for loopback and off for other binds, --secure fails closed.
• Multi-user accounts arrived on 17 September 2026. They isolate work between accounts but are not a hardened tenancy boundary.
• Serving. The docs direct enterprise and multi-user inference to vLLM.
• Updating Studio. Re-run the install script. Do not use unsloth studio update.
• Defaults that moved. GRPO loss_type is "dapo", KL beta is 0.0, Studio's default GGUF quantization is Q4_K_M.

System-design prompts

For each prompt I give the clarifying questions, a structure for the answer, and what a strong candidate adds. Do not skip the clarifying questions in a real interview. Asking them is part of the answer.

Prompt 1: Design a fine-tuning platform for ten product teams

Clarify. How many runs per week, and how large are the models? Is training data sensitive, and where may it be processed? Do teams need interactive experimentation, or only scheduled retraining? What is the serving target? Who pays for GPUs?

Structure. I would answer in layers.

1. Execution. Batch jobs on a scheduler, such as Kubernetes with GPU quotas per namespace. Each job runs a pinned image with Unsloth Core and the unsloth train command driven by a validated config. The CLI schema rejects unknown keys, so misconfiguration fails fast, and --dry-run validates before spending GPU time.
2. Environment. A base image with unsloth and unsloth_zoo pinned together and a lock file for everything else, UNSLOTH_DISABLE_AUTO_UPDATES=1 set, scanned and signed. A quarterly or monthly upgrade cadence with a regression suite.
3. Data and artifacts. Datasets referenced by immutable revision. Adapters and exports written to a per-team object-storage prefix with retention rules. Lineage recorded in an experiment tracker.
4. Quality. A mandatory held-out evaluation against the base model before promotion, and an export contract.
5. Serving. Export merged 16-bit, serve with vLLM behind a gateway with its own authentication, with adapters hot-swapped where appropriate.
6. Interactive. An optional per-team Studio, on loopback or behind your authentication, tools disabled unless requested.

What a strong answer adds. The scaling statement: Studio does not scale out, so interactive use is bounded per team, and the platform's scale comes from batch jobs. The cost levers: QLoRA and smaller GPUs, scale-to-zero node pools, checkpoint retention. The trade-off against a managed service, framed as GPU hours saved versus maintenance load and data control. A mention of where residency rules apply for Gulf and Egyptian employers, since the answer to "where does training run" can be a legal constraint rather than a technical one.

What a weak answer does. Proposes one big shared Studio, forgets the export contract, or leaves versions floating.

Prompt 2: A team wants to expose Studio to the whole company

Clarify. What do they want to do on it: chat with models, train, or call the API? Who are the users? What is on the host?

Reasoning. Start with the trust model, because it decides everything. Studio can run Python and a terminal through server-side tools, and those run as the OS user that started Studio. The docs say that anyone who can reach the server with the API key can run code on the machine. So the question is not "can we put it on the network", but "what is the blast radius if one account or key is compromised".

My answer is conditional and ordered:

1. If the need is inference, do not use Studio. Serve the model with vLLM or llama.cpp behind a gateway with proper authentication and rate limits.
2. If the need is a shared exploration environment for a trusted team, run Studio on a dedicated disposable host with no credentials beyond the job, tools disabled with --disable-tools, the bootstrap password changed, accounts created through the multi-user feature, and access through your identity-aware proxy or an SSH tunnel.
3. Never -H 0.0.0.0 with tools enabled. Prefer --secure over --no-secure if a public link is truly required, and treat the random tunnel URL as a secret.

What a strong answer adds. Details that show real use: tools default off for non-loopback binds, with --enable-tools on 0.0.0.0 prompting for confirmation; UNSLOTH_STUDIO_TRUST_FORWARDED=1 behind your reverse proxy so that rate limiting sees client addresses; the public-reachability probe to ifconfig.me and check-host.net and the variable that disables it; and the fact that the OS-level tool sandbox is a preview that leaves network access unrestricted.

Prompt 3: Design the upgrade process for a library that releases every other day

Clarify. What breaks if training changes subtly? What is the rollback story? Who consumes the artifacts?

Structure. Separate "available" from "approved". Pin exactly. Run a candidate release through a regression suite on a schedule rather than on each release: a short deterministic training run on a tiny model compared to a stored loss curve, a merge and export round trip, an exact-string chat template check including EOS and BOS, and an exported-model load in the real serving runtime with a golden prompt. Promote on pass, keep the previous image for rollback, and tag each adapter with the versions that produced it.

What a strong answer adds. Why the dependency bounds make independent upgrades dangerous. The self-updating behaviour of unsloth_zoo and why to disable it in images. That the Studio, Desktop, Docker and Core channels each upgrade differently, including the instruction not to use unsloth studio update. Reading the changelog for changed defaults, naming a real one such as the GRPO default loss type or the default GGUF quantization change.

Prompt 4: Design a pipeline that retrains a customer-support model weekly

Clarify. What is the data source and does it contain personal data? What defines a better model? How fast must a bad model be rolled back?

Structure. Data intake with filtering and a recorded dataset revision. A scheduled job running unsloth train from a versioned config with a fixed seed. A held-out evaluation set that the training process never sees, a comparison to the currently deployed model on identical prompts, and a promotion rule written down before the first run. An export step that records the template and produces a golden prompt. Canary deployment on the serving side with a quick rollback to the previous artifact. Alerting on training failures by cause.

What a strong answer adds. An honest statement that loss is not a quality metric. A guard against the all-labels-masked failure, checking that some labels differ from -100 in the first batch. Retention of the previous artifact. Data governance: personal data rules, and where training and storage occur given regional requirements.

Prompt 5: Fit a 70B fine-tune on the hardware we have

Clarify. What hardware? What task, and is a smaller model acceptable? How much context?

Reasoning. Use the documented minimums as a starting point: roughly 41 GB for QLoRA on 70B versus 164 GB for 16-bit LoRA. So QLoRA with load_in_4bit=True is the first move, with use_gradient_checkpointing="unsloth" and a modest batch size raised through gradient accumulation. If one card is not enough, device_map="balanced" splits across GPUs, and DDP applies when each GPU can hold the model. Context length is the next lever, because activations grow with it, and Unsloth's checkpointing offloads activations to system RAM.

What a strong answer adds. That the docs' figures are minimums, so plan headroom; that saving to GGUF or merged 16-bit has its own memory peak, handled with maximum_memory_usage; and the question whether a smaller model with better data would meet the bar at a tenth of the cost.

Incident-response questions

Answer these in a fixed order: contain, diagnose, fix, prevent. State what you would check first and why.

"Our fine-tuned model is great in the notebook and produces garbage in production"

Containment: stop promoting that artifact and roll back to the previous known-good one. Diagnosis: the most common cause is a chat template, EOS or BOS mismatch between training and the serving runtime. Render the same conversation in both places and diff the strings. Check for a doubled BOS token and for stop tokens. Then check that the export carried the tokenizer and template files. Fix: align the template, re-export if needed. Prevention: the export contract with a golden prompt run in the serving runtime as an acceptance test.

A strong answer notes that this is the number one documented cause, so it is the correct first hypothesis, but also names what would make you abandon it: if template strings match exactly, move on to quantization damage and a version mismatch warning at load.

"Training loss is exactly zero from step one"

Diagnosis: the message All labels in your dataset are -100. Training losses will be all 0. means train_on_responses_only masked everything, because the instruction_part and response_part strings do not match the model's template. Fix: use the exact marker strings for that model family. Prevention: a pipeline assertion on the first batch. Do not respond by removing the masking, which would hide the problem and change what the model learns.

"A job crashed with a CUDA device-side assert"

Diagnosis: this is often a compiler or fast-generation interaction. Set UNSLOTH_COMPILE_DISABLE=1 and UNSLOTH_DISABLE_FAST_GENERATION=1 before importing Unsloth, restart, and see whether it reproduces. If it does not, you have found the layer; file a bug with the version information and the minimal reproduction. A strong answer says why this matters: the same flags separate an Unsloth patch problem from a data or configuration problem on any suspicious fine-tune.

"We think someone reached Studio from outside the company"

Containment first: unsloth studio stop, isolate the host from the network. Then treat it as a compromised machine, because tools run as the OS user: rotate every credential that user could read, including Hugging Face and Weights and Biases tokens, revoke Unsloth API keys, reset the admin password, and preserve logs and studio.db. Rebuild from the image rather than cleaning. Then find the exposure: was it -H 0.0.0.0, a published Docker port, a tunnel URL that leaked? Prevention: loopback default, tools disabled on non-loopback, --secure, an SSH tunnel for Docker, and a configuration check that fails the build if 0.0.0.0 appears without --disable-tools.

A strong answer insists on rotating before cleaning up. A weak one restarts the service and moves on.

"After a routine upgrade, training behaves differently"

Compare with the changelog first. Look for changed defaults: the GRPO loss type, evaluation precision, the gradient accumulation fix, and MoE kernel defaults. Then diff the resolved dependency set against the last good image. Check whether unsloth_zoo updated itself. Roll back to the previous image to confirm the upgrade is the cause. This question tests whether you reach for the lock file and the stored loss curve, and the habit of pinning and recording environments.

Arguing trade-offs to leadership

Senior interviews often end with a role-play: you have five minutes with a director who is not technical. The skill is turning an engineering choice into a decision.

"Why not just use a managed fine-tuning service?"

State the question you are really answering: what do we get, and what do we give up? A managed service buys speed to first result, no GPU operations and vendor support. It costs per-run pricing at scale, limits on model choice and training methods, and data leaving your boundary. Unsloth costs engineer time and GPU capacity, and buys control of the data location, support for a very wide set of open models, the ability to do reinforcement learning and custom training, and a lower GPU bill by the project's own claims of faster training and lower memory. Then give a recommendation with a trigger: start managed if the volume is low and the data is not sensitive, and move to the platform when volume or residency makes the break-even obvious. Quantify with one number from your own workloads, not the vendor's claims.

"Is it risky to depend on a project that ships daily and is in Beta?"

Separate the parts. The core library is mature and widely used in production; the Studio UI is the Beta part. The risk is real, and it is manageable by design: pin versions, validate releases in a regression suite, keep rollback available, and avoid building business processes on Studio's HTTP routes. Name the residual risks honestly: a fast-moving dependency can break, and a small number of maintainers is a concentration risk. Name the mitigation for each, and the exit path: adapters and merged exports are in standard Hugging Face and GGUF formats, so switching the training tool does not strand the models.

"Can we let every team use Studio?"

Say yes to the workbench, no to the shared host. Interactive Studio per team, on isolated machines, with tools off unless a team accepts the risk. Production training as batch jobs. Explain the reason in one sentence leadership remembers: Studio's tools run code as the user who started it, so access to it is access to that machine.

"Why is the GPU bill higher than expected?"

Show the four lines: GPU hours, idle capacity, storage and egress, engineering time. Idle capacity is usually the largest surprise, so propose scale-to-zero job pools and scheduled shutdown for interactive hosts, with cost labels per team so the bill is attributable. Present the saving as a policy decision with a number, such as retaining checkpoints for seven days instead of forever.

What a strong answer covers that a weak one misses

Sixty-second self-test

Answer aloud without notes, and check against the guide.

1. Name the three products and their licences.
2. Which two packages do you pin together, and what can unsloth_zoo do on its own?
3. What are the tool defaults for a loopback bind and for 0.0.0.0?
4. What does --secure do when the tunnel fails, and why is the suggested fix a trap?
5. Why are multi-user accounts not a tenancy boundary?
6. What is the first hypothesis for "good in the notebook, garbage in production"?
7. What does All labels in your dataset are -100 mean?
8. Which flag reduces memory at export time, and what is its default?
9. Where does the project tell you to serve multi-user traffic?
10. What four lines make up the cost of the platform?
11. What is one default that changed in 2026 that could alter a job without any code change?
12. Name two situations where you would pick a different tool.

If you hesitated on more than three, reread the trust model, the incident playbooks and the section on where Unsloth stops in the senior detailed guide. For the neighbouring tools mentioned here, see the vLLM guide, the PEFT and TRL guide and the Docker guide.

Part three of three. At this level the problems are operational: a Studio port someone opened for a demo, an adapter nobody can reproduce, a release that quietly changed a default. Start with the error table, then the habits, then the review checklists you can paste into pull requests and design documents.

Production traps that appear at scale

Everything from the Beginner and Mid tips still applies. These cause incidents rather than failed notebooks, and most of them are invisible until a second team or a second month is involved.

Three ideas explain most rows. The first is that Unsloth moves faster than your change process, so anything not pinned will change. The second is that Studio is a code-execution surface, so reachability is the security decision. The third is that the hand-off between training and serving is where quality goes missing, so make it a contract.

Platform-ownership habits, in order of leverage

These are the practices that pay for themselves, ordered by how much trouble each prevents.

1. Pin as a set, and record the set

Pin unsloth and unsloth_zoo together. Freeze everything else in a lock file. Build one image and refer to it by digest. Save the freeze output, the digest, the config and the dataset revision next to each adapter you publish. This is the habit that turns "why does it behave differently" from archaeology into a diff.

2. Make the safe exposure the default one

Wrap Studio in a launcher script or a container entrypoint that you own, so that users do not type flags. The wrapper binds loopback unless told otherwise, passes --disable-tools for any other bind, reads the password from standard input with --password -, and refuses to start with the bootstrap password on a non-loopback address. Platform safety comes from defaults, not from reminders.

3. Run batch training through the validated config

unsloth train takes a YAML or JSON config and rejects unknown keys. Use that. Set lora_r and lora_alpha explicitly because the CLI defaults (64 and 16) differ from notebook defaults (16 and 16). Run --dry-run in review. Do not assume a Studio-exported YAML works as an unsloth train config, since the key names differ in places.

4. Gate promotion on a held-out evaluation

Shuffle and split with a fixed seed, evaluate against the base model on identical prompts, and write the promotion rule before training. Loss near zero suggests overfitting, but a healthy-looking loss says nothing about the task. Early stopping on evaluation loss is a convenience, not a gate.

5. Ship an export contract

Every artifact carries its tokenizer, the chat template name, the stop tokens, the responses-only marker strings if used, the sampling settings, an explicit quantization name, and a golden prompt with its expected output. The serving team runs the golden prompt in their own runtime before accepting it. This one habit removes the most frequent incident in the whole domain.

6. Rehearse the upgrade

Keep a small regression suite: a short deterministic training run compared with a stored loss curve within a tolerance, an exact-string template check, a merge and export round trip, and a model load in the serving runtime. Run each candidate release through it, keep the previous image, and read the changelog for changed defaults. Name at least the ones you know: the GRPO loss_type, the KL beta default, evaluation precision, and the GGUF default.

7. Separate training, interactive use and serving

Training is batch and containerised. Interactive Studio is per team and disposable. Serving is vLLM or similar behind a gateway. When these three are blurred, every incident gets bigger. See the vLLM guide for the serving side, and the Docker guide for hardening the container.

8. Attribute cost and set retention

Label every job with team, project and run identifier. Keep final adapters and evaluation records long term, and keep intermediate checkpoints for a short, stated period. Right-size GPUs from the documented VRAM table plus headroom, and prefer scale-to-zero pools for jobs.

9. Practise the exposure incident

Once a quarter, run a tabletop exercise: Studio with tools was reachable, what do you do? The right order is stop, isolate, rotate every credential the OS user could read, revoke API keys, preserve studio.db and logs, rebuild from the image. Rotate before cleaning, because cleaning does not un-leak a token.

10. Write down where Unsloth stops

Maintain a short decision note on when a team should move to vLLM for serving, to a multi-node framework such as DeepSpeed or Axolotl for large jobs, or to a managed service. A tool you cannot say no to is a tool you do not understand.

Practice cards

Operational recipes

Short, copyable procedures. The commands are the ones documented for the version covered by this guide; confirm flags with --help on your installed release.

Verify an install before a job starts

unsloth studio verify-install --json
echo "exit code: $?"

Exit code 0 means the install is complete, 1 means it is not. Use it as a readiness check in provisioning scripts.

Validate a training config without spending GPU time

unsloth train --config train.yaml --dry-run

An unknown key is an error, because the schema forbids extra fields. A passing dry run is a reasonable pull request requirement.

Start Studio safely on a server

printf '%s' "$UNSLOTH_STUDIO_PASSWORD" | unsloth studio run --disable-tools --password -

Reading the password from standard input keeps it out of ps and shell history. Flags go after the subcommand; unsloth studio --secure run exits with code 2. Check unsloth studio run --help on your version before relying on a flag combination.

Reach Docker Studio without publishing it

ssh -L 8000:localhost:8000 -L 8888:localhost:8888 user@server

Studio listens on 8000 and JupyterLab on 8888 inside the container, both plain HTTP. Bind published ports to 127.0.0.1 on the host and use the tunnel from your laptop.

Make a reproducible environment

export UNSLOTH_DISABLE_AUTO_UPDATES=1
uv pip install unsloth==2026.9.12 unsloth_zoo==2026.9.8 --torch-backend=auto
pip freeze > environment-lock.txt

Commit the freeze output with the artifact. Update the two version numbers only through your upgrade process.

Reduce memory during export

model.save_pretrained_merged(
    "out", tokenizer, save_method="merged_16bit", maximum_memory_usage=0.5
)

The default is 0.75. Lower it when merging or writing GGUF crashes the GPU.

Separate an Unsloth problem from a data problem

export UNSLOTH_COMPILE_DISABLE=1
export UNSLOTH_DISABLE_FAST_GENERATION=1
python train.py

If the problem disappears, the compiler or fast generation is involved and you have a bug report. If it persists, look at data, template and configuration. Set these before the import, in a fresh process.

Rotate after a suspected leak

1. Stop Studio: unsloth studio stop.
2. Revoke Unsloth API keys in Settings and reset the admin password.
3. Rotate HF_TOKEN, WANDB_API_KEY and any cloud credentials the OS user could read.
4. Preserve studio.db and logs, then rebuild the host from the image.

Review checklist for pull requests

Paste this into a pull request template for the training repository.

• unsloth and unsloth_zoo are pinned exactly, in the same change, and the lock file is updated.
• The image is referenced by digest, and the Python version is inside 3.11 to 3.13.
• from unsloth import ... precedes transformers, trl and peft in every entry point.
• The config sets lora_r, lora_alpha, learning rate, seed and gradient checkpointing explicitly and passes --dry-run.
• Only one of load_in_4bit, load_in_8bit, load_in_16bit and full_finetuning is true.
• trust_remote_code is false, or a reviewer approved the specific repository and revision.
• No token, password or sk-unsloth- key appears in code, config, compose files or logs.
• --password VALUE is not used; standard input or a secret store is.
• Any non-loopback Studio bind passes --disable-tools.
• Tool defaults were not overridden with --enable-tools and -y in a script.
• train_on_responses_only marker strings match the model family, and a test asserts some labels are not -100.
• Evaluation uses a held-out, shuffled split and compares against the base model.
• The export names its quantization explicitly and carries the tokenizer and template.
• Checkpoint frequency and retention are stated.
• Logging to Weights and Biases or TensorBoard is on, and the run is labelled with team and project.

Review checklist for design documents

Use these questions to review a proposal for a new training pipeline, a Studio rollout or an upgrade plan.

Scope and fit. Does the document say why Unsloth is the right tool for this step, and what it will not do? Does it name the serving runtime and the hand-off format? Does it identify when the design would move to another tool, such as vLLM or a multi-node framework?

Environment. Is there one pinned image, one lock file and one recorded set of versions? Is the upgrade cadence stated, with a regression suite and rollback? Does the plan say how unsloth_zoo self-updates are prevented?

Security. Is the trust model for Studio written down: bind address, tools, authentication, network path? Is the exposure mode justified, and is --secure used rather than --no-secure? Where do secrets come from, who can read them, and how are they rotated? Is the supply chain addressed: scanned images, pinned model revisions, no untrusted pickles, careful use of trust_remote_code? Is the Studio AGPL licence considered if it is embedded or offered as a service?

Tenancy. What does each tenant own: namespace, GPU quota, artifact prefix, registry namespace, cost label? Is any claim of isolation backed by something stronger than Studio accounts?

Capacity and cost. Are GPU sizes derived from the documented VRAM table with headroom? Is there a plan for idle capacity, checkpoint retention and egress? Are costs attributable by team? Where regional rules apply to the training data, does the plan say where training and storage happen?

Quality and lineage. Is there a held-out evaluation, a comparison to the base model and a written promotion rule? Does each artifact record base model revision, dataset revision, config, versions, image digest, seed, owner and evaluation result? Is there an export contract and a golden prompt?

Operations. Are there runbooks for the symptoms in the table above, with exact message text? Who is on call, and what do they check first? Is there a tabletop exercise for exposure? Are queue wait time, GPU utilisation, failure rate by cause and image age measured, perhaps through Prometheus?

Exit. If the project changed direction or a release broke, how quickly could you switch? Are adapters and exports in standard formats that other tools can load?

Habits that keep a platform calm

A few smaller routines make the difference between a platform people trust and one they work around.

Read the changelog every time you read the release list. The most dangerous changes are defaults, not features, and they are written in the same place as the features. Keep a running page of the defaults your jobs depend on, and update it at each upgrade.

Make failures cheap and specific. A job that fails in thirty seconds with a clear validation error costs nothing; the same mistake discovered after six GPU hours costs a lot. The --dry-run flag, the first-batch label assertion and the golden prompt are all ways of moving discovery earlier.

Write incidents down in the vocabulary of the tool. "The model was bad" teaches nothing. "The EOS token differed between training and vLLM, and the template was not in the export" teaches a future engineer what to check. Include the exact error text so it can be searched.

Keep an honest support matrix. Say which Unsloth version, Python version, GPU families and model families you have verified, and say plainly that Studio is Beta. People forgive limits that are stated and punish limits that are discovered.

Finally, remember the neighbours. Unsloth sits on top of Hugging Face libraries, so the PEFT and TRL guide explains what is being patched. Experiment tracking belongs in Weights and Biases or MLflow. Secrets belong in a store such as Vault. A platform owner's skill is mostly knowing which of these to reach for, and when to stop adding to the Unsloth part and reach for another tool.

A closing checklist

Before you call your Unsloth platform production-ready, you should be able to say yes to each line.

• Versions are pinned as a set, locked and recorded with every artifact.
• Studio is on loopback or behind authentication, with tools off anywhere else.
• Secrets are injected at runtime, scoped, and rotated by a written procedure.
• Training runs as containerised batch jobs from validated configs.
• Promotion is gated by a held-out evaluation against the base model.
• Every artifact ships with an export contract and passes a golden prompt in the serving runtime.
• Serving for many users is on a runtime built for it.
• Costs are labelled, idle capacity is controlled and retention is defined.
• Upgrades follow a cadence, a regression suite and a rollback.
• The incident runbooks exist, and someone has rehearsed the exposure one.
• A written note says where Unsloth stops, and what you would use instead.