{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# S9 lab notebook — fine-tune the caption model on a GPU (Kaggle T4 or RTX Pro 6000)\n",
    "\n",
    "Fine-tune a **4-bit QLoRA adapter** on 800 synthetic catalogue rows and compare the base and tuned captions with token-F1.\n",
    "\n",
    "- **Kaggle T4 (free):** one 15.6 GB card, about 18 minutes measured (855.4 s training + ~2 min load/eval, 5.78 GB peak VRAM).\n",
    "- **Faculty RTX Pro 6000 (one card per student):** 96 GB VRAM, about 4.5 minutes measured (219.8 s training, 5.76 GB peak VRAM).\n",
    "\n",
    "Run the smoke cell first (`ITERS=10`, `EVAL_ROWS=5`), then the full run (`ITERS=100`, `EVAL_ROWS=20`). Both timings are measured; record your own and compare.\n",
    "\n",
    "**You submit:** `results/cuda_results.json`, the base-vs-tuned table, three sample captions and a three-sentence *what this proves* note.\n",
    "\n",
    "<details class=\"th\" lang=\"th\">\n",
    "<summary>สรุปภาษาไทย</summary>\n",
    "\n",
    "โน้ตบุ๊กนี้ฝึก QLoRA 4 บิตบนข้อมูลแคตตาล็อกสังเคราะห์ 800 แถว และเปรียบเทียบคำบรรยายของโมเดลฐานกับโมเดลที่ปรับด้วย token-F1 เริ่มจากรอบทดสอบเล็ก ๆ ก่อน แล้วจึงรันเต็ม บน Kaggle T4 ใช้เวลาประมาณ 18 นาที (วัดจริง 855.4 วินาทีสำหรับการฝึก) และบน RTX Pro 6000 ประมาณ 4.5 นาที บันทึกเวลาหน่วยความจำและ F1 ของคุณเอง\n",
    "</details>\n",
    "\n",
    "<details class=\"zh\" lang=\"zh-Hans\"><summary>中文总结</summary>\n",
    "\n",
    "这个 notebook 在 800 行的合成目录数据上训练 4 位 QLoRA，并用 token-F1 比较基础模型与微调模型的字幕。先从小规模试运行开始，然后再完整运行。在 Kaggle T4 上大约需要 18 分钟（训练实测 855.4 秒），在 RTX Pro 6000 上大约 4.5 分钟。记录你自己的时间、内存和 F1。\n",
    "\n",
    "</details>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Install the training stack (the one exception to the no-pip standard)\n",
    "\n",
    "Kaggle's image does not ship `peft` or `bitsandbytes`, and may need a newer `transformers` for Qwen3.5. This is the single bounded install cell allowed in S9; on the faculty RTX machine the stack may already be present. **Restart the kernel after this cell**, then run the rest.\n",
    "\n",
    "<details class=\"th\" lang=\"th\">\n",
    "<summary>สรุปภาษาไทย</summary>\n",
    "\n",
    "อิมเมจของ Kaggle ไม่มี peft หรือ bitsandbytes และอาจต้องใช้ transformers ใหม่สำหรับ Qwen3.5 นี่เป็นเซลล์ติดตั้งเดียวที่อนุญาตในสัปดาห์ที่ 9 รีสตาร์ตเคอร์เนลหลังรันเซลล์นี้\n",
    "</details>\n",
    "\n",
    "<details class=\"zh\" lang=\"zh-Hans\"><summary>中文总结</summary>\n",
    "\n",
    "Kaggle 镜像没有 peft 或 bitsandbytes，并且可能需要适用于 Qwen3.5 的新版 transformers。这是第 9 周唯一允许的安装单元格。运行此单元格后重启内核。\n",
    "\n",
    "</details>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%pip install -q \"transformers>=5.2,<6\" \"peft>=0.18\" \"bitsandbytes>=0.48\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import json\n",
    "import os\n",
    "import random\n",
    "import re\n",
    "import sys\n",
    "import time\n",
    "from collections import Counter\n",
    "from pathlib import Path\n",
    "\n",
    "import torch\n",
    "from peft import LoraConfig, get_peft_model, prepare_model_for_kbit_training\n",
    "from transformers import (AutoModelForImageTextToText, AutoTokenizer, BitsAndBytesConfig,\n",
    "                          get_cosine_schedule_with_warmup)\n",
    "\n",
    "SEED = 42\n",
    "random.seed(SEED)\n",
    "torch.manual_seed(SEED)\n",
    "print(\"seed\", SEED)\n",
    "print(\"Python\", sys.version.split()[0])\n",
    "\n",
    "def data_dir():\n",
    "    # Local folder, explicit COURSE_DATA, the classic Kaggle mount, then whatever\n",
    "    # layout Kaggle actually used (it now nests attached data under /kaggle/input/datasets).\n",
    "    candidates = [Path(\"data\"), Path(\"/kaggle/input/llm-inference-finetuning\")]\n",
    "    env = os.environ.get(\"COURSE_DATA\")\n",
    "    if env:\n",
    "        candidates.append(Path(env))\n",
    "    kaggle = Path(\"/kaggle/input\")\n",
    "    if kaggle.exists():\n",
    "        candidates.extend(p.parent for p in sorted(kaggle.rglob(\"caption_catalogue.jsonl\")))\n",
    "    for path in candidates:\n",
    "        if path and (path / \"caption_catalogue.jsonl\").exists():\n",
    "            return path\n",
    "    raise FileNotFoundError(\n",
    "        \"put caption_catalogue.jsonl in data/, attach it as a Kaggle dataset, or set COURSE_DATA\")\n",
    "\n",
    "DATA = data_dir() / \"caption_catalogue.jsonl\"\n",
    "OUT = Path(\"results\")\n",
    "OUT.mkdir(exist_ok=True)\n",
    "MODEL_ID = \"Qwen/Qwen3.5-4B\"\n",
    "ITERS = int(os.environ.get(\"S9_ITERS\", \"100\"))\n",
    "EVAL_ROWS = int(os.environ.get(\"S9_EVAL_ROWS\", \"20\"))\n",
    "MAX_SEQ_LEN = 512\n",
    "HOURLY_COST = float(os.environ.get(\"S9_HOURLY_COST\", \"0.0\"))\n",
    "\n",
    "assert torch.cuda.is_available(), \"No CUDA GPU. Use notebook.ipynb (MLX) on a Mac.\"\n",
    "print(torch.cuda.get_device_name(0),\n",
    "      round(torch.cuda.get_device_properties(0).total_memory / 1e9, 1), \"GB\")\n",
    "print(\"data\", DATA, \"iters\", ITERS, \"eval rows\", EVAL_ROWS)\n",
    "assert DATA.exists()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Load the catalogue\n",
    "\n",
    "The catalogue is bundled: 1,000 rows, 800 train and 200 validation, generated with seed 42. No network fetch is required.\n",
    "\n",
    "<details class=\"th\" lang=\"th\">\n",
    "<summary>สรุปภาษาไทย</summary>\n",
    "\n",
    "แคตตาล็อกมาพร้อมโน้ตบุ๊ก: 1,000 แถว แบ่งเป็นฝึก 800 และตรวจสอบ 200 สร้างด้วย seed 42 ไม่ต้องดาวน์โหลดจากเครือข่าย\n",
    "</details>\n",
    "\n",
    "<details class=\"zh\" lang=\"zh-Hans\"><summary>中文总结</summary>\n",
    "\n",
    "目录随 notebook 一起提供：1,000 行，分为 800 行训练和 200 行验证，用 seed 42 生成，无需从网络下载。\n",
    "\n",
    "</details>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "SYSTEM = (\"You write one-sentence captions for a campus media unit's catalogue. \"\n",
    "          \"Use the metadata and keep the unit's concise English voice. \"\n",
    "          \"Answer with the caption only; do not show reasoning.\")\n",
    "\n",
    "def prompt_messages(row):\n",
    "    return [\n",
    "        {\"role\": \"system\", \"content\": SYSTEM},\n",
    "        {\"role\": \"user\", \"content\": \"Metadata:\\n\" + json.dumps(row[\"metadata\"], indent=2) +\n",
    "                                    \"\\n\\nWrite the caption.\"},\n",
    "    ]\n",
    "\n",
    "rows = [json.loads(line) for line in DATA.open(encoding=\"utf-8\") if line.strip()]\n",
    "assert len(rows) == 1000\n",
    "train_rows = [r for r in rows if r[\"split\"] == \"train\"][: ITERS * 8]\n",
    "val_rows = [r for r in rows if r[\"split\"] == \"validation\"][:EVAL_ROWS]\n",
    "assert len(train_rows) == ITERS * 8\n",
    "assert len(val_rows) == EVAL_ROWS\n",
    "print(len(train_rows), \"train rows;\", len(val_rows), \"validation rows\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Load the 4-bit model and evaluate the base\n",
    "\n",
    "The base is loaded in 4-bit NF4; the vision tower stays frozen and the text path is the only one used. Token-F1 is the teaching metric.\n",
    "\n",
    "<details class=\"th\" lang=\"th\">\n",
    "<summary>สรุปภาษาไทย</summary>\n",
    "\n",
    "โมเดลฐานโหลดแบบ 4 บิต NF4 ส่วน vision tower ถูกแช่แข็งและใช้เฉพาะเส้นทางข้อความ token-F1 เป็นตัวชี้วัดเพื่อการสอน\n",
    "</details>\n",
    "\n",
    "<details class=\"zh\" lang=\"zh-Hans\"><summary>中文总结</summary>\n",
    "\n",
    "基础模型以 4 位 NF4 加载。vision tower 被冻结，并且只使用文本路径。token-F1 是用于教学的指标。\n",
    "\n",
    "</details>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tok = AutoTokenizer.from_pretrained(MODEL_ID)\n",
    "if tok.pad_token is None:\n",
    "    tok.pad_token = tok.eos_token\n",
    "\n",
    "def chat_prompt(messages):\n",
    "    try:\n",
    "        return tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True,\n",
    "                                       enable_thinking=False)\n",
    "    except TypeError:\n",
    "        return tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)\n",
    "\n",
    "quant = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type=\"nf4\",\n",
    "                           bnb_4bit_use_double_quant=True,\n",
    "                           bnb_4bit_compute_dtype=torch.float16)\n",
    "t0 = time.time()\n",
    "model = AutoModelForImageTextToText.from_pretrained(\n",
    "    MODEL_ID, quantization_config=quant, device_map={\"\": 0}, dtype=torch.float16)\n",
    "model.eval()\n",
    "load_s = round(time.time() - t0, 1)\n",
    "print(f\"loaded in {load_s} s\")\n",
    "assert load_s > 0\n",
    "\n",
    "def generate_caption(row):\n",
    "    inputs = tok(chat_prompt(prompt_messages(row)), return_tensors=\"pt\").to(model.device)\n",
    "    with torch.no_grad():\n",
    "        out = model.generate(**inputs, max_new_tokens=48, do_sample=False,\n",
    "                             pad_token_id=tok.pad_token_id)\n",
    "    text = tok.decode(out[0][inputs[\"input_ids\"].shape[1]:], skip_special_tokens=True)\n",
    "    return re.sub(r\"<think>.*?</think>| thinking.*?</think>\", \"\", text, flags=re.S).strip()\n",
    "\n",
    "def tokens(text):\n",
    "    return re.findall(r\"[a-z0-9]+\", text.lower())\n",
    "\n",
    "def token_f1(pred, ref):\n",
    "    p, r = Counter(tokens(pred)), Counter(tokens(ref))\n",
    "    overlap = sum((p & r).values())\n",
    "    if not overlap:\n",
    "        return 0.0\n",
    "    precision = overlap / max(1, sum(p.values()))\n",
    "    recall = overlap / max(1, sum(r.values()))\n",
    "    return 2 * precision * recall / (precision + recall)\n",
    "\n",
    "def evaluate(label):\n",
    "    t = time.time()\n",
    "    scores, samples = [], []\n",
    "    for row in val_rows:\n",
    "        pred = generate_caption(row)\n",
    "        scores.append(token_f1(pred, row[\"caption\"]))\n",
    "        if len(samples) < 3:\n",
    "            samples.append({\"reference\": row[\"caption\"], \"prediction\": pred,\n",
    "                            \"f1\": round(scores[-1], 3)})\n",
    "    result = {\"f1\": round(sum(scores) / max(1, len(scores)), 4),\n",
    "              \"seconds\": round(time.time() - t, 1), \"samples\": samples}\n",
    "    print(label, \"f1\", result[\"f1\"])\n",
    "    assert 0.0 <= result[\"f1\"] <= 1.0\n",
    "    return result\n",
    "\n",
    "base = evaluate(\"base\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Attach the LoRA adapter to the text stack\n",
    "\n",
    "The adapter targets the language model's full-attention, linear-attention and MLP linear layers. The vision tower and the MTP head stay frozen.\n",
    "\n",
    "<details class=\"th\" lang=\"th\">\n",
    "<summary>สรุปภาษาไทย</summary>\n",
    "\n",
    "อะแดปเตอร์กำหนดเป้าหมายที่ชั้น full-attention, linear-attention และ MLP ของโมเดลภาษา ส่วน vision tower และ MTP head ถูกแช่แข็ง\n",
    "</details>\n",
    "\n",
    "<details class=\"zh\" lang=\"zh-Hans\"><summary>中文总结</summary>\n",
    "\n",
    "适配器作用于语言模型的 full-attention、linear-attention 和 MLP 层；vision tower 和 MTP head 则保持冻结。\n",
    "\n",
    "</details>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = prepare_model_for_kbit_training(model)\n",
    "model.config.use_cache = False\n",
    "lora = LoraConfig(\n",
    "    r=16, lora_alpha=32, lora_dropout=0.05, bias=\"none\", task_type=\"CAUSAL_LM\",\n",
    "    target_modules=(r\".*language_model\\.layers\\.\\d+\\.(?:\"\n",
    "                    r\"self_attn\\.(?:q_proj|k_proj|v_proj|o_proj)|\"\n",
    "                    r\"linear_attn\\.(?:in_proj_qkv|in_proj_a|in_proj_b|in_proj_z|out_proj)|\"\n",
    "                    r\"mlp\\.(?:gate_proj|up_proj|down_proj))$\"),\n",
    ")\n",
    "try:\n",
    "    model = get_peft_model(model, lora)\n",
    "except Exception as exc:\n",
    "    print(\"CAUSAL_LM task_type failed, retrying without:\", exc)\n",
    "    lora.task_type = None\n",
    "    model = get_peft_model(model, lora)\n",
    "model.print_trainable_parameters()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Train\n",
    "\n",
    "One optimizer step accumulates eight rows; 100 steps is one pass over 800 rows. If the T4 runs long, set `ITERS=50`; the smoke run is `ITERS=10` and `EVAL_ROWS=5`.\n",
    "\n",
    "<details class=\"th\" lang=\"th\">\n",
    "<summary>สรุปภาษาไทย</summary>\n",
    "\n",
    "หนึ่งก้าวของออปติไมเซอร์สะสมแปดแถว และ 100 ก้าวคือหนึ่งรอบของ 800 แถว ถ้า T4 ใช้เวลานานให้ตั้ง ITERS=50 ส่วนรอบทดสอบใช้ ITERS=10 และ EVAL_ROWS=5\n",
    "</details>\n",
    "\n",
    "<details class=\"zh\" lang=\"zh-Hans\"><summary>中文总结</summary>\n",
    "\n",
    "优化器的一步累积八行，100 步就是 800 行的一轮（epoch）。如果 T4 耗时过长，就设置 ITERS=50；试运行则使用 ITERS=10 和 EVAL_ROWS=5。\n",
    "\n",
    "</details>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def encode(row):\n",
    "    prompt = chat_prompt(prompt_messages(row))\n",
    "    p_ids = tok(prompt, add_special_tokens=False)[\"input_ids\"]\n",
    "    t_ids = tok(row[\"caption\"] + tok.eos_token, add_special_tokens=False)[\"input_ids\"]\n",
    "    ids = (p_ids + t_ids)[:MAX_SEQ_LEN]\n",
    "    labels = ([-100] * len(p_ids) + t_ids)[:MAX_SEQ_LEN]\n",
    "    return {\"input_ids\": ids, \"labels\": labels}\n",
    "\n",
    "encoded = [encode(r) for r in train_rows]\n",
    "\n",
    "def collate(batch):\n",
    "    maxlen = max(len(x[\"input_ids\"]) for x in batch)\n",
    "    input_ids = torch.full((len(batch), maxlen), tok.pad_token_id, dtype=torch.long)\n",
    "    labels = torch.full((len(batch), maxlen), -100, dtype=torch.long)\n",
    "    attn = torch.zeros((len(batch), maxlen), dtype=torch.long)\n",
    "    for i, x in enumerate(batch):\n",
    "        n = len(x[\"input_ids\"])\n",
    "        input_ids[i, :n] = torch.tensor(x[\"input_ids\"])\n",
    "        labels[i, :n] = torch.tensor(x[\"labels\"])\n",
    "        attn[i, :n] = 1\n",
    "    return {\"input_ids\": input_ids, \"labels\": labels, \"attention_mask\": attn}\n",
    "\n",
    "from torch.utils.data import DataLoader\n",
    "\n",
    "loader = DataLoader(encoded, batch_size=1, shuffle=True, collate_fn=collate)\n",
    "accum = 8\n",
    "params = [p for p in model.parameters() if p.requires_grad]\n",
    "optimizer = torch.optim.AdamW(params, lr=2e-4)\n",
    "scheduler = get_cosine_schedule_with_warmup(optimizer, num_warmup_steps=max(1, int(0.03 * ITERS)),\n",
    "                                            num_training_steps=ITERS)\n",
    "torch.cuda.reset_peak_memory_stats()\n",
    "t = time.time()\n",
    "model.train()\n",
    "running = 0.0\n",
    "micro = 0\n",
    "step = 0\n",
    "done = False\n",
    "while not done:\n",
    "    for batch in loader:\n",
    "        batch = {k: v.to(model.device) for k, v in batch.items()}\n",
    "        loss = model(**batch).loss / accum\n",
    "        loss.backward()\n",
    "        running += loss.item()\n",
    "        micro += 1\n",
    "        if micro % accum == 0:\n",
    "            torch.nn.utils.clip_grad_norm_(params, 1.0)\n",
    "            optimizer.step()\n",
    "            scheduler.step()\n",
    "            optimizer.zero_grad()\n",
    "            step += 1\n",
    "            if step % 10 == 0:\n",
    "                print(f\"step {step}/{ITERS} loss={running / 10:.4f} elapsed={time.time() - t:.0f}s\")\n",
    "                running = 0.0\n",
    "            if step >= ITERS:\n",
    "                done = True\n",
    "                break\n",
    "train_s = round(time.time() - t, 1)\n",
    "peak = round(torch.cuda.max_memory_allocated() / 1e9, 2)\n",
    "model.eval()\n",
    "model.config.use_cache = True\n",
    "assert train_s > 0 and peak > 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6. Evaluate the tuned model and save the result\n",
    "\n",
    "Record the GPU name, time, peak VRAM, base/tuned F1 and the cost line. On the T4 the cost is $0 (free quota); on the faculty RTX it is also $0; a RunPod fallback can set `S9_HOURLY_COST`.\n",
    "\n",
    "<details class=\"th\" lang=\"th\">\n",
    "<summary>สรุปภาษาไทย</summary>\n",
    "\n",
    "บันทึกชื่อ GPU เวลา VRAM สูงสุด F1 ของฐานและโมเดลที่ปรับ และบรรทัดต้นทุน บน T4 และ RTX ของคณะไม่มีค่าใช้จ่าย ส่วน RunPod กำหนด S9_HOURLY_COST ได้\n",
    "</details>\n",
    "\n",
    "<details class=\"zh\" lang=\"zh-Hans\"><summary>中文总结</summary>\n",
    "\n",
    "记录 GPU 名称、时间、峰值 VRAM、基础模型和微调模型的 F1，以及成本行。在 T4 和学院的 RTX 上不产生费用；在 RunPod 上可以设置 S9_HOURLY_COST。\n",
    "\n",
    "</details>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "tuned = evaluate(\"tuned\")\n",
    "result = {\n",
    "    \"runtime\": \"cuda\",\n",
    "    \"gpu\": torch.cuda.get_device_name(0),\n",
    "    \"model\": MODEL_ID,\n",
    "    \"quantization\": \"4bit-nf4\",\n",
    "    \"seed\": SEED,\n",
    "    \"load_seconds\": load_s,\n",
    "    \"train_rows\": len(train_rows),\n",
    "    \"eval_rows\": EVAL_ROWS,\n",
    "    \"iters\": ITERS,\n",
    "    \"lora\": {\"r\": 16, \"alpha\": 32, \"epochs\": 1},\n",
    "    \"train_seconds\": train_s,\n",
    "    \"peak_vram_gb\": peak,\n",
    "    \"base\": base,\n",
    "    \"tuned\": tuned,\n",
    "    \"cost_usd\": round(train_s / 3600 * HOURLY_COST, 3),\n",
    "}\n",
    "(OUT / \"cuda_results.json\").write_text(json.dumps(result, indent=2, default=str))\n",
    "print(json.dumps(result, indent=2, default=str))\n",
    "assert (OUT / \"cuda_results.json\").exists()\n",
    "assert len(tuned[\"samples\"]) >= 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What this proves\n",
    "\n",
    "Write three sentences: what the base model got wrong, what the tuned model changed, and why 4-bit QLoRA fits a 16 GB T4. Attach `results/cuda_results.json` and three sample captions.\n",
    "\n",
    "**T4 timing fallback.** If 100 steps exceed 60 minutes, set `ITERS = 50` and keep `EVAL_ROWS = 20`. On the RTX Pro 6000, keep `ITERS = 100`.\n",
    "\n",
    "## Verification\n",
    "\n",
    "**Faculty RTX Pro 6000 Blackwell (`super1`, Python 3.12, torch 2.15.0+cu132, transformers 5.22.2, peft 0.21.2, bitsandbytes 0.50.2):** verified end to end. Smoke (`S9_ITERS=10`, `S9_EVAL_ROWS=5`): 2.1 s load, 22.2 s train, **5.75 GB peak**, token-F1 **0.5152 → 0.6949**. Full (`S9_ITERS=100`, `S9_EVAL_ROWS=20`): 2.0 s load, **219.8 s train**, **5.76 GB peak**, token-F1 **0.5444 → 0.7746**. **Kaggle T4 (Python 3.13.16, CUDA 13.0.3, one T4 of the T4 ×2 accelerator):** verified end to end. Smoke (`S9_ITERS=10`, `S9_EVAL_ROWS=5`): 58.3 s load, 79.9 s train, **5.77 GB peak**, token-F1 **0.5176 → 0.6949**. Full (`S9_ITERS=100`, `S9_EVAL_ROWS=20`): 58.2 s load, **855.4 s train**, **5.78 GB peak**, token-F1 **0.5449 → 0.7749**. The 100-step run fits the lab block, so the 50-step fallback is not needed.\n",
    "\n",
    "<details class=\"th\" lang=\"th\">\n",
    "<summary>สรุปภาษาไทย</summary>\n",
    "\n",
    "เขียนสามประโยค: โมเดลฐานผิดตรงไหน โมเดลที่ปรับเปลี่ยนอะไร และทำไม QLoRA 4 บิตจึงพอดีกับ T4 16 จิกะไบต์ แนบไฟล์ผลลัพธ์และตัวอย่างสามรายการ เส้นทาง CUDA นี้ยืนยันแล้วบน Kaggle T4 และ RTX Pro 6000\n",
    "</details>\n",
    "\n",
    "<details class=\"zh\" lang=\"zh-Hans\"><summary>中文总结</summary>\n",
    "\n",
    "写三句话：基础模型错在哪里，微调模型改变了什么，以及为什么 4 位 QLoRA 刚好适合 16 GB 的 T4。附上结果文件和三个条目示例。这条 CUDA 路径已在 Kaggle T4 和 RTX Pro 6000 上验证。\n",
    "\n",
    "</details>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
