\n
",
"closed_by": null,
"reactions": {
"url": "https://api.github.com/repos/unslothai/unsloth/issues/3582/reactions",
"total_count": 0,
"+1": 0,
"-1": 0,
"laugh": 0,
"hooray": 0,
"confused": 0,
"heart": 0,
"rocket": 0,
"eyes": 0
},
"performed_via_github_app": null,
"state_reason": null,
"linked_prs": []
},
{
"url": "https://api.github.com/repos/unslothai/unsloth/issues/3581",
"id": 3612204868,
"node_id": "I_kwDOKznBOM7XTd9E",
"number": 3581,
"title": "Error saving GGUF of Gemma27B (but not Gemma4B) on DGX Spark",
"user": {
"login": "weoieoeo",
"id": 31934353,
"node_id": "MDQ6VXNlcjMxOTM0MzUz",
"avatar_url": "https://avatars.githubusercontent.com/u/31934353?v=4",
"gravatar_id": "",
"url": "https://api.github.com/users/weoieoeo",
"type": "User",
"user_view_type": "public",
"site_admin": false
},
"labels": {
"0": {
"id": 8344678723,
"node_id": "LA_kwDOKznBOM8AAAAB8WGxQw",
"url": "https://api.github.com/repos/unslothai/unsloth/labels/bug",
"name": "bug",
"color": "d93f0b",
"default": true,
"description": ""
}
},
"state": "open",
"locked": false,
"assignees": {
"0": {
"login": "rolandtannous",
"id": 115670425,
"node_id": "U_kgDOBuT9mQ",
"avatar_url": "https://avatars.githubusercontent.com/u/115670425?v=4",
"gravatar_id": "",
"url": "https://api.github.com/users/rolandtannous",
"type": "User",
"user_view_type": "public",
"site_admin": false
}
},
"milestone": null,
"comments": 6,
"created_at": "2025-11-11T12:53:33Z",
"updated_at": "2026-02-20T06:47:02Z",
"closed_at": null,
"assignee": {
"login": "rolandtannous",
"id": 115670425,
"node_id": "U_kgDOBuT9mQ",
"avatar_url": "https://avatars.githubusercontent.com/u/115670425?v=4",
"gravatar_id": "",
"url": "https://api.github.com/users/rolandtannous",
"type": "User",
"user_view_type": "public",
"site_admin": false
},
"author_association": "NONE",
"type": null,
"active_lock_reason": null,
"sub_issues_summary": {
"total": 0,
"completed": 0,
"percent_completed": 0
},
"issue_dependencies_summary": {
"blocked_by": 0,
"total_blocked_by": 0,
"blocking": 0,
"total_blocking": 0
},
"body": "After successful vision finetuning of vision model Gemma27B (4bit) I run into this error. The process utilizes only approximately 65 GB of the available 128 GB of unified RAM. This error does not occur when I finetune the smaller Gemma4B (4bit) with the same vision dataset.\n\nI am grateful for any advice\n\n> {'loss': 0.0248, 'grad_norm': 0.3881801664829254, 'learning_rate': 8.695652173913045e-09, 'epoch': 20.0} \n> {'train_runtime': 196532.9404, 'train_samples_per_second': 0.192, 'train_steps_per_second': 0.006, 'train_loss': 0.07668430322393154, 'epoch': 20.0}\n> 100%|███████████████████████████████████████████████████████████████████████████████████████████████████| 1200/1200 [54:35:32<00:00, 163.78s/it]\n> Unsloth: ##### The current model auto adds a BOS token.\n> Unsloth: ##### Your chat template has a BOS token. We shall remove it temporarily.\n> Unsloth: Merging model weights to 16-bit format...\n> Detected local model directory: /workspace/AIEngine/medgemma-27b-it\n> Copied tokenizer.model from local model directory\n> Found HuggingFace hub cache directory: /root/.cache/huggingface/hub\n> Unsloth: Preparing safetensor model files: 0%| | 0/12 [00:00 Unsloth: Preparing safetensor model files: 8%|█████▍ | 1/12 [00:02<00:22, 2.02s/it]Copied model-00006-of-00012.safetensors from local model directory\n> Unsloth: Preparing safetensor model files: 17%|██████████▊ | 2/12 [00:04<00:25, 2.52s/it]Copied model-00012-of-00012.safetensors from local model directory\n> Unsloth: Preparing safetensor model files: 25%|████████████████▎ | 3/12 [00:05<00:13, 1.45s/it]Copied model-00009-of-00012.safetensors from local model directory\n> Unsloth: Preparing safetensor model files: 33%|█████████████████████▋ | 4/12 [00:06<00:12, 1.62s/it]Copied model-00002-of-00012.safetensors from local model directory\n> Unsloth: Preparing safetensor model files: 42%|███████████████████████████ | 5/12 [00:08<00:12, 1.76s/it]Copied model-00007-of-00012.safetensors from local model directory\n> Unsloth: Preparing safetensor model files: 50%|████████████████████████████████▌ | 6/12 [00:10<00:10, 1.82s/it]Copied model-00010-of-00012.safetensors from local model directory\n> Unsloth: Preparing safetensor model files: 58%|█████████████████████████████████████▉ | 7/12 [00:13<00:09, 1.96s/it]Copied model-00008-of-00012.safetensors from local model directory\n> Unsloth: Preparing safetensor model files: 67%|███████████████████████████████████████████▎ | 8/12 [00:15<00:08, 2.00s/it]Copied model-00004-of-00012.safetensors from local model directory\n> Unsloth: Preparing safetensor model files: 75%|████████████████████████████████████████████████▊ | 9/12 [00:17<00:06, 2.00s/it]Copied model-00001-of-00012.safetensors from local model directory\n> Unsloth: Preparing safetensor model files: 83%|█████████████████████████████████████████████████████▎ | 10/12 [00:21<00:05, 2.60s/it]Copied model-00011-of-00012.safetensors from local model directory\n> Unsloth: Preparing safetensor model files: 92%|██████████████████████████████████████████████████████████▋ | 11/12 [00:23<00:02, 2.45s/it]Copied model-00005-of-00012.safetensors from local model directory\n> Unsloth: Preparing safetensor model files: 100%|████████████████████████████████████████████████████████████████| 12/12 [00:25<00:00, 2.10s/it]\n> Unsloth: Merging weights into 16bit: 100%|██████████████████████████████████████████████████████████████████████| 12/12 [07:34<00:00, 37.89s/it]\n> Unsloth: Merge process complete. Saved to `/home/ollam3/unsloth_finetune`\n> Unsloth: Converting to GGUF format...\n> ==((====))== Unsloth: Conversion from HF to GGUF information\n> \\\\ /| [0] Installing llama.cpp might take 3 minutes.\n> O^O/ \\_/ \\ [1] Converting HF to GGUF bf16 might take 3 minutes.\n> \\ / [2] Converting GGUF bf16 to ['q4_k_m'] might take 10 minutes each.\n> \"-____-\" In total, you will have to wait at least 16 minutes.\n> \n> Unsloth: llama.cpp found in the system. Skipping installation.\n> Unsloth: Preparing converter script...\n> Unsloth: [1] Converting model into bf16 GGUF format.\n> This might take 3 minutes...\n> Traceback (most recent call last):\n> File \"/usr/local/lib/python3.12/dist-packages/unsloth_zoo/llama_cpp.py\", line 991, in convert_to_gguf\n> subprocess.run(command, shell=True, check=True, capture_output=True)\n> File \"/usr/lib/python3.12/subprocess.py\", line 571, in run\n> raise CalledProcessError(retcode, process.args,\n> subprocess.CalledProcessError: Command 'python llama.cpp/unsloth_convert_hf_to_gguf.py --outfile medgemma-27b-it.BF16.gguf --outtype bf16 --split-max-size 50G unsloth_finetune' returned non-zero exit status 1.\n> \n> During handling of the above exception, another exception occurred:\n> \n> Traceback (most recent call last):\n> File \"/usr/local/lib/python3.12/dist-packages/unsloth/save.py\", line 1835, in unsloth_save_pretrained_gguf\n> all_file_locations, want_full_precision, is_vlm_update = save_to_gguf(\n> ^^^^^^^^^^^^^\n> File \"/usr/local/lib/python3.12/dist-packages/unsloth/save.py\", line 1099, in save_to_gguf\n> initial_files, is_vlm_update = convert_to_gguf(\n> ^^^^^^^^^^^^^^^^\n> File \"/usr/local/lib/python3.12/dist-packages/unsloth_zoo/llama_cpp.py\", line 995, in convert_to_gguf\n> raise RuntimeError(f\"Unsloth: Failed to convert {description} to GGUF: {e}\")\n> RuntimeError: Unsloth: Failed to convert text model to GGUF: Command 'python llama.cpp/unsloth_convert_hf_to_gguf.py --outfile medgemma-27b-it.BF16.gguf --outtype bf16 --split-max-size 50G unsloth_finetune' returned non-zero exit status 1.\n> \n> During handling of the above exception, another exception occurred:\n> \n> Traceback (most recent call last):\n> File \"/home/ollam3/finetunevisionGemma3_Herz.py\", line 217, in Error backtrace
\n\n``````\nFile \"C:\\Users\\kirschmann\\Desktop\\KI-Nachtrainieren\\finetune.py\", line 464, in export_model\n model.save_pretrained_merged(\n ~~~~~~~~~~~~~~~~~~~~~~~~~~~~^\n str(output_path / \"merged_model\"),\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n tokenizer,\n ^^^^^^^^^^\n save_method=\"merged_16bit\",\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^\n )\n ^\n File \"C:\\Users\\kirschmann\\AppData\\Roaming\\Python\\Python313\\site-packages\\unsloth\\save.py\", line 2688, in unsloth_generic_save_pretrained_merged\n unsloth_generic_save(**arguments)\n ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^\n File \"C:\\Users\\kirschmann\\AppData\\Roaming\\Python\\Python313\\site-packages\\torch\\utils\\_contextlib.py\", line 120, in decorate_context\n return func(*args, **kwargs)\n File \"C:\\Users\\kirschmann\\AppData\\Roaming\\Python\\Python313\\site-packages\\unsloth\\save.py\", line 2636, in unsloth_generic_save\n merge_and_overwrite_lora(\n ~~~~~~~~~~~~~~~~~~~~~~~~^\n get_model_name,\n ^^^^^^^^^^^^^^^\n ...<9 lines>...\n use_temp_file = False,\n ^^^^^^^^^^^^^^^^^^^^^^\n )\n ^\n File \"C:\\Users\\kirschmann\\AppData\\Roaming\\Python\\Python313\\site-packages\\torch\\utils\\_contextlib.py\", line 120, in decorate_context\n return func(*args, **kwargs)\n File \"C:\\Users\\kirschmann\\AppData\\Roaming\\Python\\Python313\\site-packages\\unsloth_zoo\\saving_utils.py\", line 1062, in merge_and_overwrite_lora\n ) = prepare_saving(\n ~~~~~~~~~~~~~~^\n model = model,\n ^^^^^^^^^^^^^^\n ...<9 lines>...\n use_temp_file = use_temp_file,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n )\n ^\n File \"C:\\Users\\kirschmann\\AppData\\Roaming\\Python\\Python313\\site-packages\\unsloth_zoo\\saving_utils.py\", line 786, in prepare_saving\n lora_weights, state_dict = create_lora_statistics(\n ~~~~~~~~~~~~~~~~~~~~~~^\n model,\n ^^^^^^\n merge_into_original = merge_into_original,\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n return_state_dict = True,\n ^^^^^^^^^^^^^^^^^^^^^^^^^\n )\n ^\n File \"C:\\Users\\kirschmann\\AppData\\Roaming\\Python\\Python313\\site-packages\\torch\\utils\\_contextlib.py\", line 120, in decorate_context\n return func(*args, **kwargs)\n File \"C:\\Users\\kirschmann\\AppData\\Roaming\\Python\\Python313\\site-packages\\unsloth_zoo\\saving_utils.py\", line 327, in create_lora_statistics\n assert(module_count == lora_A_count == lora_B_count == scaling_count)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nAssertionError\n``````\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n```\n\n\n\n\n\n\n
\n", "closed_by": null, "reactions": { "url": "https://api.github.com/repos/unslothai/unsloth/issues/3272/reactions", "total_count": 0, "+1": 0, "-1": 0, "laugh": 0, "hooray": 0, "confused": 0, "heart": 0, "rocket": 0, "eyes": 0 }, "performed_via_github_app": null, "state_reason": null, "linked_prs": [] }, { "url": "https://api.github.com/repos/unslothai/unsloth/issues/3271", "id": 3385250103, "node_id": "I_kwDOKznBOM7JxtE3", "number": 3271, "title": "[Bug] unstable training on Qwen2.5VL on text-only data", "user": { "login": "idhantgulati", "id": 84667585, "node_id": "MDQ6VXNlcjg0NjY3NTg1", "avatar_url": "https://avatars.githubusercontent.com/u/84667585?v=4", "gravatar_id": "", "url": "https://api.github.com/users/idhantgulati", "type": "User", "user_view_type": "public", "site_admin": false }, "labels": { "0": { "id": 8344678723, "node_id": "LA_kwDOKznBOM8AAAAB8WGxQw", "url": "https://api.github.com/repos/unslothai/unsloth/labels/bug", "name": "bug", "color": "d93f0b", "default": true, "description": "" } }, "state": "open", "locked": false, "assignees": {}, "milestone": null, "comments": 2, "created_at": "2025-09-04T19:57:27Z", "updated_at": "2025-09-05T04:59:06Z", "closed_at": null, "assignee": null, "author_association": "NONE", "type": null, "active_lock_reason": null, "sub_issues_summary": { "total": 0, "completed": 0, "percent_completed": 0 }, "issue_dependencies_summary": { "blocked_by": 0, "total_blocked_by": 0, "blocking": 0, "total_blocking": 0 }, "body": "1. Did you update? `pip install --upgrade unsloth unsloth_zoo`\n2. `Colab` or `Kaggle` or local / cloud- cloud (modal.com)\n3. Number GPUs used, use `nvidia-smi`- 1xA100-40gb\n4. Which notebook? Please link!- [qwen2.5vl](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Qwen2.5_VL_(7B)-Vision.ipynb) & [qwen2.5](https://colab.research.google.com/github/unslothai/notebooks/blob/main/nb/Qwen2.5_(7B)-Alpaca.ipynb). model used Qwen2.5-VL-3B vs Qwen2.5-3B\n5. Which Unsloth version, TRL version, transformers version, PyTorch version?- latest. using python3.11.x\n7. Which trainer? `SFTTrainer`, `GRPOTrainer` etc\n```python\n# Import required modules for training\nfrom trl import SFTTrainer, SFTConfig\n\ntrainer = SFTTrainer(\n model=model,\n tokenizer=tokenizer,\n # train_dataset=train_dataset, # Use the training split\n train_dataset=formatted_dataset,\n # eval_dataset=eval_dataset, # Use the evaluation split\n dataset_text_field=\"text\", # Specify the text field name\n args=SFTConfig(\n # BATCH SIZE - Increased for better gradient estimates\n per_device_train_batch_size=4, # Increased from 2\n gradient_accumulation_steps=8, # Effective batch = 32 (was 8)\n # TRAINING DURATION - Keep your original settings\n num_train_epochs=10,\n # LEARNING RATE - Optimized for LoRA fine-tuning\n learning_rate=1e-4, # Higher LR for LoRA (was 5e-5)\n max_grad_norm=0.3, # Lower gradient clipping for stability\n # OPTIMIZER - Keep your original settings\n optim=\"adamw_torch\", # stable & memory-efficient (or adamw_torch if you prefer)\n weight_decay=0.01, # no decay for LoRA matrices\n # SCHEDULING - Improved scheduling for better convergence\n warmup_steps=100, # More warmup steps for stability\n # warmup_ratio=0.1, # Using warmup_steps instead\n lr_scheduler_type=\"cosine\", # Cosine decay for better final convergence\n # LOGGING & CHECKPOINTS - Enhanced monitoring\n logging_steps=10,\n save_strategy=\"steps\",\n save_steps=100,\n # save_total_limit = 3, # Keep only best 3 checkpoints\n load_best_model_at_end=False,\n # EVALUATION - Add evaluation for monitoring\n # eval_steps=100,\n # eval_strategy=\"steps\",\n # DATA EFFICIENCY\n dataloader_num_workers=4, # Speed up data loading\n remove_unused_columns=False, # Keep all columns for debugging\n # WANDB - Keep your original settings\n report_to=\"wandb\",\n run_name=\"qwen-3b-hate-speech-lora-long\",\n # report_to = \"none\",\n # TEXT-ONLY CONFIG - Only removed vision-specific parameters\n max_seq_length=max_seq_length, # Use the max_seq_length variable\n # RANDOM SEED - Keep your original settings\n seed=3407,\n output_dir=\"outputs-qwen-3b-long\",\n gradient_checkpointing=True,\n ),\n)\n```\n\ni made some changes to the notebooks available on docs for data i was using. link: https://github.com/Idhant297/misaligned-vlm\n\n\n### ISSUE: \nwith the same hyperparameters i noticed the training was pretty unstable on Qwen2.5-VL compared to Qwen2.5\n