unsloth-llama-3-2-3b-instruct

developer_uid: Meliodia
submission_id: unsloth-llama-3-2-3b-instruct_v1
model_name: unsloth-llama-3-2-3b-instruct_v1
model_group: unsloth/Llama-3.2-3B-Ins
status: torndown
timestamp: 2025-11-23T13:21:22+00:00
num_battles: 7967
num_wins: 2962
celo_rating: 1158.9
family_friendly_score: 0.6706
family_friendly_standard_error: 0.006646738147392298
submission_type: basic
model_repo: unsloth/Llama-3.2-3B-Instruct
model_architecture: LlamaForCausalLM
model_num_parameters: 3606752256.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
reward_model: default
latencies: [{'batch_size': 1, 'throughput': 2.2485713995377195, 'latency_mean': 0.4446011996269226, 'latency_p50': 0.4456995725631714, 'latency_p90': 0.4973477840423584}, {'batch_size': 5, 'throughput': 5.9313505829933115, 'latency_mean': 0.8394657599925995, 'latency_p50': 0.844123125076294, 'latency_p90': 0.9297619581222534}, {'batch_size': 10, 'throughput': 7.357649062945704, 'latency_mean': 1.3406954443454742, 'latency_p50': 1.3385263681411743, 'latency_p90': 1.5231226682662964}, {'batch_size': 15, 'throughput': 8.0038886079907, 'latency_mean': 1.8461663687229157, 'latency_p50': 1.842139482498169, 'latency_p90': 2.105882668495178}, {'batch_size': 20, 'throughput': 8.061227213860677, 'latency_mean': 2.4238170969486235, 'latency_p50': 2.4409897327423096, 'latency_p90': 2.8093453884124755}, {'batch_size': 25, 'throughput': 8.107038944395455, 'latency_mean': 2.987551896572113, 'latency_p50': 2.9994934797286987, 'latency_p90': 3.565720725059509}, {'batch_size': 30, 'throughput': 8.093713468826182, 'latency_mean': 3.560087114572525, 'latency_p50': 3.639860153198242, 'latency_p90': 4.035198330879211}, {'batch_size': 35, 'throughput': 8.08137139217679, 'latency_mean': 4.095457398891449, 'latency_p50': 4.19767963886261, 'latency_p90': 4.607656025886535}, {'batch_size': 40, 'throughput': 8.097125881162228, 'latency_mean': 4.616721115112305, 'latency_p50': 4.789030194282532, 'latency_p90': 5.368837308883667}, {'batch_size': 45, 'throughput': 8.059581779720865, 'latency_mean': 5.159754409790039, 'latency_p50': 5.481606960296631, 'latency_p90': 6.0068511962890625}]
gpu_counts: {'NVIDIA L40S': 1}
display_name: unsloth-llama-3-2-3b-instruct_v1
ineligible_reason: num_battles<10000
is_internal_developer: True
language_model: unsloth/Llama-3.2-3B-Instruct
model_size: 4B
ranking_group: single
throughput_3p7s: 8.46
us_pacific_date: 2025-11-20
win_ratio: 0.37178360738044436
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name unsloth-llama-3-2-3b-instruct-v1-mkmlizer
Waiting for job on unsloth-llama-3-2-3b-instruct-v1-mkmlizer to finish
Failed to get response for submission mistralai-mistral-nem_93303_v569: ('http://mistralai-mistral-nem-93303-v569-predictor.tenant-chaiml-guanaco.k2.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: ║                                                                     ║
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: ║   ██████   ██████  █████   ████  ████                               ║
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: ║   ░███░█████░███   ░███  ███     ░███                               ║
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: ║   ░███░░███ ░███   ░███████      ░███                               ║
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: ║   ░███      ░███   ░███ ░░███    ░███                               ║
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: ║   █████     █████  █████ ░░████  █████                              ║
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: ║                                                                     ║
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: ║  Version: 0.30.2                                                    ║
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: ║  Features: FLYWHEEL, CUDA                                           ║
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: ║  https://mk1.ai                                                     ║
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: ║                                                                     ║
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: ║  belonging to:                                                      ║
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: ║                                                                     ║
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: ║  Chai Research Corp.                                                ║
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: ║                                                                     ║
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: Downloaded to shared memory in 11.631s
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: Checking if unsloth/Llama-3.2-3B-Instruct already exists in ChaiML
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: Creating repo ChaiML/Llama-3.2-3B-Instruct and uploading /tmp/tmp0mem12h9 to it
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
Processing Files (0 / 0)                : |          |  0.00B /  0.00B            
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
New Data Upload                         : |          |  0.00B /  0.00B            [A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors:   2%|▏         | 32.9MB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors:   2%|▏         | 32.9MB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
Processing Files (0 / 2)                :   1%|          | 33.2MB / 6.44GB,   ???B/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:   0%|          | 16.0MB / 4.97GB            [A[A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors:   4%|▍         | 64.8MB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:   0%|          | 16.0MB / 4.97GB            [A[A[A[A
Processing Files (0 / 3)                :   1%|▏         | 81.1MB / 6.44GB,  239MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors:   9%|▉         |  130MB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:   2%|▏         |  105MB / 4.97GB            [A[A[A[A
Processing Files (0 / 3)                :   4%|▎         |  235MB / 6.44GB,  505MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors:  12%|█▏        |  170MB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:   4%|▍         |  194MB / 4.97GB            [A[A[A[A
Processing Files (0 / 3)                :   6%|▌         |  364MB / 6.44GB,  552MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors:  17%|█▋        |  251MB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:   6%|▌         |  282MB / 4.97GB            [A[A[A[A
Processing Files (0 / 3)                :   8%|▊         |  533MB / 6.44GB,  625MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors:  22%|██▏       |  324MB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:   7%|▋         |  345MB / 4.97GB            [A[A[A[A
Processing Files (0 / 3)                :  10%|█         |  670MB / 6.44GB,  637MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors:  27%|██▋       |  399MB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:   9%|▊         |  425MB / 4.97GB            [A[A[A[A
Processing Files (0 / 3)                :  13%|█▎        |  825MB / 6.44GB,  660MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors:  33%|███▎      |  478MB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  10%|█         |  505MB / 4.97GB            [A[A[A[A
Processing Files (0 / 3)                :  15%|█▌        |  983MB / 6.44GB,  678MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors:  35%|███▌      |  511MB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  12%|█▏        |  592MB / 4.97GB            [A[A[A[A
Processing Files (0 / 3)                :  17%|█▋        | 1.10GB / 6.44GB,  669MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors:  41%|████▏     |  603MB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  14%|█▎        |  671MB / 4.97GB            [A[A[A[A
Processing Files (0 / 3)                :  20%|█▉        | 1.27GB / 6.44GB,  690MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors:  47%|████▋     |  686MB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  15%|█▌        |  751MB / 4.97GB            [A[A[A[A
Processing Files (0 / 3)                :  22%|██▏       | 1.44GB / 6.44GB,  702MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors:  53%|█████▎    |  778MB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  15%|█▌        |  751MB / 4.97GB            [A[A[A[A
Processing Files (0 / 3)                :  24%|██▎       | 1.53GB / 6.44GB,  680MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors:  59%|█████▉    |  866MB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  15%|█▌        |  751MB / 4.97GB            [A[A[A[A
Processing Files (0 / 3)                :  25%|██▌       | 1.62GB / 6.44GB,  660MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors:  65%|██████▌   |  953MB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  15%|█▌        |  759MB / 4.97GB            [A[A[A[A
Processing Files (0 / 3)                :  27%|██▋       | 1.71GB / 6.44GB,  646MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors:  71%|███████   | 1.04GB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  17%|█▋        |  834MB / 4.97GB            [A[A[A[A
Processing Files (0 / 3)                :  29%|██▉       | 1.87GB / 6.44GB,  658MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors:  77%|███████▋  | 1.12GB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  18%|█▊        |  918MB / 4.97GB            [A[A[A[A
Processing Files (0 / 3)                :  32%|███▏      | 2.04GB / 6.44GB,  667MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors:  83%|████████▎ | 1.21GB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  20%|██        | 1.01GB / 4.97GB            [A[A[A[A
Processing Files (0 / 3)                :  34%|███▍      | 2.21GB / 6.44GB,  682MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors:  88%|████████▊ | 1.29GB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  22%|██▏       | 1.09GB / 4.97GB            [A[A[A[A
Processing Files (0 / 3)                :  37%|███▋      | 2.38GB / 6.44GB,  689MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors:  94%|█████████▎| 1.37GB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  24%|██▎       | 1.18GB / 4.97GB            [A[A[A[A
Processing Files (0 / 3)                :  39%|███▉      | 2.54GB / 6.44GB,  698MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors:  96%|█████████▋| 1.41GB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  26%|██▌       | 1.27GB / 4.97GB            [A[A[A[A
Processing Files (0 / 3)                :  42%|████▏     | 2.68GB / 6.44GB,  696MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors: 100%|█████████▉| 1.46GB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  27%|██▋       | 1.36GB / 4.97GB            [A[A[A[A
Processing Files (0 / 3)                :  44%|████▎     | 2.82GB / 6.44GB,  696MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
New Data Upload                         : 100%|█████████▉| 52.1MB / 52.2MB, 13.0MB/s  [A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors: 100%|█████████▉| 1.46GB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  29%|██▉       | 1.45GB / 4.97GB            [A[A[A[A
Processing Files (0 / 3)                :  45%|████▌     | 2.91GB / 6.44GB,  684MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors: 100%|█████████▉| 1.46GB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  31%|███       | 1.53GB / 4.97GB            [A[A[A[A
Processing Files (0 / 3)                :  46%|████▋     | 2.99GB / 6.44GB,  673MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors: 100%|██████████| 1.46GB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  33%|███▎      | 1.62GB / 4.97GB            [A[A[A[A
Processing Files (1 / 3)                :  48%|████▊     | 3.08GB / 6.44GB,  663MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
New Data Upload                         : 100%|██████████| 52.2MB / 52.2MB, 11.4MB/s  [A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors: 100%|██████████| 1.46GB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  34%|███▍      | 1.71GB / 4.97GB            [A[A[A[A
Processing Files (1 / 3)                :  49%|████▉     | 3.17GB / 6.44GB,  654MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors: 100%|██████████| 1.46GB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  35%|███▌      | 1.75GB / 4.97GB            [A[A[A[A
Processing Files (1 / 3)                :  50%|████▉     | 3.21GB / 6.44GB,  636MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors: 100%|██████████| 1.46GB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  38%|███▊      | 1.88GB / 4.97GB            [A[A[A[A
Processing Files (1 / 3)                :  52%|█████▏    | 3.34GB / 6.44GB,  637MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
New Data Upload                         : 100%|█████████▉|  119MB /  119MB, 22.9MB/s  [A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors: 100%|██████████| 1.46GB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  40%|███▉      | 1.97GB / 4.97GB            [A[A[A[A
Processing Files (1 / 3)                :  53%|█████▎    | 3.43GB / 6.44GB,  629MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors: 100%|██████████| 1.46GB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  42%|████▏     | 2.07GB / 4.97GB            [A[A[A[A
Processing Files (1 / 3)                :  55%|█████▍    | 3.53GB / 6.44GB,  624MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors: 100%|██████████| 1.46GB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  43%|████▎     | 2.15GB / 4.97GB            [A[A[A[A
Processing Files (1 / 3)                :  56%|█████▌    | 3.61GB / 6.44GB,  617MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors: 100%|██████████| 1.46GB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  45%|████▌     | 2.25GB / 4.97GB            [A[A[A[A
Processing Files (1 / 3)                :  58%|█████▊    | 3.71GB / 6.44GB,  613MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors: 100%|██████████| 1.46GB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  47%|████▋     | 2.34GB / 4.97GB            [A[A[A[A
Processing Files (1 / 3)                :  59%|█████▉    | 3.80GB / 6.44GB,  607MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
New Data Upload                         : 100%|██████████|  119MB /  119MB, 19.2MB/s  [A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors: 100%|██████████| 1.46GB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  49%|████▉     | 2.42GB / 4.97GB            [A[A[A[A
Processing Files (1 / 3)                :  60%|██████    | 3.88GB / 6.44GB,  602MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors: 100%|██████████| 1.46GB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  51%|█████     | 2.51GB / 4.97GB            [A[A[A[A
Processing Files (1 / 3)                :  62%|██████▏   | 3.97GB / 6.44GB,  597MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors: 100%|██████████| 1.46GB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  52%|█████▏    | 2.60GB / 4.97GB            [A[A[A[A
Processing Files (1 / 3)                :  63%|██████▎   | 4.06GB / 6.44GB,  592MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors: 100%|██████████| 1.46GB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  54%|█████▍    | 2.69GB / 4.97GB            [A[A[A[A
Processing Files (1 / 3)                :  64%|██████▍   | 4.15GB / 6.44GB,  588MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors: 100%|██████████| 1.46GB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  56%|█████▌    | 2.78GB / 4.97GB            [A[A[A[A
Processing Files (1 / 3)                :  66%|██████▌   | 4.24GB / 6.44GB,  584MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors: 100%|██████████| 1.46GB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  58%|█████▊    | 2.87GB / 4.97GB            [A[A[A[A
Processing Files (1 / 3)                :  67%|██████▋   | 4.33GB / 6.44GB,  581MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors: 100%|██████████| 1.46GB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  60%|█████▉    | 2.96GB / 4.97GB            [A[A[A[A
Processing Files (1 / 3)                :  69%|██████▊   | 4.42GB / 6.44GB,  577MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors: 100%|██████████| 1.46GB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  61%|██████    | 3.03GB / 4.97GB            [A[A[A[A
Processing Files (1 / 3)                :  70%|██████▉   | 4.49GB / 6.44GB,  572MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors: 100%|██████████| 1.46GB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  61%|██████    | 3.03GB / 4.97GB            [A[A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors: 100%|██████████| 1.46GB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  62%|██████▏   | 3.06GB / 4.97GB            [A[A[A[A
Processing Files (1 / 3)                :  70%|███████   | 4.52GB / 6.44GB,  547MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors: 100%|██████████| 1.46GB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  63%|██████▎   | 3.15GB / 4.97GB            [A[A[A[A
Processing Files (1 / 3)                :  72%|███████▏  | 4.61GB / 6.44GB,  545MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors: 100%|██████████| 1.46GB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00001-of-00002.safetensors:  65%|██████▌   | 3.25GB / 4.97GB            [A[A[A[A
Processing Files (1 / 3)                :  73%|███████▎  | 4.71GB / 6.44GB,  544MB/s  
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  ...h9/model-00002-of-00002.safetensors: 100%|██████████| 1.46GB / 1.46GB            [A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: 
  /tmp/tmp0mem12h9/tokenizer.json       :   2%|▏         |  295kB / 17.2MB            [A[A[A
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: quantized model in 45.639s
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: Processed model unsloth/Llama-3.2-3B-Instruct in 80.371s
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: creating bucket guanaco-mkml-models
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/unsloth-llama-3-2-3b-instruct-v1/nvidia
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/unsloth-llama-3-2-3b-instruct-v1/nvidia/config.json
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/unsloth-llama-3-2-3b-instruct-v1/nvidia/special_tokens_map.json
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/unsloth-llama-3-2-3b-instruct-v1/nvidia/tokenizer_config.json
unsloth-llama-3-2-3b-instruct-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/unsloth-llama-3-2-3b-instruct-v1/nvidia/tokenizer.json
Job unsloth-llama-3-2-3b-instruct-v1-mkmlizer completed after 104.97s with status: succeeded
Stopping job with name unsloth-llama-3-2-3b-instruct-v1-mkmlizer
Pipeline stage MKMLizer completed in 105.55s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.18s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service unsloth-llama-3-2-3b-instruct-v1
Waiting for inference service unsloth-llama-3-2-3b-instruct-v1 to be ready
Inference service unsloth-llama-3-2-3b-instruct-v1 ready after 151.31509685516357s
Pipeline stage MKMLDeployer completed in 151.92s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 0.8634426593780518s
Received healthy response to inference request in 0.7123568058013916s
Received healthy response to inference request in 0.638131856918335s
Received healthy response to inference request in 0.43396735191345215s
Received healthy response to inference request in 0.6015219688415527s
5 requests
0 failed requests
5th percentile: 0.46747827529907227
10th percentile: 0.5009891986846924
20th percentile: 0.5680110454559326
30th percentile: 0.6088439464569092
40th percentile: 0.623487901687622
50th percentile: 0.638131856918335
60th percentile: 0.6678218364715576
70th percentile: 0.6975118160247803
80th percentile: 0.7425739765167236
90th percentile: 0.8030083179473877
95th percentile: 0.8332254886627197
99th percentile: 0.8573992252349854
mean time: 0.6498841285705567
Pipeline stage StressChecker completed in 4.86s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 1.13s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.69s
Shutdown handler de-registered
unsloth-llama-3-2-3b-instruct_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage MKMLProfilerDeleter completed in 0.14s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.11s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service unsloth-llama-3-2-3b-instruct-v1-profiler
Waiting for inference service unsloth-llama-3-2-3b-instruct-v1-profiler to be ready
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 1529.27s
Shutdown handler de-registered
unsloth-llama-3-2-3b-instruct_v1 status is now inactive due to auto deactivation removed underperforming models
unsloth-llama-3-2-3b-instruct_v1 status is now torndown due to DeploymentManager action