developer_uid: bogoconic1
submission_id: chaiml-gy-exp19-sftlora_96276_v2
model_name: chaiml-gy-exp19-sftlora_96276_v2
model_group: ChaiML/gy-exp19-sftlora-
status: torndown
timestamp: 2025-06-26T21:32:31+00:00
num_battles: 7230
num_wins: 3737
celo_rating: 1299.5
family_friendly_score: 0.5322
family_friendly_standard_error: 0.007056389445034904
submission_type: basic
model_repo: ChaiML/gy-exp19-sftlora-gy-1266-1301-7850-pref-grok-ctx2500-96tok-onlylast-ep2
model_architecture: MistralForCausalLM
model_num_parameters: 24096691200.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
latencies: [{'batch_size': 1, 'throughput': 0.5234735791512146, 'latency_mean': 1.91019238114357, 'latency_p50': 1.8930411338806152, 'latency_p90': 2.106654119491577}, {'batch_size': 3, 'throughput': 1.0483534070524059, 'latency_mean': 2.8449142110347747, 'latency_p50': 2.85868501663208, 'latency_p90': 3.182152819633484}, {'batch_size': 5, 'throughput': 1.349610954411921, 'latency_mean': 3.696244238615036, 'latency_p50': 3.6996692419052124, 'latency_p90': 4.230594110488892}, {'batch_size': 6, 'throughput': 1.4278346136825955, 'latency_mean': 4.168978453874588, 'latency_p50': 4.196207046508789, 'latency_p90': 4.578718638420105}, {'batch_size': 8, 'throughput': 1.5760951368791647, 'latency_mean': 5.027419077157974, 'latency_p50': 5.079604983329773, 'latency_p90': 5.602469420433044}, {'batch_size': 10, 'throughput': 1.6668493881784745, 'latency_mean': 5.943796718120575, 'latency_p50': 5.919687747955322, 'latency_p90': 6.713601589202881}]
gpu_counts: {'NVIDIA A100-SXM4-80GB': 1}
display_name: chaiml-gy-exp19-sftlora_96276_v2
is_internal_developer: True
language_model: ChaiML/gy-exp19-sftlora-gy-1266-1301-7850-pref-grok-ctx2500-96tok-onlylast-ep2
model_size: 24B
ranking_group: single
throughput_3p7s: 1.35
us_pacific_date: 2025-06-26
win_ratio: 0.5168741355463348
generation_params: {'temperature': 0.8, 'top_p': 0.95, 'min_p': 0.025, 'top_k': 60, 'presence_penalty': 0.4, 'frequency_penalty': 0.4, 'stopping_words': ['<|im_start|>', '<|im_end|>', '\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '<|system|>Family Friendly{memory}\n', 'prompt_template': '', 'bot_template': '<|im_start|>assistant\n{message}<|im_end|>\n', 'user_template': '<|im_start|>user\nYou:{message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-gy-exp19-sftlora-96276-v2-mkmlizer
Waiting for job on chaiml-gy-exp19-sftlora-96276-v2-mkmlizer to finish
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: ║                                                                     ║
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: ║   ██████   ██████  █████   ████  ████                               ║
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: ║   ░███░█████░███   ░███  ███     ░███                               ║
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: ║   ░███░░███ ░███   ░███████      ░███                               ║
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: ║   ░███      ░███   ░███ ░░███    ░███                               ║
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: ║   █████     █████  █████ ░░████  █████                              ║
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: ║                                                                     ║
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: ║  Version: 0.29.3                                                    ║
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: ║  Features: FLYWHEEL, CUDA                                           ║
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: ║                                                                     ║
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: ║  belonging to:                                                      ║
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: ║                                                                     ║
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: ║                                                                     ║
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: Downloaded to shared memory in 326.342s
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: Checking if ChaiML/gy-exp19-sftlora-gy-1266-1301-7850-pref-grok-ctx2500-96tok-onlylast-ep2 already exists in ChaiML
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp3c1oagpk, device:0
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: quantized model in 48.652s
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: Processed model ChaiML/gy-exp19-sftlora-gy-1266-1301-7850-pref-grok-ctx2500-96tok-onlylast-ep2 in 374.994s
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: creating bucket guanaco-mkml-models
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-gy-exp19-sftlora-96276-v2/config.json
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-gy-exp19-sftlora-96276-v2/special_tokens_map.json
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-gy-exp19-sftlora-96276-v2/tokenizer_config.json
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-gy-exp19-sftlora-96276-v2/tokenizer.json
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/chaiml-gy-exp19-sftlora-96276-v2/flywheel_model.1.safetensors
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-gy-exp19-sftlora-96276-v2/flywheel_model.0.safetensors
chaiml-gy-exp19-sftlora-96276-v2-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|          | 4/363 [00:00<00:09, 37.79it/s]
Loading 0:   2%|▏         | 8/363 [00:00<00:11, 31.36it/s]
Loading 0:   3%|▎         | 12/363 [00:00<00:10, 32.43it/s]
Loading 0:   4%|▍         | 16/363 [00:00<00:11, 29.45it/s]
Loading 0:   6%|▌         | 21/363 [00:00<00:10, 33.92it/s]
Loading 0:   7%|▋         | 25/363 [00:00<00:11, 29.77it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:08, 37.43it/s]
Loading 0:  10%|▉         | 36/363 [00:01<00:16, 20.04it/s]
Loading 0:  11%|█         | 40/363 [00:01<00:14, 22.78it/s]
Loading 0:  12%|█▏        | 44/363 [00:01<00:13, 24.51it/s]
Loading 0:  13%|█▎        | 48/363 [00:01<00:11, 26.65it/s]
Loading 0:  14%|█▍        | 52/363 [00:01<00:11, 26.05it/s]
Loading 0:  16%|█▌        | 57/363 [00:02<00:10, 28.72it/s]
Loading 0:  17%|█▋        | 61/363 [00:02<00:10, 27.67it/s]
Loading 0:  18%|█▊        | 65/363 [00:02<00:10, 28.00it/s]
Loading 0:  19%|█▉        | 70/363 [00:02<00:11, 24.96it/s]
Loading 0:  20%|██        | 73/363 [00:02<00:13, 21.51it/s]
Loading 0:  22%|██▏       | 79/363 [00:02<00:10, 26.96it/s]
Loading 0:  23%|██▎       | 82/363 [00:03<00:10, 26.22it/s]
Loading 0:  24%|██▎       | 86/363 [00:03<00:10, 27.68it/s]
Loading 0:  25%|██▍       | 89/363 [00:03<00:09, 27.85it/s]
Loading 0:  25%|██▌       | 92/363 [00:03<00:12, 22.36it/s]
Loading 0:  27%|██▋       | 99/363 [00:03<00:09, 28.99it/s]
Loading 0:  28%|██▊       | 103/363 [00:03<00:09, 26.73it/s]
Loading 0:  29%|██▉       | 107/363 [00:04<00:11, 21.80it/s]
Loading 0:  31%|███       | 111/363 [00:04<00:10, 25.03it/s]
Loading 0:  31%|███▏      | 114/363 [00:04<00:10, 22.87it/s]
Loading 0:  33%|███▎      | 120/363 [00:04<00:08, 27.48it/s]
Loading 0:  34%|███▍      | 123/363 [00:04<00:09, 24.55it/s]
Loading 0:  36%|███▌      | 129/363 [00:04<00:08, 28.59it/s]
Loading 0:  36%|███▋      | 132/363 [00:05<00:08, 25.80it/s]
Loading 0:  38%|███▊      | 138/363 [00:05<00:07, 29.81it/s]
Loading 0:  39%|███▉      | 142/363 [00:05<00:07, 28.34it/s]
Loading 0:  40%|████      | 147/363 [00:05<00:06, 32.74it/s]
Loading 0:  42%|████▏     | 151/363 [00:05<00:09, 23.42it/s]
Loading 0:  42%|████▏     | 154/363 [00:05<00:09, 22.05it/s]
Loading 0:  43%|████▎     | 157/363 [00:06<00:08, 23.20it/s]
Loading 0:  44%|████▍     | 160/363 [00:06<00:08, 22.68it/s]
Loading 0:  45%|████▍     | 163/363 [00:06<00:08, 24.10it/s]
Loading 0:  46%|████▌     | 166/363 [00:06<00:07, 25.16it/s]
Loading 0:  47%|████▋     | 169/363 [00:06<00:07, 24.99it/s]
Loading 0:  48%|████▊     | 174/363 [00:06<00:06, 27.73it/s]
Loading 0:  49%|████▉     | 177/363 [00:06<00:07, 24.34it/s]
Loading 0:  50%|█████     | 182/363 [00:06<00:06, 26.92it/s]
Loading 0:  52%|█████▏    | 187/363 [00:07<00:07, 23.44it/s]
Loading 0:  52%|█████▏    | 190/363 [00:07<00:07, 22.03it/s]
Loading 0:  53%|█████▎    | 193/363 [00:07<00:07, 23.44it/s]
Loading 0:  54%|█████▍    | 196/363 [00:07<00:06, 24.08it/s]
Loading 0:  55%|█████▌    | 200/363 [00:22<00:06, 24.08it/s]
Loading 0:  55%|█████▌    | 201/363 [00:22<03:07,  1.16s/it]
Loading 0:  56%|█████▌    | 203/363 [00:22<02:34,  1.04it/s]
Loading 0:  57%|█████▋    | 208/363 [00:22<01:32,  1.67it/s]
Loading 0:  58%|█████▊    | 211/363 [00:22<01:11,  2.14it/s]
Loading 0:  59%|█████▉    | 214/363 [00:22<00:52,  2.82it/s]
Loading 0:  60%|██████    | 218/363 [00:23<00:36,  4.01it/s]
Loading 0:  61%|██████    | 221/363 [00:23<00:27,  5.12it/s]
Loading 0:  62%|██████▏   | 224/363 [00:23<00:22,  6.13it/s]
Loading 0:  63%|██████▎   | 228/363 [00:23<00:15,  8.63it/s]
Loading 0:  64%|██████▎   | 231/363 [00:23<00:13, 10.10it/s]
Loading 0:  65%|██████▌   | 237/363 [00:23<00:08, 14.97it/s]
Loading 0:  66%|██████▌   | 240/363 [00:24<00:07, 15.50it/s]
Loading 0:  68%|██████▊   | 246/363 [00:24<00:05, 20.54it/s]
Loading 0:  69%|██████▊   | 249/363 [00:24<00:05, 20.21it/s]
Loading 0:  70%|███████   | 255/363 [00:24<00:04, 24.87it/s]
Loading 0:  71%|███████▏  | 259/363 [00:24<00:04, 24.53it/s]
Loading 0:  73%|███████▎  | 264/363 [00:24<00:03, 29.06it/s]
Loading 0:  74%|███████▍  | 268/363 [00:25<00:04, 22.29it/s]
Loading 0:  75%|███████▍  | 271/363 [00:25<00:04, 21.51it/s]
Loading 0:  75%|███████▌  | 274/363 [00:25<00:03, 22.46it/s]
Loading 0:  76%|███████▋  | 277/363 [00:25<00:04, 20.46it/s]
Loading 0:  77%|███████▋  | 280/363 [00:25<00:03, 22.31it/s]
Loading 0:  78%|███████▊  | 283/363 [00:25<00:03, 23.94it/s]
Loading 0:  79%|███████▉  | 286/363 [00:25<00:03, 24.61it/s]
Loading 0:  80%|████████  | 291/363 [00:26<00:02, 28.00it/s]
Loading 0:  81%|████████  | 294/363 [00:26<00:02, 25.35it/s]
Loading 0:  82%|████████▏ | 299/363 [00:26<00:02, 27.91it/s]
Loading 0:  84%|████████▎ | 304/363 [00:26<00:02, 25.51it/s]
Loading 0:  85%|████████▍ | 307/363 [00:26<00:02, 23.25it/s]
Loading 0:  85%|████████▌ | 310/363 [00:26<00:02, 24.20it/s]
Loading 0:  86%|████████▌ | 313/363 [00:26<00:02, 24.07it/s]
Loading 0:  88%|████████▊ | 318/363 [00:27<00:01, 26.60it/s]
Loading 0:  88%|████████▊ | 321/363 [00:27<00:01, 23.87it/s]
Loading 0:  90%|████████▉ | 325/363 [00:27<00:01, 27.17it/s]
Loading 0:  90%|█████████ | 328/363 [00:27<00:01, 27.63it/s]
Loading 0:  91%|█████████ | 331/363 [00:27<00:01, 26.64it/s]
Loading 0:  92%|█████████▏| 335/363 [00:27<00:01, 26.46it/s]
Loading 0:  93%|█████████▎| 338/363 [00:27<00:01, 24.69it/s]
Loading 0:  94%|█████████▍| 341/363 [00:28<00:01, 14.37it/s]
Loading 0:  95%|█████████▌| 346/363 [00:28<00:00, 19.97it/s]
Loading 0:  96%|█████████▌| 349/363 [00:28<00:00, 19.12it/s]
Loading 0:  97%|█████████▋| 353/363 [00:28<00:00, 22.51it/s]
Loading 0:  98%|█████████▊| 356/363 [00:28<00:00, 24.02it/s]
Loading 0:  99%|█████████▉| 359/363 [00:28<00:00, 23.90it/s]
                                                            

Job chaiml-gy-exp19-sftlora-96276-v2-mkmlizer completed after 404.48s with status: succeeded
Stopping job with name chaiml-gy-exp19-sftlora-96276-v2-mkmlizer
Pipeline stage MKMLizer completed in 404.99s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.18s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-gy-exp19-sftlora-96276-v2
Waiting for inference service chaiml-gy-exp19-sftlora-96276-v2 to be ready
Inference service chaiml-gy-exp19-sftlora-96276-v2 ready after 151.09962224960327s
Pipeline stage MKMLDeployer completed in 151.63s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 3.026214122772217s
Received healthy response to inference request in 1.966181993484497s
Received healthy response to inference request in 2.135655641555786s
Received healthy response to inference request in 2.303738832473755s
5 requests
1 failed requests
5th percentile: 2.000076723098755
10th percentile: 2.0339714527130126
20th percentile: 2.101760911941528
30th percentile: 2.16927227973938
40th percentile: 2.236505556106567
50th percentile: 2.303738832473755
60th percentile: 2.5927289485931397
70th percentile: 2.881719064712524
80th percentile: 6.445079326629642
90th percentile: 13.282809734344484
95th percentile: 16.7016749382019
99th percentile: 19.436767101287842
mean time: 5.910466146469116
%s, retrying in %s seconds...
Received healthy response to inference request in 2.234247922897339s
Received healthy response to inference request in 1.808058261871338s
Received healthy response to inference request in 1.9234318733215332s
Received healthy response to inference request in 2.117288827896118s
Received healthy response to inference request in 2.0977835655212402s
5 requests
0 failed requests
5th percentile: 1.831132984161377
10th percentile: 1.854207706451416
20th percentile: 1.900357151031494
30th percentile: 1.9583022117614746
40th percentile: 2.0280428886413575
50th percentile: 2.0977835655212402
60th percentile: 2.1055856704711915
70th percentile: 2.1133877754211428
80th percentile: 2.1406806468963624
90th percentile: 2.1874642848968504
95th percentile: 2.2108561038970946
99th percentile: 2.22956955909729
mean time: 2.036162090301514
Pipeline stage StressChecker completed in 42.36s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.65s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.69s
Shutdown handler de-registered
chaiml-gy-exp19-sftlora_96276_v2 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3242.43s
Shutdown handler de-registered
chaiml-gy-exp19-sftlora_96276_v2 status is now inactive due to auto deactivation removed underperforming models
chaiml-gy-exp19-sftlora_96276_v2 status is now torndown due to DeploymentManager action