developer_uid: bogoconic1
submission_id: chaiml-gy-exp85-simpo-e_54434_v1
model_name: chaiml-gy-exp85-simpo-e_54434_v1
model_group: ChaiML/gy-exp85-simpo-ex
status: torndown
timestamp: 2025-07-04T20:28:00+00:00
num_battles: 7655
num_wins: 4113
celo_rating: 1301.31
family_friendly_score: 0.4384
family_friendly_standard_error: 0.007017199441372605
submission_type: basic
model_repo: ChaiML/gy-exp85-simpo-exp32ep8s2-exp72data-ep1
model_architecture: MistralForCausalLM
model_num_parameters: 24096691200.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
latencies: [{'batch_size': 1, 'throughput': 0.5192862857749365, 'latency_mean': 1.9256160128116608, 'latency_p50': 1.923326015472412, 'latency_p90': 2.135530138015747}, {'batch_size': 3, 'throughput': 1.0452012099123835, 'latency_mean': 2.8624810910224916, 'latency_p50': 2.8532837629318237, 'latency_p90': 3.1561746835708617}, {'batch_size': 5, 'throughput': 1.324314884812287, 'latency_mean': 3.7512218415737153, 'latency_p50': 3.7559880018234253, 'latency_p90': 4.192740893363952}, {'batch_size': 6, 'throughput': 1.4221175713584608, 'latency_mean': 4.183492000102997, 'latency_p50': 4.172035336494446, 'latency_p90': 4.721720838546753}, {'batch_size': 8, 'throughput': 1.5610361810571576, 'latency_mean': 5.0804907631874086, 'latency_p50': 5.1193495988845825, 'latency_p90': 5.666853451728821}, {'batch_size': 10, 'throughput': 1.6455616514343978, 'latency_mean': 6.022153944969177, 'latency_p50': 6.012423872947693, 'latency_p90': 6.819515585899353}]
gpu_counts: {'NVIDIA A100-SXM4-80GB': 1}
display_name: chaiml-gy-exp85-simpo-e_54434_v1
is_internal_developer: True
language_model: ChaiML/gy-exp85-simpo-exp32ep8s2-exp72data-ep1
model_size: 24B
ranking_group: single
throughput_3p7s: 1.32
us_pacific_date: 2025-07-04
win_ratio: 0.537295885042456
generation_params: {'temperature': 0.6, 'top_p': 0.95, 'min_p': 0.025, 'top_k': 60, 'presence_penalty': 0.4, 'frequency_penalty': 0.4, 'stopping_words': ['<|im_start|>', '<|im_end|>', '\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '<|system|>Family Friendly{memory}\n', 'prompt_template': '', 'bot_template': '<|im_start|>assistant\n{message}<|im_end|>\n', 'user_template': '<|im_start|>user\nYou:{message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer
Waiting for job on chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer to finish
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: ║                                                                     ║
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: ║   ██████   ██████  █████   ████  ████                               ║
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: ║   ░███░█████░███   ░███  ███     ░███                               ║
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: ║   ░███░░███ ░███   ░███████      ░███                               ║
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: ║   ░███      ░███   ░███ ░░███    ░███                               ║
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: ║   █████     █████  █████ ░░████  █████                              ║
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: ║                                                                     ║
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: ║  Version: 0.29.15                                                   ║
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: ║  Features: FLYWHEEL, CUDA                                           ║
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: ║                                                                     ║
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: ║  belonging to:                                                      ║
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: ║                                                                     ║
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: ║                                                                     ║
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: Downloaded to shared memory in 79.969s
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: Checking if ChaiML/gy-exp85-simpo-exp32ep8s2-exp72data-ep1 already exists in ChaiML
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp1zei4_q0, device:0
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: quantized model in 48.465s
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: Processed model ChaiML/gy-exp85-simpo-exp32ep8s2-exp72data-ep1 in 128.434s
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: creating bucket guanaco-mkml-models
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-gy-exp85-simpo-e-54434-v1/nvidia
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-gy-exp85-simpo-e-54434-v1/nvidia/special_tokens_map.json
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-gy-exp85-simpo-e-54434-v1/nvidia/config.json
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-gy-exp85-simpo-e-54434-v1/nvidia/tokenizer_config.json
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-gy-exp85-simpo-e-54434-v1/nvidia/tokenizer.json
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/chaiml-gy-exp85-simpo-e-54434-v1/nvidia/flywheel_model.1.safetensors
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-gy-exp85-simpo-e-54434-v1/nvidia/flywheel_model.0.safetensors
chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|          | 4/363 [00:00<00:11, 32.17it/s]
Loading 0:   2%|▏         | 8/363 [00:00<00:13, 26.63it/s]
Loading 0:   3%|▎         | 12/363 [00:00<00:11, 29.98it/s]
Loading 0:   4%|▍         | 16/363 [00:00<00:12, 26.78it/s]
Loading 0:   6%|▌         | 21/363 [00:00<00:11, 28.90it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:13, 24.61it/s]
Loading 0:   8%|▊         | 30/363 [00:01<00:10, 31.78it/s]
Loading 0:   9%|▉         | 34/363 [00:01<00:14, 23.43it/s]
Loading 0:  10%|█         | 37/363 [00:01<00:15, 21.64it/s]
Loading 0:  11%|█         | 40/363 [00:01<00:14, 22.72it/s]
Loading 0:  12%|█▏        | 43/363 [00:01<00:13, 22.97it/s]
Loading 0:  13%|█▎        | 48/363 [00:01<00:12, 26.17it/s]
Loading 0:  14%|█▍        | 51/363 [00:02<00:13, 23.50it/s]
Loading 0:  16%|█▌        | 57/363 [00:02<00:10, 28.43it/s]
Loading 0:  17%|█▋        | 60/363 [00:02<00:12, 24.95it/s]
Loading 0:  18%|█▊        | 65/363 [00:02<00:10, 27.56it/s]
Loading 0:  19%|█▉        | 70/363 [00:02<00:11, 25.08it/s]
Loading 0:  20%|██        | 73/363 [00:02<00:13, 20.99it/s]
Loading 0:  22%|██▏       | 79/363 [00:03<00:10, 26.18it/s]
Loading 0:  23%|██▎       | 82/363 [00:03<00:11, 25.03it/s]
Loading 0:  24%|██▎       | 86/363 [00:03<00:10, 27.29it/s]
Loading 0:  25%|██▍       | 89/363 [00:03<00:10, 26.66it/s]
Loading 0:  25%|██▌       | 92/363 [00:03<00:13, 20.76it/s]
Loading 0:  27%|██▋       | 99/363 [00:03<00:09, 28.70it/s]
Loading 0:  28%|██▊       | 103/363 [00:04<00:09, 26.96it/s]
Loading 0:  29%|██▉       | 107/363 [00:04<00:11, 22.68it/s]
Loading 0:  31%|███       | 111/363 [00:04<00:09, 25.64it/s]
Loading 0:  31%|███▏      | 114/363 [00:04<00:10, 23.34it/s]
Loading 0:  33%|███▎      | 120/363 [00:04<00:08, 28.97it/s]
Loading 0:  34%|███▍      | 124/363 [00:04<00:08, 27.43it/s]
Loading 0:  36%|███▌      | 129/363 [00:04<00:07, 30.83it/s]
Loading 0:  37%|███▋      | 133/363 [00:05<00:08, 28.21it/s]
Loading 0:  38%|███▊      | 138/363 [00:05<00:07, 30.78it/s]
Loading 0:  39%|███▉      | 142/363 [00:05<00:07, 28.38it/s]
Loading 0:  41%|████      | 148/363 [00:05<00:06, 35.03it/s]
Loading 0:  42%|████▏     | 152/363 [00:05<00:08, 24.29it/s]
Loading 0:  43%|████▎     | 156/363 [00:06<00:08, 23.44it/s]
Loading 0:  44%|████▍     | 159/363 [00:06<00:09, 21.90it/s]
Loading 0:  45%|████▌     | 165/363 [00:06<00:07, 27.28it/s]
Loading 0:  47%|████▋     | 169/363 [00:06<00:07, 26.45it/s]
Loading 0:  48%|████▊     | 174/363 [00:06<00:06, 28.52it/s]
Loading 0:  49%|████▉     | 178/363 [00:06<00:06, 26.57it/s]
Loading 0:  50%|█████     | 182/363 [00:06<00:06, 27.20it/s]
Loading 0:  52%|█████▏    | 187/363 [00:07<00:07, 23.39it/s]
Loading 0:  52%|█████▏    | 190/363 [00:07<00:08, 21.56it/s]
Loading 0:  53%|█████▎    | 193/363 [00:07<00:07, 22.51it/s]
Loading 0:  54%|█████▍    | 196/363 [00:07<00:07, 22.54it/s]
Loading 0:  55%|█████▍    | 199/363 [00:07<00:06, 24.09it/s]
Loading 0:  55%|█████▌    | 200/363 [00:22<00:06, 24.09it/s]
Loading 0:  55%|█████▌    | 201/363 [00:22<04:08,  1.53s/it]
Loading 0:  56%|█████▌    | 203/363 [00:22<03:13,  1.21s/it]
Loading 0:  57%|█████▋    | 208/363 [00:22<01:45,  1.47it/s]
Loading 0:  58%|█████▊    | 211/363 [00:23<01:18,  1.94it/s]
Loading 0:  59%|█████▉    | 214/363 [00:23<00:56,  2.63it/s]
Loading 0:  60%|██████    | 218/363 [00:23<00:37,  3.85it/s]
Loading 0:  61%|██████    | 221/363 [00:23<00:28,  4.97it/s]
Loading 0:  62%|██████▏   | 224/363 [00:23<00:22,  6.05it/s]
Loading 0:  63%|██████▎   | 228/363 [00:23<00:15,  8.54it/s]
Loading 0:  64%|██████▎   | 231/363 [00:23<00:13,  9.93it/s]
Loading 0:  65%|██████▌   | 237/363 [00:24<00:08, 14.98it/s]
Loading 0:  66%|██████▌   | 240/363 [00:24<00:07, 15.75it/s]
Loading 0:  68%|██████▊   | 246/363 [00:24<00:05, 21.98it/s]
Loading 0:  69%|██████▉   | 250/363 [00:24<00:04, 22.83it/s]
Loading 0:  70%|███████   | 255/363 [00:24<00:04, 26.84it/s]
Loading 0:  71%|███████▏  | 259/363 [00:24<00:03, 26.60it/s]
Loading 0:  73%|███████▎  | 265/363 [00:24<00:02, 33.33it/s]
Loading 0:  74%|███████▍  | 269/363 [00:25<00:03, 23.72it/s]
Loading 0:  75%|███████▌  | 273/363 [00:25<00:03, 23.03it/s]
Loading 0:  76%|███████▌  | 276/363 [00:25<00:04, 21.62it/s]
Loading 0:  78%|███████▊  | 282/363 [00:25<00:03, 25.99it/s]
Loading 0:  79%|███████▊  | 285/363 [00:25<00:03, 23.70it/s]
Loading 0:  80%|████████  | 291/363 [00:26<00:02, 28.16it/s]
Loading 0:  81%|████████▏ | 295/363 [00:26<00:02, 26.82it/s]
Loading 0:  82%|████████▏ | 299/363 [00:26<00:02, 27.26it/s]
Loading 0:  84%|████████▎ | 304/363 [00:26<00:02, 24.46it/s]
Loading 0:  85%|████████▍ | 307/363 [00:26<00:02, 22.47it/s]
Loading 0:  85%|████████▌ | 310/363 [00:26<00:02, 23.41it/s]
Loading 0:  86%|████████▌ | 313/363 [00:27<00:02, 23.41it/s]
Loading 0:  88%|████████▊ | 318/363 [00:27<00:01, 26.50it/s]
Loading 0:  88%|████████▊ | 321/363 [00:27<00:01, 23.86it/s]
Loading 0:  90%|█████████ | 327/363 [00:27<00:01, 30.03it/s]
Loading 0:  91%|█████████ | 331/363 [00:27<00:01, 28.69it/s]
Loading 0:  92%|█████████▏| 335/363 [00:27<00:00, 30.31it/s]
Loading 0:  93%|█████████▎| 339/363 [00:27<00:00, 29.54it/s]
Loading 0:  94%|█████████▍| 343/363 [00:28<00:01, 17.38it/s]
Loading 0:  96%|█████████▌| 348/363 [00:28<00:00, 19.07it/s]
Loading 0:  98%|█████████▊| 355/363 [00:28<00:00, 26.10it/s]
Loading 0:  99%|█████████▉| 359/363 [00:28<00:00, 24.91it/s]
                                                            

Job chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer completed after 156.12s with status: succeeded
Stopping job with name chaiml-gy-exp85-simpo-e-54434-v1-mkmlizer
Pipeline stage MKMLizer completed in 156.59s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.20s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-gy-exp85-simpo-e-54434-v1
Waiting for inference service chaiml-gy-exp85-simpo-e-54434-v1 to be ready
Inference service chaiml-gy-exp85-simpo-e-54434-v1 ready after 211.18251729011536s
Pipeline stage MKMLDeployer completed in 211.63s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.82543683052063s
Received healthy response to inference request in 2.169297933578491s
Received healthy response to inference request in 2.0712029933929443s
Received healthy response to inference request in 1.8948454856872559s
5 requests
1 failed requests
5th percentile: 1.9301169872283936
10th percentile: 1.9653884887695312
20th percentile: 2.0359314918518066
30th percentile: 2.0908219814300537
40th percentile: 2.1300599575042725
50th percentile: 2.169297933578491
60th percentile: 2.4317534923553468
70th percentile: 2.694209051132202
80th percentile: 6.289575815200809
90th percentile: 13.217853784561159
95th percentile: 16.68199276924133
99th percentile: 19.453303956985472
mean time: 5.821382999420166
%s, retrying in %s seconds...
Received healthy response to inference request in 1.8750927448272705s
Received healthy response to inference request in 2.0983800888061523s
Received healthy response to inference request in 2.110262155532837s
Received healthy response to inference request in 1.9589767456054688s
Received healthy response to inference request in 2.3510749340057373s
5 requests
0 failed requests
5th percentile: 1.8918695449829102
10th percentile: 1.9086463451385498
20th percentile: 1.942199945449829
30th percentile: 1.9868574142456055
40th percentile: 2.042618751525879
50th percentile: 2.0983800888061523
60th percentile: 2.1031329154968263
70th percentile: 2.1078857421875
80th percentile: 2.158424711227417
90th percentile: 2.254749822616577
95th percentile: 2.302912378311157
99th percentile: 2.3414424228668214
mean time: 2.078757333755493
Pipeline stage StressChecker completed in 42.72s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.71s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.71s
Shutdown handler de-registered
chaiml-gy-exp85-simpo-e_54434_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 6049.33s
Shutdown handler de-registered
chaiml-gy-exp85-simpo-e_54434_v1 status is now inactive due to auto deactivation removed underperforming models
chaiml-gy-exp85-simpo-e_54434_v1 status is now torndown due to DeploymentManager action