developer_uid: chai_backend_admin
submission_id: chaiml-remy-lebeau-gamb_99749_v1
model_name: chaiml-remy-lebeau-gamb_99749_v1
model_group: ChaiML/Remy-LeBeau-Gambi
status: torndown
timestamp: 2025-07-07T22:22:34+00:00
num_battles: 6577
num_wins: 3498
celo_rating: 1309.78
family_friendly_score: 0.5078
family_friendly_standard_error: 0.007070207351980563
submission_type: basic
model_repo: ChaiML/Remy-LeBeau-Gambit250707133221_sft
model_architecture: MistralForCausalLM
model_num_parameters: 24096691200.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
reward_model: default
latencies: [{'batch_size': 1, 'throughput': 0.5288491343900177, 'latency_mean': 1.8907992267608642, 'latency_p50': 1.8794877529144287, 'latency_p90': 2.094868803024292}, {'batch_size': 3, 'throughput': 1.0590810995622348, 'latency_mean': 2.822960557937622, 'latency_p50': 2.8394936323165894, 'latency_p90': 3.0902799129486085}, {'batch_size': 5, 'throughput': 1.3439895090027898, 'latency_mean': 3.697134956121445, 'latency_p50': 3.6599847078323364, 'latency_p90': 4.117578864097595}, {'batch_size': 6, 'throughput': 1.4519587622641985, 'latency_mean': 4.103821303844452, 'latency_p50': 4.101654767990112, 'latency_p90': 4.592072463035583}, {'batch_size': 8, 'throughput': 1.5801816960072876, 'latency_mean': 5.016381112337112, 'latency_p50': 5.038875341415405, 'latency_p90': 5.589112114906311}, {'batch_size': 10, 'throughput': 1.6594752973828093, 'latency_mean': 5.969505922794342, 'latency_p50': 5.966747045516968, 'latency_p90': 6.702977561950684}]
gpu_counts: {'NVIDIA A100-SXM4-80GB': 1}
display_name: chaiml-remy-lebeau-gamb_99749_v1
is_internal_developer: True
language_model: ChaiML/Remy-LeBeau-Gambit250707133221_sft
model_size: 24B
ranking_group: single
throughput_3p7s: 1.35
us_pacific_date: 2025-07-07
win_ratio: 0.53185342861487
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['</s>', '####', '\n', 'You:', '####\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}</s>\n', 'user_template': 'You: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-remy-lebeau-gamb-99749-v1-mkmlizer
Waiting for job on chaiml-remy-lebeau-gamb-99749-v1-mkmlizer to finish
Retrying (%r) after connection broken by '%r': %s
chaiml-remy-lebeau-gamb-99749-v1-mkmlizer: Downloaded to shared memory in 207.412s
chaiml-remy-lebeau-gamb-99749-v1-mkmlizer: Checking if ChaiML/Remy-LeBeau-Gambit250707133221_sft already exists in ChaiML
chaiml-remy-lebeau-gamb-99749-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpwhkuev0v, device:0
chaiml-remy-lebeau-gamb-99749-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-remy-lebeau-gamb-99749-v1-mkmlizer: quantized model in 47.708s
chaiml-remy-lebeau-gamb-99749-v1-mkmlizer: Processed model ChaiML/Remy-LeBeau-Gambit250707133221_sft in 255.121s
chaiml-remy-lebeau-gamb-99749-v1-mkmlizer: creating bucket guanaco-mkml-models
chaiml-remy-lebeau-gamb-99749-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-remy-lebeau-gamb-99749-v1/nvidia/config.json
chaiml-remy-lebeau-gamb-99749-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-remy-lebeau-gamb-99749-v1/nvidia/special_tokens_map.json
chaiml-remy-lebeau-gamb-99749-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-remy-lebeau-gamb-99749-v1/nvidia/tokenizer_config.json
chaiml-remy-lebeau-gamb-99749-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-remy-lebeau-gamb-99749-v1/nvidia/tokenizer.json
chaiml-remy-lebeau-gamb-99749-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/chaiml-remy-lebeau-gamb-99749-v1/nvidia/flywheel_model.1.safetensors
chaiml-remy-lebeau-gamb-99749-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-remy-lebeau-gamb-99749-v1/nvidia/flywheel_model.0.safetensors
chaiml-remy-lebeau-gamb-99749-v1-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|          | 4/363 [00:00<00:09, 39.11it/s]
Loading 0:   2%|▏         | 8/363 [00:00<00:11, 30.72it/s]
Loading 0:   3%|▎         | 12/363 [00:00<00:10, 32.28it/s]
Loading 0:   4%|▍         | 16/363 [00:00<00:12, 28.09it/s]
Loading 0:   6%|▌         | 21/363 [00:00<00:10, 31.59it/s]
Loading 0:   7%|▋         | 25/363 [00:00<00:12, 27.96it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:09, 35.37it/s]
Loading 0:  10%|▉         | 35/363 [00:01<00:14, 22.14it/s]
Loading 0:  10%|█         | 38/363 [00:01<00:14, 22.79it/s]
Loading 0:  11%|█▏        | 41/363 [00:01<00:16, 19.72it/s]
Loading 0:  13%|█▎        | 48/363 [00:01<00:11, 26.89it/s]
Loading 0:  14%|█▍        | 52/363 [00:01<00:12, 25.20it/s]
Loading 0:  16%|█▌        | 57/363 [00:02<00:10, 28.69it/s]
Loading 0:  17%|█▋        | 61/363 [00:02<00:10, 27.52it/s]
Loading 0:  18%|█▊        | 65/363 [00:02<00:10, 28.40it/s]
Loading 0:  19%|█▉        | 70/363 [00:02<00:11, 25.44it/s]
Loading 0:  20%|██        | 73/363 [00:02<00:13, 21.63it/s]
Loading 0:  22%|██▏       | 79/363 [00:02<00:10, 26.96it/s]
Loading 0:  23%|██▎       | 82/363 [00:03<00:10, 25.64it/s]
Loading 0:  24%|██▎       | 86/363 [00:03<00:10, 27.68it/s]
Loading 0:  25%|██▍       | 89/363 [00:03<00:09, 27.81it/s]
Loading 0:  25%|██▌       | 92/363 [00:03<00:12, 21.74it/s]
Loading 0:  27%|██▋       | 99/363 [00:03<00:09, 29.14it/s]
Loading 0:  28%|██▊       | 103/363 [00:03<00:09, 26.92it/s]
Loading 0:  29%|██▉       | 107/363 [00:04<00:11, 22.74it/s]
Loading 0:  31%|███       | 112/363 [00:04<00:09, 25.69it/s]
Loading 0:  32%|███▏      | 115/363 [00:04<00:09, 25.33it/s]
Loading 0:  33%|███▎      | 120/363 [00:04<00:08, 28.10it/s]
Loading 0:  34%|███▍      | 123/363 [00:04<00:09, 25.10it/s]
Loading 0:  36%|███▌      | 129/363 [00:04<00:07, 29.97it/s]
Loading 0:  37%|███▋      | 133/363 [00:05<00:08, 27.95it/s]
Loading 0:  38%|███▊      | 138/363 [00:05<00:07, 29.92it/s]
Loading 0:  39%|███▉      | 142/363 [00:05<00:07, 27.97it/s]
Loading 0:  41%|████      | 149/363 [00:05<00:06, 33.35it/s]
Loading 0:  42%|████▏     | 153/363 [00:05<00:09, 21.14it/s]
Loading 0:  43%|████▎     | 157/363 [00:05<00:08, 23.63it/s]
Loading 0:  44%|████▍     | 160/363 [00:06<00:08, 23.83it/s]
Loading 0:  45%|████▌     | 165/363 [00:06<00:07, 27.19it/s]
Loading 0:  47%|████▋     | 169/363 [00:06<00:07, 26.53it/s]
Loading 0:  48%|████▊     | 174/363 [00:06<00:06, 29.10it/s]
Loading 0:  49%|████▉     | 178/363 [00:06<00:06, 27.89it/s]
Loading 0:  50%|█████     | 182/363 [00:06<00:06, 28.42it/s]
Loading 0:  52%|█████▏    | 187/363 [00:07<00:07, 24.58it/s]
Loading 0:  52%|█████▏    | 190/363 [00:07<00:07, 22.34it/s]
Loading 0:  53%|█████▎    | 193/363 [00:07<00:07, 23.28it/s]
Loading 0:  54%|█████▍    | 196/363 [00:07<00:07, 23.10it/s]
Loading 0:  55%|█████▌    | 200/363 [00:21<00:07, 23.10it/s]
Loading 0:  55%|█████▌    | 201/363 [00:21<03:00,  1.11s/it]
Loading 0:  56%|█████▌    | 203/363 [00:22<02:29,  1.07it/s]
Loading 0:  57%|█████▋    | 208/363 [00:22<01:30,  1.72it/s]
Loading 0:  58%|█████▊    | 211/363 [00:22<01:08,  2.21it/s]
Loading 0:  59%|█████▉    | 214/363 [00:22<00:51,  2.92it/s]
Loading 0:  60%|██████    | 218/363 [00:22<00:34,  4.17it/s]
Loading 0:  61%|██████    | 221/363 [00:22<00:26,  5.34it/s]
Loading 0:  62%|██████▏   | 224/363 [00:22<00:21,  6.38it/s]
Loading 0:  63%|██████▎   | 228/363 [00:23<00:15,  8.92it/s]
Loading 0:  64%|██████▎   | 231/363 [00:23<00:12, 10.35it/s]
Loading 0:  65%|██████▌   | 237/363 [00:23<00:08, 15.36it/s]
Loading 0:  66%|██████▌   | 240/363 [00:23<00:07, 16.08it/s]
Loading 0:  68%|██████▊   | 246/363 [00:23<00:05, 21.67it/s]
Loading 0:  69%|██████▉   | 250/363 [00:23<00:04, 22.78it/s]
Loading 0:  70%|███████   | 255/363 [00:23<00:04, 26.01it/s]
Loading 0:  71%|███████▏  | 259/363 [00:24<00:04, 25.80it/s]
Loading 0:  73%|███████▎  | 266/363 [00:24<00:03, 31.51it/s]
Loading 0:  74%|███████▍  | 270/363 [00:24<00:04, 20.99it/s]
Loading 0:  75%|███████▌  | 274/363 [00:24<00:03, 23.72it/s]
Loading 0:  77%|███████▋  | 278/363 [00:24<00:03, 24.86it/s]
Loading 0:  78%|███████▊  | 282/363 [00:25<00:03, 26.03it/s]
Loading 0:  79%|███████▊  | 285/363 [00:25<00:03, 23.75it/s]
Loading 0:  80%|████████  | 291/363 [00:25<00:02, 28.77it/s]
Loading 0:  81%|████████▏ | 295/363 [00:25<00:02, 26.80it/s]
Loading 0:  82%|████████▏ | 299/363 [00:25<00:02, 27.19it/s]
Loading 0:  83%|████████▎ | 303/363 [00:25<00:02, 29.65it/s]
Loading 0:  85%|████████▍ | 307/363 [00:26<00:02, 20.42it/s]
Loading 0:  85%|████████▌ | 310/363 [00:26<00:02, 21.76it/s]
Loading 0:  86%|████████▌ | 313/363 [00:26<00:02, 22.15it/s]
Loading 0:  88%|████████▊ | 318/363 [00:26<00:01, 25.21it/s]
Loading 0:  88%|████████▊ | 321/363 [00:26<00:01, 23.19it/s]
Loading 0:  90%|█████████ | 327/363 [00:26<00:01, 28.67it/s]
Loading 0:  91%|█████████ | 331/363 [00:26<00:01, 26.87it/s]
Loading 0:  92%|█████████▏| 335/363 [00:27<00:01, 26.54it/s]
Loading 0:  93%|█████████▎| 338/363 [00:27<00:01, 24.82it/s]
Loading 0:  94%|█████████▍| 341/363 [00:27<00:01, 15.12it/s]
Loading 0:  96%|█████████▌| 347/363 [00:27<00:00, 20.59it/s]
Loading 0:  96%|█████████▋| 350/363 [00:27<00:00, 21.18it/s]
Loading 0:  98%|█████████▊| 355/363 [00:28<00:00, 24.36it/s]
Loading 0:  99%|█████████▊| 358/363 [00:28<00:00, 22.49it/s]
                                                            

Job chaiml-remy-lebeau-gamb-99749-v1-mkmlizer completed after 289.48s with status: succeeded
Stopping job with name chaiml-remy-lebeau-gamb-99749-v1-mkmlizer
Pipeline stage MKMLizer completed in 289.97s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.17s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-remy-lebeau-gamb-99749-v1
Waiting for inference service chaiml-remy-lebeau-gamb-99749-v1 to be ready
Inference service chaiml-remy-lebeau-gamb-99749-v1 ready after 191.1465003490448s
Pipeline stage MKMLDeployer completed in 191.67s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.4876620769500732s
Received healthy response to inference request in 2.142826557159424s
Received healthy response to inference request in 2.0849311351776123s
Received healthy response to inference request in 1.9015169143676758s
5 requests
1 failed requests
5th percentile: 1.938199758529663
10th percentile: 1.9748826026916504
20th percentile: 2.048248291015625
30th percentile: 2.0965102195739744
40th percentile: 2.119668388366699
50th percentile: 2.142826557159424
60th percentile: 2.2807607650756836
70th percentile: 2.4186949729919434
80th percentile: 6.0179324150085485
90th percentile: 13.07847309112549
95th percentile: 16.608743429183956
99th percentile: 19.432959699630736
mean time: 5.751190090179444
%s, retrying in %s seconds...
Received healthy response to inference request in 2.19535756111145s
Received healthy response to inference request in 1.9275882244110107s
Received healthy response to inference request in 1.953991174697876s
Received healthy response to inference request in 2.246671438217163s
Received healthy response to inference request in 2.099865436553955s
5 requests
0 failed requests
5th percentile: 1.9328688144683839
10th percentile: 1.9381494045257568
20th percentile: 1.9487105846405028
30th percentile: 1.9831660270690918
40th percentile: 2.0415157318115233
50th percentile: 2.099865436553955
60th percentile: 2.138062286376953
70th percentile: 2.1762591361999513
80th percentile: 2.2056203365325926
90th percentile: 2.226145887374878
95th percentile: 2.2364086627960207
99th percentile: 2.244618883132935
mean time: 2.084694766998291
Pipeline stage StressChecker completed in 41.65s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.84s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.70s
Shutdown handler de-registered
chaiml-remy-lebeau-gamb_99749_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 6652.88s
Shutdown handler de-registered
chaiml-remy-lebeau-gamb_99749_v1 status is now inactive due to auto deactivation removed underperforming models
chaiml-remy-lebeau-gamb_99749_v1 status is now protected
chaiml-remy-lebeau-gamb_99749_v1 status is now inactive
chaiml-remy-lebeau-gamb_99749_v1 status is now torndown due to DeploymentManager action
chaiml-remy-lebeau-gamb_99749_v1 status is now torndown due to DeploymentManager action