chaiml-nis-proj1-exp1-sf_5836

developer_uid: NischayDnk

submission_id: chaiml-nis-proj1-exp1-sf_5836_v4

model_name: chaiml-nis-proj1-exp1-sf_5836_v4

model_group: ChaiML/nis-proj1-exp1-sf

status: torndown

timestamp: 2025-03-28T17:51:12+00:00

num_battles: 7824

num_wins: 3948

celo_rating: 1283.64

family_friendly_score: 0.5628

family_friendly_standard_error: 0.007015071774401171

submission_type: basic

model_repo: ChaiML/nis-proj1-exp1-sft-prev1-type1-deltop5k-try1

model_architecture: MistralForCausalLM

model_num_parameters: 24096691200.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

reward_model: default

latencies: [{'batch_size': 1, 'throughput': 0.37087689759389775, 'latency_mean': 2.6962174022197725, 'latency_p50': 2.688431978225708, 'latency_p90': 2.9825007200241087}, {'batch_size': 2, 'throughput': 0.576304327394759, 'latency_mean': 3.4641168999671934, 'latency_p50': 3.4969812631607056, 'latency_p90': 3.8332935094833376}, {'batch_size': 3, 'throughput': 0.7298839073880684, 'latency_mean': 4.090352638959884, 'latency_p50': 4.066642761230469, 'latency_p90': 4.5328626871109}, {'batch_size': 4, 'throughput': 0.8538032386552117, 'latency_mean': 4.661990917921067, 'latency_p50': 4.6471192836761475, 'latency_p90': 5.194668936729431}, {'batch_size': 5, 'throughput': 0.9295348694546334, 'latency_mean': 5.333283708095551, 'latency_p50': 5.357381224632263, 'latency_p90': 5.930511093139648}]

gpu_counts: {'NVIDIA RTX A6000': 1}

display_name: chaiml-nis-proj1-exp1-sf_5836_v4

is_internal_developer: False

language_model: ChaiML/nis-proj1-exp1-sft-prev1-type1-deltop5k-try1

model_size: 24B

ranking_group: single

throughput_3p7s: 0.64

us_pacific_date: 2025-03-28

win_ratio: 0.504601226993865

generation_params: {'temperature': 0.45, 'top_p': 0.9, 'min_p': 0.05, 'top_k': 40, 'presence_penalty': 0.4, 'frequency_penalty': 0.4, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': True}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer
Waiting for job on chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer to finish
chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer: ║     _____            __           __                                ║
chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer: ║       /___/                                                         ║
chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer: ║                                                                     ║
chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer: ║  Version: 0.12.8                                                    ║
chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer: ║                                                                     ║
chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer: ║  belonging to:                                                      ║
chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer: ║                                                                     ║
chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer: ║  Expiration: 2025-04-15 23:59:59                                    ║
chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer: ║                                                                     ║
chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer: Downloaded to shared memory in 65.270s
chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp5whj9pid, device:0
chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer: quantized model in 54.524s
chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer: Processed model ChaiML/nis-proj1-exp1-sft-prev1-type1-deltop5k-try1 in 119.794s
chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer: creating bucket guanaco-mkml-models
chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-nis-proj1-exp1-sf-5836-v4
chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-nis-proj1-exp1-sf-5836-v4/config.json
chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-nis-proj1-exp1-sf-5836-v4/special_tokens_map.json
chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-nis-proj1-exp1-sf-5836-v4/tokenizer_config.json
chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-nis-proj1-exp1-sf-5836-v4/tokenizer.json
chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/chaiml-nis-proj1-exp1-sf-5836-v4/flywheel_model.1.safetensors
chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-nis-proj1-exp1-sf-5836-v4/flywheel_model.0.safetensors
chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|          | 4/363 [00:00<00:09, 36.88it/s]
Loading 0:   2%|▏         | 8/363 [00:00<00:12, 29.36it/s]
Loading 0:   3%|▎         | 12/363 [00:00<00:10, 32.26it/s]
Loading 0:   4%|▍         | 16/363 [00:00<00:12, 28.91it/s]
Loading 0:   6%|▌         | 21/363 [00:00<00:10, 33.66it/s]
Loading 0:   7%|▋         | 25/363 [00:00<00:11, 30.43it/s]
Loading 0:   9%|▉         | 32/363 [00:00<00:08, 37.28it/s]
Loading 0:  10%|▉         | 36/363 [00:01<00:15, 21.46it/s]
Loading 0:  11%|█         | 40/363 [00:01<00:13, 24.08it/s]
Loading 0:  12%|█▏        | 44/363 [00:01<00:12, 25.41it/s]
Loading 0:  13%|█▎        | 48/363 [00:01<00:11, 27.59it/s]
Loading 0:  14%|█▍        | 52/363 [00:01<00:11, 26.23it/s]
Loading 0:  16%|█▌        | 57/363 [00:02<00:10, 29.16it/s]
Loading 0:  17%|█▋        | 61/363 [00:02<00:10, 27.67it/s]
Loading 0:  18%|█▊        | 65/363 [00:02<00:10, 28.66it/s]
Loading 0:  19%|█▉        | 70/363 [00:02<00:11, 25.61it/s]
Loading 0:  20%|██        | 73/363 [00:02<00:14, 20.47it/s]
Loading 0:  22%|██▏       | 79/363 [00:02<00:10, 25.97it/s]
Loading 0:  23%|██▎       | 82/363 [00:03<00:10, 25.70it/s]
Loading 0:  24%|██▎       | 86/363 [00:03<00:09, 27.80it/s]
Loading 0:  25%|██▍       | 90/363 [00:03<00:10, 27.00it/s]
Loading 0:  26%|██▌       | 93/363 [00:03<00:10, 24.62it/s]
Loading 0:  27%|██▋       | 99/363 [00:03<00:09, 29.27it/s]
Loading 0:  28%|██▊       | 103/363 [00:03<00:09, 26.94it/s]
Loading 0:  29%|██▉       | 107/363 [00:04<00:11, 23.08it/s]
Loading 0:  31%|███       | 111/363 [00:04<00:09, 26.14it/s]
Loading 0:  31%|███▏      | 114/363 [00:04<00:10, 22.96it/s]
Loading 0:  33%|███▎      | 120/363 [00:04<00:08, 27.56it/s]
Loading 0:  34%|███▍      | 123/363 [00:04<00:09, 24.74it/s]
Loading 0:  36%|███▌      | 129/363 [00:04<00:07, 29.57it/s]
Loading 0:  37%|███▋      | 133/363 [00:04<00:08, 27.89it/s]
Loading 0:  38%|███▊      | 138/363 [00:05<00:07, 30.05it/s]
Loading 0:  39%|███▉      | 142/363 [00:05<00:07, 28.33it/s]
Loading 0:  41%|████      | 148/363 [00:05<00:06, 34.93it/s]
Loading 0:  42%|████▏     | 152/363 [00:05<00:08, 24.15it/s]
Loading 0:  43%|████▎     | 156/363 [00:05<00:08, 23.60it/s]
Loading 0:  44%|████▍     | 159/363 [00:05<00:09, 22.58it/s]
Loading 0:  45%|████▌     | 165/363 [00:06<00:07, 27.71it/s]
Loading 0:  47%|████▋     | 169/363 [00:06<00:07, 27.24it/s]
Loading 0:  48%|████▊     | 174/363 [00:06<00:06, 29.71it/s]
Loading 0:  49%|████▉     | 178/363 [00:06<00:06, 28.23it/s]
Loading 0:  50%|█████     | 182/363 [00:06<00:06, 28.37it/s]
Loading 0:  52%|█████▏    | 187/363 [00:06<00:06, 26.00it/s]
Loading 0:  52%|█████▏    | 190/363 [00:07<00:07, 23.67it/s]
Loading 0:  53%|█████▎    | 193/363 [00:07<00:06, 24.34it/s]
Loading 0:  54%|█████▍    | 196/363 [00:07<00:06, 25.06it/s]
Loading 0:  55%|█████▌    | 200/363 [00:21<00:06, 25.06it/s]
Loading 0:  55%|█████▌    | 201/363 [00:21<03:02,  1.13s/it]
Loading 0:  56%|█████▌    | 203/363 [00:22<02:31,  1.06it/s]
Loading 0:  57%|█████▋    | 208/363 [00:22<01:31,  1.70it/s]
Loading 0:  58%|█████▊    | 211/363 [00:22<01:10,  2.17it/s]
Loading 0:  59%|█████▉    | 214/363 [00:22<00:52,  2.85it/s]
Loading 0:  60%|██████    | 218/363 [00:22<00:35,  4.08it/s]
Loading 0:  61%|██████    | 221/363 [00:22<00:27,  5.17it/s]
Loading 0:  62%|██████▏   | 224/363 [00:23<00:22,  6.15it/s]
Loading 0:  63%|██████▎   | 228/363 [00:23<00:15,  8.64it/s]
Loading 0:  64%|██████▎   | 231/363 [00:23<00:13,  9.94it/s]
Loading 0:  65%|██████▌   | 237/363 [00:23<00:08, 14.72it/s]
Loading 0:  66%|██████▌   | 240/363 [00:23<00:08, 15.36it/s]
Loading 0:  68%|██████▊   | 246/363 [00:23<00:05, 20.41it/s]
Loading 0:  69%|██████▊   | 249/363 [00:24<00:05, 20.20it/s]
Loading 0:  70%|███████   | 255/363 [00:24<00:04, 25.45it/s]
Loading 0:  71%|███████▏  | 259/363 [00:24<00:04, 24.47it/s]
Loading 0:  72%|███████▏  | 262/363 [00:24<00:03, 25.48it/s]
Loading 0:  74%|███████▎  | 267/363 [00:24<00:04, 21.42it/s]
Loading 0:  74%|███████▍  | 270/363 [00:24<00:04, 18.65it/s]
Loading 0:  75%|███████▌  | 274/363 [00:25<00:04, 21.78it/s]
Loading 0:  76%|███████▋  | 277/363 [00:25<00:03, 22.33it/s]
Loading 0:  78%|███████▊  | 282/363 [00:25<00:03, 25.33it/s]
Loading 0:  79%|███████▊  | 285/363 [00:25<00:03, 22.95it/s]
Loading 0:  80%|███████▉  | 289/363 [00:25<00:02, 26.22it/s]
Loading 0:  80%|████████  | 292/363 [00:25<00:02, 26.18it/s]
Loading 0:  81%|████████▏ | 295/363 [00:25<00:02, 25.41it/s]
Loading 0:  82%|████████▏ | 299/363 [00:26<00:02, 26.50it/s]
Loading 0:  84%|████████▎ | 304/363 [00:26<00:02, 24.13it/s]
Loading 0:  85%|████████▍ | 307/363 [00:26<00:02, 21.98it/s]
Loading 0:  85%|████████▌ | 310/363 [00:26<00:02, 23.00it/s]
Loading 0:  86%|████████▌ | 313/363 [00:26<00:02, 23.70it/s]
Loading 0:  88%|████████▊ | 318/363 [00:26<00:01, 26.77it/s]
Loading 0:  88%|████████▊ | 321/363 [00:26<00:01, 23.73it/s]
Loading 0:  90%|████████▉ | 325/363 [00:27<00:01, 27.09it/s]
Loading 0:  90%|█████████ | 328/363 [00:27<00:01, 26.65it/s]
Loading 0:  91%|█████████ | 331/363 [00:27<00:01, 26.06it/s]
Loading 0:  92%|█████████▏| 335/363 [00:27<00:01, 26.69it/s]
Loading 0:  93%|█████████▎| 338/363 [00:27<00:00, 25.28it/s]
Loading 0:  94%|█████████▍| 341/363 [00:34<00:14,  1.52it/s]
Loading 0:  95%|█████████▌| 346/363 [00:34<00:06,  2.47it/s]
Loading 0:  96%|█████████▌| 349/363 [00:34<00:04,  3.14it/s]
Loading 0:  98%|█████████▊| 355/363 [00:34<00:01,  5.18it/s]
Loading 0:  99%|█████████▉| 359/363 [00:35<00:00,  6.62it/s]
                                                            
Job chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer completed after 155.59s with status: succeeded
Stopping job with name chaiml-nis-proj1-exp1-sf-5836-v4-mkmlizer
Pipeline stage MKMLizer completed in 156.02s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-nis-proj1-exp1-sf-5836-v4
Waiting for inference service chaiml-nis-proj1-exp1-sf-5836-v4 to be ready
Inference service chaiml-nis-proj1-exp1-sf-5836-v4 ready after 110.37689280509949s
Pipeline stage MKMLDeployer completed in 110.81s
run pipeline stage %s
Running pipeline stage StressChecker
Failed to get response for submission jellywibble-hot-kidnapp_68276_v1: HTTPConnectionPool(host='jellywibble-hot-kidnapp-68276-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 3.174520492553711s
Received healthy response to inference request in 2.714529275894165s
Received healthy response to inference request in 2.3164303302764893s
Received healthy response to inference request in 2.25009822845459s
5 requests
1 failed requests
5th percentile: 2.26336464881897
10th percentile: 2.27663106918335
20th percentile: 2.3031639099121093
30th percentile: 2.3960501194000243
40th percentile: 2.555289697647095
50th percentile: 2.714529275894165
60th percentile: 2.898525762557983
70th percentile: 3.082522249221802
80th percentile: 6.568522739410403
90th percentile: 13.356527233123781
95th percentile: 16.750529479980464
99th percentile: 19.46573127746582
mean time: 6.120022010803223
%s, retrying in %s seconds...
Received healthy response to inference request in 2.6048078536987305s
Received healthy response to inference request in 2.1909661293029785s
Received healthy response to inference request in 2.3283205032348633s
Received healthy response to inference request in 2.1348226070404053s
Received healthy response to inference request in 2.1898934841156006s
5 requests
0 failed requests
5th percentile: 2.1458367824554445
10th percentile: 2.1568509578704833
20th percentile: 2.1788793087005613
30th percentile: 2.1901080131530763
40th percentile: 2.1905370712280274
50th percentile: 2.1909661293029785
60th percentile: 2.2459078788757325
70th percentile: 2.3008496284484865
80th percentile: 2.383617973327637
90th percentile: 2.4942129135131834
95th percentile: 2.549510383605957
99th percentile: 2.5937483596801756
mean time: 2.2897621154785157
Pipeline stage StressChecker completed in 44.74s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.67s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.71s
Shutdown handler de-registered
chaiml-nis-proj1-exp1-sf_5836_v4 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 5002.92s
Shutdown handler de-registered
chaiml-nis-proj1-exp1-sf_5836_v4 status is now inactive due to auto deactivation removed underperforming models
chaiml-nis-proj1-exp1-sf_5836_v4 status is now torndown due to DeploymentManager action