chaiml-nis-proj1-exp1-sf_5836

developer_uid: NischayDnk

submission_id: chaiml-nis-proj1-exp1-sf_5836_v2

model_name: chaiml-nis-proj1-exp1-sf_5836_v2

model_group: ChaiML/nis-proj1-exp1-sf

status: torndown

timestamp: 2025-03-27T07:19:57+00:00

num_battles: 6873

num_wins: 3391

celo_rating: 1276.29

family_friendly_score: 0.5682

family_friendly_standard_error: 0.0070049805138915275

submission_type: basic

model_repo: ChaiML/nis-proj1-exp1-sft-prev1-type1-deltop5k-try1

model_architecture: MistralForCausalLM

model_num_parameters: 24096691200.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

reward_model: default

latencies: [{'batch_size': 1, 'throughput': 0.37279933549255106, 'latency_mean': 2.682305599451065, 'latency_p50': 2.645002841949463, 'latency_p90': 3.0106319904327394}, {'batch_size': 2, 'throughput': 0.5799513832568779, 'latency_mean': 3.4429695427417757, 'latency_p50': 3.458976984024048, 'latency_p90': 3.7974451780319214}, {'batch_size': 3, 'throughput': 0.7429162797473408, 'latency_mean': 4.025282552242279, 'latency_p50': 4.025515079498291, 'latency_p90': 4.428939604759217}, {'batch_size': 4, 'throughput': 0.8574088688368877, 'latency_mean': 4.646564722061157, 'latency_p50': 4.664424300193787, 'latency_p90': 5.154303407669067}, {'batch_size': 5, 'throughput': 0.9479864794741075, 'latency_mean': 5.235447856187821, 'latency_p50': 5.210162878036499, 'latency_p90': 5.939378190040588}]

gpu_counts: {'NVIDIA RTX A6000': 1}

display_name: chaiml-nis-proj1-exp1-sf_5836_v2

is_internal_developer: False

language_model: ChaiML/nis-proj1-exp1-sft-prev1-type1-deltop5k-try1

model_size: 24B

ranking_group: single

throughput_3p7s: 0.66

us_pacific_date: 2025-03-27

win_ratio: 0.49337989233231483

generation_params: {'temperature': 0.6, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.45, 'frequency_penalty': 0.45, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer
Waiting for job on chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer to finish
chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer: ║     _____            __           __                                ║
chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer: ║       /___/                                                         ║
chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer: ║                                                                     ║
chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer: ║  Version: 0.12.8                                                    ║
chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer: ║                                                                     ║
chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer: ║  belonging to:                                                      ║
chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer: ║                                                                     ║
chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer: ║  Expiration: 2025-04-15 23:59:59                                    ║
chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer: ║                                                                     ║
chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer: Downloaded to shared memory in 71.257s
chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpz79qgs7r, device:0
chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer: quantized model in 53.556s
chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer: Processed model ChaiML/nis-proj1-exp1-sft-prev1-type1-deltop5k-try1 in 124.814s
chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer: creating bucket guanaco-mkml-models
chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-nis-proj1-exp1-sf-5836-v2
chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-nis-proj1-exp1-sf-5836-v2/config.json
chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-nis-proj1-exp1-sf-5836-v2/special_tokens_map.json
chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-nis-proj1-exp1-sf-5836-v2/tokenizer_config.json
chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-nis-proj1-exp1-sf-5836-v2/tokenizer.json
chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/chaiml-nis-proj1-exp1-sf-5836-v2/flywheel_model.1.safetensors
chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-nis-proj1-exp1-sf-5836-v2/flywheel_model.0.safetensors
chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:17, 20.53it/s]
Loading 0:   3%|▎         | 12/363 [00:00<00:09, 36.96it/s]
Loading 0:   5%|▍         | 17/363 [00:00<00:10, 33.02it/s]
Loading 0:   6%|▌         | 21/363 [00:00<00:10, 34.14it/s]
Loading 0:   7%|▋         | 25/363 [00:00<00:11, 30.41it/s]
Loading 0:   9%|▉         | 32/363 [00:00<00:09, 36.65it/s]
Loading 0:  10%|▉         | 36/363 [00:01<00:14, 21.80it/s]
Loading 0:  11%|█         | 40/363 [00:01<00:13, 24.46it/s]
Loading 0:  12%|█▏        | 44/363 [00:01<00:12, 25.72it/s]
Loading 0:  13%|█▎        | 48/363 [00:01<00:11, 27.79it/s]
Loading 0:  14%|█▍        | 52/363 [00:01<00:11, 26.02it/s]
Loading 0:  16%|█▌        | 57/363 [00:02<00:10, 29.43it/s]
Loading 0:  17%|█▋        | 61/363 [00:02<00:11, 26.90it/s]
Loading 0:  18%|█▊        | 65/363 [00:02<00:10, 27.48it/s]
Loading 0:  19%|█▉        | 70/363 [00:02<00:11, 24.84it/s]
Loading 0:  20%|██        | 73/363 [00:02<00:13, 21.11it/s]
Loading 0:  22%|██▏       | 79/363 [00:02<00:10, 26.72it/s]
Loading 0:  23%|██▎       | 82/363 [00:03<00:10, 26.04it/s]
Loading 0:  24%|██▎       | 86/363 [00:03<00:09, 27.75it/s]
Loading 0:  25%|██▍       | 89/363 [00:03<00:10, 27.05it/s]
Loading 0:  25%|██▌       | 92/363 [00:03<00:12, 21.25it/s]
Loading 0:  27%|██▋       | 99/363 [00:03<00:09, 28.64it/s]
Loading 0:  28%|██▊       | 103/363 [00:03<00:09, 27.18it/s]
Loading 0:  29%|██▉       | 107/363 [00:04<00:11, 22.98it/s]
Loading 0:  31%|███       | 111/363 [00:04<00:09, 26.13it/s]
Loading 0:  31%|███▏      | 114/363 [00:04<00:10, 23.45it/s]
Loading 0:  33%|███▎      | 120/363 [00:04<00:08, 28.46it/s]
Loading 0:  34%|███▍      | 124/363 [00:04<00:08, 27.41it/s]
Loading 0:  36%|███▌      | 129/363 [00:04<00:07, 30.17it/s]
Loading 0:  37%|███▋      | 133/363 [00:04<00:08, 28.11it/s]
Loading 0:  38%|███▊      | 138/363 [00:05<00:07, 30.22it/s]
Loading 0:  39%|███▉      | 142/363 [00:05<00:07, 27.79it/s]
Loading 0:  40%|████      | 147/363 [00:05<00:06, 32.07it/s]
Loading 0:  42%|████▏     | 151/363 [00:05<00:08, 24.72it/s]
Loading 0:  42%|████▏     | 154/363 [00:05<00:09, 22.77it/s]
Loading 0:  43%|████▎     | 157/363 [00:05<00:08, 23.07it/s]
Loading 0:  44%|████▍     | 160/363 [00:06<00:08, 23.81it/s]
Loading 0:  45%|████▌     | 165/363 [00:06<00:07, 27.55it/s]
Loading 0:  46%|████▋     | 168/363 [00:06<00:07, 25.12it/s]
Loading 0:  48%|████▊     | 174/363 [00:06<00:06, 30.41it/s]
Loading 0:  49%|████▉     | 178/363 [00:06<00:06, 28.81it/s]
Loading 0:  50%|█████     | 182/363 [00:06<00:06, 29.00it/s]
Loading 0:  52%|█████▏    | 187/363 [00:06<00:06, 26.16it/s]
Loading 0:  52%|█████▏    | 190/363 [00:07<00:07, 23.76it/s]
Loading 0:  53%|█████▎    | 193/363 [00:07<00:07, 24.21it/s]
Loading 0:  54%|█████▍    | 196/363 [00:07<00:06, 24.66it/s]
Loading 0:  55%|█████▌    | 200/363 [00:21<00:06, 24.66it/s]
Loading 0:  55%|█████▌    | 201/363 [00:21<03:00,  1.11s/it]
Loading 0:  56%|█████▌    | 203/363 [00:21<02:29,  1.07it/s]
Loading 0:  57%|█████▋    | 208/363 [00:22<01:29,  1.72it/s]
Loading 0:  58%|█████▊    | 211/363 [00:22<01:08,  2.20it/s]
Loading 0:  59%|█████▉    | 214/363 [00:22<00:51,  2.91it/s]
Loading 0:  60%|██████    | 218/363 [00:22<00:34,  4.16it/s]
Loading 0:  61%|██████    | 221/363 [00:22<00:26,  5.34it/s]
Loading 0:  62%|██████▏   | 224/363 [00:22<00:21,  6.44it/s]
Loading 0:  63%|██████▎   | 228/363 [00:22<00:15,  9.00it/s]
Loading 0:  64%|██████▎   | 231/363 [00:23<00:12, 10.27it/s]
Loading 0:  65%|██████▌   | 237/363 [00:23<00:08, 15.25it/s]
Loading 0:  66%|██████▌   | 240/363 [00:23<00:07, 16.01it/s]
Loading 0:  68%|██████▊   | 246/363 [00:23<00:05, 21.54it/s]
Loading 0:  69%|██████▉   | 250/363 [00:23<00:05, 22.33it/s]
Loading 0:  70%|███████   | 255/363 [00:23<00:04, 25.51it/s]
Loading 0:  71%|███████▏  | 259/363 [00:24<00:04, 24.85it/s]
Loading 0:  73%|███████▎  | 265/363 [00:24<00:03, 31.68it/s]
Loading 0:  74%|███████▍  | 269/363 [00:24<00:04, 22.77it/s]
Loading 0:  75%|███████▌  | 273/363 [00:24<00:03, 22.57it/s]
Loading 0:  76%|███████▌  | 276/363 [00:24<00:04, 21.29it/s]
Loading 0:  78%|███████▊  | 282/363 [00:24<00:03, 26.05it/s]
Loading 0:  79%|███████▊  | 285/363 [00:25<00:03, 23.47it/s]
Loading 0:  80%|████████  | 291/363 [00:25<00:02, 27.94it/s]
Loading 0:  81%|████████▏ | 295/363 [00:25<00:02, 26.89it/s]
Loading 0:  82%|████████▏ | 299/363 [00:25<00:02, 27.75it/s]
Loading 0:  84%|████████▎ | 304/363 [00:25<00:02, 24.14it/s]
Loading 0:  85%|████████▍ | 307/363 [00:25<00:02, 22.40it/s]
Loading 0:  85%|████████▌ | 310/363 [00:26<00:02, 23.43it/s]
Loading 0:  86%|████████▌ | 313/363 [00:26<00:02, 23.69it/s]
Loading 0:  88%|████████▊ | 318/363 [00:26<00:01, 27.54it/s]
Loading 0:  88%|████████▊ | 321/363 [00:26<00:01, 24.93it/s]
Loading 0:  90%|█████████ | 327/363 [00:26<00:01, 29.86it/s]
Loading 0:  91%|█████████ | 331/363 [00:26<00:01, 28.28it/s]
Loading 0:  92%|█████████▏| 335/363 [00:26<00:00, 28.84it/s]
Loading 0:  93%|█████████▎| 338/363 [00:27<00:00, 27.73it/s]
Loading 0:  94%|█████████▍| 341/363 [00:33<00:13,  1.67it/s]
Loading 0:  96%|█████████▌| 347/363 [00:34<00:05,  2.79it/s]
Loading 0:  96%|█████████▋| 350/363 [00:34<00:03,  3.51it/s]
Loading 0:  98%|█████████▊| 355/363 [00:34<00:01,  5.19it/s]
Loading 0:  99%|█████████▉| 359/363 [00:34<00:00,  6.70it/s]
                                                            
Job chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer completed after 155.75s with status: succeeded
Stopping job with name chaiml-nis-proj1-exp1-sf-5836-v2-mkmlizer
Pipeline stage MKMLizer completed in 156.22s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-nis-proj1-exp1-sf-5836-v2
Waiting for inference service chaiml-nis-proj1-exp1-sf-5836-v2 to be ready
Inference service chaiml-nis-proj1-exp1-sf-5836-v2 ready after 110.404301404953s
Pipeline stage MKMLDeployer completed in 110.84s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 3.287393093109131s
Received healthy response to inference request in 2.4938547611236572s
Received healthy response to inference request in 2.2495672702789307s
Received healthy response to inference request in 2.4934921264648438s
5 requests
1 failed requests
5th percentile: 2.2983522415161133
10th percentile: 2.347137212753296
20th percentile: 2.444707155227661
30th percentile: 2.4935646533966063
40th percentile: 2.4937097072601317
50th percentile: 2.4938547611236572
60th percentile: 2.8112700939178468
70th percentile: 3.128685426712036
80th percentile: 6.657056045532229
90th percentile: 13.39638195037842
95th percentile: 16.76604490280151
99th percentile: 19.46177526473999
mean time: 6.1320030212402346
%s, retrying in %s seconds...
Received healthy response to inference request in 2.509673595428467s
Received healthy response to inference request in 2.7334184646606445s
Received healthy response to inference request in 2.499898910522461s
Received healthy response to inference request in 2.141940116882324s
Received healthy response to inference request in 2.6196203231811523s
5 requests
0 failed requests
5th percentile: 2.2135318756103515
10th percentile: 2.2851236343383787
20th percentile: 2.4283071517944337
30th percentile: 2.501853847503662
40th percentile: 2.5057637214660646
50th percentile: 2.509673595428467
60th percentile: 2.553652286529541
70th percentile: 2.5976309776306152
80th percentile: 2.642379951477051
90th percentile: 2.6878992080688477
95th percentile: 2.710658836364746
99th percentile: 2.728866539001465
mean time: 2.5009102821350098
Pipeline stage StressChecker completed in 46.07s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.66s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.70s
Shutdown handler de-registered
chaiml-nis-proj1-exp1-sf_5836_v2 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
%s, retrying in %s seconds...
%s, retrying in %s seconds...
%s, retrying in %s seconds...
%s, retrying in %s seconds...
%s, retrying in %s seconds...
%s, retrying in %s seconds...
%s, retrying in %s seconds...
Received signal 15, running shutdown handler
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2682.72s
Shutdown handler de-registered
chaiml-nis-proj1-exp1-sf_5836_v2 status is now inactive due to auto deactivation removed underperforming models
chaiml-nis-proj1-exp1-sf_5836_v2 status is now torndown due to DeploymentManager action