rica40325-kto-1017

developer_uid: rica40325

submission_id: rica40325-kto-1017_v1

model_name: rica40325-kto-1017_v1

model_group: rica40325/kto_1017

status: torndown

timestamp: 2024-10-17T14:04:08+00:00

num_battles: 9605

num_wins: 5380

celo_rating: 1295.4

family_friendly_score: 0.5744714275153397

family_friendly_standard_error: 0.005178281823652083

submission_type: basic

model_repo: rica40325/kto_1017

model_architecture: MistralForCausalLM

model_num_parameters: 12772070400.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

display_name: rica40325-kto-1017_v1

is_internal_developer: False

language_model: rica40325/kto_1017

model_size: 13B

ranking_group: single

us_pacific_date: 2024-10-17

win_ratio: 0.5601249349297241

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name rica40325-kto-1017-v1-mkmlizer
Waiting for job on rica40325-kto-1017-v1-mkmlizer to finish
rica40325-kto-1017-v1-mkmlizer: quantized model in 41.601s
rica40325-kto-1017-v1-mkmlizer: Processed model rica40325/kto_1017 in 326.939s
rica40325-kto-1017-v1-mkmlizer: creating bucket guanaco-mkml-models
rica40325-kto-1017-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
rica40325-kto-1017-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/rica40325-kto-1017-v1
rica40325-kto-1017-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/rica40325-kto-1017-v1/config.json
rica40325-kto-1017-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/rica40325-kto-1017-v1/special_tokens_map.json
rica40325-kto-1017-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/rica40325-kto-1017-v1/tokenizer_config.json
rica40325-kto-1017-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/rica40325-kto-1017-v1/tokenizer.json
rica40325-kto-1017-v1-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:16, 21.96it/s]
Loading 0:   3%|▎         | 10/363 [00:00<00:12, 28.24it/s]
Loading 0:   4%|▍         | 14/363 [00:00<00:14, 24.26it/s]
Loading 0:   6%|▌         | 20/363 [00:00<00:10, 32.85it/s]
Loading 0:   7%|▋         | 24/363 [00:01<00:15, 21.59it/s]
Loading 0:   7%|▋         | 27/363 [00:01<00:15, 21.14it/s]
Loading 0:   9%|▊         | 31/363 [00:01<00:13, 24.70it/s]
Loading 0:   9%|▉         | 34/363 [00:01<00:12, 25.36it/s]
Loading 0:  11%|█         | 39/363 [00:01<00:11, 28.82it/s]
Loading 0:  12%|█▏        | 43/363 [00:01<00:11, 27.99it/s]
Loading 0:  13%|█▎        | 48/363 [00:01<00:10, 30.70it/s]
Loading 0:  14%|█▍        | 52/363 [00:01<00:10, 28.59it/s]
Loading 0:  15%|█▌        | 56/363 [00:02<00:10, 28.07it/s]
Loading 0:  17%|█▋        | 61/363 [00:02<00:12, 24.53it/s]
Loading 0:  18%|█▊        | 64/363 [00:02<00:13, 21.76it/s]
Loading 0:  20%|█▉        | 71/363 [00:02<00:10, 29.07it/s]
Loading 0:  21%|██        | 75/363 [00:02<00:10, 28.32it/s]
Loading 0:  22%|██▏       | 79/363 [00:02<00:10, 27.70it/s]
Loading 0:  23%|██▎       | 84/363 [00:03<00:09, 30.33it/s]
Loading 0:  24%|██▍       | 88/363 [00:03<00:09, 28.03it/s]
Loading 0:  25%|██▌       | 91/363 [00:03<00:09, 28.23it/s]
Loading 0:  26%|██▌       | 95/363 [00:03<00:10, 25.19it/s]
Loading 0:  28%|██▊       | 101/363 [00:03<00:10, 24.27it/s]
Loading 0:  29%|██▊       | 104/363 [00:04<00:12, 21.42it/s]
Loading 0:  31%|███       | 111/363 [00:04<00:08, 28.19it/s]
Loading 0:  32%|███▏      | 115/363 [00:04<00:08, 27.79it/s]
Loading 0:  33%|███▎      | 120/363 [00:04<00:08, 29.09it/s]
Loading 0:  34%|███▍      | 124/363 [00:04<00:08, 27.64it/s]
Loading 0:  36%|███▌      | 129/363 [00:04<00:07, 29.88it/s]
Loading 0:  37%|███▋      | 133/363 [00:04<00:08, 28.70it/s]
Loading 0:  38%|███▊      | 137/363 [00:05<00:07, 29.13it/s]
Loading 0:  39%|███▉      | 142/363 [00:05<00:08, 24.58it/s]
Loading 0:  40%|███▉      | 145/363 [00:05<00:09, 23.36it/s]
Loading 0:  41%|████      | 149/363 [00:05<00:09, 22.54it/s]
Loading 0:  43%|████▎     | 156/363 [00:05<00:07, 29.17it/s]
Loading 0:  44%|████▍     | 160/363 [00:06<00:07, 27.28it/s]
Loading 0:  45%|████▌     | 165/363 [00:06<00:06, 29.72it/s]
Loading 0:  47%|████▋     | 169/363 [00:06<00:06, 28.69it/s]
Loading 0:  48%|████▊     | 174/363 [00:06<00:06, 31.05it/s]
Loading 0:  49%|████▉     | 178/363 [00:06<00:06, 29.51it/s]
Loading 0:  50%|█████     | 182/363 [00:06<00:07, 24.03it/s]
Loading 0:  51%|█████     | 185/363 [00:07<00:08, 20.83it/s]
Loading 0:  53%|█████▎    | 192/363 [00:07<00:06, 27.70it/s]
Loading 0:  54%|█████▍    | 196/363 [00:07<00:05, 27.86it/s]
Loading 0:  55%|█████▌    | 201/363 [00:07<00:05, 30.53it/s]
Loading 0:  56%|█████▋    | 205/363 [00:07<00:05, 28.65it/s]
Loading 0:  58%|█████▊    | 210/363 [00:07<00:04, 31.44it/s]
Loading 0:  59%|█████▉    | 214/363 [00:07<00:04, 30.24it/s]
Loading 0:  60%|██████    | 218/363 [00:08<00:04, 30.18it/s]
Loading 0:  61%|██████    | 222/363 [00:08<00:04, 31.93it/s]
Loading 0:  62%|██████▏   | 226/363 [00:08<00:06, 22.16it/s]
Loading 0:  63%|██████▎   | 230/363 [00:08<00:06, 21.64it/s]
Loading 0:  65%|██████▌   | 237/363 [00:08<00:04, 28.01it/s]
Loading 0:  66%|██████▋   | 241/363 [00:08<00:04, 27.29it/s]
Loading 0:  68%|██████▊   | 246/363 [00:09<00:03, 29.90it/s]
Loading 0:  69%|██████▉   | 250/363 [00:09<00:03, 28.64it/s]
Loading 0:  70%|███████   | 255/363 [00:09<00:03, 30.70it/s]
Loading 0:  71%|███████▏  | 259/363 [00:09<00:03, 28.92it/s]
Loading 0:  72%|███████▏  | 263/363 [00:09<00:04, 23.91it/s]
Loading 0:  73%|███████▎  | 266/363 [00:09<00:04, 21.00it/s]
Loading 0:  75%|███████▌  | 273/363 [00:10<00:03, 28.15it/s]
Loading 0:  76%|███████▋  | 277/363 [00:10<00:03, 26.89it/s]
Loading 0:  77%|███████▋  | 280/363 [00:10<00:03, 27.48it/s]
Loading 0:  78%|███████▊  | 284/363 [00:10<00:03, 24.01it/s]
Loading 0:  80%|███████▉  | 289/363 [00:10<00:02, 28.25it/s]
Loading 0:  81%|████████  | 293/363 [00:10<00:02, 25.41it/s]
Loading 0:  82%|████████▏ | 299/363 [00:11<00:02, 29.85it/s]
Loading 0:  84%|████████▎ | 304/363 [00:11<00:02, 26.37it/s]
Loading 0:  85%|████████▍ | 307/363 [00:11<00:02, 24.87it/s]
Loading 0:  86%|████████▌ | 311/363 [00:11<00:02, 23.85it/s]
Loading 0:  88%|████████▊ | 318/363 [00:11<00:01, 30.57it/s]
Loading 0:  89%|████████▊ | 322/363 [00:11<00:01, 28.83it/s]
Loading 0:  90%|█████████ | 327/363 [00:12<00:01, 30.74it/s]
Loading 0:  91%|█████████ | 331/363 [00:12<00:01, 29.36it/s]
Loading 0:  93%|█████████▎| 336/363 [00:12<00:00, 31.95it/s]
Loading 0:  94%|█████████▎| 340/363 [00:12<00:00, 30.29it/s]
Loading 0:  95%|█████████▍| 344/363 [00:19<00:09,  1.97it/s]
Loading 0:  96%|█████████▌| 348/363 [00:19<00:05,  2.64it/s]
Loading 0:  97%|█████████▋| 353/363 [00:19<00:02,  3.84it/s]
Loading 0:  98%|█████████▊| 357/363 [00:20<00:01,  4.98it/s]
                                                            
Job rica40325-kto-1017-v1-mkmlizer completed after 358.35s with status: succeeded
Stopping job with name rica40325-kto-1017-v1-mkmlizer
Pipeline stage MKMLizer completed in 358.94s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.21s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service rica40325-kto-1017-v1
Waiting for inference service rica40325-kto-1017-v1 to be ready
Inference service rica40325-kto-1017-v1 ready after 170.71959686279297s
Pipeline stage MKMLDeployer completed in 171.30s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.2264554500579834s
Received healthy response to inference request in 1.7281198501586914s
Received healthy response to inference request in 1.7739191055297852s
Received healthy response to inference request in 2.3262462615966797s
Received healthy response to inference request in 1.7094709873199463s
5 requests
0 failed requests
5th percentile: 1.7132007598876953
10th percentile: 1.7169305324554442
20th percentile: 1.7243900775909424
30th percentile: 1.7372797012329102
40th percentile: 1.7555994033813476
50th percentile: 1.7739191055297852
60th percentile: 1.9549336433410645
70th percentile: 2.1359481811523438
80th percentile: 2.2464136123657226
90th percentile: 2.2863299369812013
95th percentile: 2.3062880992889405
99th percentile: 2.322254629135132
mean time: 1.9528423309326173
Pipeline stage StressChecker completed in 11.37s
Shutdown handler de-registered
rica40325-kto-1017_v1 status is now deployed due to DeploymentManager action
rica40325-kto-1017_v1 status is now inactive due to auto deactivation removed underperforming models
rica40325-kto-1017_v1 status is now torndown due to DeploymentManager action