developer_uid: Trace2333
submission_id: trace2333-ultra-dol-aug-_7639_v2
model_name: trace2333-ultra-dol-aug-_7639_v2
model_group: Trace2333/ultra_dol_aug_
status: torndown
timestamp: 2024-08-22T06:21:42+00:00
num_battles: 10582
num_wins: 3847
celo_rating: 1140.3
family_friendly_score: 0.0
submission_type: basic
model_repo: Trace2333/ultra_dol_aug_r32a16_qkvo_epoch6_v1
model_architecture: LlamaForCausalLM
model_num_parameters: 8030261248.0
best_of: 1
max_input_tokens: 512
max_output_tokens: 64
display_name: trace2333-ultra-dol-aug-_7639_v2
is_internal_developer: False
language_model: Trace2333/ultra_dol_aug_r32a16_qkvo_epoch6_v1
model_size: 8B
ranking_group: single
us_pacific_date: 2024-08-21
win_ratio: 0.3635418635418635
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|eot_id|>'], 'max_input_tokens': 512, 'best_of': 1, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Running pipeline stage MKMLizer
Starting job with name trace2333-ultra-dol-aug-7639-v2-mkmlizer
Waiting for job on trace2333-ultra-dol-aug-7639-v2-mkmlizer to finish
trace2333-ultra-dol-aug-7639-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
trace2333-ultra-dol-aug-7639-v2-mkmlizer: ║     _____            __           __                                ║
trace2333-ultra-dol-aug-7639-v2-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
trace2333-ultra-dol-aug-7639-v2-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
trace2333-ultra-dol-aug-7639-v2-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
trace2333-ultra-dol-aug-7639-v2-mkmlizer: ║       /___/                                                         ║
trace2333-ultra-dol-aug-7639-v2-mkmlizer: ║                                                                     ║
trace2333-ultra-dol-aug-7639-v2-mkmlizer: ║  Version: 0.10.1                                                    ║
trace2333-ultra-dol-aug-7639-v2-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
trace2333-ultra-dol-aug-7639-v2-mkmlizer: ║  https://mk1.ai                                                     ║
trace2333-ultra-dol-aug-7639-v2-mkmlizer: ║                                                                     ║
trace2333-ultra-dol-aug-7639-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
trace2333-ultra-dol-aug-7639-v2-mkmlizer: ║  belonging to:                                                      ║
trace2333-ultra-dol-aug-7639-v2-mkmlizer: ║                                                                     ║
trace2333-ultra-dol-aug-7639-v2-mkmlizer: ║  Chai Research Corp.                                                ║
trace2333-ultra-dol-aug-7639-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
trace2333-ultra-dol-aug-7639-v2-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
trace2333-ultra-dol-aug-7639-v2-mkmlizer: ║                                                                     ║
trace2333-ultra-dol-aug-7639-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
trace2333-ultra-dol-aug-7639-v2-mkmlizer: Downloaded to shared memory in 63.912s
trace2333-ultra-dol-aug-7639-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmppw2k5du8, device:0
trace2333-ultra-dol-aug-7639-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Failed to get response for submission blend_lubas_2024-08-22: ('http://chaiml-elo-alignment-run-3-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:41878->127.0.0.1:8080: read: connection reset by peer\n')
trace2333-ultra-dol-aug-7639-v2-mkmlizer: quantized model in 28.483s
trace2333-ultra-dol-aug-7639-v2-mkmlizer: Processed model Trace2333/ultra_dol_aug_r32a16_qkvo_epoch6_v1 in 92.396s
trace2333-ultra-dol-aug-7639-v2-mkmlizer: creating bucket guanaco-mkml-models
trace2333-ultra-dol-aug-7639-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
trace2333-ultra-dol-aug-7639-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/trace2333-ultra-dol-aug-7639-v2
trace2333-ultra-dol-aug-7639-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/trace2333-ultra-dol-aug-7639-v2/config.json
trace2333-ultra-dol-aug-7639-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/trace2333-ultra-dol-aug-7639-v2/special_tokens_map.json
trace2333-ultra-dol-aug-7639-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/trace2333-ultra-dol-aug-7639-v2/tokenizer_config.json
trace2333-ultra-dol-aug-7639-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/trace2333-ultra-dol-aug-7639-v2/tokenizer.json
trace2333-ultra-dol-aug-7639-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/trace2333-ultra-dol-aug-7639-v2/flywheel_model.0.safetensors
trace2333-ultra-dol-aug-7639-v2-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 5/291 [00:00<00:10, 27.86it/s]
Loading 0:   4%|▍         | 12/291 [00:00<00:07, 37.10it/s]
Loading 0:   5%|▌         | 16/291 [00:00<00:07, 34.73it/s]
Loading 0:   7%|▋         | 21/291 [00:00<00:07, 37.47it/s]
Loading 0:   9%|▊         | 25/291 [00:00<00:07, 35.04it/s]
Loading 0:  11%|█         | 31/291 [00:00<00:06, 41.34it/s]
Loading 0:  12%|█▏        | 36/291 [00:01<00:10, 24.09it/s]
Loading 0:  14%|█▍        | 41/291 [00:01<00:09, 25.96it/s]
Loading 0:  16%|█▋        | 48/291 [00:01<00:07, 32.50it/s]
Loading 0:  18%|█▊        | 52/291 [00:01<00:07, 31.88it/s]
Loading 0:  20%|█▉        | 57/291 [00:01<00:06, 33.55it/s]
Loading 0:  21%|██        | 61/291 [00:01<00:06, 33.15it/s]
Loading 0:  23%|██▎       | 66/291 [00:02<00:06, 35.82it/s]
Loading 0:  24%|██▍       | 70/291 [00:02<00:06, 34.45it/s]
Loading 0:  25%|██▌       | 74/291 [00:02<00:06, 34.73it/s]
Loading 0:  27%|██▋       | 78/291 [00:02<00:06, 35.00it/s]
Loading 0:  28%|██▊       | 82/291 [00:02<00:08, 24.76it/s]
Loading 0:  30%|██▉       | 86/291 [00:02<00:07, 27.52it/s]
Loading 0:  31%|███       | 90/291 [00:02<00:06, 29.76it/s]
Loading 0:  32%|███▏      | 94/291 [00:02<00:06, 29.83it/s]
Loading 0:  34%|███▍      | 99/291 [00:03<00:05, 33.30it/s]
Loading 0:  35%|███▌      | 103/291 [00:03<00:05, 32.88it/s]
Loading 0:  37%|███▋      | 108/291 [00:03<00:05, 36.35it/s]
Loading 0:  38%|███▊      | 112/291 [00:03<00:05, 34.82it/s]
Loading 0:  40%|███▉      | 116/291 [00:03<00:05, 34.73it/s]
Loading 0:  42%|████▏     | 122/291 [00:03<00:04, 39.02it/s]
Loading 0:  44%|████▎     | 127/291 [00:03<00:04, 36.30it/s]
Loading 0:  46%|████▌     | 133/291 [00:04<00:05, 30.79it/s]
Loading 0:  47%|████▋     | 137/291 [00:04<00:04, 31.15it/s]
Loading 0:  48%|████▊     | 141/291 [00:04<00:05, 29.12it/s]
Loading 0:  51%|█████     | 147/291 [00:04<00:04, 34.14it/s]
Loading 0:  52%|█████▏    | 151/291 [00:04<00:04, 33.61it/s]
Loading 0:  54%|█████▎    | 156/291 [00:04<00:03, 35.12it/s]
Loading 0:  55%|█████▍    | 160/291 [00:04<00:03, 33.76it/s]
Loading 0:  57%|█████▋    | 165/291 [00:05<00:03, 35.80it/s]
Loading 0:  58%|█████▊    | 169/291 [00:05<00:03, 34.11it/s]
Loading 0:  60%|█████▉    | 174/291 [00:05<00:03, 36.07it/s]
Loading 0:  61%|██████    | 178/291 [00:05<00:03, 34.67it/s]
Loading 0:  63%|██████▎   | 184/291 [00:05<00:02, 40.93it/s]
Loading 0:  65%|██████▍   | 189/291 [00:05<00:04, 25.22it/s]
Loading 0:  67%|██████▋   | 194/291 [00:06<00:03, 26.96it/s]
Loading 0:  69%|██████▉   | 201/291 [00:06<00:02, 33.24it/s]
Loading 0:  70%|███████   | 205/291 [00:06<00:02, 32.60it/s]
Loading 0:  72%|███████▏  | 210/291 [00:06<00:02, 35.31it/s]
Loading 0:  74%|███████▎  | 214/291 [00:06<00:02, 34.37it/s]
Loading 0:  75%|███████▌  | 219/291 [00:06<00:01, 36.50it/s]
Loading 0:  77%|███████▋  | 223/291 [00:06<00:01, 35.13it/s]
Loading 0:  78%|███████▊  | 227/291 [00:06<00:01, 35.07it/s]
Loading 0:  79%|███████▉  | 231/291 [00:07<00:01, 35.33it/s]
Loading 0:  81%|████████  | 235/291 [00:07<00:02, 26.02it/s]
Loading 0:  82%|████████▏ | 239/291 [00:07<00:01, 26.08it/s]
Loading 0:  85%|████████▍ | 246/291 [00:07<00:01, 33.70it/s]
Loading 0:  86%|████████▌ | 250/291 [00:07<00:01, 33.44it/s]
Loading 0:  88%|████████▊ | 255/291 [00:07<00:00, 36.08it/s]
Loading 0:  89%|████████▉ | 259/291 [00:07<00:00, 34.64it/s]
Loading 0:  91%|█████████ | 264/291 [00:08<00:00, 36.79it/s]
Loading 0:  92%|█████████▏| 268/291 [00:08<00:00, 34.81it/s]
Loading 0:  94%|█████████▍| 273/291 [00:08<00:00, 36.89it/s]
Loading 0:  95%|█████████▌| 277/291 [00:08<00:00, 34.73it/s]
Loading 0:  97%|█████████▋| 281/291 [00:08<00:00, 34.42it/s]
Loading 0:  98%|█████████▊| 286/291 [00:13<00:01,  2.62it/s]
Loading 0:  99%|█████████▉| 289/291 [00:14<00:00,  3.26it/s]
                                                            

Job trace2333-ultra-dol-aug-7639-v2-mkmlizer completed after 117.55s with status: succeeded
Stopping job with name trace2333-ultra-dol-aug-7639-v2-mkmlizer
Pipeline stage MKMLizer completed in 118.66s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.11s
Running pipeline stage ISVCDeployer
Creating inference service trace2333-ultra-dol-aug-7639-v2
Waiting for inference service trace2333-ultra-dol-aug-7639-v2 to be ready
Inference service trace2333-ultra-dol-aug-7639-v2 ready after 263.12414169311523s
Pipeline stage ISVCDeployer completed in 264.01s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.898411750793457s
Received healthy response to inference request in 1.1578714847564697s
Received healthy response to inference request in 1.7303335666656494s
Received healthy response to inference request in 1.1454472541809082s
Received healthy response to inference request in 1.017263412475586s
5 requests
0 failed requests
5th percentile: 1.0429001808166505
10th percentile: 1.0685369491577148
20th percentile: 1.1198104858398437
30th percentile: 1.1479321002960206
40th percentile: 1.152901792526245
50th percentile: 1.1578714847564697
60th percentile: 1.3868563175201416
70th percentile: 1.6158411502838135
80th percentile: 1.763949203491211
90th percentile: 1.831180477142334
95th percentile: 1.8647961139678955
99th percentile: 1.8916886234283448
mean time: 1.3898654937744142
Pipeline stage StressChecker completed in 7.82s
trace2333-ultra-dol-aug-_7639_v2 status is now deployed due to DeploymentManager action
trace2333-ultra-dol-aug-_7639_v2 status is now inactive due to auto deactivation removed underperforming models
trace2333-ultra-dol-aug-_7639_v2 status is now torndown due to DeploymentManager action