chaiml-viral-ss-v2-12b1e5

developer_uid: azuruce
submission_id: chaiml-viral-ss-v2-12b1e5_v5
model_name: chaiml-viral-ss-v2-12b1e5_v5
model_group: ChaiML/Viral-ss-v2-12b1e
status: torndown
timestamp: 2024-10-18T17:25:34+00:00
num_battles: 7979
num_wins: 3316
celo_rating: 1182.93
family_friendly_score: 0.6540686210069673
family_friendly_standard_error: 0.005424643743902396
submission_type: basic
model_repo: ChaiML/Viral-ss-v2-12b1e5
model_architecture: MistralForCausalLM
model_num_parameters: 12772070400.0
best_of: 4
max_input_tokens: 1024
max_output_tokens: 64
display_name: chaiml-viral-ss-v2-12b1e5_v5
is_internal_developer: True
language_model: ChaiML/Viral-ss-v2-12b1e5
model_size: 13B
ranking_group: single
us_pacific_date: 2024-10-18
win_ratio: 0.41559092618122573
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|eot_id|>', '<|end_of_text|>', 'You:'], 'max_input_tokens': 1024, 'best_of': 4, 'max_output_tokens': 64}
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '### Response:\n{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-viral-ss-v2-12b1e5-v5-mkmlizer
Waiting for job on chaiml-viral-ss-v2-12b1e5-v5-mkmlizer to finish
chaiml-viral-ss-v2-12b1e5-v5-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-viral-ss-v2-12b1e5-v5-mkmlizer: ║     _____            __           __                                ║
chaiml-viral-ss-v2-12b1e5-v5-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
chaiml-viral-ss-v2-12b1e5-v5-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
chaiml-viral-ss-v2-12b1e5-v5-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
chaiml-viral-ss-v2-12b1e5-v5-mkmlizer: ║       /___/                                                         ║
chaiml-viral-ss-v2-12b1e5-v5-mkmlizer: ║                                                                     ║
chaiml-viral-ss-v2-12b1e5-v5-mkmlizer: ║  Version: 0.11.12                                                   ║
chaiml-viral-ss-v2-12b1e5-v5-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
chaiml-viral-ss-v2-12b1e5-v5-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-viral-ss-v2-12b1e5-v5-mkmlizer: ║                                                                     ║
chaiml-viral-ss-v2-12b1e5-v5-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-viral-ss-v2-12b1e5-v5-mkmlizer: ║  belonging to:                                                      ║
chaiml-viral-ss-v2-12b1e5-v5-mkmlizer: ║                                                                     ║
chaiml-viral-ss-v2-12b1e5-v5-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-viral-ss-v2-12b1e5-v5-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-viral-ss-v2-12b1e5-v5-mkmlizer: ║  Expiration: 2025-01-15 23:59:59                                    ║
chaiml-viral-ss-v2-12b1e5-v5-mkmlizer: ║                                                                     ║
chaiml-viral-ss-v2-12b1e5-v5-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-viral-ss-v2-12b1e5-v5-mkmlizer: Downloaded to shared memory in 29.878s
chaiml-viral-ss-v2-12b1e5-v5-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp3odi3ihk, device:0
chaiml-viral-ss-v2-12b1e5-v5-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-viral-ss-v2-12b1e5-v5-mkmlizer: quantized model in 37.401s
chaiml-viral-ss-v2-12b1e5-v5-mkmlizer: Processed model ChaiML/Viral-ss-v2-12b1e5 in 67.278s
chaiml-viral-ss-v2-12b1e5-v5-mkmlizer: creating bucket guanaco-mkml-models
chaiml-viral-ss-v2-12b1e5-v5-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-viral-ss-v2-12b1e5-v5-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-viral-ss-v2-12b1e5-v5
chaiml-viral-ss-v2-12b1e5-v5-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-viral-ss-v2-12b1e5-v5/config.json
chaiml-viral-ss-v2-12b1e5-v5-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-viral-ss-v2-12b1e5-v5/special_tokens_map.json
chaiml-viral-ss-v2-12b1e5-v5-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-viral-ss-v2-12b1e5-v5/tokenizer_config.json
chaiml-viral-ss-v2-12b1e5-v5-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-viral-ss-v2-12b1e5-v5/tokenizer.json
chaiml-viral-ss-v2-12b1e5-v5-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-viral-ss-v2-12b1e5-v5/flywheel_model.0.safetensors
chaiml-viral-ss-v2-12b1e5-v5-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:13, 26.54it/s]
Loading 0:   3%|▎         | 12/363 [00:00<00:07, 44.34it/s]
Loading 0:   5%|▍         | 18/363 [00:00<00:07, 44.20it/s]
Loading 0:   6%|▋         | 23/363 [00:00<00:09, 34.60it/s]
Loading 0:   8%|▊         | 30/363 [00:00<00:07, 42.44it/s]
Loading 0:  10%|▉         | 35/363 [00:00<00:07, 41.50it/s]
Loading 0:  11%|█         | 40/363 [00:00<00:07, 40.71it/s]
Loading 0:  12%|█▏        | 45/363 [00:01<00:07, 40.71it/s]
Loading 0:  14%|█▍        | 50/363 [00:01<00:09, 32.46it/s]
Loading 0:  15%|█▌        | 55/363 [00:01<00:08, 36.11it/s]
Loading 0:  17%|█▋        | 60/363 [00:01<00:07, 38.03it/s]
Loading 0:  18%|█▊        | 65/363 [00:01<00:11, 25.36it/s]
Loading 0:  20%|█▉        | 71/363 [00:02<00:09, 30.51it/s]
Loading 0:  21%|██        | 75/363 [00:02<00:09, 31.25it/s]
Loading 0:  22%|██▏       | 80/363 [00:02<00:08, 33.06it/s]
Loading 0:  23%|██▎       | 84/363 [00:02<00:08, 32.67it/s]
Loading 0:  25%|██▍       | 89/363 [00:02<00:07, 36.27it/s]
Loading 0:  26%|██▌       | 93/363 [00:02<00:07, 35.03it/s]
Loading 0:  27%|██▋       | 98/363 [00:02<00:07, 37.34it/s]
Loading 0:  28%|██▊       | 102/363 [00:02<00:07, 35.65it/s]
Loading 0:  29%|██▉       | 106/363 [00:02<00:07, 35.35it/s]
Loading 0:  31%|███       | 112/363 [00:03<00:06, 40.49it/s]
Loading 0:  32%|███▏      | 117/363 [00:03<00:06, 38.27it/s]
Loading 0:  33%|███▎      | 121/363 [00:03<00:06, 37.99it/s]
Loading 0:  34%|███▍      | 125/363 [00:03<00:06, 37.96it/s]
Loading 0:  36%|███▌      | 129/363 [00:03<00:06, 36.20it/s]
Loading 0:  37%|███▋      | 135/363 [00:03<00:05, 40.59it/s]
Loading 0:  39%|███▉      | 141/363 [00:03<00:05, 40.65it/s]
Loading 0:  40%|████      | 146/363 [00:04<00:07, 28.99it/s]
Loading 0:  41%|████▏     | 150/363 [00:04<00:07, 28.59it/s]
Loading 0:  43%|████▎     | 156/363 [00:04<00:06, 34.25it/s]
Loading 0:  44%|████▍     | 160/363 [00:04<00:06, 33.36it/s]
Loading 0:  45%|████▌     | 165/363 [00:04<00:05, 35.74it/s]
Loading 0:  47%|████▋     | 169/363 [00:04<00:05, 34.37it/s]
Loading 0:  48%|████▊     | 174/363 [00:04<00:05, 37.40it/s]
Loading 0:  49%|████▉     | 178/363 [00:05<00:05, 36.15it/s]
Loading 0:  50%|█████     | 183/363 [00:05<00:04, 38.64it/s]
Loading 0:  52%|█████▏    | 187/363 [00:05<00:04, 38.41it/s]
Loading 0:  53%|█████▎    | 193/363 [00:05<00:03, 43.37it/s]
Loading 0:  55%|█████▍    | 199/363 [00:05<00:03, 42.62it/s]
Loading 0:  56%|█████▌    | 204/363 [00:05<00:03, 41.98it/s]
Loading 0:  58%|█████▊    | 211/363 [00:05<00:03, 47.65it/s]
Loading 0:  60%|█████▉    | 216/363 [00:05<00:03, 48.22it/s]
Loading 0:  61%|██████    | 222/363 [00:05<00:03, 44.16it/s]
Loading 0:  63%|██████▎   | 227/363 [00:06<00:04, 30.94it/s]
Loading 0:  64%|██████▎   | 231/363 [00:06<00:04, 31.11it/s]
Loading 0:  66%|██████▌   | 238/363 [00:06<00:03, 38.18it/s]
Loading 0:  67%|██████▋   | 244/363 [00:06<00:03, 37.99it/s]
Loading 0:  69%|██████▊   | 249/363 [00:06<00:03, 36.96it/s]
Loading 0:  70%|███████   | 255/363 [00:06<00:02, 41.05it/s]
Loading 0:  72%|███████▏  | 260/363 [00:07<00:02, 40.03it/s]
Loading 0:  73%|███████▎  | 265/363 [00:07<00:02, 40.76it/s]
Loading 0:  74%|███████▍  | 270/363 [00:07<00:02, 42.88it/s]
Loading 0:  76%|███████▌  | 275/363 [00:07<00:02, 36.98it/s]
Loading 0:  78%|███████▊  | 282/363 [00:07<00:01, 44.46it/s]
Loading 0:  79%|███████▉  | 287/363 [00:07<00:01, 44.02it/s]
Loading 0:  80%|████████  | 292/363 [00:07<00:01, 43.03it/s]
Loading 0:  82%|████████▏ | 298/363 [00:07<00:01, 41.38it/s]
Loading 0:  83%|████████▎ | 303/363 [00:08<00:01, 43.47it/s]
Loading 0:  85%|████████▍ | 308/363 [00:15<00:22,  2.42it/s]
Loading 0:  86%|████████▌ | 312/363 [00:15<00:16,  3.13it/s]
Loading 0:  88%|████████▊ | 319/363 [00:15<00:08,  4.90it/s]
Loading 0:  89%|████████▉ | 324/363 [00:15<00:06,  6.48it/s]
Loading 0:  91%|█████████ | 329/363 [00:15<00:03,  8.54it/s]
Loading 0:  92%|█████████▏| 334/363 [00:15<00:02, 11.18it/s]
Loading 0:  93%|█████████▎| 339/363 [00:15<00:01, 13.41it/s]
Loading 0:  96%|█████████▌| 347/363 [00:15<00:00, 19.74it/s]
Loading 0:  97%|█████████▋| 353/363 [00:16<00:00, 23.37it/s]
Loading 0:  99%|█████████▊| 358/363 [00:16<00:00, 26.33it/s]
                                                            
Job chaiml-viral-ss-v2-12b1e5-v5-mkmlizer completed after 93.99s with status: succeeded
Stopping job with name chaiml-viral-ss-v2-12b1e5-v5-mkmlizer
Pipeline stage MKMLizer completed in 94.49s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.18s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-viral-ss-v2-12b1e5-v5
Waiting for inference service chaiml-viral-ss-v2-12b1e5-v5 to be ready
Failed to get response for submission zonemercy-viral-ss-v16-12b1e5_v2: ('http://zonemercy-viral-ss-v16-12b1e5-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:49536->127.0.0.1:8080: read: connection reset by peer\n')
Inference service chaiml-viral-ss-v2-12b1e5-v5 ready after 181.18857383728027s
Pipeline stage MKMLDeployer completed in 181.69s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.0567617416381836s
Received healthy response to inference request in 1.5238211154937744s
Received healthy response to inference request in 1.5071995258331299s
Received healthy response to inference request in 1.462803840637207s
5 requests
1 failed requests
5th percentile: 1.4716829776763916
10th percentile: 1.480562114715576
20th percentile: 1.4983203887939454
30th percentile: 1.5105238437652588
40th percentile: 1.5171724796295165
50th percentile: 1.5238211154937744
60th percentile: 1.736997365951538
70th percentile: 1.9501736164093018
80th percentile: 5.671849441528323
90th percentile: 12.902024841308595
95th percentile: 16.517112541198728
99th percentile: 19.40918270111084
mean time: 5.336557292938233
%s, retrying in %s seconds...
Received healthy response to inference request in 1.383171558380127s
Received healthy response to inference request in 1.4931888580322266s
Received healthy response to inference request in 1.6213371753692627s
Received healthy response to inference request in 2.9158966541290283s
Received healthy response to inference request in 1.7804787158966064s
5 requests
0 failed requests
5th percentile: 1.405175018310547
10th percentile: 1.4271784782409669
20th percentile: 1.4711853981018066
30th percentile: 1.5188185214996337
40th percentile: 1.5700778484344482
50th percentile: 1.6213371753692627
60th percentile: 1.6849937915802002
70th percentile: 1.7486504077911376
80th percentile: 2.007562303543091
90th percentile: 2.4617294788360597
95th percentile: 2.6888130664825436
99th percentile: 2.8704799365997316
mean time: 1.8388145923614503
Pipeline stage StressChecker completed in 39.02s
Shutdown handler de-registered
chaiml-viral-ss-v2-12b1e5_v5 status is now deployed due to DeploymentManager action
chaiml-viral-ss-v2-12b1e5_v5 status is now inactive due to auto deactivation removed underperforming models
chaiml-viral-ss-v2-12b1e5_v5 status is now torndown due to DeploymentManager action