chaiml-viral-ss-v2-12b1e5

submission_id: chaiml-viral-ss-v2-12b1e5_v1
developer_uid: chai_backend_admin
best_of: 8
celo_rating: 1192.91
display_name: chaiml-viral-ss-v2-12b1e5_v1
family_friendly_score: 0.6105475746609643
family_friendly_standard_error: 0.005070364298085825
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '### Response:\n{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '###', 'Bot:', 'User:', 'You:', '<|im_end|>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
is_internal_developer: True
language_model: ChaiML/Viral-ss-v2-12b1e5
max_input_tokens: 1024
max_output_tokens: 64
model_architecture: MistralForCausalLM
model_group: ChaiML/Viral-ss-v2-12b1e
model_name: chaiml-viral-ss-v2-12b1e5_v1
model_num_parameters: 12772070400.0
model_repo: ChaiML/Viral-ss-v2-12b1e5
model_size: 13B
num_battles: 9776
num_wins: 4191
ranking_group: single
status: inactive
submission_type: basic
timestamp: 2024-10-17T22:32:01+00:00
us_pacific_date: 2024-10-17
win_ratio: 0.42870294599018005
Download Preference Data
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-viral-ss-v2-12b1e5-v1-mkmlizer
Waiting for job on chaiml-viral-ss-v2-12b1e5-v1-mkmlizer to finish
chaiml-viral-ss-v2-12b1e5-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-viral-ss-v2-12b1e5-v1-mkmlizer: ║     _____            __           __                                ║
chaiml-viral-ss-v2-12b1e5-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
chaiml-viral-ss-v2-12b1e5-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
chaiml-viral-ss-v2-12b1e5-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
chaiml-viral-ss-v2-12b1e5-v1-mkmlizer: ║       /___/                                                         ║
chaiml-viral-ss-v2-12b1e5-v1-mkmlizer: ║                                                                     ║
chaiml-viral-ss-v2-12b1e5-v1-mkmlizer: ║  Version: 0.11.12                                                   ║
chaiml-viral-ss-v2-12b1e5-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
chaiml-viral-ss-v2-12b1e5-v1-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-viral-ss-v2-12b1e5-v1-mkmlizer: ║                                                                     ║
chaiml-viral-ss-v2-12b1e5-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-viral-ss-v2-12b1e5-v1-mkmlizer: ║  belonging to:                                                      ║
chaiml-viral-ss-v2-12b1e5-v1-mkmlizer: ║                                                                     ║
chaiml-viral-ss-v2-12b1e5-v1-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-viral-ss-v2-12b1e5-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-viral-ss-v2-12b1e5-v1-mkmlizer: ║  Expiration: 2025-01-15 23:59:59                                    ║
chaiml-viral-ss-v2-12b1e5-v1-mkmlizer: ║                                                                     ║
chaiml-viral-ss-v2-12b1e5-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-viral-ss-v2-12b1e5-v1-mkmlizer: Downloaded to shared memory in 53.997s
chaiml-viral-ss-v2-12b1e5-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpy0gxvy9v, device:0
chaiml-viral-ss-v2-12b1e5-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-viral-ss-v2-12b1e5-v1-mkmlizer: quantized model in 37.692s
chaiml-viral-ss-v2-12b1e5-v1-mkmlizer: Processed model ChaiML/Viral-ss-v2-12b1e5 in 91.689s
chaiml-viral-ss-v2-12b1e5-v1-mkmlizer: creating bucket guanaco-mkml-models
chaiml-viral-ss-v2-12b1e5-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-viral-ss-v2-12b1e5-v1/special_tokens_map.json
chaiml-viral-ss-v2-12b1e5-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-viral-ss-v2-12b1e5-v1/config.json
chaiml-viral-ss-v2-12b1e5-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-viral-ss-v2-12b1e5-v1/tokenizer_config.json
chaiml-viral-ss-v2-12b1e5-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-viral-ss-v2-12b1e5-v1/tokenizer.json
chaiml-viral-ss-v2-12b1e5-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-viral-ss-v2-12b1e5-v1/flywheel_model.0.safetensors
chaiml-viral-ss-v2-12b1e5-v1-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:13, 26.26it/s]
Loading 0:   3%|▎         | 12/363 [00:00<00:07, 44.04it/s]
Loading 0:   5%|▍         | 17/363 [00:00<00:08, 42.47it/s]
Loading 0:   6%|▌         | 22/363 [00:00<00:08, 42.40it/s]
Loading 0:   7%|▋         | 27/363 [00:00<00:07, 43.21it/s]
Loading 0:   9%|▉         | 32/363 [00:00<00:09, 33.19it/s]
Loading 0:  11%|█         | 39/363 [00:00<00:07, 41.58it/s]
Loading 0:  12%|█▏        | 44/363 [00:01<00:07, 41.19it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:07, 41.42it/s]
Loading 0:  15%|█▍        | 54/363 [00:01<00:07, 41.98it/s]
Loading 0:  16%|█▋        | 59/363 [00:01<00:07, 42.99it/s]
Loading 0:  18%|█▊        | 64/363 [00:01<00:12, 24.68it/s]
Loading 0:  20%|█▉        | 71/363 [00:01<00:09, 31.90it/s]
Loading 0:  21%|██        | 76/363 [00:02<00:08, 33.29it/s]
Loading 0:  22%|██▏       | 81/363 [00:02<00:07, 35.57it/s]
Loading 0:  24%|██▎       | 86/363 [00:02<00:07, 37.61it/s]
Loading 0:  25%|██▌       | 91/363 [00:02<00:08, 32.21it/s]
Loading 0:  27%|██▋       | 98/363 [00:02<00:06, 38.78it/s]
Loading 0:  28%|██▊       | 103/363 [00:02<00:06, 38.47it/s]
Loading 0:  30%|██▉       | 108/363 [00:02<00:06, 40.37it/s]
Loading 0:  31%|███       | 113/363 [00:03<00:07, 34.56it/s]
Loading 0:  33%|███▎      | 118/363 [00:03<00:07, 34.54it/s]
Loading 0:  34%|███▍      | 125/363 [00:03<00:05, 41.47it/s]
Loading 0:  36%|███▌      | 130/363 [00:03<00:05, 40.76it/s]
Loading 0:  37%|███▋      | 135/363 [00:03<00:05, 41.05it/s]
Loading 0:  39%|███▊      | 140/363 [00:03<00:05, 42.46it/s]
Loading 0:  40%|███▉      | 145/363 [00:04<00:08, 26.34it/s]
Loading 0:  41%|████      | 149/363 [00:04<00:08, 26.58it/s]
Loading 0:  43%|████▎     | 156/363 [00:04<00:06, 34.33it/s]
Loading 0:  44%|████▍     | 161/363 [00:04<00:05, 35.39it/s]
Loading 0:  46%|████▌     | 166/363 [00:04<00:05, 37.20it/s]
Loading 0:  47%|████▋     | 171/363 [00:04<00:04, 38.57it/s]
Loading 0:  48%|████▊     | 176/363 [00:04<00:05, 32.47it/s]
Loading 0:  50%|█████     | 183/363 [00:05<00:04, 39.38it/s]
Loading 0:  52%|█████▏    | 188/363 [00:05<00:04, 39.06it/s]
Loading 0:  53%|█████▎    | 193/363 [00:05<00:04, 39.86it/s]
Loading 0:  55%|█████▍    | 198/363 [00:05<00:04, 40.49it/s]
Loading 0:  56%|█████▌    | 203/363 [00:05<00:04, 33.75it/s]
Loading 0:  58%|█████▊    | 210/363 [00:05<00:03, 41.29it/s]
Loading 0:  59%|█████▉    | 215/363 [00:05<00:03, 40.59it/s]
Loading 0:  61%|██████    | 220/363 [00:05<00:03, 42.02it/s]
Loading 0:  62%|██████▏   | 225/363 [00:06<00:05, 26.15it/s]
Loading 0:  63%|██████▎   | 230/363 [00:06<00:04, 28.04it/s]
Loading 0:  65%|██████▌   | 237/363 [00:06<00:03, 35.45it/s]
Loading 0:  67%|██████▋   | 242/363 [00:06<00:03, 36.04it/s]
Loading 0:  68%|██████▊   | 247/363 [00:06<00:03, 37.46it/s]
Loading 0:  69%|██████▉   | 252/363 [00:06<00:02, 39.12it/s]
Loading 0:  71%|███████   | 257/363 [00:07<00:03, 32.92it/s]
Loading 0:  73%|███████▎  | 264/363 [00:07<00:02, 39.60it/s]
Loading 0:  74%|███████▍  | 269/363 [00:07<00:02, 39.29it/s]
Loading 0:  75%|███████▌  | 274/363 [00:07<00:02, 39.65it/s]
Loading 0:  77%|███████▋  | 279/363 [00:07<00:02, 39.26it/s]
Loading 0:  78%|███████▊  | 284/363 [00:07<00:02, 33.22it/s]
Loading 0:  80%|███████▉  | 289/363 [00:07<00:02, 36.66it/s]
Loading 0:  81%|████████  | 293/363 [00:08<00:02, 33.92it/s]
Loading 0:  82%|████████▏ | 299/363 [00:08<00:01, 38.92it/s]
Loading 0:  84%|████████▎ | 304/363 [00:15<00:24,  2.36it/s]
Loading 0:  85%|████████▍ | 307/363 [00:15<00:19,  2.88it/s]
Loading 0:  86%|████████▌ | 312/363 [00:15<00:12,  4.11it/s]
Loading 0:  88%|████████▊ | 319/363 [00:15<00:06,  6.52it/s]
Loading 0:  89%|████████▉ | 324/363 [00:15<00:04,  8.57it/s]
Loading 0:  91%|█████████ | 329/363 [00:15<00:03, 11.16it/s]
Loading 0:  92%|█████████▏| 334/363 [00:15<00:02, 14.37it/s]
Loading 0:  93%|█████████▎| 339/363 [00:16<00:01, 16.35it/s]
Loading 0:  95%|█████████▌| 346/363 [00:16<00:00, 22.56it/s]
Loading 0:  97%|█████████▋| 351/363 [00:16<00:00, 25.42it/s]
Loading 0:  98%|█████████▊| 356/363 [00:16<00:00, 28.66it/s]
Loading 0:  99%|█████████▉| 361/363 [00:16<00:00, 31.57it/s]
                                                            
Job chaiml-viral-ss-v2-12b1e5-v1-mkmlizer completed after 114.19s with status: succeeded
Stopping job with name chaiml-viral-ss-v2-12b1e5-v1-mkmlizer
Pipeline stage MKMLizer completed in 114.80s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.19s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-viral-ss-v2-12b1e5-v1
Waiting for inference service chaiml-viral-ss-v2-12b1e5-v1 to be ready
Inference service chaiml-viral-ss-v2-12b1e5-v1 ready after 170.60518598556519s
Pipeline stage MKMLDeployer completed in 171.15s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 1.9002723693847656s
Received healthy response to inference request in 1.5207598209381104s
Received healthy response to inference request in 1.6388564109802246s
Received healthy response to inference request in 1.6234843730926514s
5 requests
1 failed requests
5th percentile: 1.5413047313690185
10th percentile: 1.5618496417999268
20th percentile: 1.6029394626617433
30th percentile: 1.626558780670166
40th percentile: 1.6327075958251953
50th percentile: 1.6388564109802246
60th percentile: 1.743422794342041
70th percentile: 1.8479891777038573
80th percentile: 5.552189922332767
90th percentile: 12.85602502822876
95th percentile: 16.507942581176756
99th percentile: 19.429476623535155
mean time: 5.368646621704102
%s, retrying in %s seconds...
Received healthy response to inference request in 1.4950473308563232s
Received healthy response to inference request in 1.4596891403198242s
Received healthy response to inference request in 1.9246397018432617s
Received healthy response to inference request in 1.5298888683319092s
Received healthy response to inference request in 1.524047613143921s
5 requests
0 failed requests
5th percentile: 1.466760778427124
10th percentile: 1.4738324165344239
20th percentile: 1.4879756927490235
30th percentile: 1.5008473873138428
40th percentile: 1.5124475002288817
50th percentile: 1.524047613143921
60th percentile: 1.5263841152191162
70th percentile: 1.5287206172943115
80th percentile: 1.6088390350341797
90th percentile: 1.7667393684387207
95th percentile: 1.8456895351409912
99th percentile: 1.9088496685028076
mean time: 1.5866625308990479
Pipeline stage StressChecker completed in 37.57s
Shutdown handler de-registered
chaiml-viral-ss-v2-12b1e5_v1 status is now deployed due to DeploymentManager action
chaiml-viral-ss-v2-12b1e5_v1 status is now inactive due to auto deactivation removed underperforming models