zonemercy-super-ss-v1-22_1691

developer_uid: chai_backend_admin
submission_id: zonemercy-super-ss-v1-22_1691_v6
model_name: tempv1-5
model_group: zonemercy/Super-ss-v1-22
status: torndown
timestamp: 2024-10-29T17:36:58+00:00
num_battles: 9895
num_wins: 5280
celo_rating: 1263.64
family_friendly_score: 0.5636
family_friendly_standard_error: 0.007013630158484265
submission_type: basic
model_repo: zonemercy/Super-ss-v1-22b0-sft1e5
model_architecture: MistralForCausalLM
model_num_parameters: 22247282688.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
display_name: tempv1-5
is_internal_developer: True
language_model: zonemercy/Super-ss-v1-22b0-sft1e5
model_size: 22B
ranking_group: single
us_pacific_date: 2024-10-29
win_ratio: 0.5336028297119757
generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '####', 'Bot:', 'User:', 'You:', '<|im_end|>', '<|eot_id|>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': 'Bot: {message}\n', 'user_template': 'User: {message}\n', 'response_template': 'Bot:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name zonemercy-super-ss-v1-22-1691-v6-mkmlizer
Waiting for job on zonemercy-super-ss-v1-22-1691-v6-mkmlizer to finish
zonemercy-super-ss-v1-22-1691-v6-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
zonemercy-super-ss-v1-22-1691-v6-mkmlizer: ║     _____            __           __                                ║
zonemercy-super-ss-v1-22-1691-v6-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
zonemercy-super-ss-v1-22-1691-v6-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
zonemercy-super-ss-v1-22-1691-v6-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
zonemercy-super-ss-v1-22-1691-v6-mkmlizer: ║       /___/                                                         ║
zonemercy-super-ss-v1-22-1691-v6-mkmlizer: ║                                                                     ║
zonemercy-super-ss-v1-22-1691-v6-mkmlizer: ║  Version: 0.11.12                                                   ║
zonemercy-super-ss-v1-22-1691-v6-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
zonemercy-super-ss-v1-22-1691-v6-mkmlizer: ║  https://mk1.ai                                                     ║
zonemercy-super-ss-v1-22-1691-v6-mkmlizer: ║                                                                     ║
zonemercy-super-ss-v1-22-1691-v6-mkmlizer: ║  The license key for the current software has been verified as      ║
zonemercy-super-ss-v1-22-1691-v6-mkmlizer: ║  belonging to:                                                      ║
zonemercy-super-ss-v1-22-1691-v6-mkmlizer: ║                                                                     ║
zonemercy-super-ss-v1-22-1691-v6-mkmlizer: ║  Chai Research Corp.                                                ║
zonemercy-super-ss-v1-22-1691-v6-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
zonemercy-super-ss-v1-22-1691-v6-mkmlizer: ║  Expiration: 2025-01-15 23:59:59                                    ║
zonemercy-super-ss-v1-22-1691-v6-mkmlizer: ║                                                                     ║
zonemercy-super-ss-v1-22-1691-v6-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
zonemercy-super-ss-v1-22-1691-v6-mkmlizer: Downloaded to shared memory in 55.683s
zonemercy-super-ss-v1-22-1691-v6-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp0s2xfmi8, device:0
zonemercy-super-ss-v1-22-1691-v6-mkmlizer: Saving flywheel model at /dev/shm/model_cache
zonemercy-super-ss-v1-22-1691-v6-mkmlizer: quantized model in 43.264s
zonemercy-super-ss-v1-22-1691-v6-mkmlizer: Processed model zonemercy/Super-ss-v1-22b0-sft1e5 in 98.947s
zonemercy-super-ss-v1-22-1691-v6-mkmlizer: creating bucket guanaco-mkml-models
zonemercy-super-ss-v1-22-1691-v6-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
zonemercy-super-ss-v1-22-1691-v6-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/zonemercy-super-ss-v1-22-1691-v6
zonemercy-super-ss-v1-22-1691-v6-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/zonemercy-super-ss-v1-22-1691-v6/config.json
zonemercy-super-ss-v1-22-1691-v6-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/zonemercy-super-ss-v1-22-1691-v6/special_tokens_map.json
zonemercy-super-ss-v1-22-1691-v6-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/zonemercy-super-ss-v1-22-1691-v6/tokenizer_config.json
zonemercy-super-ss-v1-22-1691-v6-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/zonemercy-super-ss-v1-22-1691-v6/tokenizer.json
zonemercy-super-ss-v1-22-1691-v6-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/zonemercy-super-ss-v1-22-1691-v6/flywheel_model.1.safetensors
zonemercy-super-ss-v1-22-1691-v6-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/zonemercy-super-ss-v1-22-1691-v6/flywheel_model.0.safetensors
zonemercy-super-ss-v1-22-1691-v6-mkmlizer: 
Loading 0:   0%|          | 0/507 [00:00<?, ?it/s]
Loading 0:   1%|          | 5/507 [00:00<00:21, 23.07it/s]
Loading 0:   2%|▏         | 12/507 [00:00<00:12, 39.71it/s]
Loading 0:   3%|▎         | 17/507 [00:00<00:12, 39.64it/s]
Loading 0:   4%|▍         | 22/507 [00:00<00:11, 40.59it/s]
Loading 0:   5%|▌         | 27/507 [00:00<00:11, 42.20it/s]
Loading 0:   6%|▋         | 32/507 [00:00<00:13, 35.56it/s]
Loading 0:   8%|▊         | 39/507 [00:00<00:10, 43.36it/s]
Loading 0:   9%|▊         | 44/507 [00:01<00:10, 43.42it/s]
Loading 0:  10%|▉         | 49/507 [00:01<00:12, 37.98it/s]
Loading 0:  11%|█         | 54/507 [00:01<00:16, 28.10it/s]
Loading 0:  11%|█▏        | 58/507 [00:01<00:15, 29.03it/s]
Loading 0:  12%|█▏        | 63/507 [00:01<00:13, 32.70it/s]
Loading 0:  13%|█▎        | 67/507 [00:01<00:13, 33.25it/s]
Loading 0:  14%|█▍        | 73/507 [00:02<00:12, 36.00it/s]
Loading 0:  16%|█▌        | 80/507 [00:02<00:11, 38.49it/s]
Loading 0:  17%|█▋        | 87/507 [00:02<00:09, 43.63it/s]
Loading 0:  18%|█▊        | 92/507 [00:02<00:09, 41.68it/s]
Loading 0:  19%|█▉        | 97/507 [00:02<00:09, 41.18it/s]
Loading 0:  20%|██        | 102/507 [00:02<00:09, 42.58it/s]
Loading 0:  21%|██        | 107/507 [00:02<00:11, 36.25it/s]
Loading 0:  22%|██▏       | 113/507 [00:03<00:12, 31.06it/s]
Loading 0:  23%|██▎       | 117/507 [00:03<00:13, 29.48it/s]
Loading 0:  24%|██▍       | 122/507 [00:03<00:12, 30.25it/s]
Loading 0:  25%|██▌       | 129/507 [00:03<00:10, 36.83it/s]
Loading 0:  26%|██▌       | 133/507 [00:03<00:10, 36.46it/s]
Loading 0:  27%|██▋       | 138/507 [00:03<00:09, 37.81it/s]
Loading 0:  28%|██▊       | 142/507 [00:03<00:09, 37.31it/s]
Loading 0:  29%|██▉       | 147/507 [00:04<00:08, 40.30it/s]
Loading 0:  30%|██▉       | 152/507 [00:04<00:08, 39.77it/s]
Loading 0:  31%|███       | 157/507 [00:04<00:08, 40.46it/s]
Loading 0:  32%|███▏      | 162/507 [00:04<00:08, 42.06it/s]
Loading 0:  33%|███▎      | 167/507 [00:04<00:07, 44.18it/s]
Loading 0:  34%|███▍      | 172/507 [00:04<00:12, 27.58it/s]
Loading 0:  35%|███▍      | 176/507 [00:04<00:11, 27.88it/s]
Loading 0:  36%|███▌      | 183/507 [00:05<00:09, 35.86it/s]
Loading 0:  37%|███▋      | 188/507 [00:05<00:08, 36.93it/s]
Loading 0:  38%|███▊      | 193/507 [00:05<00:08, 38.70it/s]
Loading 0:  39%|███▉      | 198/507 [00:05<00:07, 40.90it/s]
Loading 0:  40%|████      | 203/507 [00:05<00:08, 35.85it/s]
Loading 0:  41%|████▏     | 210/507 [00:05<00:06, 43.37it/s]
Loading 0:  42%|████▏     | 215/507 [00:05<00:06, 43.40it/s]
Loading 0:  43%|████▎     | 220/507 [00:05<00:07, 38.41it/s]
Loading 0:  44%|████▍     | 225/507 [00:06<00:09, 29.46it/s]
Loading 0:  45%|████▌     | 230/507 [00:06<00:09, 29.63it/s]
Loading 0:  47%|████▋     | 237/507 [00:06<00:07, 36.27it/s]
Loading 0:  48%|████▊     | 242/507 [00:06<00:07, 36.48it/s]
Loading 0:  49%|████▊     | 246/507 [00:06<00:07, 36.82it/s]
Loading 0:  49%|████▉     | 250/507 [00:06<00:07, 35.78it/s]
Loading 0:  50%|█████     | 255/507 [00:07<00:06, 36.68it/s]
Loading 0:  51%|█████     | 259/507 [00:07<00:06, 35.83it/s]
Loading 0:  52%|█████▏    | 264/507 [00:07<00:06, 36.92it/s]
Loading 0:  53%|█████▎    | 268/507 [00:07<00:06, 34.49it/s]
Loading 0:  54%|█████▍    | 273/507 [00:07<00:06, 36.46it/s]
Loading 0:  55%|█████▍    | 277/507 [00:07<00:06, 35.16it/s]
Loading 0:  56%|█████▌    | 283/507 [00:07<00:06, 36.74it/s]
Loading 0:  57%|█████▋    | 287/507 [00:08<00:09, 24.01it/s]
Loading 0:  58%|█████▊    | 293/507 [00:08<00:07, 26.88it/s]
Loading 0:  59%|█████▉    | 299/507 [00:22<00:07, 26.88it/s]
Loading 0:  59%|█████▉    | 300/507 [00:22<02:45,  1.25it/s]
Loading 0:  60%|█████▉    | 302/507 [00:22<02:23,  1.42it/s]
Loading 0:  61%|██████    | 307/507 [00:23<01:37,  2.06it/s]
Loading 0:  61%|██████    | 310/507 [00:23<01:17,  2.54it/s]
Loading 0:  62%|██████▏   | 313/507 [00:23<00:59,  3.24it/s]
Loading 0:  63%|██████▎   | 319/507 [00:23<00:36,  5.17it/s]
Loading 0:  64%|██████▍   | 325/507 [00:23<00:23,  7.60it/s]
Loading 0:  65%|██████▍   | 329/507 [00:23<00:19,  9.36it/s]
Loading 0:  66%|██████▌   | 335/507 [00:23<00:12, 13.29it/s]
Loading 0:  67%|██████▋   | 340/507 [00:24<00:11, 14.76it/s]
Loading 0:  68%|██████▊   | 344/507 [00:24<00:09, 16.97it/s]
Loading 0:  69%|██████▊   | 348/507 [00:24<00:08, 18.56it/s]
Loading 0:  70%|██████▉   | 354/507 [00:24<00:06, 24.54it/s]
Loading 0:  71%|███████   | 358/507 [00:24<00:05, 26.88it/s]
Loading 0:  72%|███████▏  | 364/507 [00:24<00:04, 31.71it/s]
Loading 0:  73%|███████▎  | 369/507 [00:24<00:03, 35.03it/s]
Loading 0:  74%|███████▍  | 374/507 [00:25<00:04, 31.95it/s]
Loading 0:  75%|███████▌  | 381/507 [00:25<00:03, 40.05it/s]
Loading 0:  76%|███████▌  | 386/507 [00:25<00:02, 40.67it/s]
Loading 0:  77%|███████▋  | 391/507 [00:25<00:03, 36.68it/s]
Loading 0:  78%|███████▊  | 396/507 [00:25<00:03, 28.06it/s]
Loading 0:  79%|███████▉  | 401/507 [00:25<00:03, 28.72it/s]
Loading 0:  80%|████████  | 408/507 [00:26<00:02, 35.56it/s]
Loading 0:  81%|████████▏ | 413/507 [00:26<00:02, 36.87it/s]
Loading 0:  82%|████████▏ | 418/507 [00:26<00:02, 38.38it/s]
Loading 0:  83%|████████▎ | 423/507 [00:26<00:02, 40.37it/s]
Loading 0:  84%|████████▍ | 428/507 [00:26<00:02, 35.73it/s]
Loading 0:  86%|████████▌ | 435/507 [00:26<00:01, 42.87it/s]
Loading 0:  87%|████████▋ | 440/507 [00:26<00:01, 41.27it/s]
Loading 0:  88%|████████▊ | 445/507 [00:26<00:01, 40.67it/s]
Loading 0:  89%|████████▉ | 450/507 [00:27<00:01, 41.64it/s]
Loading 0:  90%|████████▉ | 455/507 [00:29<00:07,  6.70it/s]
Loading 0:  91%|█████████ | 459/507 [00:29<00:05,  8.24it/s]
Loading 0:  92%|█████████▏| 465/507 [00:29<00:03, 11.37it/s]
Loading 0:  93%|█████████▎| 472/507 [00:29<00:02, 16.19it/s]
Loading 0:  94%|█████████▍| 477/507 [00:29<00:01, 19.32it/s]
Loading 0:  95%|█████████▌| 482/507 [00:29<00:01, 23.00it/s]
Loading 0:  96%|█████████▌| 487/507 [00:30<00:00, 27.06it/s]
Loading 0:  97%|█████████▋| 492/507 [00:30<00:00, 26.83it/s]
Loading 0:  98%|█████████▊| 499/507 [00:30<00:00, 33.87it/s]
Loading 0:  99%|█████████▉| 504/507 [00:30<00:00, 35.56it/s]
                                                            
Job zonemercy-super-ss-v1-22-1691-v6-mkmlizer completed after 125.13s with status: succeeded
Stopping job with name zonemercy-super-ss-v1-22-1691-v6-mkmlizer
Pipeline stage MKMLizer completed in 125.70s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.20s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service zonemercy-super-ss-v1-22-1691-v6
Waiting for inference service zonemercy-super-ss-v1-22-1691-v6 to be ready
Inference service zonemercy-super-ss-v1-22-1691-v6 ready after 120.51912569999695s
Pipeline stage MKMLDeployer completed in 121.44s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.886833429336548s
Received healthy response to inference request in 3.2520320415496826s
Received healthy response to inference request in 2.4284751415252686s
Received healthy response to inference request in 2.58577036857605s
5 requests
1 failed requests
5th percentile: 2.459934186935425
10th percentile: 2.491393232345581
20th percentile: 2.5543113231658934
30th percentile: 2.6459829807281494
40th percentile: 2.7664082050323486
50th percentile: 2.886833429336548
60th percentile: 3.0329128742218017
70th percentile: 3.1789923191070555
80th percentile: 6.632038688659671
90th percentile: 13.39205198287964
95th percentile: 16.772058629989623
99th percentile: 19.476063947677613
mean time: 6.2610352516174315
%s, retrying in %s seconds...
Received healthy response to inference request in 2.581130266189575s
Received healthy response to inference request in 2.506284236907959s
Received healthy response to inference request in 2.766826629638672s
Received healthy response to inference request in 2.485802412033081s
Received healthy response to inference request in 2.6347033977508545s
5 requests
0 failed requests
5th percentile: 2.4898987770080567
10th percentile: 2.4939951419830324
20th percentile: 2.5021878719329833
30th percentile: 2.5212534427642823
40th percentile: 2.5511918544769285
50th percentile: 2.581130266189575
60th percentile: 2.602559518814087
70th percentile: 2.6239887714385985
80th percentile: 2.661128044128418
90th percentile: 2.713977336883545
95th percentile: 2.7404019832611084
99th percentile: 2.7615417003631593
mean time: 2.5949493885040282
Pipeline stage StressChecker completed in 47.32s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.75s
Shutdown handler de-registered
zonemercy-super-ss-v1-22_1691_v6 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3168.73s
Shutdown handler de-registered
zonemercy-super-ss-v1-22_1691_v6 status is now inactive due to auto deactivation removed underperforming models
zonemercy-super-ss-v1-22_1691_v6 status is now torndown due to DeploymentManager action