zonemercy-lexical-viral_1979

developer_uid: Jellywibble

submission_id: zonemercy-lexical-viral_1979_v16

model_name: zonemercy-lexical-viral_1979_v16

model_group: zonemercy/Lexical-Viral-

status: inactive

timestamp: 2024-11-30T18:04:45+00:00

num_battles: 12699

num_wins: 6146

celo_rating: 1253.14

family_friendly_score: 0.6044

family_friendly_standard_error: 0.006915209902815677

submission_type: basic

model_repo: zonemercy/Lexical-Viral-v6ava-22b11e5r128

model_architecture: MistralForCausalLM

model_num_parameters: 22247282688.0

best_of: 4

max_input_tokens: 768

max_output_tokens: 64

latencies: [{'batch_size': 1, 'throughput': 0.41048437575993807, 'latency_mean': 2.436059491634369, 'latency_p50': 2.448710322380066, 'latency_p90': 2.6988261699676515}, {'batch_size': 3, 'throughput': 0.8948723961942103, 'latency_mean': 3.331004012823105, 'latency_p50': 3.3377184867858887, 'latency_p90': 3.672817850112915}, {'batch_size': 5, 'throughput': 1.2263079058160347, 'latency_mean': 4.060347625017166, 'latency_p50': 4.071827411651611, 'latency_p90': 4.519102239608765}, {'batch_size': 6, 'throughput': 1.3435048153797422, 'latency_mean': 4.4531832957267765, 'latency_p50': 4.456693649291992, 'latency_p90': 5.043470478057861}, {'batch_size': 8, 'throughput': 1.5160077550646187, 'latency_mean': 5.227090052366257, 'latency_p50': 5.240728855133057, 'latency_p90': 5.902246975898742}, {'batch_size': 10, 'throughput': 1.6499170850877396, 'latency_mean': 6.010663371086121, 'latency_p50': 5.966573715209961, 'latency_p90': 6.823694729804992}]

gpu_counts: {'NVIDIA RTX A6000': 1}

display_name: zonemercy-lexical-viral_1979_v16

is_internal_developer: True

language_model: zonemercy/Lexical-Viral-v6ava-22b11e5r128

model_size: 22B

ranking_group: single

throughput_3p7s: 1.08

us_pacific_date: 2024-11-30

win_ratio: 0.483975116150878

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '####', 'Bot:', 'User:', 'You:', '<|im_end|>', '<|eot_id|>'], 'max_input_tokens': 768, 'best_of': 4, 'max_output_tokens': 64}

formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name zonemercy-lexical-viral-1979-v16-mkmlizer
Waiting for job on zonemercy-lexical-viral-1979-v16-mkmlizer to finish
zonemercy-lexical-viral-1979-v16-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
zonemercy-lexical-viral-1979-v16-mkmlizer: ║     _____            __           __                                ║
zonemercy-lexical-viral-1979-v16-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
zonemercy-lexical-viral-1979-v16-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
zonemercy-lexical-viral-1979-v16-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
zonemercy-lexical-viral-1979-v16-mkmlizer: ║       /___/                                                         ║
zonemercy-lexical-viral-1979-v16-mkmlizer: ║                                                                     ║
zonemercy-lexical-viral-1979-v16-mkmlizer: ║  Version: 0.11.12                                                   ║
zonemercy-lexical-viral-1979-v16-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
zonemercy-lexical-viral-1979-v16-mkmlizer: ║  https://mk1.ai                                                     ║
zonemercy-lexical-viral-1979-v16-mkmlizer: ║                                                                     ║
zonemercy-lexical-viral-1979-v16-mkmlizer: ║  The license key for the current software has been verified as      ║
zonemercy-lexical-viral-1979-v16-mkmlizer: ║  belonging to:                                                      ║
zonemercy-lexical-viral-1979-v16-mkmlizer: ║                                                                     ║
zonemercy-lexical-viral-1979-v16-mkmlizer: ║  Chai Research Corp.                                                ║
zonemercy-lexical-viral-1979-v16-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
zonemercy-lexical-viral-1979-v16-mkmlizer: ║  Expiration: 2025-01-15 23:59:59                                    ║
zonemercy-lexical-viral-1979-v16-mkmlizer: ║                                                                     ║
zonemercy-lexical-viral-1979-v16-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
zonemercy-lexical-viral-1979-v16-mkmlizer: Downloaded to shared memory in 57.385s
zonemercy-lexical-viral-1979-v16-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpng1mv91f, device:0
zonemercy-lexical-viral-1979-v16-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
zonemercy-lexical-viral-1979-v16-mkmlizer: quantized model in 42.944s
zonemercy-lexical-viral-1979-v16-mkmlizer: Processed model zonemercy/Lexical-Viral-v6ava-22b11e5r128 in 100.329s
zonemercy-lexical-viral-1979-v16-mkmlizer: creating bucket guanaco-mkml-models
zonemercy-lexical-viral-1979-v16-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
zonemercy-lexical-viral-1979-v16-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/zonemercy-lexical-viral-1979-v16
zonemercy-lexical-viral-1979-v16-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/zonemercy-lexical-viral-1979-v16/config.json
zonemercy-lexical-viral-1979-v16-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/zonemercy-lexical-viral-1979-v16/special_tokens_map.json
zonemercy-lexical-viral-1979-v16-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/zonemercy-lexical-viral-1979-v16/tokenizer_config.json
zonemercy-lexical-viral-1979-v16-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/zonemercy-lexical-viral-1979-v16/tokenizer.json
zonemercy-lexical-viral-1979-v16-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/zonemercy-lexical-viral-1979-v16/flywheel_model.1.safetensors
zonemercy-lexical-viral-1979-v16-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/zonemercy-lexical-viral-1979-v16/flywheel_model.0.safetensors
zonemercy-lexical-viral-1979-v16-mkmlizer: 
Loading 0:   0%|          | 0/507 [00:00<?, ?it/s]
Loading 0:   1%|          | 5/507 [00:00<00:20, 24.64it/s]
Loading 0:   2%|▏         | 12/507 [00:00<00:12, 41.01it/s]
Loading 0:   3%|▎         | 17/507 [00:00<00:12, 39.69it/s]
Loading 0:   4%|▍         | 22/507 [00:00<00:11, 40.84it/s]
Loading 0:   5%|▌         | 27/507 [00:00<00:11, 42.52it/s]
Loading 0:   6%|▋         | 32/507 [00:00<00:13, 35.61it/s]
Loading 0:   8%|▊         | 39/507 [00:00<00:10, 43.45it/s]
Loading 0:   9%|▊         | 44/507 [00:01<00:10, 43.40it/s]
Loading 0:  10%|▉         | 49/507 [00:01<00:12, 37.99it/s]
Loading 0:  11%|█         | 54/507 [00:01<00:15, 29.34it/s]
Loading 0:  11%|█▏        | 58/507 [00:01<00:15, 29.70it/s]
Loading 0:  12%|█▏        | 63/507 [00:01<00:13, 32.92it/s]
Loading 0:  13%|█▎        | 67/507 [00:01<00:13, 33.33it/s]
Loading 0:  14%|█▍        | 73/507 [00:02<00:11, 36.23it/s]
Loading 0:  16%|█▌        | 80/507 [00:02<00:11, 38.23it/s]
Loading 0:  17%|█▋        | 85/507 [00:02<00:10, 40.01it/s]
Loading 0:  18%|█▊        | 90/507 [00:02<00:10, 38.36it/s]
Loading 0:  19%|█▉        | 96/507 [00:02<00:09, 41.91it/s]
Loading 0:  20%|█▉        | 101/507 [00:02<00:09, 40.77it/s]
Loading 0:  21%|██        | 106/507 [00:02<00:09, 40.37it/s]
Loading 0:  22%|██▏       | 111/507 [00:02<00:09, 40.83it/s]
Loading 0:  23%|██▎       | 116/507 [00:03<00:15, 25.71it/s]
Loading 0:  24%|██▍       | 122/507 [00:03<00:13, 28.38it/s]
Loading 0:  25%|██▌       | 129/507 [00:03<00:10, 34.86it/s]
Loading 0:  26%|██▋       | 134/507 [00:03<00:10, 35.29it/s]
Loading 0:  27%|██▋       | 138/507 [00:03<00:10, 36.19it/s]
Loading 0:  28%|██▊       | 142/507 [00:03<00:10, 35.71it/s]
Loading 0:  29%|██▉       | 147/507 [00:04<00:09, 39.00it/s]
Loading 0:  30%|██▉       | 152/507 [00:04<00:09, 38.83it/s]
Loading 0:  31%|███       | 157/507 [00:04<00:08, 39.58it/s]
Loading 0:  32%|███▏      | 162/507 [00:04<00:08, 39.89it/s]
Loading 0:  33%|███▎      | 167/507 [00:04<00:08, 40.83it/s]
Loading 0:  34%|███▍      | 172/507 [00:04<00:13, 25.60it/s]
Loading 0:  35%|███▍      | 176/507 [00:05<00:12, 26.07it/s]
Loading 0:  36%|███▌      | 183/507 [00:05<00:09, 33.75it/s]
Loading 0:  37%|███▋      | 188/507 [00:05<00:09, 35.06it/s]
Loading 0:  38%|███▊      | 193/507 [00:05<00:08, 37.06it/s]
Loading 0:  39%|███▉      | 198/507 [00:05<00:07, 39.48it/s]
Loading 0:  40%|████      | 203/507 [00:05<00:08, 35.16it/s]
Loading 0:  42%|████▏     | 211/507 [00:05<00:06, 42.92it/s]
Loading 0:  43%|████▎     | 217/507 [00:05<00:06, 41.85it/s]
Loading 0:  44%|████▍     | 222/507 [00:06<00:07, 39.33it/s]
Loading 0:  45%|████▍     | 227/507 [00:06<00:08, 31.75it/s]
Loading 0:  46%|████▌     | 231/507 [00:06<00:09, 30.32it/s]
Loading 0:  47%|████▋     | 237/507 [00:06<00:07, 35.04it/s]
Loading 0:  48%|████▊     | 241/507 [00:06<00:07, 35.35it/s]
Loading 0:  49%|████▊     | 246/507 [00:06<00:06, 38.23it/s]
Loading 0:  50%|████▉     | 251/507 [00:06<00:06, 38.87it/s]
Loading 0:  50%|█████     | 256/507 [00:07<00:06, 39.28it/s]
Loading 0:  51%|█████▏    | 261/507 [00:07<00:05, 41.26it/s]
Loading 0:  52%|█████▏    | 266/507 [00:07<00:07, 32.94it/s]
Loading 0:  53%|█████▎    | 271/507 [00:07<00:06, 36.07it/s]
Loading 0:  54%|█████▍    | 275/507 [00:07<00:07, 32.01it/s]
Loading 0:  55%|█████▌    | 280/507 [00:07<00:06, 35.73it/s]
Loading 0:  56%|█████▌    | 284/507 [00:08<00:07, 28.61it/s]
Loading 0:  57%|█████▋    | 288/507 [00:08<00:07, 27.49it/s]
Loading 0:  58%|█████▊    | 293/507 [00:08<00:07, 28.46it/s]
Loading 0:  59%|█████▉    | 299/507 [00:22<00:07, 28.46it/s]
Loading 0:  59%|█████▉    | 300/507 [00:22<02:53,  1.19it/s]
Loading 0:  60%|█████▉    | 302/507 [00:22<02:30,  1.36it/s]
Loading 0:  61%|██████    | 307/507 [00:23<01:39,  2.00it/s]
Loading 0:  61%|██████    | 310/507 [00:23<01:19,  2.49it/s]
Loading 0:  62%|██████▏   | 314/507 [00:23<00:55,  3.45it/s]
Loading 0:  63%|██████▎   | 319/507 [00:23<00:37,  5.06it/s]
Loading 0:  64%|██████▍   | 324/507 [00:23<00:25,  7.18it/s]
Loading 0:  65%|██████▍   | 328/507 [00:23<00:19,  9.16it/s]
Loading 0:  65%|██████▌   | 332/507 [00:23<00:15, 11.49it/s]
Loading 0:  66%|██████▋   | 336/507 [00:23<00:11, 14.37it/s]
Loading 0:  67%|██████▋   | 340/507 [00:24<00:11, 14.55it/s]
Loading 0:  68%|██████▊   | 343/507 [00:24<00:10, 15.98it/s]
Loading 0:  68%|██████▊   | 347/507 [00:24<00:08, 18.44it/s]
Loading 0:  70%|██████▉   | 354/507 [00:24<00:05, 26.98it/s]
Loading 0:  71%|███████   | 358/507 [00:24<00:05, 29.07it/s]
Loading 0:  72%|███████▏  | 363/507 [00:24<00:04, 33.33it/s]
Loading 0:  73%|███████▎  | 368/507 [00:24<00:04, 33.40it/s]
Loading 0:  74%|███████▎  | 373/507 [00:25<00:03, 35.58it/s]
Loading 0:  75%|███████▍  | 378/507 [00:25<00:03, 38.68it/s]
Loading 0:  76%|███████▌  | 383/507 [00:25<00:03, 34.28it/s]
Loading 0:  77%|███████▋  | 389/507 [00:25<00:02, 40.15it/s]
Loading 0:  78%|███████▊  | 394/507 [00:25<00:03, 37.29it/s]
Loading 0:  79%|███████▊  | 399/507 [00:25<00:03, 31.69it/s]
Loading 0:  79%|███████▉  | 403/507 [00:25<00:03, 31.51it/s]
Loading 0:  80%|████████  | 408/507 [00:26<00:02, 34.65it/s]
Loading 0:  81%|████████▏ | 412/507 [00:26<00:02, 34.60it/s]
Loading 0:  82%|████████▏ | 417/507 [00:26<00:02, 37.56it/s]
Loading 0:  83%|████████▎ | 421/507 [00:26<00:02, 35.50it/s]
Loading 0:  84%|████████▍ | 426/507 [00:26<00:02, 39.14it/s]
Loading 0:  85%|████████▌ | 431/507 [00:26<00:01, 39.69it/s]
Loading 0:  86%|████████▌ | 436/507 [00:26<00:01, 40.81it/s]
Loading 0:  87%|████████▋ | 442/507 [00:26<00:01, 41.40it/s]
Loading 0:  88%|████████▊ | 447/507 [00:27<00:01, 40.53it/s]
Loading 0:  90%|████████▉ | 454/507 [00:27<00:01, 44.63it/s]
Loading 0:  91%|█████████ | 459/507 [00:29<00:06,  7.03it/s]
Loading 0:  92%|█████████▏| 465/507 [00:29<00:04,  9.47it/s]
Loading 0:  93%|█████████▎| 472/507 [00:29<00:02, 13.37it/s]
Loading 0:  94%|█████████▍| 477/507 [00:29<00:01, 16.07it/s]
Loading 0:  95%|█████████▌| 482/507 [00:29<00:01, 19.32it/s]
Loading 0:  96%|█████████▌| 487/507 [00:30<00:00, 22.92it/s]
Loading 0:  97%|█████████▋| 492/507 [00:30<00:00, 23.61it/s]
Loading 0:  98%|█████████▊| 499/507 [00:30<00:00, 29.99it/s]
Loading 0:  99%|█████████▉| 504/507 [00:30<00:00, 31.35it/s]
                                                            
Job zonemercy-lexical-viral-1979-v16-mkmlizer completed after 134.58s with status: succeeded
Stopping job with name zonemercy-lexical-viral-1979-v16-mkmlizer
Pipeline stage MKMLizer completed in 135.08s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service zonemercy-lexical-viral-1979-v16
Waiting for inference service zonemercy-lexical-viral-1979-v16 to be ready
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Inference service zonemercy-lexical-viral-1979-v16 ready after 140.77839875221252s
Pipeline stage MKMLDeployer completed in 141.31s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.0498507022857666s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Received healthy response to inference request in 2.801923990249634s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Received healthy response to inference request in 2.3581490516662598s
Received healthy response to inference request in 2.2183752059936523s
Received healthy response to inference request in 2.406497001647949s
5 requests
0 failed requests
5th percentile: 2.246329975128174
10th percentile: 2.2742847442626952
20th percentile: 2.330194282531738
30th percentile: 2.3678186416625975
40th percentile: 2.3871578216552733
50th percentile: 2.406497001647949
60th percentile: 2.564667797088623
70th percentile: 2.7228385925292966
80th percentile: 2.8515093326568604
90th percentile: 2.9506800174713135
95th percentile: 3.00026535987854
99th percentile: 3.0399336338043215
mean time: 2.566959190368652
Pipeline stage StressChecker completed in 14.24s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.33s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 2.04s
Shutdown handler de-registered
zonemercy-lexical-viral_1979_v16 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3561.18s
Shutdown handler de-registered
zonemercy-lexical-viral_1979_v16 status is now inactive due to auto deactivation removed underperforming models