submission_id: zonemercy-lexical-viral-_1616_v5
developer_uid: chai_backend_admin
best_of: 4
celo_rating: 1271.25
display_name: tempv1-4
family_friendly_score: 0.5932
family_friendly_standard_error: 0.006947139843129689
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '####', 'Bot:', 'User:', 'You:', '<|im_end|>', '<|eot_id|>'], 'max_input_tokens': 1024, 'best_of': 4, 'max_output_tokens': 64}
gpu_counts: {'NVIDIA RTX A6000': 1}
is_internal_developer: True
language_model: zonemercy/Lexical-Viral-v6ava-22b11e5r256d01
latencies: [{'batch_size': 1, 'throughput': 0.38908166967060737, 'latency_mean': 2.5700636637210845, 'latency_p50': 2.5747530460357666, 'latency_p90': 2.8389044523239133}, {'batch_size': 3, 'throughput': 0.8257571423175062, 'latency_mean': 3.6172175657749177, 'latency_p50': 3.6344447135925293, 'latency_p90': 3.98761043548584}, {'batch_size': 5, 'throughput': 1.1061590419411735, 'latency_mean': 4.495504372119903, 'latency_p50': 4.494860768318176, 'latency_p90': 4.941921901702881}, {'batch_size': 6, 'throughput': 1.1848550443257768, 'latency_mean': 5.0366358160972595, 'latency_p50': 5.008782505989075, 'latency_p90': 5.788398337364197}, {'batch_size': 10, 'throughput': 1.3962269289328244, 'latency_mean': 7.08331800699234, 'latency_p50': 7.042068719863892, 'latency_p90': 8.09586741924286}]
max_input_tokens: 1024
max_output_tokens: 64
model_architecture: MistralForCausalLM
model_group: zonemercy/Lexical-Viral-
model_name: tempv1-4
model_num_parameters: 22247282688.0
model_repo: zonemercy/Lexical-Viral-v6ava-22b11e5r256d01
model_size: 22B
num_battles: 9049
num_wins: 4746
ranking_group: single
status: inactive
submission_type: basic
throughput_3p7s: 0.86
timestamp: 2024-11-20T14:50:28+00:00
us_pacific_date: 2024-11-20
win_ratio: 0.5244778428555642
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name zonemercy-lexical-viral-1616-v5-mkmlizer
Waiting for job on zonemercy-lexical-viral-1616-v5-mkmlizer to finish
zonemercy-lexical-viral-1616-v5-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
zonemercy-lexical-viral-1616-v5-mkmlizer: ║ _____ __ __ ║
zonemercy-lexical-viral-1616-v5-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
zonemercy-lexical-viral-1616-v5-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
zonemercy-lexical-viral-1616-v5-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
zonemercy-lexical-viral-1616-v5-mkmlizer: ║ /___/ ║
zonemercy-lexical-viral-1616-v5-mkmlizer: ║ ║
zonemercy-lexical-viral-1616-v5-mkmlizer: ║ Version: 0.11.12 ║
zonemercy-lexical-viral-1616-v5-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
zonemercy-lexical-viral-1616-v5-mkmlizer: ║ https://mk1.ai ║
zonemercy-lexical-viral-1616-v5-mkmlizer: ║ ║
zonemercy-lexical-viral-1616-v5-mkmlizer: ║ The license key for the current software has been verified as ║
zonemercy-lexical-viral-1616-v5-mkmlizer: ║ belonging to: ║
zonemercy-lexical-viral-1616-v5-mkmlizer: ║ ║
zonemercy-lexical-viral-1616-v5-mkmlizer: ║ Chai Research Corp. ║
zonemercy-lexical-viral-1616-v5-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
zonemercy-lexical-viral-1616-v5-mkmlizer: ║ Expiration: 2025-01-15 23:59:59 ║
zonemercy-lexical-viral-1616-v5-mkmlizer: ║ ║
zonemercy-lexical-viral-1616-v5-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
zonemercy-lexical-viral-1616-v5-mkmlizer: Downloaded to shared memory in 62.859s
zonemercy-lexical-viral-1616-v5-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpklc_iv93, device:0
zonemercy-lexical-viral-1616-v5-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
zonemercy-lexical-viral-1616-v5-mkmlizer: creating bucket guanaco-mkml-models
zonemercy-lexical-viral-1616-v5-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
zonemercy-lexical-viral-1616-v5-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/zonemercy-lexical-viral-1616-v5
zonemercy-lexical-viral-1616-v5-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/zonemercy-lexical-viral-1616-v5/config.json
zonemercy-lexical-viral-1616-v5-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/zonemercy-lexical-viral-1616-v5/special_tokens_map.json
zonemercy-lexical-viral-1616-v5-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/zonemercy-lexical-viral-1616-v5/tokenizer_config.json
zonemercy-lexical-viral-1616-v5-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/zonemercy-lexical-viral-1616-v5/tokenizer.json
zonemercy-lexical-viral-1616-v5-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/zonemercy-lexical-viral-1616-v5/flywheel_model.0.safetensors
zonemercy-lexical-viral-1616-v5-mkmlizer: Loading 0: 0%| | 0/507 [00:00<?, ?it/s] Loading 0: 1%| | 5/507 [00:00<00:21, 23.47it/s] Loading 0: 2%|▏ | 12/507 [00:00<00:13, 37.52it/s] Loading 0: 3%|▎ | 17/507 [00:00<00:13, 35.93it/s] Loading 0: 4%|▍ | 21/507 [00:00<00:13, 36.48it/s] Loading 0: 5%|▍ | 25/507 [00:00<00:14, 34.16it/s] Loading 0: 6%|▌ | 30/507 [00:00<00:12, 37.59it/s] Loading 0: 7%|▋ | 34/507 [00:00<00:13, 34.93it/s] Loading 0: 8%|▊ | 39/507 [00:01<00:12, 37.45it/s] Loading 0: 8%|▊ | 43/507 [00:01<00:12, 37.03it/s] Loading 0: 9%|▉ | 47/507 [00:01<00:12, 37.11it/s] Loading 0: 10%|█ | 51/507 [00:01<00:13, 34.52it/s] Loading 0: 11%|█ | 55/507 [00:01<00:19, 23.66it/s] Loading 0: 12%|█▏ | 59/507 [00:01<00:17, 26.26it/s] Loading 0: 12%|█▏ | 63/507 [00:01<00:15, 28.05it/s] Loading 0: 13%|█▎ | 67/507 [00:02<00:15, 29.23it/s] Loading 0: 14%|█▍ | 72/507 [00:02<00:12, 33.88it/s] Loading 0: 15%|█▌ | 78/507 [00:02<00:11, 37.51it/s] Loading 0: 16%|█▌ | 82/507 [00:02<00:12, 34.76it/s] Loading 0: 17%|█▋ | 87/507 [00:02<00:11, 36.01it/s] Loading 0: 18%|█▊ | 91/507 [00:02<00:11, 35.23it/s] Loading 0: 19%|█▉ | 96/507 [00:02<00:11, 36.25it/s] Loading 0: 20%|█▉ | 100/507 [00:02<00:11, 33.93it/s] Loading 0: 21%|██ | 105/507 [00:03<00:11, 35.79it/s] Loading 0: 21%|██▏ | 109/507 [00:03<00:11, 34.55it/s] Loading 0: 22%|██▏ | 113/507 [00:03<00:16, 24.56it/s] Loading 0: 23%|██▎ | 116/507 [00:03<00:17, 22.68it/s] Loading 0: 24%|██▍ | 121/507 [00:03<00:13, 28.08it/s] Loading 0: 25%|██▍ | 125/507 [00:03<00:12, 29.71it/s] Loading 0: 25%|██▌ | 129/507 [00:04<00:12, 31.06it/s] Loading 0: 26%|██▌ | 133/507 [00:04<00:12, 30.73it/s] Loading 0: 27%|██▋ | 138/507 [00:04<00:11, 33.22it/s] Loading 0: 28%|██▊ | 142/507 [00:04<00:11, 32.18it/s] Loading 0: 29%|██▉ | 147/507 [00:04<00:10, 34.72it/s] Loading 0: 30%|██▉ | 151/507 [00:04<00:10, 33.55it/s] Loading 0: 31%|███ | 156/507 [00:04<00:09, 36.91it/s] Loading 0: 32%|███▏ | 160/507 [00:04<00:09, 35.72it/s] Loading 0: 32%|███▏ | 164/507 [00:05<00:09, 34.95it/s] Loading 0: 33%|███▎ | 169/507 [00:05<00:12, 27.92it/s] Loading 0: 34%|███▍ | 173/507 [00:05<00:11, 28.75it/s] Loading 0: 35%|███▍ | 177/507 [00:05<00:11, 28.17it/s] Loading 0: 36%|███▌ | 183/507 [00:05<00:09, 33.08it/s] Loading 0: 37%|███▋ | 187/507 [00:05<00:10, 31.67it/s] Loading 0: 38%|███▊ | 192/507 [00:05<00:09, 34.43it/s] Loading 0: 39%|███▊ | 196/507 [00:06<00:09, 33.72it/s] Loading 0: 40%|███▉ | 201/507 [00:06<00:08, 36.51it/s] Loading 0: 40%|████ | 205/507 [00:06<00:08, 35.06it/s] Loading 0: 41%|████ | 209/507 [00:06<00:08, 35.76it/s] Loading 0: 42%|████▏ | 213/507 [00:06<00:09, 32.02it/s] Loading 0: 43%|████▎ | 218/507 [00:06<00:08, 35.08it/s] Loading 0: 44%|████▍ | 222/507 [00:06<00:08, 33.79it/s] Loading 0: 45%|████▍ | 226/507 [00:07<00:10, 25.75it/s] Loading 0: 45%|████▌ | 230/507 [00:07<00:10, 25.62it/s] Loading 0: 47%|████▋ | 237/507 [00:07<00:08, 32.66it/s] Loading 0: 48%|████▊ | 241/507 [00:07<00:08, 32.40it/s] Loading 0: 49%|████▊ | 246/507 [00:07<00:07, 35.00it/s] Loading 0: 49%|████▉ | 250/507 [00:07<00:07, 34.29it/s] Loading 0: 50%|█████ | 255/507 [00:07<00:07, 35.36it/s] Loading 0: 51%|█████ | 259/507 [00:07<00:07, 34.63it/s] Loading 0: 52%|█████▏ | 264/507 [00:08<00:06, 36.44it/s] Loading 0: 53%|█████▎ | 268/507 [00:08<00:06, 34.21it/s] Loading 0: 54%|█████▍ | 273/507 [00:08<00:06, 35.90it/s] Loading 0: 55%|█████▍ | 277/507 [00:08<00:06, 34.27it/s] Loading 0: 56%|█████▌ | 283/507 [00:08<00:06, 34.24it/s] Loading 0: 57%|█████▋ | 287/507 [00:08<00:09, 22.77it/s] Loading 0: 58%|█████▊ | 293/507 [00:09<00:08, 25.88it/s] Loading 0: 59%|█████▉ | 299/507 [00:23<00:08, 25.88it/s] Loading 0: 59%|█████▉ | 300/507 [00:23<02:42, 1.28it/s] Loading 0: 60%|█████▉ | 302/507 [00:23<02:21, 1.45it/s] Loading 0: 61%|██████ | 307/507 [00:23<01:35, 2.09it/s] Loading 0: 61%|██████ | 310/507 [00:23<01:16, 2.58it/s] Loading 0: 62%|██████▏ | 313/507 [00:23<00:59, 3.29it/s] Loading 0: 63%|██████▎ | 318/507 [00:24<00:38, 4.91it/s] Loading 0: 64%|██████▎ | 322/507 [00:24<00:28, 6.50it/s] Loading 0: 64%|██████▍ | 327/507 [00:24<00:19, 9.14it/s] Loading 0: 65%|██████▌ | 331/507 [00:24<00:15, 11.37it/s] Loading 0: 66%|██████▌ | 335/507 [00:24<00:12, 14.04it/s] Loading 0: 67%|██████▋ | 340/507 [00:24<00:10, 15.59it/s] Loading 0: 68%|██████▊ | 343/507 [00:25<00:09, 16.73it/s] Loading 0: 68%|██████▊ | 347/507 [00:25<00:08, 18.49it/s] Loading 0: 70%|██████▉ | 354/507 [00:25<00:05, 26.21it/s] Loading 0: 71%|███████ | 358/507 [00:25<00:05, 27.73it/s] Loading 0: 72%|███████▏ | 363/507 [00:25<00:04, 32.01it/s] Loading 0: 72%|███████▏ | 367/507 [00:25<00:04, 31.63it/s] Loading 0: 73%|███████▎ | 372/507 [00:25<00:03, 34.19it/s] Loading 0: 74%|███████▍ | 376/507 [00:25<00:03, 33.22it/s] Loading 0: 75%|███████▌ | 381/507 [00:26<00:03, 36.18it/s] Loading 0: 76%|███████▌ | 385/507 [00:26<00:03, 34.92it/s] Loading 0: 77%|███████▋ | 389/507 [00:26<00:03, 34.44it/s] Loading 0: 78%|███████▊ | 393/507 [00:26<00:03, 33.51it/s] Loading 0: 78%|███████▊ | 397/507 [00:26<00:04, 25.49it/s] Loading 0: 79%|███████▉ | 401/507 [00:26<00:04, 25.35it/s] Loading 0: 80%|████████ | 408/507 [00:26<00:03, 32.63it/s] Loading 0: 81%|████████▏ | 412/507 [00:27<00:02, 31.78it/s] Loading 0: 82%|████████▏ | 416/507 [00:27<00:02, 33.14it/s] Loading 0: 83%|████████▎ | 420/507 [00:27<00:02, 30.22it/s] Loading 0: 84%|████████▍ | 426/507 [00:27<00:02, 35.27it/s] Loading 0: 85%|████████▍ | 430/507 [00:27<00:02, 34.18it/s] Loading 0: 86%|████████▌ | 435/507 [00:27<00:01, 36.52it/s] Loading 0: 87%|████████▋ | 439/507 [00:27<00:01, 35.52it/s] Loading 0: 88%|████████▊ | 444/507 [00:27<00:01, 37.89it/s] Loading 0: 88%|████████▊ | 448/507 [00:28<00:01, 36.78it/s] Loading 0: 90%|████████▉ | 454/507 [00:28<00:01, 37.72it/s] Loading 0: 90%|█████████ | 458/507 [00:30<00:08, 6.02it/s] Loading 0: 91%|█████████ | 461/507 [00:30<00:06, 7.23it/s] Loading 0: 92%|█████████▏| 465/507 [00:30<00:04, 9.13it/s] Loading 0: 93%|█████████▎| 472/507 [00:30<00:02, 14.04it/s] Loading 0: 94%|█████████▍| 476/507 [00:30<00:01, 16.19it/s] Loading 0: 95%|█████████▍| 481/507 [00:31<00:01, 19.80it/s] Loading 0: 96%|█████████▌| 485/507 [00:31<00:00, 22.00it/s] Loading 0: 97%|█████████▋| 490/507 [00:31<00:00, 25.54it/s] Loading 0: 97%|█████████▋| 494/507 [00:31<00:00, 26.54it/s] Loading 0: 98%|█████████▊| 499/507 [00:31<00:00, 29.46it/s] Loading 0: 99%|█████████▉| 503/507 [00:31<00:00, 29.99it/s]
Job zonemercy-lexical-viral-1616-v5-mkmlizer completed after 145.56s with status: succeeded
Stopping job with name zonemercy-lexical-viral-1616-v5-mkmlizer
Pipeline stage MKMLizer completed in 146.12s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.18s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service zonemercy-lexical-viral-1616-v5
Waiting for inference service zonemercy-lexical-viral-1616-v5 to be ready
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Inference service zonemercy-lexical-viral-1616-v5 ready after 230.95651769638062s
Pipeline stage MKMLDeployer completed in 231.56s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.9047963619232178s
Received healthy response to inference request in 2.486405611038208s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Received healthy response to inference request in 2.294457197189331s
Received healthy response to inference request in 2.353567600250244s
Received healthy response to inference request in 3.1027779579162598s
5 requests
0 failed requests
5th percentile: 2.3062792778015138
10th percentile: 2.3181013584136965
20th percentile: 2.3417455196380614
30th percentile: 2.380135202407837
40th percentile: 2.4332704067230226
50th percentile: 2.486405611038208
60th percentile: 2.6537619113922117
70th percentile: 2.821118211746216
80th percentile: 2.944392681121826
90th percentile: 3.023585319519043
95th percentile: 3.0631816387176514
99th percentile: 3.094858694076538
mean time: 2.628400945663452
Pipeline stage StressChecker completed in 15.32s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.66s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 2.87s
Shutdown handler de-registered
zonemercy-lexical-viral-_1616_v5 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3170.00s
Shutdown handler de-registered
zonemercy-lexical-viral-_1616_v5 status is now inactive due to auto deactivation removed underperforming models