chaiml-lexical-viral-v5a_2693

developer_uid: zonemercy

submission_id: chaiml-lexical-viral-v5a_2693_v2

model_name: tempv1-1

model_group: ChaiML/Lexical-Viral-v5a

status: inactive

timestamp: 2024-12-10T17:34:21+00:00

num_battles: 12932

num_wins: 6428

celo_rating: 1260.92

family_friendly_score: 0.5784

family_friendly_standard_error: 0.006983601363193635

submission_type: basic

model_repo: ChaiML/Lexical-Viral-v5ava-12b01e5r128

model_architecture: MistralForCausalLM

model_num_parameters: 12772070400.0

best_of: 4

max_input_tokens: 1024

max_output_tokens: 64

latencies: [{'batch_size': 1, 'throughput': 0.6369428903734503, 'latency_mean': 1.5699004518985749, 'latency_p50': 1.5675791501998901, 'latency_p90': 1.7416449069976807}, {'batch_size': 4, 'throughput': 1.4720442117526717, 'latency_mean': 2.704244297742844, 'latency_p50': 2.709717869758606, 'latency_p90': 2.980261206626892}, {'batch_size': 5, 'throughput': 1.614190751736841, 'latency_mean': 3.087519578933716, 'latency_p50': 3.0972968339920044, 'latency_p90': 3.480712914466858}, {'batch_size': 8, 'throughput': 1.8904452484243954, 'latency_mean': 4.1942414999008175, 'latency_p50': 4.184686064720154, 'latency_p90': 4.747991967201233}, {'batch_size': 10, 'throughput': 1.997280081693025, 'latency_mean': 4.970580780506134, 'latency_p50': 4.971385717391968, 'latency_p90': 5.554191470146179}, {'batch_size': 12, 'throughput': 2.034730344116971, 'latency_mean': 5.843570767641068, 'latency_p50': 5.822869300842285, 'latency_p90': 6.539579439163208}, {'batch_size': 15, 'throughput': 2.0763097227017075, 'latency_mean': 7.146173708438873, 'latency_p50': 7.276411175727844, 'latency_p90': 8.165987730026245}]

gpu_counts: {'NVIDIA RTX A5000': 1}

display_name: tempv1-1

is_internal_developer: True

language_model: ChaiML/Lexical-Viral-v5ava-12b01e5r128

model_size: 13B

ranking_group: single

throughput_3p7s: 1.8

us_pacific_date: 2024-12-10

win_ratio: 0.4970615527373956

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '####', 'Bot:', 'User:', 'You:', '<|im_end|>', '<|eot_id|>'], 'max_input_tokens': 1024, 'best_of': 4, 'max_output_tokens': 64}

formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-lexical-viral-v5a-2693-v2-mkmlizer
Waiting for job on chaiml-lexical-viral-v5a-2693-v2-mkmlizer to finish
chaiml-lexical-viral-v5a-2693-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-lexical-viral-v5a-2693-v2-mkmlizer: ║     _____            __           __                                ║
chaiml-lexical-viral-v5a-2693-v2-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
chaiml-lexical-viral-v5a-2693-v2-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
chaiml-lexical-viral-v5a-2693-v2-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
chaiml-lexical-viral-v5a-2693-v2-mkmlizer: ║       /___/                                                         ║
chaiml-lexical-viral-v5a-2693-v2-mkmlizer: ║                                                                     ║
chaiml-lexical-viral-v5a-2693-v2-mkmlizer: ║  Version: 0.11.12                                                   ║
chaiml-lexical-viral-v5a-2693-v2-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
chaiml-lexical-viral-v5a-2693-v2-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-lexical-viral-v5a-2693-v2-mkmlizer: ║                                                                     ║
chaiml-lexical-viral-v5a-2693-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-lexical-viral-v5a-2693-v2-mkmlizer: ║  belonging to:                                                      ║
chaiml-lexical-viral-v5a-2693-v2-mkmlizer: ║                                                                     ║
chaiml-lexical-viral-v5a-2693-v2-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-lexical-viral-v5a-2693-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-lexical-viral-v5a-2693-v2-mkmlizer: ║  Expiration: 2025-01-15 23:59:59                                    ║
chaiml-lexical-viral-v5a-2693-v2-mkmlizer: ║                                                                     ║
chaiml-lexical-viral-v5a-2693-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-lexical-viral-v5a-2693-v2-mkmlizer: Downloaded to shared memory in 33.111s
chaiml-lexical-viral-v5a-2693-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpft1e6c7f, device:0
chaiml-lexical-viral-v5a-2693-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-lexical-viral-v5a-2693-v2-mkmlizer: quantized model in 39.321s
chaiml-lexical-viral-v5a-2693-v2-mkmlizer: Processed model ChaiML/Lexical-Viral-v5ava-12b01e5r128 in 72.433s
chaiml-lexical-viral-v5a-2693-v2-mkmlizer: creating bucket guanaco-mkml-models
chaiml-lexical-viral-v5a-2693-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-lexical-viral-v5a-2693-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-lexical-viral-v5a-2693-v2
chaiml-lexical-viral-v5a-2693-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-lexical-viral-v5a-2693-v2/config.json
chaiml-lexical-viral-v5a-2693-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-lexical-viral-v5a-2693-v2/special_tokens_map.json
chaiml-lexical-viral-v5a-2693-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-lexical-viral-v5a-2693-v2/tokenizer_config.json
chaiml-lexical-viral-v5a-2693-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-lexical-viral-v5a-2693-v2/tokenizer.json
chaiml-lexical-viral-v5a-2693-v2-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:15, 23.60it/s]
Loading 0:   3%|▎         | 12/363 [00:00<00:08, 41.18it/s]
Loading 0:   5%|▍         | 17/363 [00:00<00:08, 42.17it/s]
Loading 0:   6%|▌         | 22/363 [00:00<00:08, 41.95it/s]
Loading 0:   7%|▋         | 27/363 [00:00<00:07, 42.74it/s]
Loading 0:   9%|▉         | 32/363 [00:00<00:10, 32.89it/s]
Loading 0:  11%|█         | 39/363 [00:01<00:08, 40.19it/s]
Loading 0:  12%|█▏        | 44/363 [00:01<00:08, 38.74it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:08, 38.39it/s]
Loading 0:  15%|█▍        | 54/363 [00:01<00:07, 39.03it/s]
Loading 0:  16%|█▋        | 59/363 [00:01<00:07, 40.21it/s]
Loading 0:  18%|█▊        | 64/363 [00:01<00:13, 22.26it/s]
Loading 0:  19%|█▉        | 69/363 [00:02<00:11, 26.52it/s]
Loading 0:  20%|██        | 73/363 [00:02<00:11, 26.06it/s]
Loading 0:  22%|██▏       | 79/363 [00:02<00:08, 32.52it/s]
Loading 0:  23%|██▎       | 84/363 [00:02<00:08, 32.03it/s]
Loading 0:  24%|██▍       | 88/363 [00:02<00:08, 33.24it/s]
Loading 0:  25%|██▌       | 92/363 [00:02<00:08, 30.44it/s]
Loading 0:  27%|██▋       | 98/363 [00:02<00:07, 34.92it/s]
Loading 0:  28%|██▊       | 102/363 [00:03<00:07, 33.61it/s]
Loading 0:  29%|██▉       | 106/363 [00:03<00:07, 33.62it/s]
Loading 0:  31%|███       | 112/363 [00:03<00:06, 37.53it/s]
Loading 0:  32%|███▏      | 116/363 [00:03<00:07, 34.30it/s]
Loading 0:  33%|███▎      | 120/363 [00:03<00:07, 32.29it/s]
Loading 0:  34%|███▍      | 125/363 [00:03<00:06, 35.12it/s]
Loading 0:  36%|███▌      | 129/363 [00:03<00:06, 33.81it/s]
Loading 0:  37%|███▋      | 134/363 [00:03<00:06, 36.46it/s]
Loading 0:  38%|███▊      | 138/363 [00:04<00:06, 34.03it/s]
Loading 0:  39%|███▉      | 142/363 [00:04<00:09, 23.17it/s]
Loading 0:  40%|███▉      | 145/363 [00:04<00:09, 23.33it/s]
Loading 0:  41%|████      | 149/363 [00:04<00:08, 24.21it/s]
Loading 0:  43%|████▎     | 156/363 [00:04<00:06, 33.02it/s]
Loading 0:  44%|████▍     | 160/363 [00:04<00:06, 33.42it/s]
Loading 0:  45%|████▌     | 165/363 [00:04<00:05, 36.93it/s]
Loading 0:  47%|████▋     | 170/363 [00:05<00:05, 36.95it/s]
Loading 0:  48%|████▊     | 175/363 [00:05<00:04, 38.19it/s]
Loading 0:  49%|████▉     | 179/363 [00:05<00:04, 37.20it/s]
Loading 0:  50%|█████     | 183/363 [00:05<00:04, 37.53it/s]
Loading 0:  52%|█████▏    | 187/363 [00:05<00:04, 35.75it/s]
Loading 0:  53%|█████▎    | 192/363 [00:05<00:04, 39.13it/s]
Loading 0:  54%|█████▍    | 196/363 [00:05<00:04, 36.79it/s]
Loading 0:  55%|█████▌    | 201/363 [00:05<00:04, 38.76it/s]
Loading 0:  56%|█████▋    | 205/363 [00:06<00:04, 36.45it/s]
Loading 0:  58%|█████▊    | 210/363 [00:06<00:03, 39.47it/s]
Loading 0:  59%|█████▉    | 215/363 [00:06<00:03, 39.15it/s]
Loading 0:  60%|██████    | 219/363 [00:06<00:03, 38.34it/s]
Loading 0:  61%|██████▏   | 223/363 [00:06<00:05, 24.82it/s]
Loading 0:  63%|██████▎   | 227/363 [00:06<00:05, 26.15it/s]
Loading 0:  64%|██████▎   | 231/363 [00:07<00:05, 25.96it/s]
Loading 0:  65%|██████▌   | 237/363 [00:07<00:03, 31.95it/s]
Loading 0:  66%|██████▋   | 241/363 [00:07<00:03, 31.62it/s]
Loading 0:  68%|██████▊   | 246/363 [00:07<00:03, 34.37it/s]
Loading 0:  69%|██████▉   | 250/363 [00:07<00:03, 33.34it/s]
Loading 0:  70%|███████   | 255/363 [00:07<00:03, 35.89it/s]
Loading 0:  71%|███████▏  | 259/363 [00:07<00:03, 34.22it/s]
Loading 0:  73%|███████▎  | 264/363 [00:07<00:02, 36.42it/s]
Loading 0:  74%|███████▍  | 268/363 [00:08<00:02, 34.33it/s]
Loading 0:  75%|███████▌  | 273/363 [00:08<00:02, 36.94it/s]
Loading 0:  76%|███████▋  | 277/363 [00:08<00:02, 35.08it/s]
Loading 0:  78%|███████▊  | 282/363 [00:08<00:02, 38.07it/s]
Loading 0:  79%|███████▉  | 286/363 [00:08<00:02, 36.15it/s]
Loading 0:  80%|████████  | 291/363 [00:08<00:01, 38.06it/s]
Loading 0:  81%|████████▏ | 295/363 [00:08<00:01, 35.91it/s]
Loading 0:  82%|████████▏ | 299/363 [00:08<00:01, 35.84it/s]
Loading 0:  84%|████████▎ | 304/363 [00:15<00:28,  2.04it/s]
Loading 0:  85%|████████▍ | 307/363 [00:16<00:21,  2.56it/s]
Loading 0:  86%|████████▌ | 312/363 [00:16<00:13,  3.80it/s]
Loading 0:  88%|████████▊ | 320/363 [00:16<00:06,  6.53it/s]
Loading 0:  90%|████████▉ | 326/363 [00:16<00:04,  9.00it/s]
Loading 0:  91%|█████████ | 331/363 [00:16<00:02, 11.54it/s]
Loading 0:  93%|█████████▎| 338/363 [00:16<00:01, 16.23it/s]
Loading 0:  95%|█████████▍| 344/363 [00:16<00:00, 20.03it/s]
Loading 0:  96%|█████████▌| 349/363 [00:16<00:00, 23.35it/s]
Loading 0:  98%|█████████▊| 356/363 [00:16<00:00, 29.56it/s]
Loading 0:  99%|█████████▉| 361/363 [00:17<00:00, 32.94it/s]
                                                            
Job chaiml-lexical-viral-v5a-2693-v2-mkmlizer completed after 104.17s with status: succeeded
Stopping job with name chaiml-lexical-viral-v5a-2693-v2-mkmlizer
Pipeline stage MKMLizer completed in 104.71s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.14s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-lexical-viral-v5a-2693-v2
Waiting for inference service chaiml-lexical-viral-v5a-2693-v2 to be ready
Inference service chaiml-lexical-viral-v5a-2693-v2 ready after 180.71931195259094s
Pipeline stage MKMLDeployer completed in 181.25s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.0940873622894287s
Received healthy response to inference request in 1.6406426429748535s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Received healthy response to inference request in 1.7895939350128174s
Received healthy response to inference request in 1.5432348251342773s
Received healthy response to inference request in 1.58207368850708s
5 requests
0 failed requests
5th percentile: 1.551002597808838
10th percentile: 1.5587703704833984
20th percentile: 1.5743059158325194
30th percentile: 1.5937874794006348
40th percentile: 1.617215061187744
50th percentile: 1.6406426429748535
60th percentile: 1.700223159790039
70th percentile: 1.7598036766052245
80th percentile: 1.8504926204681398
90th percentile: 1.9722899913787841
95th percentile: 2.0331886768341065
99th percentile: 2.0819076251983644
mean time: 1.7299264907836913
Pipeline stage StressChecker completed in 9.93s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.20s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 2.07s
Shutdown handler de-registered
chaiml-lexical-viral-v5a_2693_v2 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage MKMLProfilerDeleter completed in 0.10s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.09s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service chaiml-lexical-viral-v5a-2693-v2-profiler
Waiting for inference service chaiml-lexical-viral-v5a-2693-v2-profiler to be ready
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2483.43s
Shutdown handler de-registered
chaiml-lexical-viral-v5a_2693_v2 status is now inactive due to auto deactivation removed underperforming models