developer_uid: Jellywibble
submission_id: zonemercy-lexical-viral_1979_v17
model_name: zonemercy-lexical-viral_1979_v17
model_group: zonemercy/Lexical-Viral-
status: inactive
timestamp: 2024-11-30T18:05:47+00:00
num_battles: 12176
num_wins: 6164
celo_rating: 1268.48
family_friendly_score: 0.6052
family_friendly_standard_error: 0.006912784677682359
submission_type: basic
model_repo: zonemercy/Lexical-Viral-v6ava-22b11e5r128
model_architecture: MistralForCausalLM
model_num_parameters: 22247282688.0
best_of: 8
max_input_tokens: 768
max_output_tokens: 64
latencies: [{'batch_size': 1, 'throughput': 0.40389571144764946, 'latency_mean': 2.475795269012451, 'latency_p50': 2.4762803316116333, 'latency_p90': 2.7609493017196653}, {'batch_size': 3, 'throughput': 0.8474528267545219, 'latency_mean': 3.527431491613388, 'latency_p50': 3.525572657585144, 'latency_p90': 3.8737319231033323}, {'batch_size': 5, 'throughput': 1.1286441581838527, 'latency_mean': 4.415311014652252, 'latency_p50': 4.415125012397766, 'latency_p90': 4.969506359100341}, {'batch_size': 6, 'throughput': 1.2124893248182897, 'latency_mean': 4.9128923976421355, 'latency_p50': 4.933930516242981, 'latency_p90': 5.541410803794861}, {'batch_size': 8, 'throughput': 1.3418714664193059, 'latency_mean': 5.925771027803421, 'latency_p50': 5.885978102684021, 'latency_p90': 6.671813178062439}, {'batch_size': 10, 'throughput': 1.384681085030283, 'latency_mean': 7.148692475557327, 'latency_p50': 7.154859662055969, 'latency_p90': 7.9841862440109255}]
gpu_counts: {'NVIDIA RTX A6000': 1}
display_name: zonemercy-lexical-viral_1979_v17
is_internal_developer: True
language_model: zonemercy/Lexical-Viral-v6ava-22b11e5r128
model_size: 22B
ranking_group: single
throughput_3p7s: 0.92
us_pacific_date: 2024-11-30
win_ratio: 0.5062417871222076
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '####', 'Bot:', 'User:', 'You:', '<|im_end|>', '<|eot_id|>'], 'max_input_tokens': 768, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name zonemercy-lexical-viral-1979-v17-mkmlizer
Waiting for job on zonemercy-lexical-viral-1979-v17-mkmlizer to finish
zonemercy-lexical-viral-1979-v17-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
zonemercy-lexical-viral-1979-v17-mkmlizer: ║     _____            __           __                                ║
zonemercy-lexical-viral-1979-v17-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
zonemercy-lexical-viral-1979-v17-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
zonemercy-lexical-viral-1979-v17-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
zonemercy-lexical-viral-1979-v17-mkmlizer: ║       /___/                                                         ║
zonemercy-lexical-viral-1979-v17-mkmlizer: ║                                                                     ║
zonemercy-lexical-viral-1979-v17-mkmlizer: ║  Version: 0.11.12                                                   ║
zonemercy-lexical-viral-1979-v17-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
zonemercy-lexical-viral-1979-v17-mkmlizer: ║  https://mk1.ai                                                     ║
zonemercy-lexical-viral-1979-v17-mkmlizer: ║                                                                     ║
zonemercy-lexical-viral-1979-v17-mkmlizer: ║  The license key for the current software has been verified as      ║
zonemercy-lexical-viral-1979-v17-mkmlizer: ║  belonging to:                                                      ║
zonemercy-lexical-viral-1979-v17-mkmlizer: ║                                                                     ║
zonemercy-lexical-viral-1979-v17-mkmlizer: ║  Chai Research Corp.                                                ║
zonemercy-lexical-viral-1979-v17-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
zonemercy-lexical-viral-1979-v17-mkmlizer: ║  Expiration: 2025-01-15 23:59:59                                    ║
zonemercy-lexical-viral-1979-v17-mkmlizer: ║                                                                     ║
zonemercy-lexical-viral-1979-v17-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
zonemercy-lexical-viral-1979-v17-mkmlizer: Downloaded to shared memory in 56.627s
zonemercy-lexical-viral-1979-v17-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp207dt0jr, device:0
zonemercy-lexical-viral-1979-v17-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
zonemercy-lexical-viral-1979-v17-mkmlizer: quantized model in 48.806s
zonemercy-lexical-viral-1979-v17-mkmlizer: Processed model zonemercy/Lexical-Viral-v6ava-22b11e5r128 in 105.432s
zonemercy-lexical-viral-1979-v17-mkmlizer: creating bucket guanaco-mkml-models
zonemercy-lexical-viral-1979-v17-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
zonemercy-lexical-viral-1979-v17-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/zonemercy-lexical-viral-1979-v17
zonemercy-lexical-viral-1979-v17-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/zonemercy-lexical-viral-1979-v17/tokenizer_config.json
zonemercy-lexical-viral-1979-v17-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/zonemercy-lexical-viral-1979-v17/special_tokens_map.json
zonemercy-lexical-viral-1979-v17-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/zonemercy-lexical-viral-1979-v17/tokenizer.json
zonemercy-lexical-viral-1979-v17-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/zonemercy-lexical-viral-1979-v17/flywheel_model.1.safetensors
Job zonemercy-lexical-viral-1979-v17-mkmlizer completed after 154.86s with status: succeeded
Stopping job with name zonemercy-lexical-viral-1979-v17-mkmlizer
Pipeline stage MKMLizer completed in 155.42s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.16s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service zonemercy-lexical-viral-1979-v17
Waiting for inference service zonemercy-lexical-viral-1979-v17 to be ready
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Inference service zonemercy-lexical-viral-1979-v17 ready after 140.52830410003662s
Pipeline stage MKMLDeployer completed in 141.08s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.913292646408081s
Received healthy response to inference request in 2.381788730621338s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Received healthy response to inference request in 2.400106191635132s
Received healthy response to inference request in 2.2464795112609863s
Received healthy response to inference request in 2.3246662616729736s
5 requests
0 failed requests
5th percentile: 2.262116861343384
10th percentile: 2.277754211425781
20th percentile: 2.309028911590576
30th percentile: 2.3360907554626467
40th percentile: 2.3589397430419923
50th percentile: 2.381788730621338
60th percentile: 2.3891157150268554
70th percentile: 2.396442699432373
80th percentile: 2.5027434825897217
90th percentile: 2.7080180644989014
95th percentile: 2.810655355453491
99th percentile: 2.892765188217163
mean time: 2.453266668319702
Pipeline stage StressChecker completed in 13.89s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.48s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 2.30s
Shutdown handler de-registered
zonemercy-lexical-viral_1979_v17 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3727.93s
Shutdown handler de-registered
zonemercy-lexical-viral_1979_v17 status is now inactive due to auto deactivation removed underperforming models