submission_id: zonemercy-acute-nemo-v1_4488_v11
developer_uid: zonemercy
alignment_samples: 128622
alignment_score: 1.6154372970072102
best_of: 32
celo_rating: 1281.94
display_name: 0815v3-1
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.08, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '###', 'Bot:', 'User:', 'You:', '<|im_end|>'], 'max_input_tokens': 512, 'best_of': 32, 'max_output_tokens': 64}
is_internal_developer: True
language_model: zonemercy/Acute-Nemo-v1-1e5ep1
max_input_tokens: 512
max_output_tokens: 64
model_architecture: MistralForCausalLM
model_group: zonemercy/Acute-Nemo-v1-
model_name: 0815v3-1
model_num_parameters: 12772070400.0
model_repo: zonemercy/Acute-Nemo-v1-1e5ep1
model_size: 13B
num_battles: 128622
num_wins: 64696
propriety_score: 0.7221388054721388
propriety_total_count: 11988.0
ranking_group: single
status: torndown
submission_type: basic
timestamp: 2024-08-17T05:26:28+00:00
us_pacific_date: 2024-08-16
win_ratio: 0.5029932670927213
Download Preference Data
Resubmit model
Running pipeline stage MKMLizer
Starting job with name zonemercy-acute-nemo-v1-4488-v11-mkmlizer
Waiting for job on zonemercy-acute-nemo-v1-4488-v11-mkmlizer to finish
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ║ _____ __ __ ║
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ║ /___/ ║
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ║ ║
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ║ Version: 0.9.11 ║
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ║ https://mk1.ai ║
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ║ ║
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ║ The license key for the current software has been verified as ║
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ║ belonging to: ║
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ║ ║
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ║ Chai Research Corp. ║
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ║ ║
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Failed to get response for submission neversleep-noromaid-v0-_8068_v36: ('http://neversleep-noromaid-v0-8068-v36-predictor-default.tenant-chaiml-guanaco.knative.ord1.coreweave.cloud/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:42964->127.0.0.1:8080: read: connection reset by peer\n')
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: Downloaded to shared memory in 71.691s
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpkfwu_qa8, device:0
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Failed to get response for submission chaiml-llama-8b-pairwis_8189_v14: ('http://chaiml-llama-8b-pairwis-8189-v14-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission undi95-meta-llama-3-70b_6209_v19: ('http://undi95-meta-llama-3-70b-6209-v19-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: quantized model in 43.269s
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: Processed model zonemercy/Acute-Nemo-v1-1e5ep1 in 114.960s
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/zonemercy-acute-nemo-v1-4488-v11
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/zonemercy-acute-nemo-v1-4488-v11/config.json
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/zonemercy-acute-nemo-v1-4488-v11/special_tokens_map.json
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/zonemercy-acute-nemo-v1-4488-v11/tokenizer_config.json
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/zonemercy-acute-nemo-v1-4488-v11/tokenizer.json
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/zonemercy-acute-nemo-v1-4488-v11/flywheel_model.0.safetensors
Job zonemercy-acute-nemo-v1-4488-v11-mkmlizer completed after 146.06s with status: succeeded
Stopping job with name zonemercy-acute-nemo-v1-4488-v11-mkmlizer
Pipeline stage MKMLizer completed in 146.93s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.10s
Running pipeline stage ISVCDeployer
Creating inference service zonemercy-acute-nemo-v1-4488-v11
Waiting for inference service zonemercy-acute-nemo-v1-4488-v11 to be ready
Failed to get response for submission chaiml-llama-8b-pairwis_8189_v14: ('http://chaiml-llama-8b-pairwis-8189-v14-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission undi95-meta-llama-3-70b_6209_v19: ('http://undi95-meta-llama-3-70b-6209-v19-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission undi95-meta-llama-3-70b_6209_v19: ('http://undi95-meta-llama-3-70b-6209-v19-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission undi95-meta-llama-3-70b_6209_v19: ('http://undi95-meta-llama-3-70b-6209-v19-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission chaiml-llama-8b-pairwis_8189_v14: ('http://chaiml-llama-8b-pairwis-8189-v14-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission chaiml-llama-8b-pairwis_8189_v14: ('http://chaiml-llama-8b-pairwis-8189-v14-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Inference service zonemercy-acute-nemo-v1-4488-v11 ready after 241.57556581497192s
Pipeline stage ISVCDeployer completed in 242.83s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.15925931930542s
Received healthy response to inference request in 2.628145694732666s
Failed to get response for submission undi95-meta-llama-3-70b_6209_v19: ('http://undi95-meta-llama-3-70b-6209-v19-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
Received healthy response to inference request in 2.5470690727233887s
Received healthy response to inference request in 2.3861489295959473s
Received healthy response to inference request in 2.3393607139587402s
5 requests
0 failed requests
5th percentile: 2.348718357086182
10th percentile: 2.358076000213623
20th percentile: 2.3767912864685057
30th percentile: 2.4183329582214355
40th percentile: 2.4827010154724123
50th percentile: 2.5470690727233887
60th percentile: 2.5794997215270996
70th percentile: 2.6119303703308105
80th percentile: 2.734368419647217
90th percentile: 2.946813869476318
95th percentile: 3.053036594390869
99th percentile: 3.1380147743225097
mean time: 2.6119967460632325
Pipeline stage StressChecker completed in 13.75s
zonemercy-acute-nemo-v1_4488_v11 status is now deployed due to DeploymentManager action
zonemercy-acute-nemo-v1_4488_v11 status is now inactive due to auto deactivation removed underperforming models
zonemercy-acute-nemo-v1_4488_v11 status is now torndown due to DeploymentManager action

Usage Metrics

Latency Metrics