zonemercy-acute-nemo-v1_4488

developer_uid: zonemercy

submission_id: zonemercy-acute-nemo-v1_4488_v11

model_name: 0815v3-1

model_group: zonemercy/Acute-Nemo-v1-

status: torndown

timestamp: 2024-08-17T05:26:28+00:00

num_battles: 128622

num_wins: 64696

celo_rating: 1281.96

family_friendly_score: 0.0

submission_type: basic

model_repo: zonemercy/Acute-Nemo-v1-1e5ep1

model_architecture: MistralForCausalLM

model_num_parameters: 12772070400.0

best_of: 32

max_input_tokens: 512

max_output_tokens: 64

display_name: 0815v3-1

is_internal_developer: True

language_model: zonemercy/Acute-Nemo-v1-1e5ep1

model_size: 13B

ranking_group: single

us_pacific_date: 2024-08-16

win_ratio: 0.5029932670927213

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.08, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '###', 'Bot:', 'User:', 'You:', '<|im_end|>'], 'max_input_tokens': 512, 'best_of': 32, 'max_output_tokens': 64}

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Running pipeline stage MKMLizer
Starting job with name zonemercy-acute-nemo-v1-4488-v11-mkmlizer
Waiting for job on zonemercy-acute-nemo-v1-4488-v11-mkmlizer to finish
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ║     _____            __           __                                ║
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ║       /___/                                                         ║
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ║                                                                     ║
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ║  Version: 0.9.11                                                    ║
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ║  https://mk1.ai                                                     ║
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ║                                                                     ║
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ║  The license key for the current software has been verified as      ║
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ║  belonging to:                                                      ║
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ║                                                                     ║
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ║  Chai Research Corp.                                                ║
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ║                                                                     ║
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Failed to get response for submission neversleep-noromaid-v0-_8068_v36: ('http://neversleep-noromaid-v0-8068-v36-predictor-default.tenant-chaiml-guanaco.knative.ord1.coreweave.cloud/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:42964->127.0.0.1:8080: read: connection reset by peer\n')
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: Downloaded to shared memory in 71.691s
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpkfwu_qa8, device:0
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Failed to get response for submission chaiml-llama-8b-pairwis_8189_v14: ('http://chaiml-llama-8b-pairwis-8189-v14-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission undi95-meta-llama-3-70b_6209_v19: ('http://undi95-meta-llama-3-70b-6209-v19-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: quantized model in 43.269s
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: Processed model zonemercy/Acute-Nemo-v1-1e5ep1 in 114.960s
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/zonemercy-acute-nemo-v1-4488-v11
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/zonemercy-acute-nemo-v1-4488-v11/config.json
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/zonemercy-acute-nemo-v1-4488-v11/special_tokens_map.json
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/zonemercy-acute-nemo-v1-4488-v11/tokenizer_config.json
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/zonemercy-acute-nemo-v1-4488-v11/tokenizer.json
zonemercy-acute-nemo-v1-4488-v11-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/zonemercy-acute-nemo-v1-4488-v11/flywheel_model.0.safetensors
Job zonemercy-acute-nemo-v1-4488-v11-mkmlizer completed after 146.06s with status: succeeded
Stopping job with name zonemercy-acute-nemo-v1-4488-v11-mkmlizer
Pipeline stage MKMLizer completed in 146.93s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.10s
Running pipeline stage ISVCDeployer
Creating inference service zonemercy-acute-nemo-v1-4488-v11
Waiting for inference service zonemercy-acute-nemo-v1-4488-v11 to be ready
Failed to get response for submission chaiml-llama-8b-pairwis_8189_v14: ('http://chaiml-llama-8b-pairwis-8189-v14-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission undi95-meta-llama-3-70b_6209_v19: ('http://undi95-meta-llama-3-70b-6209-v19-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission undi95-meta-llama-3-70b_6209_v19: ('http://undi95-meta-llama-3-70b-6209-v19-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission undi95-meta-llama-3-70b_6209_v19: ('http://undi95-meta-llama-3-70b-6209-v19-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission chaiml-llama-8b-pairwis_8189_v14: ('http://chaiml-llama-8b-pairwis-8189-v14-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission chaiml-llama-8b-pairwis_8189_v14: ('http://chaiml-llama-8b-pairwis-8189-v14-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Inference service zonemercy-acute-nemo-v1-4488-v11 ready after 241.57556581497192s
Pipeline stage ISVCDeployer completed in 242.83s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.15925931930542s
Received healthy response to inference request in 2.628145694732666s
Failed to get response for submission undi95-meta-llama-3-70b_6209_v19: ('http://undi95-meta-llama-3-70b-6209-v19-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
Received healthy response to inference request in 2.5470690727233887s
Received healthy response to inference request in 2.3861489295959473s
Received healthy response to inference request in 2.3393607139587402s
5 requests
0 failed requests
5th percentile: 2.348718357086182
10th percentile: 2.358076000213623
20th percentile: 2.3767912864685057
30th percentile: 2.4183329582214355
40th percentile: 2.4827010154724123
50th percentile: 2.5470690727233887
60th percentile: 2.5794997215270996
70th percentile: 2.6119303703308105
80th percentile: 2.734368419647217
90th percentile: 2.946813869476318
95th percentile: 3.053036594390869
99th percentile: 3.1380147743225097
mean time: 2.6119967460632325
Pipeline stage StressChecker completed in 13.75s
zonemercy-acute-nemo-v1_4488_v11 status is now deployed due to DeploymentManager action
zonemercy-acute-nemo-v1_4488_v11 status is now inactive due to auto deactivation removed underperforming models
zonemercy-acute-nemo-v1_4488_v11 status is now torndown due to DeploymentManager action