developer_uid: zonemercy
submission_id: zonemercy-lexical-nemo-_1518_v20
model_name: 0815v2-0
model_group: zonemercy/Lexical-Nemo-v
status: torndown
timestamp: 2024-08-16T00:51:41+00:00
num_battles: 9715
num_wins: 5443
celo_rating: 1278.16
family_friendly_score: 0.0
submission_type: basic
model_repo: zonemercy/Lexical-Nemo-v4-1k1e5
model_architecture: MistralForCausalLM
model_num_parameters: 12772070400.0
best_of: 16
max_input_tokens: 512
max_output_tokens: 64
display_name: 0815v2-0
is_internal_developer: True
language_model: zonemercy/Lexical-Nemo-v4-1k1e5
model_size: 13B
ranking_group: single
us_pacific_date: 2024-08-15
win_ratio: 0.5602676273803396
generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '###', 'Bot:', 'User:', 'You:', '<|im_end|>'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Running pipeline stage MKMLizer
Starting job with name zonemercy-lexical-nemo-1518-v20-mkmlizer
Waiting for job on zonemercy-lexical-nemo-1518-v20-mkmlizer to finish
Stopping job with name zonemercy-lexical-nemo-1518-v20-mkmlizer
%s, retrying in %s seconds...
Starting job with name zonemercy-lexical-nemo-1518-v20-mkmlizer
Waiting for job on zonemercy-lexical-nemo-1518-v20-mkmlizer to finish
Failed to get response for submission zonemercy-cogent-nemo-v2-5e6_v13: ('http://zonemercy-cogent-nemo-v2-5e6-v13-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission zonemercy-cogent-nemo-v2-5e6_v14: ('http://zonemercy-cogent-nemo-v2-5e6-v14-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission chaiml-elo-alignment-run-3_v34: ('http://chaiml-elo-alignment-run-3-v34-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
zonemercy-lexical-nemo-1518-v20-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
zonemercy-lexical-nemo-1518-v20-mkmlizer: ║     _____            __           __                                ║
zonemercy-lexical-nemo-1518-v20-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
zonemercy-lexical-nemo-1518-v20-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
zonemercy-lexical-nemo-1518-v20-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
zonemercy-lexical-nemo-1518-v20-mkmlizer: ║       /___/                                                         ║
zonemercy-lexical-nemo-1518-v20-mkmlizer: ║                                                                     ║
zonemercy-lexical-nemo-1518-v20-mkmlizer: ║  Version: 0.9.11                                                    ║
zonemercy-lexical-nemo-1518-v20-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
zonemercy-lexical-nemo-1518-v20-mkmlizer: ║  https://mk1.ai                                                     ║
zonemercy-lexical-nemo-1518-v20-mkmlizer: ║                                                                     ║
zonemercy-lexical-nemo-1518-v20-mkmlizer: ║  The license key for the current software has been verified as      ║
zonemercy-lexical-nemo-1518-v20-mkmlizer: ║  belonging to:                                                      ║
zonemercy-lexical-nemo-1518-v20-mkmlizer: ║                                                                     ║
zonemercy-lexical-nemo-1518-v20-mkmlizer: ║  Chai Research Corp.                                                ║
zonemercy-lexical-nemo-1518-v20-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
zonemercy-lexical-nemo-1518-v20-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
zonemercy-lexical-nemo-1518-v20-mkmlizer: ║                                                                     ║
zonemercy-lexical-nemo-1518-v20-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Failed to get response for submission zonemercy-cogent-nemo-v2-5e6_v13: ('http://zonemercy-cogent-nemo-v2-5e6-v13-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
zonemercy-lexical-nemo-1518-v20-mkmlizer: Downloaded to shared memory in 75.080s
zonemercy-lexical-nemo-1518-v20-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpx3398vti, device:0
zonemercy-lexical-nemo-1518-v20-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
Failed to get response for submission zonemercy-cogent-nemo-v2-5e6_v14: ('http://zonemercy-cogent-nemo-v2-5e6-v14-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission zonemercy-cogent-nemo-v2-5e6_v14: ('http://zonemercy-cogent-nemo-v2-5e6-v14-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
zonemercy-lexical-nemo-1518-v20-mkmlizer: quantized model in 41.766s
zonemercy-lexical-nemo-1518-v20-mkmlizer: Processed model zonemercy/Lexical-Nemo-v4-1k1e5 in 116.847s
zonemercy-lexical-nemo-1518-v20-mkmlizer: creating bucket guanaco-mkml-models
zonemercy-lexical-nemo-1518-v20-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
zonemercy-lexical-nemo-1518-v20-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/zonemercy-lexical-nemo-1518-v20
zonemercy-lexical-nemo-1518-v20-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/zonemercy-lexical-nemo-1518-v20/config.json
zonemercy-lexical-nemo-1518-v20-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/zonemercy-lexical-nemo-1518-v20/special_tokens_map.json
zonemercy-lexical-nemo-1518-v20-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/zonemercy-lexical-nemo-1518-v20/tokenizer_config.json
zonemercy-lexical-nemo-1518-v20-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/zonemercy-lexical-nemo-1518-v20/tokenizer.json
zonemercy-lexical-nemo-1518-v20-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/zonemercy-lexical-nemo-1518-v20/flywheel_model.0.safetensors
Job zonemercy-lexical-nemo-1518-v20-mkmlizer completed after 177.33s with status: succeeded
Stopping job with name zonemercy-lexical-nemo-1518-v20-mkmlizer
Pipeline stage MKMLizer completed in 178.88s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.11s
Running pipeline stage ISVCDeployer
Creating inference service zonemercy-lexical-nemo-1518-v20
Waiting for inference service zonemercy-lexical-nemo-1518-v20 to be ready
Failed to get response for submission chaiml-elo-alignment-run-3_v34: ('http://chaiml-elo-alignment-run-3-v34-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
Failed to get response for submission zonemercy-cogent-nemo-v2-5e6_v13: ('http://zonemercy-cogent-nemo-v2-5e6-v13-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
Failed to get response for submission jic062-instruct-g32-lr25_5927_v2: ('http://jic062-instruct-g32-lr25-5927-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:50856->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission chaiml-elo-alignment-run-3_v34: ('http://chaiml-elo-alignment-run-3-v34-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission zonemercy-cogent-nemo-v2-5e6_v13: ('http://zonemercy-cogent-nemo-v2-5e6-v13-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission chaiml-elo-alignment-run-3_v34: ('http://chaiml-elo-alignment-run-3-v34-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission chaiml-elo-alignment-run-3_v34: ('http://chaiml-elo-alignment-run-3-v34-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission zonemercy-cogent-nemo-v2-5e6_v14: ('http://zonemercy-cogent-nemo-v2-5e6-v14-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
Failed to get response for submission zonemercy-cogent-nemo-v2-5e6_v14: ('http://zonemercy-cogent-nemo-v2-5e6-v14-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Inference service zonemercy-lexical-nemo-1518-v20 ready after 372.41024518013s
Pipeline stage ISVCDeployer completed in 374.07s
Running pipeline stage StressChecker
Failed to get response for submission function_fahor_2024-08-15: no entry with id "function_fahor_2024-08-15" found on database!
Received healthy response to inference request in 3.138559341430664s
Received healthy response to inference request in 3.3853976726531982s
Received healthy response to inference request in 2.318357229232788s
Received healthy response to inference request in 1.8984155654907227s
Received healthy response to inference request in 2.1314847469329834s
5 requests
0 failed requests
5th percentile: 1.9450294017791747
10th percentile: 1.991643238067627
20th percentile: 2.0848709106445313
30th percentile: 2.1688592433929443
40th percentile: 2.243608236312866
50th percentile: 2.318357229232788
60th percentile: 2.646438074111938
70th percentile: 2.974518918991089
80th percentile: 3.187927007675171
90th percentile: 3.2866623401641846
95th percentile: 3.3360300064086914
99th percentile: 3.375524139404297
mean time: 2.574442911148071
Pipeline stage StressChecker completed in 13.69s
zonemercy-lexical-nemo-_1518_v20 status is now deployed due to DeploymentManager action
zonemercy-lexical-nemo-_1518_v20 status is now inactive due to auto deactivation removed underperforming models
zonemercy-lexical-nemo-_1518_v20 status is now torndown due to DeploymentManager action