developer_uid: chace9580
submission_id: jic062-instruct-v18-4-5w-c2_v3
model_name: jic062-instruct-v18-4-5w-c2_v3
model_group: jic062/instruct_v18_4.5w
status: torndown
timestamp: 2024-08-19T00:45:58+00:00
num_battles: 10780
num_wins: 5050
celo_rating: 1205.64
family_friendly_score: 0.0
submission_type: basic
model_repo: jic062/instruct_v18_4.5w_c2
model_architecture: LlamaForCausalLM
model_num_parameters: 8030261248.0
best_of: 16
max_input_tokens: 512
max_output_tokens: 64
display_name: jic062-instruct-v18-4-5w-c2_v3
is_internal_developer: False
language_model: jic062/instruct_v18_4.5w_c2
model_size: 8B
ranking_group: single
us_pacific_date: 2024-08-18
win_ratio: 0.4684601113172542
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|end_of_text|>', '|eot_id|'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}
formatter: {'memory_template': "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n{bot_name}'s Persona: {memory}\n\n", 'prompt_template': '{prompt}<|eot_id|>', 'bot_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}: {message}<|eot_id|>', 'user_template': '<|start_header_id|>user<|end_header_id|>\n\n{user_name}: {message}<|eot_id|>', 'response_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}:', 'truncate_by_message': False}
Resubmit model
Running pipeline stage MKMLizer
Starting job with name jic062-instruct-v18-4-5w-c2-v3-mkmlizer
Waiting for job on jic062-instruct-v18-4-5w-c2-v3-mkmlizer to finish
jic062-instruct-v18-4-5w-c2-v3-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
jic062-instruct-v18-4-5w-c2-v3-mkmlizer: ║     _____            __           __                                ║
jic062-instruct-v18-4-5w-c2-v3-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
jic062-instruct-v18-4-5w-c2-v3-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
jic062-instruct-v18-4-5w-c2-v3-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
jic062-instruct-v18-4-5w-c2-v3-mkmlizer: ║       /___/                                                         ║
jic062-instruct-v18-4-5w-c2-v3-mkmlizer: ║                                                                     ║
jic062-instruct-v18-4-5w-c2-v3-mkmlizer: ║  Version: 0.9.11                                                    ║
jic062-instruct-v18-4-5w-c2-v3-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
jic062-instruct-v18-4-5w-c2-v3-mkmlizer: ║  https://mk1.ai                                                     ║
jic062-instruct-v18-4-5w-c2-v3-mkmlizer: ║                                                                     ║
jic062-instruct-v18-4-5w-c2-v3-mkmlizer: ║  The license key for the current software has been verified as      ║
jic062-instruct-v18-4-5w-c2-v3-mkmlizer: ║  belonging to:                                                      ║
jic062-instruct-v18-4-5w-c2-v3-mkmlizer: ║                                                                     ║
jic062-instruct-v18-4-5w-c2-v3-mkmlizer: ║  Chai Research Corp.                                                ║
jic062-instruct-v18-4-5w-c2-v3-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
jic062-instruct-v18-4-5w-c2-v3-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
jic062-instruct-v18-4-5w-c2-v3-mkmlizer: ║                                                                     ║
jic062-instruct-v18-4-5w-c2-v3-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
jic062-instruct-v18-4-5w-c2-v3-mkmlizer: Downloaded to shared memory in 21.498s
jic062-instruct-v18-4-5w-c2-v3-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpdtzdq2ms, device:0
jic062-instruct-v18-4-5w-c2-v3-mkmlizer: Saving flywheel model at /dev/shm/model_cache
jic062-instruct-v18-4-5w-c2-v3-mkmlizer: quantized model in 26.470s
jic062-instruct-v18-4-5w-c2-v3-mkmlizer: Processed model jic062/instruct_v18_4.5w_c2 in 47.968s
jic062-instruct-v18-4-5w-c2-v3-mkmlizer: creating bucket guanaco-mkml-models
jic062-instruct-v18-4-5w-c2-v3-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
jic062-instruct-v18-4-5w-c2-v3-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/jic062-instruct-v18-4-5w-c2-v3
jic062-instruct-v18-4-5w-c2-v3-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/jic062-instruct-v18-4-5w-c2-v3/config.json
jic062-instruct-v18-4-5w-c2-v3-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/jic062-instruct-v18-4-5w-c2-v3/special_tokens_map.json
jic062-instruct-v18-4-5w-c2-v3-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/jic062-instruct-v18-4-5w-c2-v3/tokenizer_config.json
jic062-instruct-v18-4-5w-c2-v3-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/jic062-instruct-v18-4-5w-c2-v3/tokenizer.json
jic062-instruct-v18-4-5w-c2-v3-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/jic062-instruct-v18-4-5w-c2-v3/flywheel_model.0.safetensors
jic062-instruct-v18-4-5w-c2-v3-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 7/291 [00:00<00:05, 51.68it/s]
Loading 0:   5%|▌         | 16/291 [00:00<00:04, 67.52it/s]
Loading 0:   9%|▊         | 25/291 [00:00<00:03, 74.75it/s]
Loading 0:  12%|█▏        | 34/291 [00:00<00:03, 79.29it/s]
Loading 0:  15%|█▍        | 43/291 [00:00<00:03, 79.92it/s]
Loading 0:  18%|█▊        | 52/291 [00:00<00:02, 82.66it/s]
Loading 0:  21%|██        | 61/291 [00:00<00:02, 84.45it/s]
Loading 0:  26%|██▌       | 75/291 [00:00<00:02, 101.47it/s]
Loading 0:  30%|██▉       | 86/291 [00:02<00:08, 23.83it/s] 
Loading 0:  33%|███▎      | 96/291 [00:02<00:06, 30.71it/s]
Loading 0:  36%|███▌      | 105/291 [00:02<00:05, 36.55it/s]
Loading 0:  39%|███▉      | 113/291 [00:02<00:04, 42.38it/s]
Loading 0:  42%|████▏     | 121/291 [00:02<00:03, 48.39it/s]
Loading 0:  45%|████▍     | 130/291 [00:02<00:02, 54.26it/s]
Loading 0:  48%|████▊     | 139/291 [00:02<00:02, 60.93it/s]
Loading 0:  51%|█████     | 148/291 [00:02<00:02, 63.82it/s]
Loading 0:  54%|█████▍    | 157/291 [00:02<00:01, 67.23it/s]
Loading 0:  57%|█████▋    | 166/291 [00:03<00:01, 71.60it/s]
Loading 0:  60%|██████    | 175/291 [00:03<00:01, 74.62it/s]
Loading 0:  63%|██████▎   | 184/291 [00:03<00:01, 75.86it/s]
Loading 0:  66%|██████▌   | 192/291 [00:04<00:04, 21.03it/s]
Loading 0:  68%|██████▊   | 198/291 [00:04<00:03, 24.65it/s]
Loading 0:  70%|███████   | 205/291 [00:04<00:02, 29.93it/s]
Loading 0:  74%|███████▎  | 214/291 [00:04<00:02, 37.65it/s]
Loading 0:  77%|███████▋  | 223/291 [00:04<00:01, 46.05it/s]
Loading 0:  80%|███████▉  | 232/291 [00:04<00:01, 54.16it/s]
Loading 0:  83%|████████▎ | 241/291 [00:05<00:00, 59.24it/s]
Loading 0:  86%|████████▌ | 250/291 [00:05<00:00, 63.72it/s]
Loading 0:  89%|████████▉ | 259/291 [00:05<00:00, 68.19it/s]
Loading 0:  92%|█████████▏| 268/291 [00:05<00:00, 72.22it/s]
Loading 0:  95%|█████████▌| 277/291 [00:05<00:00, 75.55it/s]
Loading 0:  98%|█████████▊| 286/291 [00:05<00:00, 76.55it/s]
                                                            

Job jic062-instruct-v18-4-5w-c2-v3-mkmlizer completed after 73.62s with status: succeeded
Stopping job with name jic062-instruct-v18-4-5w-c2-v3-mkmlizer
Pipeline stage MKMLizer completed in 74.55s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.10s
Running pipeline stage ISVCDeployer
Creating inference service jic062-instruct-v18-4-5w-c2-v3
Waiting for inference service jic062-instruct-v18-4-5w-c2-v3 to be ready
Inference service jic062-instruct-v18-4-5w-c2-v3 ready after 252.29956364631653s
Pipeline stage ISVCDeployer completed in 253.46s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.00327205657959s
Received healthy response to inference request in 1.9558207988739014s
Received healthy response to inference request in 2.0711536407470703s
Received healthy response to inference request in 1.5182065963745117s
Received healthy response to inference request in 2.375579833984375s
5 requests
0 failed requests
5th percentile: 1.6057294368743897
10th percentile: 1.6932522773742675
20th percentile: 1.8682979583740233
30th percentile: 1.965311050415039
40th percentile: 1.9842915534973145
50th percentile: 2.00327205657959
60th percentile: 2.030424690246582
70th percentile: 2.0575773239135744
80th percentile: 2.1320388793945315
90th percentile: 2.253809356689453
95th percentile: 2.314694595336914
99th percentile: 2.3634027862548828
mean time: 1.9848065853118897
Pipeline stage StressChecker completed in 10.63s
jic062-instruct-v18-4-5w-c2_v3 status is now deployed due to DeploymentManager action
jic062-instruct-v18-4-5w-c2_v3 status is now inactive due to auto deactivation removed underperforming models
jic062-instruct-v18-4-5w-c2_v3 status is now torndown due to DeploymentManager action