rica40325-my-second-chai

developer_uid: rica40325

submission_id: rica40325-my-second-chai_v8

model_name: rica40325-my-second-chai_v5

model_group: rica40325/my-second-chai

status: torndown

timestamp: 2024-08-29T06:23:02+00:00

num_battles: 10857

num_wins: 5370

celo_rating: 1232.01

family_friendly_score: 0.0

submission_type: basic

model_repo: rica40325/my-second-chai

model_architecture: LlamaForCausalLM

model_num_parameters: 8030261248.0

best_of: 16

max_input_tokens: 512

max_output_tokens: 64

display_name: rica40325-my-second-chai_v5

is_internal_developer: False

language_model: rica40325/my-second-chai

model_size: 8B

ranking_group: single

us_pacific_date: 2024-08-28

win_ratio: 0.49461177120751587

generation_params: {'temperature': 0.95, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}

formatter: {'memory_template': "<|start_header_id|>system<|end_header_id|>\n\n{bot_name}'s Persona: {memory}\n\n", 'prompt_template': '{prompt}<|eot_id|>', 'bot_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}: {message}<|eot_id|>', 'user_template': '<|start_header_id|>user<|end_header_id|>\n\nYou: {message}<|eot_id|>', 'response_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}:', 'truncate_by_message': False}

Resubmit model

Running pipeline stage MKMLizer
Starting job with name rica40325-my-second-chai-v8-mkmlizer
Waiting for job on rica40325-my-second-chai-v8-mkmlizer to finish
Stopping job with name rica40325-my-second-chai-v8-mkmlizer
%s, retrying in %s seconds...
Starting job with name rica40325-my-second-chai-v8-mkmlizer
Waiting for job on rica40325-my-second-chai-v8-mkmlizer to finish
rica40325-my-second-chai-v8-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
rica40325-my-second-chai-v8-mkmlizer: ║     _____            __           __                                ║
rica40325-my-second-chai-v8-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
rica40325-my-second-chai-v8-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
rica40325-my-second-chai-v8-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
rica40325-my-second-chai-v8-mkmlizer: ║       /___/                                                         ║
rica40325-my-second-chai-v8-mkmlizer: ║                                                                     ║
rica40325-my-second-chai-v8-mkmlizer: ║  Version: 0.10.1                                                    ║
rica40325-my-second-chai-v8-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
rica40325-my-second-chai-v8-mkmlizer: ║  https://mk1.ai                                                     ║
rica40325-my-second-chai-v8-mkmlizer: ║                                                                     ║
rica40325-my-second-chai-v8-mkmlizer: ║  The license key for the current software has been verified as      ║
rica40325-my-second-chai-v8-mkmlizer: ║  belonging to:                                                      ║
rica40325-my-second-chai-v8-mkmlizer: ║                                                                     ║
rica40325-my-second-chai-v8-mkmlizer: ║  Chai Research Corp.                                                ║
rica40325-my-second-chai-v8-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
rica40325-my-second-chai-v8-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
rica40325-my-second-chai-v8-mkmlizer: ║                                                                     ║
rica40325-my-second-chai-v8-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
rica40325-my-second-chai-v8-mkmlizer: Downloaded to shared memory in 37.219s
rica40325-my-second-chai-v8-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp7w7id2dv, device:0
rica40325-my-second-chai-v8-mkmlizer: Saving flywheel model at /dev/shm/model_cache
rica40325-my-second-chai-v8-mkmlizer: quantized model in 25.665s
rica40325-my-second-chai-v8-mkmlizer: Processed model rica40325/my-second-chai in 62.884s
rica40325-my-second-chai-v8-mkmlizer: creating bucket guanaco-mkml-models
rica40325-my-second-chai-v8-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
rica40325-my-second-chai-v8-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/rica40325-my-second-chai-v8
rica40325-my-second-chai-v8-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/rica40325-my-second-chai-v8/config.json
rica40325-my-second-chai-v8-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/rica40325-my-second-chai-v8/special_tokens_map.json
rica40325-my-second-chai-v8-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/rica40325-my-second-chai-v8/tokenizer_config.json
rica40325-my-second-chai-v8-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/rica40325-my-second-chai-v8/tokenizer.json
rica40325-my-second-chai-v8-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/rica40325-my-second-chai-v8/flywheel_model.0.safetensors
rica40325-my-second-chai-v8-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 7/291 [00:00<00:06, 43.84it/s]
Loading 0:   5%|▌         | 16/291 [00:00<00:04, 58.27it/s]
Loading 0:   9%|▊         | 25/291 [00:00<00:03, 69.92it/s]
Loading 0:  12%|█▏        | 34/291 [00:00<00:03, 71.40it/s]
Loading 0:  15%|█▍        | 43/291 [00:00<00:03, 73.64it/s]
Loading 0:  18%|█▊        | 52/291 [00:00<00:03, 69.24it/s]
Loading 0:  21%|██        | 61/291 [00:00<00:03, 73.67it/s]
Loading 0:  24%|██▍       | 71/291 [00:00<00:02, 80.80it/s]
Loading 0:  27%|██▋       | 80/291 [00:01<00:02, 80.42it/s]
Loading 0:  31%|███       | 89/291 [00:02<00:09, 20.89it/s]
Loading 0:  33%|███▎      | 97/291 [00:02<00:07, 25.85it/s]
Loading 0:  37%|███▋      | 107/291 [00:02<00:05, 34.28it/s]
Loading 0:  42%|████▏     | 121/291 [00:02<00:03, 46.51it/s]
Loading 0:  45%|████▌     | 131/291 [00:02<00:02, 54.87it/s]
Loading 0:  48%|████▊     | 140/291 [00:02<00:02, 61.24it/s]
Loading 0:  51%|█████     | 149/291 [00:02<00:02, 63.84it/s]
Loading 0:  54%|█████▍    | 158/291 [00:03<00:01, 66.93it/s]
Loading 0:  58%|█████▊    | 169/291 [00:03<00:01, 69.63it/s]
Loading 0:  62%|██████▏   | 179/291 [00:03<00:01, 76.55it/s]
Loading 0:  65%|██████▍   | 188/291 [00:04<00:04, 22.83it/s]
Loading 0:  69%|██████▉   | 202/291 [00:04<00:02, 32.36it/s]
Loading 0:  74%|███████▎  | 214/291 [00:04<00:01, 40.58it/s]
Loading 0:  77%|███████▋  | 224/291 [00:04<00:01, 48.28it/s]
Loading 0:  80%|████████  | 233/291 [00:04<00:01, 52.26it/s]
Loading 0:  85%|████████▍ | 247/291 [00:05<00:00, 63.55it/s]
Loading 0:  88%|████████▊ | 257/291 [00:05<00:00, 70.46it/s]
Loading 0:  92%|█████████▏| 267/291 [00:05<00:00, 76.63it/s]
Loading 0:  95%|█████████▌| 277/291 [00:05<00:00, 76.07it/s]
Loading 0:  99%|█████████▊| 287/291 [00:05<00:00, 46.25it/s]
                                                            
Job rica40325-my-second-chai-v8-mkmlizer completed after 83.61s with status: succeeded
Stopping job with name rica40325-my-second-chai-v8-mkmlizer
Pipeline stage MKMLizer completed in 85.14s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.10s
Running pipeline stage ISVCDeployer
Creating inference service rica40325-my-second-chai-v8
Waiting for inference service rica40325-my-second-chai-v8 to be ready
Failed to get response for submission neversleep-noromaid-v0_8068_v150: ('http://chaiml-llama-8b-pairwis-8189-v19-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '{"error":"ValueError : [TypeError(\\"\'numpy.int64\' object is not iterable\\"), TypeError(\'vars() argument must have __dict__ attribute\')]"}')
Inference service rica40325-my-second-chai-v8 ready after 180.61957383155823s
Pipeline stage ISVCDeployer completed in 181.63s
Running pipeline stage StressChecker
Received healthy response to inference request in 5.446141004562378s
Received healthy response to inference request in 4.972785472869873s
Received healthy response to inference request in 1.784764051437378s
Received healthy response to inference request in 5.055232524871826s
Received healthy response to inference request in 1.4413535594940186s
5 requests
0 failed requests
5th percentile: 1.5100356578826903
10th percentile: 1.5787177562713623
20th percentile: 1.7160819530487061
30th percentile: 2.4223683357238768
40th percentile: 3.6975769042968754
50th percentile: 4.972785472869873
60th percentile: 5.005764293670654
70th percentile: 5.038743114471435
80th percentile: 5.133414220809937
90th percentile: 5.289777612686157
95th percentile: 5.367959308624267
99th percentile: 5.430504665374756
mean time: 3.7400553226470947
%s, retrying in %s seconds...
Received healthy response to inference request in 1.787771463394165s
Received healthy response to inference request in 1.7177329063415527s
Received healthy response to inference request in 1.8740417957305908s
Received healthy response to inference request in 5.439385414123535s
Received healthy response to inference request in 2.1976027488708496s
5 requests
0 failed requests
5th percentile: 1.731740617752075
10th percentile: 1.7457483291625977
20th percentile: 1.7737637519836427
30th percentile: 1.8050255298614502
40th percentile: 1.8395336627960206
50th percentile: 1.8740417957305908
60th percentile: 2.003466176986694
70th percentile: 2.132890558242798
80th percentile: 2.8459592819213873
90th percentile: 4.142672348022462
95th percentile: 4.7910288810729975
99th percentile: 5.309714107513428
mean time: 2.6033068656921388
Pipeline stage StressChecker completed in 33.21s
rica40325-my-second-chai_v8 status is now deployed due to DeploymentManager action
rica40325-my-second-chai_v8 status is now inactive due to auto deactivation removed underperforming models
rica40325-my-second-chai_v8 status is now torndown due to DeploymentManager action
rica40325-my-second-chai_v8 status is now torndown due to DeploymentManager action