developer_uid: shiroe40
submission_id: cycy233-l3-bpa-v0-c1_v1
model_name: auto
model_group: cycy233/L3-bpa-v0-c1
status: torndown
timestamp: 2024-09-04T02:44:36+00:00
num_battles: 12991
num_wins: 6468
celo_rating: 1239.61
family_friendly_score: 0.0
submission_type: basic
model_repo: cycy233/L3-bpa-v0-c1
model_architecture: LlamaForCausalLM
model_num_parameters: 8030261248.0
best_of: 16
max_input_tokens: 512
max_output_tokens: 64
display_name: auto
is_internal_developer: False
language_model: cycy233/L3-bpa-v0-c1
model_size: 8B
ranking_group: single
us_pacific_date: 2024-09-03
win_ratio: 0.497883149872989
generation_params: {'temperature': 1.0, 'top_p': 0.9, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['<|end_header_id|>', '<|eot_id|>'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}
formatter: {'memory_template': "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n{bot_name}'s Persona: {memory}\n\n", 'prompt_template': '{prompt}<|eot_id|>', 'bot_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}: {message}<|eot_id|>', 'user_template': '<|start_header_id|>user<|end_header_id|>\n\n{user_name}: {message}<|eot_id|>', 'response_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}:', 'truncate_by_message': False}
Resubmit model
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name cycy233-l3-bpa-v0-c1-v1-mkmlizer
Waiting for job on cycy233-l3-bpa-v0-c1-v1-mkmlizer to finish
cycy233-l3-bpa-v0-c1-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
cycy233-l3-bpa-v0-c1-v1-mkmlizer: ║     _____            __           __                                ║
cycy233-l3-bpa-v0-c1-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
cycy233-l3-bpa-v0-c1-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
cycy233-l3-bpa-v0-c1-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
cycy233-l3-bpa-v0-c1-v1-mkmlizer: ║       /___/                                                         ║
cycy233-l3-bpa-v0-c1-v1-mkmlizer: ║                                                                     ║
cycy233-l3-bpa-v0-c1-v1-mkmlizer: ║  Version: 0.10.1                                                    ║
cycy233-l3-bpa-v0-c1-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
cycy233-l3-bpa-v0-c1-v1-mkmlizer: ║  https://mk1.ai                                                     ║
cycy233-l3-bpa-v0-c1-v1-mkmlizer: ║                                                                     ║
cycy233-l3-bpa-v0-c1-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
cycy233-l3-bpa-v0-c1-v1-mkmlizer: ║  belonging to:                                                      ║
cycy233-l3-bpa-v0-c1-v1-mkmlizer: ║                                                                     ║
cycy233-l3-bpa-v0-c1-v1-mkmlizer: ║  Chai Research Corp.                                                ║
cycy233-l3-bpa-v0-c1-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
cycy233-l3-bpa-v0-c1-v1-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
cycy233-l3-bpa-v0-c1-v1-mkmlizer: ║                                                                     ║
cycy233-l3-bpa-v0-c1-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
cycy233-l3-bpa-v0-c1-v1-mkmlizer: Downloaded to shared memory in 32.593s
cycy233-l3-bpa-v0-c1-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmps1rtm2on, device:0
cycy233-l3-bpa-v0-c1-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
cycy233-l3-bpa-v0-c1-v1-mkmlizer: quantized model in 26.245s
cycy233-l3-bpa-v0-c1-v1-mkmlizer: Processed model cycy233/L3-bpa-v0-c1 in 58.838s
cycy233-l3-bpa-v0-c1-v1-mkmlizer: creating bucket guanaco-mkml-models
cycy233-l3-bpa-v0-c1-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
cycy233-l3-bpa-v0-c1-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/cycy233-l3-bpa-v0-c1-v1
cycy233-l3-bpa-v0-c1-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/cycy233-l3-bpa-v0-c1-v1/config.json
cycy233-l3-bpa-v0-c1-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/cycy233-l3-bpa-v0-c1-v1/special_tokens_map.json
cycy233-l3-bpa-v0-c1-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/cycy233-l3-bpa-v0-c1-v1/tokenizer_config.json
cycy233-l3-bpa-v0-c1-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/cycy233-l3-bpa-v0-c1-v1/tokenizer.json
cycy233-l3-bpa-v0-c1-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/cycy233-l3-bpa-v0-c1-v1/flywheel_model.0.safetensors
cycy233-l3-bpa-v0-c1-v1-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 4/291 [00:00<00:08, 35.76it/s]
Loading 0:   4%|▍         | 13/291 [00:00<00:04, 59.93it/s]
Loading 0:   8%|▊         | 22/291 [00:00<00:03, 70.52it/s]
Loading 0:  11%|█         | 31/291 [00:00<00:03, 67.26it/s]
Loading 0:  14%|█▎        | 40/291 [00:00<00:03, 72.35it/s]
Loading 0:  18%|█▊        | 52/291 [00:00<00:03, 79.49it/s]
Loading 0:  23%|██▎       | 67/291 [00:00<00:02, 89.54it/s]
Loading 0:  27%|██▋       | 79/291 [00:00<00:02, 88.24it/s]
Loading 0:  30%|███       | 88/291 [00:02<00:08, 23.80it/s]
Loading 0:  33%|███▎      | 97/291 [00:02<00:06, 29.29it/s]
Loading 0:  38%|███▊      | 112/291 [00:02<00:04, 40.49it/s]
Loading 0:  42%|████▏     | 121/291 [00:02<00:03, 45.24it/s]
Loading 0:  45%|████▍     | 130/291 [00:02<00:03, 51.80it/s]
Loading 0:  48%|████▊     | 139/291 [00:02<00:02, 54.93it/s]
Loading 0:  51%|█████     | 148/291 [00:02<00:02, 58.11it/s]
Loading 0:  54%|█████▍    | 158/291 [00:03<00:01, 66.73it/s]
Loading 0:  58%|█████▊    | 169/291 [00:03<00:01, 69.78it/s]
Loading 0:  62%|██████▏   | 180/291 [00:03<00:01, 78.96it/s]
Loading 0:  65%|██████▍   | 189/291 [00:04<00:04, 22.63it/s]
Loading 0:  67%|██████▋   | 196/291 [00:04<00:03, 26.29it/s]
Loading 0:  70%|███████   | 205/291 [00:04<00:02, 32.91it/s]
Loading 0:  74%|███████▎  | 214/291 [00:04<00:01, 39.97it/s]
Loading 0:  77%|███████▋  | 223/291 [00:04<00:01, 46.01it/s]
Loading 0:  82%|████████▏ | 238/291 [00:05<00:00, 59.36it/s]
Loading 0:  85%|████████▍ | 247/291 [00:05<00:00, 60.65it/s]
Loading 0:  88%|████████▊ | 256/291 [00:05<00:00, 65.92it/s]
Loading 0:  92%|█████████▏| 267/291 [00:05<00:00, 75.59it/s]
Loading 0:  95%|█████████▌| 277/291 [00:05<00:00, 74.32it/s]
Loading 0:  98%|█████████▊| 286/291 [00:05<00:00, 75.62it/s]
                                                            

Job cycy233-l3-bpa-v0-c1-v1-mkmlizer completed after 84.89s with status: succeeded
Stopping job with name cycy233-l3-bpa-v0-c1-v1-mkmlizer
Pipeline stage MKMLizer completed in 86.02s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.08s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service cycy233-l3-bpa-v0-c1-v1
Waiting for inference service cycy233-l3-bpa-v0-c1-v1 to be ready
Failed to get response for submission blend_susol_2024-08-22: ('http://chaiml-elo-alignment-run-3-v34-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '{"error":"ValueError : [TypeError(\\"\'numpy.int64\' object is not iterable\\"), TypeError(\'vars() argument must have __dict__ attribute\')]"}')
Inference service cycy233-l3-bpa-v0-c1-v1 ready after 131.1568157672882s
Pipeline stage MKMLDeployer completed in 131.53s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.0337586402893066s
Received healthy response to inference request in 1.5700244903564453s
Received healthy response to inference request in 1.8724610805511475s
Received healthy response to inference request in 1.9025466442108154s
Received healthy response to inference request in 2.4497876167297363s
5 requests
0 failed requests
5th percentile: 1.6305118083953858
10th percentile: 1.6909991264343263
20th percentile: 1.811973762512207
30th percentile: 1.878478193283081
40th percentile: 1.8905124187469482
50th percentile: 1.9025466442108154
60th percentile: 1.955031442642212
70th percentile: 2.0075162410736085
80th percentile: 2.1169644355773927
90th percentile: 2.2833760261535643
95th percentile: 2.3665818214416503
99th percentile: 2.433146457672119
mean time: 1.9657156944274903
Pipeline stage StressChecker completed in 10.43s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
starting trigger_guanaco_pipeline %s
Pipeline stage TriggerMKMLProfilingPipeline completed in 6.44s
cycy233-l3-bpa-v0-c1_v1 status is now deployed due to DeploymentManager action
cycy233-l3-bpa-v0-c1_v1 status is now inactive due to auto deactivation removed underperforming models
cycy233-l3-bpa-v0-c1_v1 status is now torndown due to DeploymentManager action