developer_uid: shiroe40
submission_id: cycy233-l3-bpc-v0-c1_v1
model_name: auto
model_group: cycy233/L3-bpc-v0-c1
status: torndown
timestamp: 2024-08-30T01:56:56+00:00
num_battles: 10967
num_wins: 5608
celo_rating: 1242.52
family_friendly_score: 0.0
submission_type: basic
model_repo: cycy233/L3-bpc-v0-c1
model_architecture: LlamaForCausalLM
model_num_parameters: 8030261248.0
best_of: 16
max_input_tokens: 512
max_output_tokens: 64
display_name: auto
is_internal_developer: False
language_model: cycy233/L3-bpc-v0-c1
model_size: 8B
ranking_group: single
us_pacific_date: 2024-08-29
win_ratio: 0.5113522385337832
generation_params: {'temperature': 1.0, 'top_p': 0.9, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['<|end_header_id|>', '<|eot_id|>'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}
formatter: {'memory_template': "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n{bot_name}'s Persona: {memory}\n\n", 'prompt_template': '{prompt}<|eot_id|>', 'bot_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}: {message}<|eot_id|>', 'user_template': '<|start_header_id|>user<|end_header_id|>\n\n{user_name}: {message}<|eot_id|>', 'response_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}:', 'truncate_by_message': False}
Resubmit model
Running pipeline stage MKMLizer
Starting job with name cycy233-l3-bpc-v0-c1-v1-mkmlizer
Waiting for job on cycy233-l3-bpc-v0-c1-v1-mkmlizer to finish
Stopping job with name cycy233-l3-bpc-v0-c1-v1-mkmlizer
%s, retrying in %s seconds...
Starting job with name cycy233-l3-bpc-v0-c1-v1-mkmlizer
Waiting for job on cycy233-l3-bpc-v0-c1-v1-mkmlizer to finish
Stopping job with name cycy233-l3-bpc-v0-c1-v1-mkmlizer
%s, retrying in %s seconds...
Starting job with name cycy233-l3-bpc-v0-c1-v1-mkmlizer
Waiting for job on cycy233-l3-bpc-v0-c1-v1-mkmlizer to finish
cycy233-l3-bpc-v0-c1-v1-mkmlizer: quantized model in 25.854s
cycy233-l3-bpc-v0-c1-v1-mkmlizer: Processed model cycy233/L3-bpc-v0-c1 in 61.050s
cycy233-l3-bpc-v0-c1-v1-mkmlizer: creating bucket guanaco-mkml-models
cycy233-l3-bpc-v0-c1-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
cycy233-l3-bpc-v0-c1-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/cycy233-l3-bpc-v0-c1-v1
cycy233-l3-bpc-v0-c1-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/cycy233-l3-bpc-v0-c1-v1/config.json
cycy233-l3-bpc-v0-c1-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/cycy233-l3-bpc-v0-c1-v1/special_tokens_map.json
cycy233-l3-bpc-v0-c1-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/cycy233-l3-bpc-v0-c1-v1/tokenizer_config.json
cycy233-l3-bpc-v0-c1-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/cycy233-l3-bpc-v0-c1-v1/tokenizer.json
cycy233-l3-bpc-v0-c1-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/cycy233-l3-bpc-v0-c1-v1/flywheel_model.0.safetensors
cycy233-l3-bpc-v0-c1-v1-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 4/291 [00:00<00:07, 38.67it/s]
Loading 0:   4%|▍         | 13/291 [00:00<00:04, 67.81it/s]
Loading 0:   9%|▊         | 25/291 [00:00<00:03, 77.19it/s]
Loading 0:  14%|█▎        | 40/291 [00:00<00:02, 84.93it/s]
Loading 0:  17%|█▋        | 49/291 [00:00<00:02, 84.76it/s]
Loading 0:  20%|█▉        | 58/291 [00:00<00:02, 85.78it/s]
Loading 0:  23%|██▎       | 67/291 [00:00<00:02, 81.53it/s]
Loading 0:  26%|██▌       | 76/291 [00:00<00:02, 79.30it/s]
Loading 0:  29%|██▉       | 84/291 [00:01<00:09, 22.33it/s]
Loading 0:  31%|███       | 90/291 [00:02<00:07, 25.94it/s]
Loading 0:  35%|███▌      | 103/291 [00:02<00:05, 36.47it/s]
Loading 0:  40%|███▉      | 115/291 [00:02<00:03, 44.81it/s]
Loading 0:  45%|████▍     | 130/291 [00:02<00:02, 57.68it/s]
Loading 0:  48%|████▊     | 141/291 [00:02<00:02, 66.79it/s]
Loading 0:  52%|█████▏    | 150/291 [00:02<00:02, 67.61it/s]
Loading 0:  55%|█████▍    | 159/291 [00:02<00:01, 70.46it/s]
Loading 0:  58%|█████▊    | 168/291 [00:03<00:01, 71.74it/s]
Loading 0:  61%|██████    | 177/291 [00:03<00:01, 69.16it/s]
Loading 0:  64%|██████▎   | 185/291 [00:03<00:01, 71.71it/s]
Loading 0:  66%|██████▋   | 193/291 [00:04<00:04, 21.12it/s]
Loading 0:  69%|██████▉   | 202/291 [00:04<00:03, 26.83it/s]
Loading 0:  73%|███████▎  | 211/291 [00:04<00:02, 33.62it/s]
Loading 0:  76%|███████▌  | 220/291 [00:04<00:01, 40.91it/s]
Loading 0:  79%|███████▊  | 229/291 [00:04<00:01, 47.05it/s]
Loading 0:  82%|████████▏ | 238/291 [00:04<00:01, 51.45it/s]
Loading 0:  85%|████████▍ | 247/291 [00:05<00:00, 54.91it/s]
Loading 0:  88%|████████▊ | 256/291 [00:05<00:00, 62.03it/s]
Loading 0:  91%|█████████ | 265/291 [00:05<00:00, 64.22it/s]
Loading 0:  94%|█████████▍| 274/291 [00:05<00:00, 63.96it/s]
Loading 0:  97%|█████████▋| 283/291 [00:05<00:00, 68.55it/s]
Loading 0: 100%|██████████| 291/291 [00:10<00:00,  5.15it/s]
                                                            

Job cycy233-l3-bpc-v0-c1-v1-mkmlizer completed after 83.78s with status: succeeded
Stopping job with name cycy233-l3-bpc-v0-c1-v1-mkmlizer
Pipeline stage MKMLizer completed in 114.57s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.13s
Running pipeline stage ISVCDeployer
Creating inference service cycy233-l3-bpc-v0-c1-v1
Waiting for inference service cycy233-l3-bpc-v0-c1-v1 to be ready
Inference service cycy233-l3-bpc-v0-c1-v1 ready after 321.602979183197s
Pipeline stage ISVCDeployer completed in 321.94s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.8645851612091064s
Received healthy response to inference request in 2.0738894939422607s
Received healthy response to inference request in 1.6539547443389893s
Received healthy response to inference request in 1.9269180297851562s
Received healthy response to inference request in 1.9128530025482178s
5 requests
0 failed requests
5th percentile: 1.6960808277130126
10th percentile: 1.7382069110870362
20th percentile: 1.822459077835083
30th percentile: 1.8742387294769287
40th percentile: 1.8935458660125732
50th percentile: 1.9128530025482178
60th percentile: 1.9184790134429932
70th percentile: 1.9241050243377686
80th percentile: 1.9563123226165773
90th percentile: 2.015100908279419
95th percentile: 2.04449520111084
99th percentile: 2.0680106353759764
mean time: 1.8864400863647461
Pipeline stage StressChecker completed in 11.18s
cycy233-l3-bpc-v0-c1_v1 status is now deployed due to DeploymentManager action
cycy233-l3-bpc-v0-c1_v1 status is now inactive due to auto deactivation removed underperforming models
cycy233-l3-bpc-v0-c1_v1 status is now torndown due to DeploymentManager action