cycy233-l3-ba-q-v0-c4

developer_uid: shiroe40
submission_id: cycy233-l3-ba-q-v0-c4_v1
model_name: auto
model_group: cycy233/L3-ba-q-v0-c4
status: torndown
timestamp: 2024-08-24T03:00:37+00:00
num_battles: 9241
num_wins: 4750
celo_rating: 1248.24
family_friendly_score: 0.0
submission_type: basic
model_repo: cycy233/L3-ba-q-v0-c4
model_architecture: LlamaForCausalLM
model_num_parameters: 8030261248.0
best_of: 16
max_input_tokens: 512
max_output_tokens: 64
display_name: auto
is_internal_developer: False
language_model: cycy233/L3-ba-q-v0-c4
model_size: 8B
ranking_group: single
us_pacific_date: 2024-08-23
win_ratio: 0.5140136348879991
generation_params: {'temperature': 1.0, 'top_p': 0.9, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['<|end_header_id|>', '<|eot_id|>'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}
formatter: {'memory_template': "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n{bot_name}'s Persona: {memory}\n\n", 'prompt_template': '{prompt}<|eot_id|>', 'bot_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}: {message}<|eot_id|>', 'user_template': '<|start_header_id|>user<|end_header_id|>\n\n{user_name}: {message}<|eot_id|>', 'response_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}:', 'truncate_by_message': False}
Resubmit model
Running pipeline stage MKMLizer
Starting job with name cycy233-l3-ba-q-v0-c4-v1-mkmlizer
Waiting for job on cycy233-l3-ba-q-v0-c4-v1-mkmlizer to finish
cycy233-l3-ba-q-v0-c4-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
cycy233-l3-ba-q-v0-c4-v1-mkmlizer: ║     _____            __           __                                ║
cycy233-l3-ba-q-v0-c4-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
cycy233-l3-ba-q-v0-c4-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
cycy233-l3-ba-q-v0-c4-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
cycy233-l3-ba-q-v0-c4-v1-mkmlizer: ║       /___/                                                         ║
cycy233-l3-ba-q-v0-c4-v1-mkmlizer: ║                                                                     ║
cycy233-l3-ba-q-v0-c4-v1-mkmlizer: ║  Version: 0.10.1                                                    ║
cycy233-l3-ba-q-v0-c4-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
cycy233-l3-ba-q-v0-c4-v1-mkmlizer: ║  https://mk1.ai                                                     ║
cycy233-l3-ba-q-v0-c4-v1-mkmlizer: ║                                                                     ║
cycy233-l3-ba-q-v0-c4-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
cycy233-l3-ba-q-v0-c4-v1-mkmlizer: ║  belonging to:                                                      ║
cycy233-l3-ba-q-v0-c4-v1-mkmlizer: ║                                                                     ║
cycy233-l3-ba-q-v0-c4-v1-mkmlizer: ║  Chai Research Corp.                                                ║
cycy233-l3-ba-q-v0-c4-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
cycy233-l3-ba-q-v0-c4-v1-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
cycy233-l3-ba-q-v0-c4-v1-mkmlizer: ║                                                                     ║
cycy233-l3-ba-q-v0-c4-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
cycy233-l3-ba-q-v0-c3-v1-mkmlizer: Downloaded to shared memory in 37.252s
cycy233-l3-ba-q-v0-c3-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpo9_83uh3, device:0
cycy233-l3-ba-q-v0-c3-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
cycy233-l3-ba-q-v0-c4-v1-mkmlizer: Downloaded to shared memory in 30.592s
cycy233-l3-ba-q-v0-c4-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpm3d5u21b, device:0
cycy233-l3-ba-q-v0-c4-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
cycy233-l3-ba-q-v0-c3-v1-mkmlizer: quantized model in 28.573s
cycy233-l3-ba-q-v0-c3-v1-mkmlizer: Processed model cycy233/L3-ba-q-v0-c3 in 65.825s
cycy233-l3-ba-q-v0-c3-v1-mkmlizer: creating bucket guanaco-mkml-models
cycy233-l3-ba-q-v0-c3-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
cycy233-l3-ba-q-v0-c3-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/cycy233-l3-ba-q-v0-c3-v1
cycy233-l3-ba-q-v0-c3-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/cycy233-l3-ba-q-v0-c3-v1/config.json
cycy233-l3-ba-q-v0-c3-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/cycy233-l3-ba-q-v0-c3-v1/special_tokens_map.json
cycy233-l3-ba-q-v0-c3-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/cycy233-l3-ba-q-v0-c3-v1/tokenizer_config.json
cycy233-l3-ba-q-v0-c3-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/cycy233-l3-ba-q-v0-c3-v1/tokenizer.json
cycy233-l3-ba-q-v0-c3-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/cycy233-l3-ba-q-v0-c3-v1/flywheel_model.0.safetensors
cycy233-l3-ba-q-v0-c3-v1-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 5/291 [00:00<00:08, 32.33it/s]
Loading 0:   4%|▍         | 13/291 [00:00<00:05, 52.02it/s]
Loading 0:   7%|▋         | 19/291 [00:00<00:05, 45.91it/s]
Loading 0:   8%|▊         | 24/291 [00:00<00:05, 44.52it/s]
Loading 0:  11%|█         | 31/291 [00:00<00:05, 51.64it/s]
Loading 0:  13%|█▎        | 37/291 [00:00<00:05, 45.14it/s]
Loading 0:  14%|█▍        | 42/291 [00:00<00:05, 43.45it/s]
Loading 0:  17%|█▋        | 49/291 [00:01<00:05, 48.20it/s]
Loading 0:  19%|█▊        | 54/291 [00:01<00:05, 47.28it/s]
Loading 0:  20%|██        | 59/291 [00:01<00:06, 36.77it/s]
Loading 0:  23%|██▎       | 67/291 [00:01<00:04, 45.22it/s]
Loading 0:  25%|██▌       | 73/291 [00:01<00:05, 43.12it/s]
Loading 0:  27%|██▋       | 78/291 [00:01<00:05, 42.02it/s]
Loading 0:  29%|██▊       | 83/291 [00:02<00:06, 31.44it/s]
Loading 0:  30%|██▉       | 87/291 [00:02<00:06, 31.41it/s]
Loading 0:  32%|███▏      | 94/291 [00:02<00:05, 37.95it/s]
Loading 0:  34%|███▍      | 99/291 [00:02<00:04, 39.72it/s]
Loading 0:  36%|███▌      | 104/291 [00:02<00:05, 33.69it/s]
Loading 0:  38%|███▊      | 111/291 [00:02<00:04, 40.85it/s]
Loading 0:  40%|███▉      | 116/291 [00:02<00:04, 41.34it/s]
Loading 0:  42%|████▏     | 121/291 [00:02<00:04, 41.96it/s]
Loading 0:  44%|████▎     | 127/291 [00:03<00:04, 40.67it/s]
Loading 0:  45%|████▌     | 132/291 [00:03<00:04, 39.73it/s]
Loading 0:  47%|████▋     | 138/291 [00:03<00:03, 44.07it/s]
Loading 0:  49%|████▉     | 143/291 [00:03<00:03, 42.82it/s]
Loading 0:  51%|█████     | 148/291 [00:03<00:03, 42.89it/s]
Loading 0:  53%|█████▎    | 153/291 [00:03<00:03, 43.24it/s]
Loading 0:  54%|█████▍    | 158/291 [00:03<00:03, 35.85it/s]
Loading 0:  57%|█████▋    | 165/291 [00:03<00:02, 42.77it/s]
Loading 0:  58%|█████▊    | 170/291 [00:04<00:02, 42.33it/s]
Loading 0:  60%|██████    | 175/291 [00:04<00:02, 44.05it/s]
Loading 0:  62%|██████▏   | 180/291 [00:04<00:02, 41.83it/s]
Loading 0:  64%|██████▎   | 185/291 [00:04<00:02, 42.88it/s]
Loading 0:  65%|██████▌   | 190/291 [00:04<00:04, 24.86it/s]
Loading 0:  67%|██████▋   | 194/291 [00:05<00:03, 24.96it/s]
Loading 0:  69%|██████▉   | 201/291 [00:05<00:02, 32.43it/s]
Loading 0:  71%|███████   | 206/291 [00:05<00:02, 34.21it/s]
Loading 0:  73%|███████▎  | 211/291 [00:05<00:02, 35.66it/s]
Loading 0:  74%|███████▍  | 216/291 [00:05<00:01, 37.93it/s]
Loading 0:  76%|███████▌  | 221/291 [00:05<00:02, 31.94it/s]
Loading 0:  78%|███████▊  | 228/291 [00:05<00:01, 38.49it/s]
Loading 0:  80%|████████  | 233/291 [00:05<00:01, 38.52it/s]
Loading 0:  82%|████████▏ | 238/291 [00:06<00:01, 38.18it/s]
Loading 0:  84%|████████▎ | 243/291 [00:06<00:01, 39.22it/s]
Loading 0:  85%|████████▌ | 248/291 [00:06<00:01, 31.88it/s]
Loading 0:  88%|████████▊ | 255/291 [00:06<00:00, 38.24it/s]
Loading 0:  89%|████████▉ | 260/291 [00:06<00:00, 37.67it/s]
Loading 0:  91%|█████████ | 265/291 [00:06<00:00, 37.34it/s]
Loading 0:  92%|█████████▏| 269/291 [00:06<00:00, 36.71it/s]
Loading 0:  94%|█████████▍| 273/291 [00:07<00:00, 37.13it/s]
Loading 0:  95%|█████████▌| 277/291 [00:07<00:00, 36.48it/s]
Loading 0:  97%|█████████▋| 281/291 [00:07<00:00, 35.89it/s]
Loading 0:  98%|█████████▊| 285/291 [00:07<00:00, 35.58it/s]
Loading 0:  99%|█████████▉| 289/291 [00:13<00:00,  2.32it/s]
                                                            
cycy233-l3-ba-q-v0-c4-v1-mkmlizer: quantized model in 28.093s
cycy233-l3-ba-q-v0-c4-v1-mkmlizer: Processed model cycy233/L3-ba-q-v0-c4 in 58.686s
cycy233-l3-ba-q-v0-c4-v1-mkmlizer: creating bucket guanaco-mkml-models
cycy233-l3-ba-q-v0-c4-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
cycy233-l3-ba-q-v0-c4-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/cycy233-l3-ba-q-v0-c4-v1
cycy233-l3-ba-q-v0-c4-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/cycy233-l3-ba-q-v0-c4-v1/config.json
cycy233-l3-ba-q-v0-c4-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/cycy233-l3-ba-q-v0-c4-v1/special_tokens_map.json
cycy233-l3-ba-q-v0-c4-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/cycy233-l3-ba-q-v0-c4-v1/tokenizer_config.json
cycy233-l3-ba-q-v0-c4-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/cycy233-l3-ba-q-v0-c4-v1/tokenizer.json
cycy233-l3-ba-q-v0-c4-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/cycy233-l3-ba-q-v0-c4-v1/flywheel_model.0.safetensors
cycy233-l3-ba-q-v0-c4-v1-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 5/291 [00:00<00:08, 31.95it/s]
Loading 0:   4%|▍         | 13/291 [00:00<00:05, 50.99it/s]
Loading 0:   7%|▋         | 19/291 [00:00<00:06, 43.41it/s]
Loading 0:   8%|▊         | 24/291 [00:00<00:06, 42.07it/s]
Loading 0:  10%|█         | 30/291 [00:00<00:05, 46.68it/s]
Loading 0:  12%|█▏        | 35/291 [00:00<00:05, 45.39it/s]
Loading 0:  14%|█▎        | 40/291 [00:00<00:05, 44.57it/s]
Loading 0:  15%|█▌        | 45/291 [00:01<00:05, 45.56it/s]
Loading 0:  17%|█▋        | 50/291 [00:01<00:06, 36.18it/s]
Loading 0:  20%|█▉        | 57/291 [00:01<00:05, 43.78it/s]
Loading 0:  21%|██▏       | 62/291 [00:01<00:05, 42.16it/s]
Loading 0:  23%|██▎       | 67/291 [00:01<00:05, 41.69it/s]
Loading 0:  25%|██▍       | 72/291 [00:01<00:05, 42.31it/s]
Loading 0:  26%|██▋       | 77/291 [00:01<00:06, 34.70it/s]
Loading 0:  29%|██▊       | 83/291 [00:02<00:07, 27.58it/s]
Loading 0:  30%|██▉       | 87/291 [00:02<00:07, 27.53it/s]
Loading 0:  32%|███▏      | 93/291 [00:02<00:05, 33.15it/s]
Loading 0:  33%|███▎      | 97/291 [00:02<00:05, 33.14it/s]
Loading 0:  35%|███▌      | 102/291 [00:02<00:05, 36.41it/s]
Loading 0:  36%|███▋      | 106/291 [00:02<00:05, 36.17it/s]
Loading 0:  38%|███▊      | 111/291 [00:02<00:04, 39.30it/s]
Loading 0:  40%|███▉      | 116/291 [00:03<00:04, 39.28it/s]
Loading 0:  42%|████▏     | 121/291 [00:03<00:04, 40.00it/s]
Loading 0:  43%|████▎     | 126/291 [00:03<00:04, 39.96it/s]
Loading 0:  45%|████▌     | 131/291 [00:03<00:04, 33.48it/s]
Loading 0:  47%|████▋     | 138/291 [00:03<00:03, 41.26it/s]
Loading 0:  49%|████▉     | 143/291 [00:03<00:03, 42.61it/s]
Loading 0:  51%|█████     | 149/291 [00:03<00:03, 39.44it/s]
Loading 0:  54%|█████▍    | 157/291 [00:03<00:02, 46.61it/s]
Loading 0:  56%|█████▌    | 162/291 [00:04<00:02, 47.02it/s]
Loading 0:  57%|█████▋    | 167/291 [00:04<00:03, 41.25it/s]
Loading 0:  60%|██████    | 176/291 [00:04<00:02, 52.75it/s]
Loading 0:  63%|██████▎   | 182/291 [00:04<00:02, 45.91it/s]
Loading 0:  65%|██████▍   | 188/291 [00:04<00:02, 36.43it/s]
Loading 0:  66%|██████▋   | 193/291 [00:04<00:02, 37.04it/s]
Loading 0:  68%|██████▊   | 199/291 [00:05<00:02, 37.87it/s]
Loading 0:  70%|███████   | 204/291 [00:05<00:02, 39.12it/s]
Loading 0:  73%|███████▎  | 211/291 [00:05<00:01, 44.53it/s]
Loading 0:  74%|███████▍  | 216/291 [00:05<00:01, 45.77it/s]
Loading 0:  76%|███████▌  | 221/291 [00:05<00:01, 36.53it/s]
Loading 0:  78%|███████▊  | 228/291 [00:05<00:01, 43.75it/s]
Loading 0:  80%|████████  | 233/291 [00:05<00:01, 44.03it/s]
Loading 0:  82%|████████▏ | 238/291 [00:05<00:01, 44.01it/s]
Loading 0:  84%|████████▎ | 243/291 [00:06<00:01, 45.04it/s]
Loading 0:  85%|████████▌ | 248/291 [00:06<00:01, 35.86it/s]
Loading 0:  87%|████████▋ | 253/291 [00:06<00:00, 39.04it/s]
Loading 0:  89%|████████▊ | 258/291 [00:06<00:00, 37.30it/s]
Loading 0:  91%|█████████ | 264/291 [00:06<00:00, 40.66it/s]
Loading 0:  92%|█████████▏| 269/291 [00:06<00:00, 41.12it/s]
Loading 0:  94%|█████████▍| 274/291 [00:06<00:00, 41.71it/s]
Loading 0:  96%|█████████▌| 279/291 [00:06<00:00, 40.68it/s]
Loading 0:  98%|█████████▊| 284/291 [00:07<00:00, 33.23it/s]
Loading 0:  99%|█████████▉| 288/291 [00:12<00:01,  2.59it/s]
                                                            
Job cycy233-l3-ba-q-v0-c3-v1-mkmlizer completed after 96.06s with status: succeeded
Stopping job with name cycy233-l3-ba-q-v0-c3-v1-mkmlizer
Pipeline stage MKMLizer completed in 97.11s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.17s
Running pipeline stage ISVCDeployer
Creating inference service cycy233-l3-ba-q-v0-c3-v1
Waiting for inference service cycy233-l3-ba-q-v0-c3-v1 to be ready
Job cycy233-l3-ba-q-v0-c4-v1-mkmlizer completed after 85.22s with status: succeeded
Stopping job with name cycy233-l3-ba-q-v0-c4-v1-mkmlizer
Pipeline stage MKMLizer completed in 85.94s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.14s
Running pipeline stage ISVCDeployer
Creating inference service cycy233-l3-ba-q-v0-c4-v1
Waiting for inference service cycy233-l3-ba-q-v0-c4-v1 to be ready
Inference service cycy233-l3-ba-q-v0-c3-v1 ready after 211.80721473693848s
Pipeline stage ISVCDeployer completed in 213.08s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.20780611038208s
Received healthy response to inference request in 2.2011096477508545s
Received healthy response to inference request in 1.8050239086151123s
Received healthy response to inference request in 2.07828426361084s
Received healthy response to inference request in 2.0274291038513184s
5 requests
0 failed requests
5th percentile: 1.8495049476623535
10th percentile: 1.8939859867095947
20th percentile: 1.9829480648040771
30th percentile: 2.0376001358032227
40th percentile: 2.057942199707031
50th percentile: 2.07828426361084
60th percentile: 2.127414417266846
70th percentile: 2.1765445709228515
80th percentile: 2.2024489402770997
90th percentile: 2.2051275253295897
95th percentile: 2.206466817855835
99th percentile: 2.207538251876831
mean time: 2.063930606842041
Pipeline stage StressChecker completed in 12.66s
cycy233-l3-ba-q-v0-c3_v1 status is now deployed due to DeploymentManager action
Inference service cycy233-l3-ba-q-v0-c4-v1 ready after 222.17279720306396s
Pipeline stage ISVCDeployer completed in 223.11s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.092038154602051s
Received healthy response to inference request in 1.9236056804656982s
Received healthy response to inference request in 2.1096408367156982s
Received healthy response to inference request in 1.6468474864959717s
Received healthy response to inference request in 2.832148551940918s
5 requests
0 failed requests
5th percentile: 1.702199125289917
10th percentile: 1.7575507640838623
20th percentile: 1.868254041671753
30th percentile: 1.9572921752929688
40th percentile: 2.0246651649475096
50th percentile: 2.092038154602051
60th percentile: 2.09907922744751
70th percentile: 2.106120300292969
80th percentile: 2.2541423797607423
90th percentile: 2.54314546585083
95th percentile: 2.687647008895874
99th percentile: 2.8032482433319093
mean time: 2.1208561420440675
Pipeline stage StressChecker completed in 11.62s
cycy233-l3-ba-q-v0-c4_v1 status is now deployed due to DeploymentManager action
cycy233-l3-ba-q-v0-c4_v1 status is now inactive due to auto deactivation removed underperforming models
cycy233-l3-ba-q-v0-c4_v1 status is now torndown due to DeploymentManager action