developer_uid: Bbbrun0
submission_id: jellywibble-wibblewobble_v5
model_name: test
model_group: Jellywibble/WibbleWobble
status: torndown
timestamp: 2024-08-28T02:39:48+00:00
num_battles: 10476
num_wins: 5633
celo_rating: 1262.53
family_friendly_score: 0.0
submission_type: basic
model_repo: Jellywibble/WibbleWobble
model_architecture: LlamaForCausalLM
model_num_parameters: 8030261248.0
best_of: 16
max_input_tokens: 512
max_output_tokens: 64
display_name: test
is_internal_developer: False
language_model: Jellywibble/WibbleWobble
model_size: 8B
ranking_group: single
us_pacific_date: 2024-08-27
win_ratio: 0.5377052310042001
generation_params: {'temperature': 1.4, 'top_p': 1.0, 'min_p': 0.2, 'top_k': 50, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|end_header_id|>', '<|eot_id|>', '\n\n{user_name}'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}
formatter: {'memory_template': "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n{bot_name}'s Persona: {memory}\n\n", 'prompt_template': '{prompt}<|eot_id|>', 'bot_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}: {message}<|eot_id|>', 'user_template': '<|start_header_id|>user<|end_header_id|>\n\n{user_name}: {message}<|eot_id|>', 'response_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}:', 'truncate_by_message': False}
Resubmit model
Running pipeline stage MKMLizer
Starting job with name jellywibble-wibblewobble-v5-mkmlizer
Waiting for job on jellywibble-wibblewobble-v5-mkmlizer to finish
Stopping job with name jellywibble-wibblewobble-v5-mkmlizer
%s, retrying in %s seconds...
Starting job with name jellywibble-wibblewobble-v5-mkmlizer
Waiting for job on jellywibble-wibblewobble-v5-mkmlizer to finish
jellywibble-wibblewobble-v5-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
jellywibble-wibblewobble-v5-mkmlizer: ║     _____            __           __                                ║
jellywibble-wibblewobble-v5-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
jellywibble-wibblewobble-v5-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
jellywibble-wibblewobble-v5-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
jellywibble-wibblewobble-v5-mkmlizer: ║       /___/                                                         ║
jellywibble-wibblewobble-v5-mkmlizer: ║                                                                     ║
jellywibble-wibblewobble-v5-mkmlizer: ║  Version: 0.10.1                                                    ║
jellywibble-wibblewobble-v5-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
jellywibble-wibblewobble-v5-mkmlizer: ║  https://mk1.ai                                                     ║
jellywibble-wibblewobble-v5-mkmlizer: ║                                                                     ║
jellywibble-wibblewobble-v5-mkmlizer: ║  The license key for the current software has been verified as      ║
jellywibble-wibblewobble-v5-mkmlizer: ║  belonging to:                                                      ║
jellywibble-wibblewobble-v5-mkmlizer: ║                                                                     ║
jellywibble-wibblewobble-v5-mkmlizer: ║  Chai Research Corp.                                                ║
jellywibble-wibblewobble-v5-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
jellywibble-wibblewobble-v5-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
jellywibble-wibblewobble-v5-mkmlizer: ║                                                                     ║
jellywibble-wibblewobble-v5-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
jellywibble-wibblewobble-v5-mkmlizer: Downloaded to shared memory in 62.816s
jellywibble-wibblewobble-v5-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpthrn023p, device:0
jellywibble-wibblewobble-v5-mkmlizer: Saving flywheel model at /dev/shm/model_cache
jellywibble-wibblewobble-v5-mkmlizer: quantized model in 28.907s
jellywibble-wibblewobble-v5-mkmlizer: Processed model Jellywibble/WibbleWobble in 91.723s
jellywibble-wibblewobble-v5-mkmlizer: creating bucket guanaco-mkml-models
jellywibble-wibblewobble-v5-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
jellywibble-wibblewobble-v5-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/jellywibble-wibblewobble-v5
jellywibble-wibblewobble-v5-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/jellywibble-wibblewobble-v5/config.json
jellywibble-wibblewobble-v5-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/jellywibble-wibblewobble-v5/special_tokens_map.json
jellywibble-wibblewobble-v5-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/jellywibble-wibblewobble-v5/tokenizer_config.json
jellywibble-wibblewobble-v5-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/jellywibble-wibblewobble-v5/tokenizer.json
jellywibble-wibblewobble-v5-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/jellywibble-wibblewobble-v5/flywheel_model.0.safetensors
jellywibble-wibblewobble-v5-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 5/291 [00:00<00:10, 26.96it/s]
Loading 0:   4%|▍         | 12/291 [00:00<00:06, 45.20it/s]
Loading 0:   6%|▌         | 18/291 [00:00<00:06, 40.85it/s]
Loading 0:   8%|▊         | 23/291 [00:00<00:08, 31.67it/s]
Loading 0:  10%|▉         | 29/291 [00:00<00:06, 38.39it/s]
Loading 0:  12%|█▏        | 34/291 [00:01<00:09, 28.02it/s]
Loading 0:  13%|█▎        | 38/291 [00:01<00:08, 28.94it/s]
Loading 0:  14%|█▍        | 42/291 [00:01<00:09, 27.09it/s]
Loading 0:  16%|█▋        | 48/291 [00:01<00:07, 32.17it/s]
Loading 0:  18%|█▊        | 52/291 [00:01<00:07, 31.80it/s]
Loading 0:  20%|█▉        | 57/291 [00:01<00:06, 35.12it/s]
Loading 0:  21%|██        | 61/291 [00:01<00:06, 34.43it/s]
Loading 0:  23%|██▎       | 67/291 [00:01<00:05, 37.98it/s]
Loading 0:  24%|██▍       | 71/291 [00:02<00:05, 36.80it/s]
Loading 0:  26%|██▌       | 75/291 [00:02<00:07, 29.89it/s]
Loading 0:  27%|██▋       | 80/291 [00:02<00:07, 26.62it/s]
Loading 0:  29%|██▊       | 83/291 [00:02<00:08, 24.49it/s]
Loading 0:  31%|███       | 90/291 [00:02<00:06, 31.93it/s]
Loading 0:  32%|███▏      | 94/291 [00:02<00:06, 31.75it/s]
Loading 0:  34%|███▍      | 99/291 [00:03<00:05, 35.70it/s]
Loading 0:  35%|███▌      | 103/291 [00:03<00:05, 33.30it/s]
Loading 0:  37%|███▋      | 108/291 [00:03<00:05, 36.00it/s]
Loading 0:  38%|███▊      | 112/291 [00:03<00:05, 34.94it/s]
Loading 0:  40%|███▉      | 116/291 [00:03<00:04, 35.72it/s]
Loading 0:  42%|████▏     | 122/291 [00:03<00:04, 40.49it/s]
Loading 0:  44%|████▎     | 127/291 [00:03<00:04, 36.69it/s]
Loading 0:  46%|████▌     | 133/291 [00:04<00:04, 33.23it/s]
Loading 0:  47%|████▋     | 137/291 [00:04<00:04, 32.44it/s]
Loading 0:  48%|████▊     | 141/291 [00:04<00:05, 29.84it/s]
Loading 0:  50%|████▉     | 145/291 [00:04<00:04, 31.72it/s]
Loading 0:  51%|█████     | 149/291 [00:04<00:04, 28.67it/s]
Loading 0:  54%|█████▎    | 156/291 [00:04<00:03, 36.81it/s]
Loading 0:  55%|█████▍    | 160/291 [00:04<00:03, 35.38it/s]
Loading 0:  57%|█████▋    | 165/291 [00:04<00:03, 37.42it/s]
Loading 0:  58%|█████▊    | 169/291 [00:05<00:03, 33.56it/s]
Loading 0:  60%|█████▉    | 174/291 [00:05<00:03, 34.84it/s]
Loading 0:  61%|██████    | 178/291 [00:05<00:03, 32.35it/s]
Loading 0:  63%|██████▎   | 182/291 [00:05<00:03, 33.97it/s]
Loading 0:  64%|██████▍   | 186/291 [00:05<00:03, 26.91it/s]
Loading 0:  65%|██████▍   | 189/291 [00:05<00:04, 24.11it/s]
Loading 0:  67%|██████▋   | 194/291 [00:06<00:03, 25.18it/s]
Loading 0:  69%|██████▉   | 201/291 [00:06<00:02, 31.85it/s]
Loading 0:  70%|███████   | 205/291 [00:06<00:02, 31.88it/s]
Loading 0:  72%|███████▏  | 210/291 [00:06<00:02, 35.32it/s]
Loading 0:  74%|███████▎  | 214/291 [00:06<00:02, 34.32it/s]
Loading 0:  75%|███████▌  | 219/291 [00:06<00:01, 37.23it/s]
Loading 0:  77%|███████▋  | 223/291 [00:06<00:01, 34.03it/s]
Loading 0:  78%|███████▊  | 227/291 [00:06<00:01, 34.27it/s]
Loading 0:  79%|███████▉  | 231/291 [00:07<00:01, 35.48it/s]
Loading 0:  81%|████████  | 235/291 [00:07<00:02, 26.98it/s]
Loading 0:  82%|████████▏ | 239/291 [00:07<00:02, 25.61it/s]
Loading 0:  85%|████████▍ | 246/291 [00:07<00:01, 33.49it/s]
Loading 0:  86%|████████▌ | 250/291 [00:07<00:01, 32.08it/s]
Loading 0:  88%|████████▊ | 255/291 [00:07<00:01, 34.02it/s]
Loading 0:  89%|████████▉ | 259/291 [00:07<00:00, 32.90it/s]
Loading 0:  91%|█████████ | 264/291 [00:08<00:00, 36.34it/s]
Loading 0:  92%|█████████▏| 268/291 [00:08<00:00, 34.96it/s]
Loading 0:  94%|█████████▍| 273/291 [00:08<00:00, 36.43it/s]
Loading 0:  95%|█████████▌| 277/291 [00:08<00:00, 34.01it/s]
Loading 0:  97%|█████████▋| 281/291 [00:08<00:00, 34.74it/s]
Loading 0:  98%|█████████▊| 286/291 [00:14<00:01,  2.58it/s]
Loading 0:  99%|█████████▉| 289/291 [00:14<00:00,  3.22it/s]
                                                            

Job jellywibble-wibblewobble-v5-mkmlizer completed after 116.59s with status: succeeded
Stopping job with name jellywibble-wibblewobble-v5-mkmlizer
Pipeline stage MKMLizer completed in 118.44s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.09s
Running pipeline stage ISVCDeployer
Creating inference service jellywibble-wibblewobble-v5
Waiting for inference service jellywibble-wibblewobble-v5 to be ready
Failed to get response for submission blend_koran_2024-08-16: ('http://chaiml-elo-alignment-run-3-v34-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '{"error":"ValueError : [TypeError(\\"\'numpy.int64\' object is not iterable\\"), TypeError(\'vars() argument must have __dict__ attribute\')]"}')
Inference service jellywibble-wibblewobble-v5 ready after 170.66428136825562s
Pipeline stage ISVCDeployer completed in 171.21s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.3407034873962402s
Received healthy response to inference request in 1.781771183013916s
Received healthy response to inference request in 2.0375940799713135s
Received healthy response to inference request in 1.5011279582977295s
Received healthy response to inference request in 2.17655611038208s
5 requests
0 failed requests
5th percentile: 1.5572566032409667
10th percentile: 1.6133852481842041
20th percentile: 1.7256425380706788
30th percentile: 1.8329357624053955
40th percentile: 1.9352649211883546
50th percentile: 2.0375940799713135
60th percentile: 2.09317889213562
70th percentile: 2.1487637042999266
80th percentile: 2.2093855857849123
90th percentile: 2.2750445365905763
95th percentile: 2.307874011993408
99th percentile: 2.334137592315674
mean time: 1.967550563812256
Pipeline stage StressChecker completed in 11.55s
jellywibble-wibblewobble_v5 status is now deployed due to DeploymentManager action
jellywibble-wibblewobble_v5 status is now inactive due to auto deactivation removed underperforming models
jellywibble-wibblewobble_v5 status is now torndown due to DeploymentManager action
jellywibble-wibblewobble_v5 status is now torndown due to DeploymentManager action