hastagaras-cupang-12b-test-7

submission_id: hastagaras-cupang-12b-test-7_v2
developer_uid: Hastagaras
best_of: 4
celo_rating: 1196.32
display_name: hastagaras-cupang-12b-test-7_v2
family_friendly_score: 0.0
formatter: {'memory_template': "<s>[INST] {bot_name}'s Persona: {memory}\n\n", 'prompt_template': '{prompt} [/INST]\n', 'bot_template': '{bot_name}: {message}</s>\n', 'user_template': '[INST] {user_name}: {message} [/INST]\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.0, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 60, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<', '[TOOL_CALLS]', '[/TOOL_RESULTS]', 'You:'], 'max_input_tokens': 1024, 'best_of': 4, 'max_output_tokens': 64, 'reward_max_token_input': 256}
is_internal_developer: False
language_model: Hastagaras/Cupang-12B-Test-7
max_input_tokens: 1024
max_output_tokens: 64
model_architecture: MistralForCausalLM
model_group: Hastagaras/Cupang-12B-Te
model_name: hastagaras-cupang-12b-test-7_v2
model_num_parameters: 12772070400.0
model_repo: Hastagaras/Cupang-12B-Test-7
model_size: 13B
num_battles: 16680
num_wins: 8155
ranking_group: single
reward_formatter: {'bot_template': '{bot_name}: {message}\n', 'memory_template': '', 'prompt_template': '', 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}\n'}
reward_repo: ChaiML/gpt2_xl_pairwise_89m_step_347634
status: torndown
submission_type: basic
timestamp: 2024-08-03T21:45:47+00:00
us_pacific_date: 2024-08-03
win_ratio: 0.48890887290167867
Resubmit model
Running pipeline stage MKMLizer
Starting job with name hastagaras-cupang-12b-test-7-v2-mkmlizer
Waiting for job on hastagaras-cupang-12b-test-7-v2-mkmlizer to finish
Stopping job with name hastagaras-cupang-12b-test-7-v2-mkmlizer
%s, retrying in %s seconds...
Starting job with name hastagaras-cupang-12b-test-7-v2-mkmlizer
Waiting for job on hastagaras-cupang-12b-test-7-v2-mkmlizer to finish
hastagaras-cupang-12b-test-7-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
hastagaras-cupang-12b-test-7-v2-mkmlizer: ║     _____            __           __                                ║
hastagaras-cupang-12b-test-7-v2-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
hastagaras-cupang-12b-test-7-v2-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
hastagaras-cupang-12b-test-7-v2-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
hastagaras-cupang-12b-test-7-v2-mkmlizer: ║       /___/                                                         ║
hastagaras-cupang-12b-test-7-v2-mkmlizer: ║                                                                     ║
hastagaras-cupang-12b-test-7-v2-mkmlizer: ║  Version: 0.9.9                                                     ║
hastagaras-cupang-12b-test-7-v2-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
hastagaras-cupang-12b-test-7-v2-mkmlizer: ║  https://mk1.ai                                                     ║
hastagaras-cupang-12b-test-7-v2-mkmlizer: ║                                                                     ║
hastagaras-cupang-12b-test-7-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
hastagaras-cupang-12b-test-7-v2-mkmlizer: ║  belonging to:                                                      ║
hastagaras-cupang-12b-test-7-v2-mkmlizer: ║                                                                     ║
hastagaras-cupang-12b-test-7-v2-mkmlizer: ║  Chai Research Corp.                                                ║
hastagaras-cupang-12b-test-7-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
hastagaras-cupang-12b-test-7-v2-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
hastagaras-cupang-12b-test-7-v2-mkmlizer: ║                                                                     ║
hastagaras-cupang-12b-test-7-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
hastagaras-cupang-12b-test-7-v2-mkmlizer: Downloaded to shared memory in 29.480s
hastagaras-cupang-12b-test-7-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpx3t04ctz, device:0
hastagaras-cupang-12b-test-7-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
hastagaras-cupang-12b-test-7-v2-mkmlizer: quantized model in 35.353s
hastagaras-cupang-12b-test-7-v2-mkmlizer: Processed model Hastagaras/Cupang-12B-Test-7 in 64.833s
hastagaras-cupang-12b-test-7-v2-mkmlizer: creating bucket guanaco-mkml-models
hastagaras-cupang-12b-test-7-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
hastagaras-cupang-12b-test-7-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/hastagaras-cupang-12b-test-7-v2
hastagaras-cupang-12b-test-7-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/hastagaras-cupang-12b-test-7-v2/config.json
hastagaras-cupang-12b-test-7-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/hastagaras-cupang-12b-test-7-v2/special_tokens_map.json
hastagaras-cupang-12b-test-7-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/hastagaras-cupang-12b-test-7-v2/tokenizer_config.json
hastagaras-cupang-12b-test-7-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/hastagaras-cupang-12b-test-7-v2/flywheel_model.0.safetensors
hastagaras-cupang-12b-test-7-v2-mkmlizer: loading reward model from ChaiML/gpt2_xl_pairwise_89m_step_347634
hastagaras-cupang-12b-test-7-v2-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:11, 31.92it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:06, 51.54it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:07, 46.89it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:07, 45.27it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:06, 51.26it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:06, 46.92it/s]
Loading 0:  12%|█▏        | 42/363 [00:00<00:07, 44.04it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:06, 48.71it/s]
Loading 0:  15%|█▌        | 55/363 [00:01<00:06, 45.94it/s]
Loading 0:  17%|█▋        | 61/363 [00:01<00:08, 34.39it/s]
Loading 0:  18%|█▊        | 65/363 [00:01<00:08, 33.91it/s]
Loading 0:  20%|█▉        | 72/363 [00:01<00:07, 40.21it/s]
Loading 0:  21%|██▏       | 78/363 [00:01<00:07, 39.56it/s]
Loading 0:  23%|██▎       | 83/363 [00:01<00:07, 39.94it/s]
Loading 0:  25%|██▍       | 90/363 [00:02<00:06, 45.00it/s]
Loading 0:  26%|██▌       | 95/363 [00:02<00:05, 46.21it/s]
Loading 0:  28%|██▊       | 100/363 [00:02<00:06, 38.31it/s]
Loading 0:  29%|██▉       | 107/363 [00:02<00:05, 45.44it/s]
Loading 0:  31%|███       | 113/363 [00:02<00:05, 42.00it/s]
Loading 0:  33%|███▎      | 118/363 [00:02<00:05, 41.86it/s]
Loading 0:  35%|███▍      | 126/363 [00:02<00:04, 49.19it/s]
Loading 0:  36%|███▋      | 132/363 [00:03<00:05, 44.94it/s]
Loading 0:  38%|███▊      | 137/363 [00:03<00:05, 44.26it/s]
Loading 0:  39%|███▉      | 142/363 [00:03<00:06, 34.04it/s]
Loading 0:  40%|████      | 147/363 [00:03<00:06, 35.45it/s]
Loading 0:  42%|████▏     | 151/363 [00:03<00:05, 36.33it/s]
Loading 0:  43%|████▎     | 156/363 [00:03<00:05, 38.98it/s]
Loading 0:  44%|████▍     | 161/363 [00:03<00:04, 40.69it/s]
Loading 0:  46%|████▌     | 167/363 [00:04<00:05, 38.28it/s]
Loading 0:  48%|████▊     | 175/363 [00:04<00:04, 45.90it/s]
Loading 0:  50%|████▉     | 181/363 [00:04<00:04, 43.89it/s]
Loading 0:  51%|█████     | 186/363 [00:04<00:04, 42.46it/s]
Loading 0:  53%|█████▎    | 193/363 [00:04<00:03, 47.45it/s]
Loading 0:  55%|█████▍    | 199/363 [00:04<00:03, 45.29it/s]
Loading 0:  56%|█████▌    | 204/363 [00:04<00:03, 43.25it/s]
Loading 0:  58%|█████▊    | 211/363 [00:04<00:03, 47.60it/s]
Loading 0:  60%|█████▉    | 216/363 [00:05<00:03, 48.18it/s]
Loading 0:  61%|██████    | 222/363 [00:05<00:03, 45.08it/s]
Loading 0:  63%|██████▎   | 227/363 [00:05<00:04, 32.38it/s]
Loading 0:  64%|██████▎   | 231/363 [00:05<00:04, 31.53it/s]
Loading 0:  66%|██████▌   | 238/363 [00:05<00:03, 38.32it/s]
Loading 0:  67%|██████▋   | 244/363 [00:05<00:03, 38.98it/s]
Loading 0:  69%|██████▊   | 249/363 [00:06<00:02, 38.87it/s]
Loading 0:  70%|███████   | 255/363 [00:06<00:02, 43.70it/s]
Loading 0:  72%|███████▏  | 260/363 [00:06<00:02, 44.34it/s]
Loading 0:  73%|███████▎  | 265/363 [00:06<00:02, 45.22it/s]
Loading 0:  75%|███████▍  | 271/363 [00:06<00:02, 43.53it/s]
Loading 0:  76%|███████▌  | 276/363 [00:06<00:02, 42.77it/s]
Loading 0:  78%|███████▊  | 283/363 [00:06<00:01, 47.66it/s]
Loading 0:  80%|███████▉  | 289/363 [00:06<00:01, 45.09it/s]
Loading 0:  81%|████████  | 294/363 [00:06<00:01, 43.46it/s]
Loading 0:  83%|████████▎ | 300/363 [00:07<00:01, 47.42it/s]
Loading 0:  84%|████████▍ | 305/363 [00:13<00:22,  2.63it/s]
Loading 0:  85%|████████▌ | 309/363 [00:13<00:16,  3.37it/s]
Loading 0:  86%|████████▌ | 313/363 [00:14<00:11,  4.36it/s]
Loading 0:  88%|████████▊ | 320/363 [00:14<00:06,  6.82it/s]
Loading 0:  90%|████████▉ | 326/363 [00:14<00:03,  9.32it/s]
Loading 0:  91%|█████████ | 331/363 [00:14<00:02, 11.91it/s]
Loading 0:  93%|█████████▎| 338/363 [00:14<00:01, 16.77it/s]
Loading 0:  95%|█████████▍| 344/363 [00:14<00:00, 20.31it/s]
Loading 0:  96%|█████████▌| 349/363 [00:14<00:00, 23.25it/s]
Loading 0:  98%|█████████▊| 356/363 [00:14<00:00, 29.61it/s]
Loading 0: 100%|█████████▉| 362/363 [00:15<00:00, 32.19it/s]
                                                            
/opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:957: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
hastagaras-cupang-12b-test-7-v2-mkmlizer:   warnings.warn(
hastagaras-cupang-12b-test-7-v2-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:785: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
hastagaras-cupang-12b-test-7-v2-mkmlizer:   warnings.warn(
hastagaras-cupang-12b-test-7-v2-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
hastagaras-cupang-12b-test-7-v2-mkmlizer:   warnings.warn(
hastagaras-cupang-12b-test-7-v2-mkmlizer: 
Downloading shards:   0%|          | 0/2 [00:00<?, ?it/s]
Downloading shards:  50%|█████     | 1/2 [00:05<00:05,  5.01s/it]
Downloading shards: 100%|██████████| 2/2 [00:07<00:00,  3.65s/it]
Downloading shards: 100%|██████████| 2/2 [00:07<00:00,  3.86s/it]
hastagaras-cupang-12b-test-7-v2-mkmlizer: 
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:00<00:00,  2.38it/s]
Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00,  3.86it/s]
Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00,  3.53it/s]
hastagaras-cupang-12b-test-7-v2-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
hastagaras-cupang-12b-test-7-v2-mkmlizer: Saving duration: 1.303s
hastagaras-cupang-12b-test-7-v2-mkmlizer: Processed model ChaiML/gpt2_xl_pairwise_89m_step_347634 in 12.341s
hastagaras-cupang-12b-test-7-v2-mkmlizer: creating bucket guanaco-reward-models
hastagaras-cupang-12b-test-7-v2-mkmlizer: Bucket 's3://guanaco-reward-models/' created
hastagaras-cupang-12b-test-7-v2-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/hastagaras-cupang-12b-test-7-v2_reward
hastagaras-cupang-12b-test-7-v2-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/hastagaras-cupang-12b-test-7-v2_reward/config.json
hastagaras-cupang-12b-test-7-v2-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/hastagaras-cupang-12b-test-7-v2_reward/special_tokens_map.json
hastagaras-cupang-12b-test-7-v2-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/hastagaras-cupang-12b-test-7-v2_reward/tokenizer_config.json
hastagaras-cupang-12b-test-7-v2-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/hastagaras-cupang-12b-test-7-v2_reward/merges.txt
hastagaras-cupang-12b-test-7-v2-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/hastagaras-cupang-12b-test-7-v2_reward/vocab.json
hastagaras-cupang-12b-test-7-v2-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/hastagaras-cupang-12b-test-7-v2_reward/tokenizer.json
hastagaras-cupang-12b-test-7-v2-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/hastagaras-cupang-12b-test-7-v2_reward/reward.tensors
Job hastagaras-cupang-12b-test-7-v2-mkmlizer completed after 115.97s with status: succeeded
Stopping job with name hastagaras-cupang-12b-test-7-v2-mkmlizer
Pipeline stage MKMLizer completed in 117.65s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.09s
Running pipeline stage ISVCDeployer
Creating inference service hastagaras-cupang-12b-test-7-v2
Failed to get response for submission mistralai-mixtral-8x7b_3473_v107: ('http://mistralai-mixtral-8x7b-3473-v107-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:32796->127.0.0.1:8080: read: connection reset by peer\n')
Waiting for inference service hastagaras-cupang-12b-test-7-v2 to be ready
Inference service hastagaras-cupang-12b-test-7-v2 ready after 150.9126889705658s
Pipeline stage ISVCDeployer completed in 152.86s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.1324944496154785s
Received healthy response to inference request in 1.5648677349090576s
Received healthy response to inference request in 1.5268197059631348s
Received healthy response to inference request in 1.4506757259368896s
Received healthy response to inference request in 1.5624668598175049s
5 requests
0 failed requests
5th percentile: 1.4659045219421387
10th percentile: 1.4811333179473878
20th percentile: 1.5115909099578857
30th percentile: 1.5339491367340088
40th percentile: 1.5482079982757568
50th percentile: 1.5624668598175049
60th percentile: 1.563427209854126
70th percentile: 1.564387559890747
80th percentile: 1.678393077850342
90th percentile: 1.9054437637329102
95th percentile: 2.018969106674194
99th percentile: 2.1097893810272215
mean time: 1.647464895248413
Pipeline stage StressChecker completed in 8.93s
hastagaras-cupang-12b-test-7_v2 status is now deployed due to DeploymentManager action
hastagaras-cupang-12b-test-7_v2 status is now inactive due to auto deactivation removed underperforming models
hastagaras-cupang-12b-test-7_v2 status is now torndown due to DeploymentManager action
admin requested tearing down of hastagaras-cupang-12b-test-7_v2
Running pipeline stage ISVCDeleter
Pipeline stage %s skipped, reason=%s
Pipeline stage ISVCDeleter completed in 0.67s
Running pipeline stage MKMLModelDeleter
Pipeline stage %s skipped, reason=%s
Pipeline stage MKMLModelDeleter completed in 0.41s
hastagaras-cupang-12b-test-7_v2 status is now torndown due to DeploymentManager action