usernamejustanother-nemo_7622

submission_id: usernamejustanother-nemo_7622_v1
developer_uid: Azazelle
best_of: 8
celo_rating: 1232.98
display_name: Nemo-12B-Marlin-v5
family_friendly_score: 0.0
formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.05, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<', '<|'], 'max_input_tokens': 512, 'best_of': 8, 'max_output_tokens': 64, 'reward_max_token_input': 256}
is_internal_developer: False
language_model: UsernameJustAnother/Nemo-12B-Marlin-v5
max_input_tokens: 512
max_output_tokens: 64
model_architecture: MistralForCausalLM
model_group: UsernameJustAnother/Nemo
model_name: Nemo-12B-Marlin-v5
model_num_parameters: 12772070400.0
model_repo: UsernameJustAnother/Nemo-12B-Marlin-v5
model_size: 13B
num_battles: 12549
num_wins: 6596
ranking_group: single
reward_formatter: {'bot_template': '{bot_name}: {message}\n', 'memory_template': '', 'prompt_template': '', 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}\n'}
reward_repo: ChaiML/gpt2_xl_pairwise_89m_step_347634
status: torndown
submission_type: basic
timestamp: 2024-08-06T20:36:44+00:00
us_pacific_date: 2024-08-06
win_ratio: 0.5256195712805801
Resubmit model
Running pipeline stage MKMLizer
Starting job with name usernamejustanother-nemo-7622-v1-mkmlizer
Waiting for job on usernamejustanother-nemo-7622-v1-mkmlizer to finish
usernamejustanother-nemo-7622-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
usernamejustanother-nemo-7622-v1-mkmlizer: ║     _____            __           __                                ║
usernamejustanother-nemo-7622-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
usernamejustanother-nemo-7622-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
usernamejustanother-nemo-7622-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
usernamejustanother-nemo-7622-v1-mkmlizer: ║       /___/                                                         ║
usernamejustanother-nemo-7622-v1-mkmlizer: ║                                                                     ║
usernamejustanother-nemo-7622-v1-mkmlizer: ║  Version: 0.9.9                                                     ║
usernamejustanother-nemo-7622-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
usernamejustanother-nemo-7622-v1-mkmlizer: ║  https://mk1.ai                                                     ║
usernamejustanother-nemo-7622-v1-mkmlizer: ║                                                                     ║
usernamejustanother-nemo-7622-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
usernamejustanother-nemo-7622-v1-mkmlizer: ║  belonging to:                                                      ║
usernamejustanother-nemo-7622-v1-mkmlizer: ║                                                                     ║
usernamejustanother-nemo-7622-v1-mkmlizer: ║  Chai Research Corp.                                                ║
usernamejustanother-nemo-7622-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
usernamejustanother-nemo-7622-v1-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
usernamejustanother-nemo-7622-v1-mkmlizer: ║                                                                     ║
usernamejustanother-nemo-7622-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
usernamejustanother-nemo-7622-v1-mkmlizer: Downloaded to shared memory in 35.302s
usernamejustanother-nemo-7622-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpkjsbs7gz, device:0
usernamejustanother-nemo-7622-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
usernamejustanother-nemo-7622-v1-mkmlizer: quantized model in 36.034s
usernamejustanother-nemo-7622-v1-mkmlizer: Processed model UsernameJustAnother/Nemo-12B-Marlin-v5 in 71.337s
usernamejustanother-nemo-7622-v1-mkmlizer: creating bucket guanaco-mkml-models
usernamejustanother-nemo-7622-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
usernamejustanother-nemo-7622-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/usernamejustanother-nemo-7622-v1
usernamejustanother-nemo-7622-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/usernamejustanother-nemo-7622-v1/config.json
usernamejustanother-nemo-7622-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/usernamejustanother-nemo-7622-v1/special_tokens_map.json
usernamejustanother-nemo-7622-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/usernamejustanother-nemo-7622-v1/tokenizer_config.json
usernamejustanother-nemo-7622-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/usernamejustanother-nemo-7622-v1/flywheel_model.0.safetensors
usernamejustanother-nemo-7622-v1-mkmlizer: loading reward model from ChaiML/gpt2_xl_pairwise_89m_step_347634
usernamejustanother-nemo-7622-v1-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:10, 33.15it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:06, 51.96it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:07, 43.74it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:08, 42.05it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:07, 47.37it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:07, 43.12it/s]
Loading 0:  12%|█▏        | 42/363 [00:00<00:07, 42.25it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:06, 47.26it/s]
Loading 0:  15%|█▌        | 55/363 [00:01<00:06, 45.42it/s]
Loading 0:  17%|█▋        | 61/363 [00:01<00:08, 33.95it/s]
Loading 0:  18%|█▊        | 65/363 [00:01<00:08, 33.31it/s]
Loading 0:  20%|█▉        | 71/363 [00:01<00:07, 38.61it/s]
Loading 0:  21%|██        | 76/363 [00:01<00:07, 39.60it/s]
Loading 0:  22%|██▏       | 81/363 [00:01<00:06, 41.49it/s]
Loading 0:  24%|██▎       | 86/363 [00:02<00:06, 43.48it/s]
Loading 0:  25%|██▌       | 91/363 [00:02<00:07, 36.29it/s]
Loading 0:  27%|██▋       | 99/363 [00:02<00:05, 44.48it/s]
Loading 0:  29%|██▊       | 104/363 [00:02<00:05, 45.34it/s]
Loading 0:  30%|███       | 109/363 [00:02<00:05, 46.27it/s]
Loading 0:  31%|███▏      | 114/363 [00:02<00:06, 36.95it/s]
Loading 0:  33%|███▎      | 119/363 [00:02<00:06, 37.55it/s]
Loading 0:  34%|███▍      | 125/363 [00:03<00:05, 42.27it/s]
Loading 0:  36%|███▌      | 130/363 [00:03<00:05, 42.23it/s]
Loading 0:  37%|███▋      | 135/363 [00:03<00:05, 42.90it/s]
Loading 0:  39%|███▉      | 141/363 [00:03<00:05, 41.83it/s]
Loading 0:  40%|████      | 146/363 [00:03<00:07, 30.42it/s]
Loading 0:  41%|████▏     | 150/363 [00:03<00:06, 30.78it/s]
Loading 0:  43%|████▎     | 156/363 [00:03<00:05, 36.47it/s]
Loading 0:  44%|████▍     | 161/363 [00:04<00:05, 38.19it/s]
Loading 0:  46%|████▌     | 166/363 [00:04<00:05, 39.26it/s]
Loading 0:  47%|████▋     | 171/363 [00:04<00:04, 41.13it/s]
Loading 0:  48%|████▊     | 176/363 [00:04<00:05, 35.12it/s]
Loading 0:  50%|█████     | 183/363 [00:04<00:04, 42.75it/s]
Loading 0:  52%|█████▏    | 188/363 [00:04<00:04, 43.12it/s]
Loading 0:  53%|█████▎    | 193/363 [00:04<00:04, 41.86it/s]
Loading 0:  55%|█████▍    | 199/363 [00:04<00:04, 40.72it/s]
Loading 0:  56%|█████▌    | 204/363 [00:05<00:03, 40.79it/s]
Loading 0:  58%|█████▊    | 210/363 [00:05<00:03, 44.30it/s]
Loading 0:  59%|█████▉    | 215/363 [00:05<00:03, 43.68it/s]
Loading 0:  61%|██████    | 220/363 [00:05<00:03, 43.62it/s]
Loading 0:  62%|██████▏   | 225/363 [00:05<00:05, 26.96it/s]
Loading 0:  63%|██████▎   | 230/363 [00:05<00:04, 29.66it/s]
Loading 0:  65%|██████▌   | 237/363 [00:06<00:03, 37.10it/s]
Loading 0:  67%|██████▋   | 242/363 [00:06<00:03, 38.87it/s]
Loading 0:  68%|██████▊   | 247/363 [00:06<00:02, 40.03it/s]
Loading 0:  70%|██████▉   | 253/363 [00:06<00:02, 39.82it/s]
Loading 0:  71%|███████   | 258/363 [00:06<00:02, 39.70it/s]
Loading 0:  72%|███████▏  | 263/363 [00:06<00:02, 41.51it/s]
Loading 0:  74%|███████▍  | 268/363 [00:06<00:02, 40.39it/s]
Loading 0:  75%|███████▌  | 274/363 [00:06<00:02, 43.24it/s]
Loading 0:  77%|███████▋  | 279/363 [00:06<00:01, 44.88it/s]
Loading 0:  78%|███████▊  | 284/363 [00:07<00:02, 36.93it/s]
Loading 0:  80%|████████  | 291/363 [00:07<00:01, 43.51it/s]
Loading 0:  82%|████████▏ | 296/363 [00:07<00:01, 43.89it/s]
Loading 0:  83%|████████▎ | 301/363 [00:07<00:01, 44.02it/s]
Loading 0:  84%|████████▍ | 306/363 [00:14<00:23,  2.47it/s]
Loading 0:  85%|████████▌ | 310/363 [00:14<00:16,  3.21it/s]
Loading 0:  87%|████████▋ | 314/363 [00:14<00:11,  4.21it/s]
Loading 0:  88%|████████▊ | 320/363 [00:14<00:06,  6.27it/s]
Loading 0:  90%|████████▉ | 326/363 [00:14<00:04,  8.74it/s]
Loading 0:  91%|█████████ | 330/363 [00:14<00:03, 10.63it/s]
Loading 0:  93%|█████████▎| 337/363 [00:15<00:01, 15.63it/s]
Loading 0:  94%|█████████▍| 342/363 [00:15<00:01, 19.21it/s]
Loading 0:  96%|█████████▌| 347/363 [00:15<00:00, 23.29it/s]
Loading 0:  97%|█████████▋| 353/363 [00:15<00:00, 27.11it/s]
Loading 0:  99%|█████████▊| 358/363 [00:15<00:00, 30.07it/s]
                                                            
/opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:957: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
usernamejustanother-nemo-7622-v1-mkmlizer:   warnings.warn(
usernamejustanother-nemo-7622-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:785: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
usernamejustanother-nemo-7622-v1-mkmlizer:   warnings.warn(
usernamejustanother-nemo-7622-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
usernamejustanother-nemo-7622-v1-mkmlizer:   warnings.warn(
usernamejustanother-nemo-7622-v1-mkmlizer: 
Downloading shards:   0%|          | 0/2 [00:00<?, ?it/s]
Downloading shards:  50%|█████     | 1/2 [00:05<00:05,  5.40s/it]
Downloading shards: 100%|██████████| 2/2 [00:08<00:00,  3.98s/it]
Downloading shards: 100%|██████████| 2/2 [00:08<00:00,  4.20s/it]
usernamejustanother-nemo-7622-v1-mkmlizer: 
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:00<00:00,  2.40it/s]
Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00,  3.95it/s]
Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00,  3.60it/s]
usernamejustanother-nemo-7622-v1-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
usernamejustanother-nemo-7622-v1-mkmlizer: Saving duration: 1.346s
usernamejustanother-nemo-7622-v1-mkmlizer: Processed model ChaiML/gpt2_xl_pairwise_89m_step_347634 in 13.472s
usernamejustanother-nemo-7622-v1-mkmlizer: creating bucket guanaco-reward-models
usernamejustanother-nemo-7622-v1-mkmlizer: Bucket 's3://guanaco-reward-models/' created
usernamejustanother-nemo-7622-v1-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/usernamejustanother-nemo-7622-v1_reward
usernamejustanother-nemo-7622-v1-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/usernamejustanother-nemo-7622-v1_reward/config.json
usernamejustanother-nemo-7622-v1-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/usernamejustanother-nemo-7622-v1_reward/special_tokens_map.json
usernamejustanother-nemo-7622-v1-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/usernamejustanother-nemo-7622-v1_reward/tokenizer_config.json
usernamejustanother-nemo-7622-v1-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/usernamejustanother-nemo-7622-v1_reward/merges.txt
usernamejustanother-nemo-7622-v1-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/usernamejustanother-nemo-7622-v1_reward/vocab.json
usernamejustanother-nemo-7622-v1-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/usernamejustanother-nemo-7622-v1_reward/tokenizer.json
usernamejustanother-nemo-7622-v1-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/usernamejustanother-nemo-7622-v1_reward/reward.tensors
Job usernamejustanother-nemo-7622-v1-mkmlizer completed after 125.77s with status: succeeded
Stopping job with name usernamejustanother-nemo-7622-v1-mkmlizer
Pipeline stage MKMLizer completed in 126.79s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.09s
Running pipeline stage ISVCDeployer
Creating inference service usernamejustanother-nemo-7622-v1
Waiting for inference service usernamejustanother-nemo-7622-v1 to be ready
Inference service usernamejustanother-nemo-7622-v1 ready after 171.27239227294922s
Pipeline stage ISVCDeployer completed in 173.34s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.5934674739837646s
Received healthy response to inference request in 1.5840568542480469s
Received healthy response to inference request in 1.5902352333068848s
Received healthy response to inference request in 1.5892627239227295s
%s, retrying in %s seconds...
Received healthy response to inference request in 1.8190462589263916s
Received healthy response to inference request in 1.8608171939849854s
Received healthy response to inference request in 8.73722529411316s
Received healthy response to inference request in 1.6450011730194092s
Received healthy response to inference request in 14.150760889053345s
5 requests
0 failed requests
5th percentile: 1.6798101902008056
10th percentile: 1.7146192073822022
20th percentile: 1.7842372417449952
30th percentile: 1.8274004459381104
40th percentile: 1.8441088199615479
50th percentile: 1.8608171939849854
60th percentile: 4.6113804340362545
70th percentile: 7.361943674087524
80th percentile: 9.819932413101197
90th percentile: 11.985346651077272
95th percentile: 13.068053770065307
99th percentile: 13.934219465255737
mean time: 5.642570161819458
%s, retrying in %s seconds...
Received healthy response to inference request in 1.5903279781341553s
Received healthy response to inference request in 2.7802443504333496s
Received healthy response to inference request in 1.2648074626922607s
Received healthy response to inference request in 1.5768277645111084s
Received healthy response to inference request in 5.798621892929077s
5 requests
0 failed requests
5th percentile: 1.3272115230560302
10th percentile: 1.3896155834197998
20th percentile: 1.514423704147339
30th percentile: 1.5795278072357177
40th percentile: 1.5849278926849366
50th percentile: 1.5903279781341553
60th percentile: 2.0662945270538327
70th percentile: 2.5422610759735105
80th percentile: 3.3839198589324955
90th percentile: 4.591270875930786
95th percentile: 5.194946384429931
99th percentile: 5.677886791229248
mean time: 2.6021658897399904
Pipeline stage StressChecker completed in 70.19s
usernamejustanother-nemo_7622_v1 status is now deployed due to DeploymentManager action
usernamejustanother-nemo_7622_v1 status is now inactive due to auto deactivation removed underperforming models
usernamejustanother-nemo_7622_v1 status is now torndown due to DeploymentManager action