submission_id: neversleep-noromaid-v0_8068_v146
developer_uid: Jellywibble
alignment_samples: 0
best_of: 4
celo_rating: 1179.53
display_name: noromaid-rolling
formatter: {'memory_template': '', 'prompt_template': '<s>[INST] This is an entertaining conversation. You are {bot_name} who has the persona: {memory}.\nPlay the role of {bot_name}. Engage in a chat with {user_name} while staying in character. You should create a fun dialogue which entertains {user_name}.\n{prompt}\n', 'bot_template': '{bot_name}: {message}</s>', 'user_template': '[INST] {user_name}: {message} [/INST]', 'response_template': '{bot_name}:', 'truncate_by_message': True}
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 50, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '<|user|>', '###'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64, 'reward_max_token_input': 256}
is_internal_developer: True
language_model: NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3
max_input_tokens: 512
max_output_tokens: 64
model_architecture: MixtralForCausalLM
model_group: NeverSleep/Noromaid-v0.1
model_name: noromaid-rolling
model_num_parameters: 46702792704.0
model_repo: NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3
model_size: 47B
num_battles: 9271
num_wins: 4198
propriety_score: 0.7274853801169591
propriety_total_count: 855.0
ranking_group: single
reward_formatter: {'bot_template': 'Bot: {message}\n', 'memory_template': 'Memory: {memory}\n', 'prompt_template': '{prompt}\n', 'response_template': 'Bot:', 'truncate_by_message': False, 'user_template': 'User: {message}\n'}
reward_repo: ChaiML/gpt2_xl_pairwise_89m_step_347634
status: torndown
submission_type: basic
timestamp: 2024-08-05T23:04:31+00:00
us_pacific_date: 2024-08-05
win_ratio: 0.45280983712652356
Download Preference Data
Resubmit model
Running pipeline stage MKMLizer
Starting job with name neversleep-noromaid-v0-8068-v146-mkmlizer
Waiting for job on neversleep-noromaid-v0-8068-v146-mkmlizer to finish
neversleep-noromaid-v0-8068-v146-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
neversleep-noromaid-v0-8068-v146-mkmlizer: ║ _____ __ __ ║
neversleep-noromaid-v0-8068-v146-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
neversleep-noromaid-v0-8068-v146-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
neversleep-noromaid-v0-8068-v146-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
neversleep-noromaid-v0-8068-v146-mkmlizer: ║ /___/ ║
neversleep-noromaid-v0-8068-v146-mkmlizer: ║ ║
neversleep-noromaid-v0-8068-v146-mkmlizer: ║ Version: 0.9.9 ║
neversleep-noromaid-v0-8068-v146-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
neversleep-noromaid-v0-8068-v146-mkmlizer: ║ https://mk1.ai ║
neversleep-noromaid-v0-8068-v146-mkmlizer: ║ ║
neversleep-noromaid-v0-8068-v146-mkmlizer: ║ The license key for the current software has been verified as ║
neversleep-noromaid-v0-8068-v146-mkmlizer: ║ belonging to: ║
neversleep-noromaid-v0-8068-v146-mkmlizer: ║ ║
neversleep-noromaid-v0-8068-v146-mkmlizer: ║ Chai Research Corp. ║
neversleep-noromaid-v0-8068-v146-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
neversleep-noromaid-v0-8068-v146-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
neversleep-noromaid-v0-8068-v146-mkmlizer: ║ ║
neversleep-noromaid-v0-8068-v146-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
neversleep-noromaid-v0-8068-v146-mkmlizer: Downloaded to shared memory in 114.781s
neversleep-noromaid-v0-8068-v146-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpaannwvix, device:0
neversleep-noromaid-v0-8068-v146-mkmlizer: Saving flywheel model at /dev/shm/model_cache
neversleep-noromaid-v0-8068-v146-mkmlizer: quantized model in 91.062s
neversleep-noromaid-v0-8068-v146-mkmlizer: Processed model NeverSleep/Noromaid-v0.1-mixtral-8x7b-Instruct-v3 in 205.843s
neversleep-noromaid-v0-8068-v146-mkmlizer: creating bucket guanaco-mkml-models
neversleep-noromaid-v0-8068-v146-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
neversleep-noromaid-v0-8068-v146-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v146
neversleep-noromaid-v0-8068-v146-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v146/special_tokens_map.json
neversleep-noromaid-v0-8068-v146-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v146/config.json
neversleep-noromaid-v0-8068-v146-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v146/tokenizer_config.json
neversleep-noromaid-v0-8068-v146-mkmlizer: cp /dev/shm/model_cache/tokenizer.model s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v146/tokenizer.model
neversleep-noromaid-v0-8068-v146-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v146/tokenizer.json
neversleep-noromaid-v0-8068-v146-mkmlizer: cp /dev/shm/model_cache/flywheel_model.3.safetensors s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v146/flywheel_model.3.safetensors
neversleep-noromaid-v0-8068-v146-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v146/flywheel_model.0.safetensors
neversleep-noromaid-v0-8068-v146-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v146/flywheel_model.1.safetensors
neversleep-noromaid-v0-8068-v146-mkmlizer: cp /dev/shm/model_cache/flywheel_model.2.safetensors s3://guanaco-mkml-models/neversleep-noromaid-v0-8068-v146/flywheel_model.2.safetensors
neversleep-noromaid-v0-8068-v146-mkmlizer: loading reward model from ChaiML/gpt2_xl_pairwise_89m_step_347634
neversleep-noromaid-v0-8068-v146-mkmlizer: Loading 0: 0%| | 0/995 [00:00<?, ?it/s] Loading 0: 1%| | 9/995 [00:00<00:15, 62.01it/s] Loading 0: 2%|▏ | 16/995 [00:00<00:17, 57.39it/s] Loading 0: 2%|▏ | 22/995 [00:00<00:17, 54.56it/s] Loading 0: 3%|▎ | 28/995 [00:00<00:18, 52.67it/s] Loading 0: 4%|▎ | 35/995 [00:00<00:17, 55.94it/s] Loading 0: 4%|▍ | 43/995 [00:00<00:15, 60.05it/s] Loading 0: 5%|▌ | 50/995 [00:00<00:15, 60.28it/s] Loading 0: 6%|▌ | 57/995 [00:01<00:58, 16.11it/s] Loading 0: 6%|▌ | 62/995 [00:02<00:49, 19.01it/s] Loading 0: 7%|▋ | 71/995 [00:02<00:34, 26.75it/s] Loading 0: 8%|▊ | 77/995 [00:02<00:29, 30.68it/s] Loading 0: 8%|▊ | 83/995 [00:02<00:26, 34.00it/s] Loading 0: 9%|▉ | 89/995 [00:02<00:23, 37.94it/s] Loading 0: 10%|▉ | 97/995 [00:02<00:19, 45.40it/s] Loading 0: 11%|█ | 105/995 [00:02<00:18, 48.86it/s] Loading 0: 11%|█ | 111/995 [00:03<00:57, 15.43it/s] Loading 0: 12%|█▏ | 117/995 [00:04<00:46, 19.08it/s] Loading 0: 12%|█▏ | 123/995 [00:04<00:38, 22.77it/s] Loading 0: 13%|█▎ | 133/995 [00:04<00:26, 32.21it/s] Loading 0: 14%|█▍ | 139/995 [00:04<00:24, 35.53it/s] Loading 0: 15%|█▍ | 145/995 [00:04<00:21, 39.01it/s] Loading 0: 15%|█▌ | 151/995 [00:04<00:20, 41.60it/s] Loading 0: 16%|█▌ | 159/995 [00:04<00:17, 48.55it/s] Loading 0: 17%|█▋ | 165/995 [00:05<00:51, 16.17it/s] Loading 0: 17%|█▋ | 170/995 [00:05<00:43, 19.00it/s] Loading 0: 18%|█▊ | 176/995 [00:06<00:35, 23.06it/s] Loading 0: 18%|█▊ | 182/995 [00:06<00:29, 27.57it/s] Loading 0: 19%|█▉ | 190/995 [00:06<00:22, 35.56it/s] Loading 0: 20%|█▉ | 198/995 [00:06<00:18, 42.04it/s] Loading 0: 21%|██ | 204/995 [00:06<00:18, 43.77it/s] Loading 0: 21%|██ | 210/995 [00:07<00:58, 13.42it/s] Loading 0: 22%|██▏ | 216/995 [00:07<00:46, 16.92it/s] Loading 0: 23%|██▎ | 226/995 [00:08<00:30, 24.99it/s] Loading 0: 23%|██▎ | 232/995 [00:08<00:26, 28.28it/s] Loading 0: 24%|██▍ | 238/995 [00:08<00:23, 31.64it/s] Loading 0: 25%|██▍ | 244/995 [00:08<00:21, 35.10it/s] Loading 0: 25%|██▌ | 252/995 [00:08<00:17, 42.62it/s] Loading 0: 26%|██▌ | 260/995 [00:08<00:15, 47.59it/s] Loading 0: 27%|██▋ | 266/995 [00:09<00:49, 14.77it/s] Loading 0: 27%|██▋ | 272/995 [00:10<00:40, 18.01it/s] Loading 0: 28%|██▊ | 277/995 [00:20<00:39, 18.01it/s] Loading 0: 28%|██▊ | 278/995 [00:24<08:31, 1.40it/s] Loading 0: 29%|██▉ | 288/995 [00:24<05:05, 2.31it/s] Loading 0: 30%|██▉ | 294/995 [00:24<03:48, 3.06it/s] Loading 0: 30%|███ | 300/995 [00:24<02:50, 4.09it/s] Loading 0: 31%|███ | 306/995 [00:24<02:06, 5.45it/s] Loading 0: 32%|███▏ | 314/995 [00:25<01:25, 7.93it/s] Loading 0: 32%|███▏ | 320/995 [00:26<01:34, 7.11it/s] Loading 0: 33%|███▎ | 325/995 [00:26<01:15, 8.89it/s] Loading 0: 33%|███▎ | 331/995 [00:26<00:56, 11.79it/s] Loading 0: 34%|███▍ | 337/995 [00:26<00:43, 15.16it/s] Loading 0: 35%|███▍ | 345/995 [00:26<00:30, 21.06it/s] Loading 0: 35%|███▌ | 353/995 [00:26<00:22, 27.92it/s] Loading 0: 36%|███▌ | 359/995 [00:26<00:19, 32.10it/s] Loading 0: 37%|███▋ | 365/995 [00:26<00:17, 35.83it/s] Loading 0: 37%|███▋ | 371/995 [00:28<00:46, 13.32it/s] Loading 0: 38%|███▊ | 381/995 [00:28<00:30, 20.32it/s] Loading 0: 39%|███▉ | 387/995 [00:28<00:25, 23.96it/s] Loading 0: 39%|███▉ | 393/995 [00:28<00:21, 27.84it/s] Loading 0: 40%|████ | 399/995 [00:28<00:18, 32.06it/s] Loading 0: 41%|████ | 407/995 [00:28<00:14, 39.57it/s] Loading 0: 42%|████▏ | 415/995 [00:28<00:12, 45.63it/s] Loading 0: 42%|████▏ | 422/995 [00:28<00:11, 48.18it/s] Loading 0: 43%|████▎ | 428/995 [00:30<00:37, 15.21it/s] Loading 0: 44%|████▎ | 433/995 [00:30<00:31, 17.92it/s] Loading 0: 45%|████▍ | 443/995 [00:30<00:21, 26.26it/s] Loading 0: 45%|████▌ | 449/995 [00:30<00:18, 29.66it/s] Loading 0: 46%|████▌ | 455/995 [00:30<00:16, 33.00it/s] Loading 0: 46%|████▋ | 461/995 [00:30<00:14, 36.61it/s] Loading 0: 47%|████▋ | 469/995 [00:30<00:12, 43.76it/s] Loading 0: 48%|████▊ | 477/995 [00:30<00:10, 50.31it/s] Loading 0: 49%|████▊ | 484/995 [00:32<00:30, 16.51it/s] Loading 0: 49%|████▉ | 489/995 [00:32<00:26, 19.35it/s] Loading 0: 50%|████▉ | 495/995 [00:32<00:21, 23.61it/s] Loading 0: 51%|█████ | 505/995 [00:32<00:14, 33.27it/s] Loading 0: 51%|█████▏ | 511/995 [00:32<00:13, 36.77it/s] Loading 0: 52%|█████▏ | 517/995 [00:32<00:11, 40.34it/s] Loading 0: 53%|█████▎ | 523/995 [00:32<00:10, 43.31it/s] Loading 0: 53%|█████▎ | 529/995 [00:33<00:31, 14.81it/s] Loading 0: 54%|█████▍ | 536/995 [00:33<00:23, 19.63it/s] Loading 0: 54%|█████▍ | 542/995 [00:34<00:19, 23.63it/s] Loading 0: 55%|█████▌ | 548/995 [00:34<00:16, 27.78it/s] Loading 0: 56%|█████▌ | 554/995 [00:34<00:13, 32.11it/s] Loading 0: 56%|█████▋ | 562/995 [00:34<00:10, 40.28it/s] Loading 0: 56%|█████▋ | 562/995 [00:44<00:10, 40.28it/s] Loading 0: 57%|█████▋ | 563/995 [00:48<06:27, 1.12it/s] Loading 0: 57%|█████▋ | 567/995 [00:49<04:48, 1.48it/s] Loading 0: 58%|█████▊ | 573/995 [00:49<03:07, 2.25it/s] Loading 0: 58%|█████▊ | 579/995 [00:49<02:05, 3.33it/s] Loading 0: 59%|█████▉ | 585/995 [00:50<01:48, 3.79it/s] Loading 0: 60%|█████▉ | 593/995 [00:50<01:07, 5.94it/s] Loading 0: 60%|██████ | 601/995 [00:50<00:44, 8.76it/s] Loading 0: 61%|██████ | 607/995 [00:50<00:34, 11.36it/s] Loading 0: 62%|██████▏ | 613/995 [00:50<00:26, 14.46it/s] Loading 0: 62%|██████▏ | 619/995 [00:50<00:20, 18.01it/s] Loading 0: 63%|██████▎ | 629/995 [00:51<00:13, 26.72it/s] Loading 0: 64%|██████▍ | 636/995 [00:52<00:25, 13.99it/s] Loading 0: 64%|██████▍ | 641/995 [00:52<00:21, 16.39it/s] Loading 0: 65%|██████▌ | 647/995 [00:52<00:17, 20.23it/s] Loading 0: 66%|██████▌ | 655/995 [00:52<00:12, 27.16it/s] Loading 0: 67%|██████▋ | 663/995 [00:52<00:09, 34.32it/s] Loading 0: 67%|██████▋ | 670/995 [00:52<00:08, 38.44it/s] Loading 0: 68%|██████▊ | 676/995 [00:52<00:07, 41.10it/s] Loading 0: 69%|██████▊ | 682/995 [00:53<00:07, 43.56it/s] Loading 0: 69%|██████▉ | 691/995 [00:54<00:18, 16.86it/s] Loading 0: 70%|███████ | 697/995 [00:54<00:14, 20.48it/s] Loading 0: 71%|███████ | 703/995 [00:54<00:11, 24.50it/s] Loading 0: 71%|███████▏ | 709/995 [00:54<00:10, 28.53it/s] Loading 0: 72%|███████▏ | 717/995 [00:54<00:07, 35.50it/s] Loading 0: 73%|███████▎ | 725/995 [00:54<00:06, 42.79it/s] Loading 0: 73%|███████▎ | 731/995 [00:54<00:05, 45.16it/s] Loading 0: 74%|███████▍ | 737/995 [00:54<00:05, 46.96it/s] Loading 0: 75%|███████▍ | 743/995 [00:56<00:16, 14.85it/s] Loading 0: 76%|███████▌ | 753/995 [00:56<00:10, 22.29it/s] Loading 0: 76%|███████▋ | 759/995 [00:56<00:09, 25.94it/s] Loading 0: 77%|███████▋ | 765/995 [00:56<00:07, 29.63it/s] Loading 0: 77%|███████▋ | 771/995 [00:56<00:06, 33.44it/s] Loading 0: 78%|███████▊ | 779/995 [00:56<00:05, 40.59it/s] Loading 0: 79%|███████▉ | 787/995 [00:56<00:04, 46.74it/s] Loading 0: 80%|███████▉ | 793/995 [00:56<00:04, 46.32it/s] Loading 0: 80%|████████ | 799/995 [00:57<00:12, 15.32it/s] Loading 0: 81%|████████ | 805/995 [00:58<00:10, 18.57it/s] Loading 0: 82%|████████▏ | 815/995 [00:58<00:06, 27.25it/s] Loading 0: 83%|████████▎ | 821/995 [00:58<00:05, 30.39it/s] Loading 0: 83%|████████▎ | 827/995 [00:58<00:05, 33.37it/s] Loading 0: 84%|████████▎ | 833/995 [00:58<00:04, 36.93it/s] Loading 0: 85%|████████▍ | 841/995 [00:58<00:03, 44.31it/s] Loading 0: 85%|████████▍ | 845/995 [01:13<00:03, 44.31it/s] Loading 0: 85%|████████▌ | 846/995 [01:13<01:45, 1.41it/s] Loading 0: 85%|████████▌ | 849/995 [01:14<01:35, 1.52it/s] Loading 0: 86%|████████▌ | 855/995 [01:14<01:03, 2.22it/s] Loading 0: 87%|████████▋ | 861/995 [01:14<00:42, 3.19it/s] Loading 0: 87%|████████▋ | 867/995 [01:14<00:28, 4.50it/s] Loading 0: 88%|████████▊ | 877/995 [01:15<00:15, 7.53it/s] Loading 0: 89%|████████▊ | 883/995 [01:15<00:11, 9.74it/s] Loading 0: 89%|████████▉ | 889/995 [01:15<00:08, 12.51it/s] Loading 0: 90%|████████▉ | 895/995 [01:15<00:06, 15.87it/s] Loading 0: 91%|█████████ | 901/995 [01:16<00:08, 10.47it/s] Loading 0: 91%|█████████▏| 908/995 [01:16<00:06, 14.28it/s] Loading 0: 92%|█████████▏| 914/995 [01:16<00:04, 17.94it/s] Loading 0: 92%|█████████▏| 920/995 [01:16<00:03, 21.91it/s] Loading 0: 93%|█████████▎| 926/995 [01:16<00:02, 26.46it/s] Loading 0: 94%|█████████▍| 934/995 [01:17<00:01, 34.13it/s] Loading 0: 95%|█████████▍| 942/995 [01:17<00:01, 41.32it/s] Loading 0: 95%|█████████▌| 948/995 [01:17<00:01, 42.85it/s] Loading 0: 96%|█████████▌| 954/995 [01:18<00:02, 16.15it/s] Loading 0: 96%|█████████▋| 960/995 [01:18<00:01, 20.06it/s] Loading 0: 97%|█████████▋| 970/995 [01:18<00:00, 29.27it/s] Loading 0: 98%|█████████▊| 976/995 [01:18<00:00, 32.49it/s] Loading 0: 99%|█████████▊| 982/995 [01:18<00:00, 36.09it/s] Loading 0: 99%|█████████▉| 988/995 [01:18<00:00, 39.49it/s] Loading 0: 100%|██████████| 995/995 [01:20<00:00, 12.98it/s] /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:957: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
neversleep-noromaid-v0-8068-v146-mkmlizer: warnings.warn(
neversleep-noromaid-v0-8068-v146-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:785: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
neversleep-noromaid-v0-8068-v146-mkmlizer: warnings.warn(
neversleep-noromaid-v0-8068-v146-mkmlizer: Downloading shards: 0%| | 0/2 [00:00<?, ?it/s] Downloading shards: 50%|█████ | 1/2 [00:05<00:05, 5.81s/it] Downloading shards: 100%|██████████| 2/2 [00:08<00:00, 4.10s/it] Downloading shards: 100%|██████████| 2/2 [00:08<00:00, 4.36s/it]
neversleep-noromaid-v0-8068-v146-mkmlizer: Saving duration: 1.344s
neversleep-noromaid-v0-8068-v146-mkmlizer: Processed model ChaiML/gpt2_xl_pairwise_89m_step_347634 in 13.761s
neversleep-noromaid-v0-8068-v146-mkmlizer: creating bucket guanaco-reward-models
neversleep-noromaid-v0-8068-v146-mkmlizer: Bucket 's3://guanaco-reward-models/' created
neversleep-noromaid-v0-8068-v146-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v146_reward
neversleep-noromaid-v0-8068-v146-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v146_reward/config.json
neversleep-noromaid-v0-8068-v146-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v146_reward/special_tokens_map.json
neversleep-noromaid-v0-8068-v146-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v146_reward/tokenizer_config.json
neversleep-noromaid-v0-8068-v146-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v146_reward/merges.txt
neversleep-noromaid-v0-8068-v146-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v146_reward/vocab.json
neversleep-noromaid-v0-8068-v146-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v146_reward/tokenizer.json
neversleep-noromaid-v0-8068-v146-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/neversleep-noromaid-v0-8068-v146_reward/reward.tensors
Job neversleep-noromaid-v0-8068-v146-mkmlizer completed after 290.24s with status: succeeded
Stopping job with name neversleep-noromaid-v0-8068-v146-mkmlizer
Pipeline stage MKMLizer completed in 291.13s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.10s
Running pipeline stage ISVCDeployer
Creating inference service neversleep-noromaid-v0-8068-v146
Waiting for inference service neversleep-noromaid-v0-8068-v146 to be ready
Failed to get response for submission mistralai-mixtral-8x7b_3473_v114: ('http://mistralai-mixtral-8x7b-3473-v114-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:53138->127.0.0.1:8080: read: connection reset by peer\n')
Inference service neversleep-noromaid-v0-8068-v146 ready after 181.4355664253235s
Pipeline stage ISVCDeployer completed in 183.45s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.2639172077178955s
Received healthy response to inference request in 1.7923123836517334s
Received healthy response to inference request in 1.571730136871338s
Received healthy response to inference request in 2.0935218334198s
Received healthy response to inference request in 1.1788990497589111s
5 requests
0 failed requests
5th percentile: 1.2574652671813964
10th percentile: 1.3360314846038819
20th percentile: 1.4931639194488526
30th percentile: 1.615846586227417
40th percentile: 1.7040794849395753
50th percentile: 1.7923123836517334
60th percentile: 1.9127961635589599
70th percentile: 2.0332799434661863
80th percentile: 2.327600908279419
90th percentile: 2.7957590579986573
95th percentile: 3.029838132858276
99th percentile: 3.2171013927459717
mean time: 1.9800761222839356
Pipeline stage StressChecker completed in 10.70s
neversleep-noromaid-v0_8068_v146 status is now deployed due to DeploymentManager action
neversleep-noromaid-v0_8068_v146 status is now inactive due to auto deactivation removed underperforming models
neversleep-noromaid-v0_8068_v146 status is now torndown due to DeploymentManager action

Usage Metrics

Latency Metrics