Running pipeline stage MKMLizer
Starting job with name mistralai-mistral-nemo-9330-v47-mkmlizer
Waiting for job on mistralai-mistral-nemo-9330-v47-mkmlizer to finish
mistralai-mistral-nemo-9330-v47-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mistralai-mistral-nemo-9330-v47-mkmlizer: ║ _____ __ __ ║
mistralai-mistral-nemo-9330-v47-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
mistralai-mistral-nemo-9330-v47-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
mistralai-mistral-nemo-9330-v47-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
mistralai-mistral-nemo-9330-v47-mkmlizer: ║ /___/ ║
mistralai-mistral-nemo-9330-v47-mkmlizer: ║ ║
mistralai-mistral-nemo-9330-v47-mkmlizer: ║ Version: 0.9.9 ║
mistralai-mistral-nemo-9330-v47-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
mistralai-mistral-nemo-9330-v47-mkmlizer: ║ https://mk1.ai ║
mistralai-mistral-nemo-9330-v47-mkmlizer: ║ ║
mistralai-mistral-nemo-9330-v47-mkmlizer: ║ The license key for the current software has been verified as ║
mistralai-mistral-nemo-9330-v47-mkmlizer: ║ belonging to: ║
mistralai-mistral-nemo-9330-v47-mkmlizer: ║ ║
mistralai-mistral-nemo-9330-v47-mkmlizer: ║ Chai Research Corp. ║
mistralai-mistral-nemo-9330-v47-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
mistralai-mistral-nemo-9330-v47-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
mistralai-mistral-nemo-9330-v47-mkmlizer: ║ ║
mistralai-mistral-nemo-9330-v47-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Inference service bbchicago-brt-v1-12-narr-4893-v2 ready after 300.9702067375183s
Pipeline stage ISVCDeployer completed in 303.14s
Running pipeline stage StressChecker
mistralai-mistral-nemo-9330-v47-mkmlizer: Downloaded to shared memory in 53.486s
mistralai-mistral-nemo-9330-v47-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpcruv1veu, device:0
mistralai-mistral-nemo-9330-v47-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Received healthy response to inference request in 2.0866687297821045s
Received healthy response to inference request in 1.1396889686584473s
Received healthy response to inference request in 1.1749725341796875s
Received healthy response to inference request in 1.142134666442871s
Received healthy response to inference request in 1.1460075378417969s
5 requests
0 failed requests
5th percentile: 1.140178108215332
10th percentile: 1.1406672477722168
20th percentile: 1.1416455268859864
30th percentile: 1.1429092407226562
40th percentile: 1.1444583892822267
50th percentile: 1.1460075378417969
60th percentile: 1.1575935363769532
70th percentile: 1.1691795349121095
80th percentile: 1.3573117733001712
90th percentile: 1.7219902515411378
95th percentile: 1.9043294906616208
99th percentile: 2.0502008819580078
mean time: 1.3378944873809815
Pipeline stage StressChecker completed in 8.10s
bbchicago-brt-v1-12-narr_4893_v2 status is now deployed due to DeploymentManager action
mistralai-mistral-nemo-9330-v47-mkmlizer: quantized model in 36.241s
mistralai-mistral-nemo-9330-v47-mkmlizer: Processed model mistralai/Mistral-Nemo-Instruct-2407 in 89.728s
mistralai-mistral-nemo-9330-v47-mkmlizer: creating bucket guanaco-mkml-models
mistralai-mistral-nemo-9330-v47-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
mistralai-mistral-nemo-9330-v47-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v47
mistralai-mistral-nemo-9330-v47-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v47/config.json
mistralai-mistral-nemo-9330-v47-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v47/special_tokens_map.json
mistralai-mistral-nemo-9330-v47-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v47/tokenizer_config.json
mistralai-mistral-nemo-9330-v47-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v47/tokenizer.json
mistralai-mistral-nemo-9330-v47-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v47/flywheel_model.0.safetensors
mistralai-mistral-nemo-9330-v47-mkmlizer: loading reward model from ChaiML/gpt2_xl_pairwise_89m_step_347634
mistralai-mistral-nemo-9330-v47-mkmlizer:
Loading 0: 0%| | 0/363 [00:00<?, ?it/s]
Loading 0: 1%|▏ | 5/363 [00:00<00:10, 33.07it/s]
Loading 0: 4%|▎ | 13/363 [00:00<00:06, 51.95it/s]
Loading 0: 5%|▌ | 19/363 [00:00<00:07, 45.84it/s]
Loading 0: 7%|▋ | 24/363 [00:00<00:07, 43.60it/s]
Loading 0: 9%|▊ | 31/363 [00:00<00:06, 49.13it/s]
Loading 0: 10%|█ | 37/363 [00:00<00:07, 44.15it/s]
Loading 0: 12%|█▏ | 42/363 [00:00<00:07, 43.05it/s]
Loading 0: 13%|█▎ | 49/363 [00:01<00:06, 48.16it/s]
Loading 0: 15%|█▌ | 55/363 [00:01<00:06, 45.47it/s]
Loading 0: 17%|█▋ | 61/363 [00:01<00:08, 34.91it/s]
Loading 0: 18%|█▊ | 65/363 [00:01<00:08, 34.31it/s]
Loading 0: 20%|█▉ | 72/363 [00:01<00:07, 40.74it/s]
Loading 0: 21%|██▏ | 78/363 [00:01<00:07, 40.41it/s]
Loading 0: 23%|██▎ | 83/363 [00:01<00:06, 40.55it/s]
Loading 0: 25%|██▍ | 90/363 [00:02<00:05, 45.56it/s]
Loading 0: 26%|██▋ | 96/363 [00:02<00:06, 43.32it/s]
Loading 0: 28%|██▊ | 101/363 [00:02<00:06, 42.32it/s]
Loading 0: 29%|██▉ | 106/363 [00:02<00:05, 44.11it/s]
Loading 0: 31%|███ | 112/363 [00:02<00:05, 46.92it/s]
Loading 0: 32%|███▏ | 117/363 [00:02<00:05, 44.97it/s]
Loading 0: 34%|███▍ | 123/363 [00:02<00:05, 42.89it/s]
Loading 0: 35%|███▌ | 128/363 [00:02<00:05, 42.03it/s]
Loading 0: 37%|███▋ | 134/363 [00:03<00:04, 46.21it/s]
Loading 0: 38%|███▊ | 139/363 [00:03<00:05, 44.72it/s]
Loading 0: 40%|███▉ | 144/363 [00:03<00:07, 28.34it/s]
Loading 0: 41%|████ | 149/363 [00:03<00:07, 30.50it/s]
Loading 0: 43%|████▎ | 156/363 [00:03<00:05, 37.72it/s]
Loading 0: 44%|████▍ | 161/363 [00:03<00:05, 39.52it/s]
Loading 0: 46%|████▌ | 166/363 [00:04<00:04, 41.09it/s]
Loading 0: 47%|████▋ | 172/363 [00:04<00:04, 39.56it/s]
Loading 0: 49%|████▉ | 177/363 [00:04<00:04, 39.00it/s]
Loading 0: 50%|█████ | 183/363 [00:04<00:04, 43.72it/s]
Loading 0: 52%|█████▏ | 188/363 [00:04<00:03, 44.09it/s]
Loading 0: 53%|█████▎ | 193/363 [00:04<00:03, 43.44it/s]
Loading 0: 55%|█████▍ | 199/363 [00:04<00:03, 42.33it/s]
Loading 0: 56%|█████▌ | 204/363 [00:04<00:03, 41.80it/s]
Loading 0: 58%|█████▊ | 210/363 [00:05<00:03, 45.44it/s]
Loading 0: 59%|█████▉ | 215/363 [00:05<00:03, 45.17it/s]
Loading 0: 61%|██████ | 221/363 [00:05<00:02, 48.76it/s]
Loading 0: 62%|██████▏ | 226/363 [00:05<00:04, 29.13it/s]
Loading 0: 63%|██████▎ | 230/363 [00:05<00:04, 29.82it/s]
Loading 0: 65%|██████▌ | 237/363 [00:05<00:03, 37.61it/s]
Loading 0: 67%|██████▋ | 242/363 [00:05<00:03, 38.99it/s]
Loading 0: 68%|██████▊ | 247/363 [00:06<00:02, 39.64it/s]
Loading 0: 70%|██████▉ | 253/363 [00:06<00:02, 38.94it/s]
Loading 0: 71%|███████ | 258/363 [00:06<00:02, 39.03it/s]
Loading 0: 73%|███████▎ | 264/363 [00:06<00:02, 42.97it/s]
Loading 0: 74%|███████▍ | 269/363 [00:06<00:02, 42.61it/s]
Loading 0: 75%|███████▌ | 274/363 [00:06<00:02, 42.55it/s]
Loading 0: 77%|███████▋ | 279/363 [00:06<00:01, 44.39it/s]
Loading 0: 78%|███████▊ | 284/363 [00:06<00:02, 37.76it/s]
Loading 0: 80%|████████ | 291/363 [00:07<00:01, 45.12it/s]
Loading 0: 82%|████████▏ | 296/363 [00:07<00:01, 44.74it/s]
Loading 0: 83%|████████▎ | 302/363 [00:07<00:01, 48.00it/s]
Loading 0: 85%|████████▍ | 307/363 [00:14<00:21, 2.55it/s]
Loading 0: 86%|████████▌ | 312/363 [00:14<00:14, 3.46it/s]
Loading 0: 88%|████████▊ | 320/363 [00:14<00:07, 5.52it/s]
Loading 0: 90%|████████▉ | 326/363 [00:14<00:04, 7.42it/s]
Loading 0: 91%|█████████ | 331/363 [00:14<00:03, 9.44it/s]
Loading 0: 93%|█████████▎| 337/363 [00:14<00:02, 12.74it/s]
Loading 0: 94%|█████████▍| 342/363 [00:14<00:01, 15.86it/s]
Loading 0: 96%|█████████▌| 347/363 [00:14<00:00, 19.43it/s]
Loading 0: 97%|█████████▋| 353/363 [00:15<00:00, 23.44it/s]
Loading 0: 99%|█████████▊| 358/363 [00:15<00:00, 26.61it/s]
/opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:957: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mistralai-mistral-nemo-9330-v47-mkmlizer: warnings.warn(
mistralai-mistral-nemo-9330-v47-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:785: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mistralai-mistral-nemo-9330-v47-mkmlizer: warnings.warn(
mistralai-mistral-nemo-9330-v47-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mistralai-mistral-nemo-9330-v47-mkmlizer: warnings.warn(
mistralai-mistral-nemo-9330-v47-mkmlizer:
Downloading shards: 0%| | 0/2 [00:00<?, ?it/s]
Downloading shards: 50%|█████ | 1/2 [00:07<00:07, 7.03s/it]
Downloading shards: 100%|██████████| 2/2 [00:10<00:00, 4.67s/it]
Downloading shards: 100%|██████████| 2/2 [00:10<00:00, 5.03s/it]
mistralai-mistral-nemo-9330-v47-mkmlizer:
Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards: 50%|█████ | 1/2 [00:00<00:00, 2.36it/s]
Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 3.95it/s]
Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 3.58it/s]
mistralai-mistral-nemo-9330-v47-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
mistralai-mistral-nemo-9330-v47-mkmlizer: Saving duration: 1.350s
mistralai-mistral-nemo-9330-v47-mkmlizer: Processed model ChaiML/gpt2_xl_pairwise_89m_step_347634 in 15.030s
mistralai-mistral-nemo-9330-v47-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/mistralai-mistral-nemo-9330-v47_reward/config.json
mistralai-mistral-nemo-9330-v47-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/mistralai-mistral-nemo-9330-v47_reward/special_tokens_map.json
mistralai-mistral-nemo-9330-v47-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/mistralai-mistral-nemo-9330-v47_reward/tokenizer_config.json
mistralai-mistral-nemo-9330-v47-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/mistralai-mistral-nemo-9330-v47_reward/merges.txt
mistralai-mistral-nemo-9330-v47-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/mistralai-mistral-nemo-9330-v47_reward/vocab.json
mistralai-mistral-nemo-9330-v47-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/mistralai-mistral-nemo-9330-v47_reward/tokenizer.json
mistralai-mistral-nemo-9330-v47-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/mistralai-mistral-nemo-9330-v47_reward/reward.tensors
Job mistralai-mistral-nemo-9330-v47-mkmlizer completed after 135.47s with status: succeeded
Stopping job with name mistralai-mistral-nemo-9330-v47-mkmlizer
Pipeline stage MKMLizer completed in 137.18s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.10s
Running pipeline stage ISVCDeployer
Creating inference service mistralai-mistral-nemo-9330-v47
Waiting for inference service mistralai-mistral-nemo-9330-v47 to be ready
Inference service mistralai-mistral-nemo-9330-v47 ready after 323.38906049728394s
Pipeline stage ISVCDeployer completed in 325.47s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.6750988960266113s
Received healthy response to inference request in 1.861332654953003s
Received healthy response to inference request in 1.907867193222046s
Received healthy response to inference request in 1.9211676120758057s
Received healthy response to inference request in 1.9552550315856934s
5 requests
0 failed requests
5th percentile: 1.8706395626068115
10th percentile: 1.8799464702606201
20th percentile: 1.8985602855682373
30th percentile: 1.910527276992798
40th percentile: 1.9158474445343017
50th percentile: 1.9211676120758057
60th percentile: 1.9348025798797608
70th percentile: 1.948437547683716
80th percentile: 2.099223804473877
90th percentile: 2.3871613502502442
95th percentile: 2.5311301231384276
99th percentile: 2.6463051414489747
mean time: 2.0641442775726317
Pipeline stage StressChecker completed in 11.25s
mistralai-mistral-nemo-_9330_v47 status is now deployed due to DeploymentManager action
mistralai-mistral-nemo-_9330_v47 status is now inactive due to auto deactivation removed underperforming models
mistralai-mistral-nemo-_9330_v47 status is now torndown due to DeploymentManager action