gryphe-pantheon-rp-1-6-1_6536

developer_uid: Gryphe

submission_id: gryphe-pantheon-rp-1-6-1_6536_v1

model_name: gryphe-pantheon-rp-1-6-1_6536_v1

model_group: Gryphe/Pantheon-RP-1.6-1

status: torndown

timestamp: 2024-08-21T20:09:09+00:00

num_battles: 11303

num_wins: 5618

celo_rating: 1231.51

family_friendly_score: 0.0

submission_type: basic

model_repo: Gryphe/Pantheon-RP-1.6-12b-Nemo

model_architecture: MistralForCausalLM

model_num_parameters: 12772070400.0

best_of: 8

max_input_tokens: 512

max_output_tokens: 64

display_name: gryphe-pantheon-rp-1-6-1_6536_v1

is_internal_developer: False

language_model: Gryphe/Pantheon-RP-1.6-12b-Nemo

model_size: 13B

ranking_group: single

us_pacific_date: 2024-08-21

win_ratio: 0.49703618508360614

generation_params: {'temperature': 0.75, 'top_p': 1.0, 'min_p': 0.025, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 1.05, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': False}

Resubmit model

Running pipeline stage MKMLizer
Starting job with name gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer
Waiting for job on gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer to finish
gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer: ║     _____            __           __                                ║
gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer: ║       /___/                                                         ║
gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer: ║                                                                     ║
gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer: ║  Version: 0.10.1                                                    ║
gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer: ║  https://mk1.ai                                                     ║
gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer: ║                                                                     ║
gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer: ║  belonging to:                                                      ║
gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer: ║                                                                     ║
gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer: ║  Chai Research Corp.                                                ║
gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer: ║                                                                     ║
gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer: Downloaded to shared memory in 54.154s
gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp5fp2mz3a, device:0
gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer: quantized model in 36.118s
gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer: Processed model Gryphe/Pantheon-RP-1.6-12b-Nemo in 90.272s
gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer: creating bucket guanaco-mkml-models
gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/gryphe-pantheon-rp-1-6-1-6536-v1
gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/gryphe-pantheon-rp-1-6-1-6536-v1/config.json
gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/gryphe-pantheon-rp-1-6-1-6536-v1/special_tokens_map.json
gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/gryphe-pantheon-rp-1-6-1-6536-v1/tokenizer_config.json
gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/gryphe-pantheon-rp-1-6-1-6536-v1/tokenizer.json
gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:10, 33.72it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:06, 54.44it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:07, 47.76it/s]
Loading 0:   7%|▋         | 25/363 [00:00<00:06, 48.49it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:06, 50.80it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:06, 47.04it/s]
Loading 0:  12%|█▏        | 42/363 [00:00<00:07, 45.30it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:06, 50.35it/s]
Loading 0:  15%|█▌        | 55/363 [00:01<00:06, 47.22it/s]
Loading 0:  17%|█▋        | 61/363 [00:01<00:08, 35.29it/s]
Loading 0:  18%|█▊        | 66/363 [00:01<00:08, 36.62it/s]
Loading 0:  20%|█▉        | 72/363 [00:01<00:07, 40.52it/s]
Loading 0:  21%|██▏       | 78/363 [00:01<00:07, 40.00it/s]
Loading 0:  23%|██▎       | 83/363 [00:01<00:06, 40.13it/s]
Loading 0:  25%|██▍       | 90/363 [00:02<00:06, 45.26it/s]
Loading 0:  26%|██▌       | 95/363 [00:02<00:05, 46.32it/s]
Loading 0:  28%|██▊       | 100/363 [00:02<00:06, 38.20it/s]
Loading 0:  29%|██▉       | 106/363 [00:02<00:05, 43.14it/s]
Loading 0:  31%|███       | 112/363 [00:02<00:05, 47.09it/s]
Loading 0:  33%|███▎      | 118/363 [00:02<00:06, 39.24it/s]
Loading 0:  34%|███▍      | 125/363 [00:02<00:05, 46.04it/s]
Loading 0:  36%|███▌      | 131/363 [00:02<00:04, 46.94it/s]
Loading 0:  38%|███▊      | 137/363 [00:03<00:05, 40.96it/s]
Loading 0:  39%|███▉      | 142/363 [00:03<00:07, 31.51it/s]
Loading 0:  40%|████      | 146/363 [00:03<00:06, 33.02it/s]
Loading 0:  41%|████▏     | 150/363 [00:03<00:06, 32.23it/s]
Loading 0:  43%|████▎     | 156/363 [00:03<00:05, 37.89it/s]
Loading 0:  44%|████▍     | 161/363 [00:03<00:05, 39.78it/s]
Loading 0:  46%|████▌     | 166/363 [00:03<00:04, 41.26it/s]
Loading 0:  47%|████▋     | 172/363 [00:04<00:04, 39.41it/s]
Loading 0:  49%|████▉     | 177/363 [00:04<00:04, 38.91it/s]
Loading 0:  50%|█████     | 183/363 [00:04<00:04, 43.48it/s]
Loading 0:  52%|█████▏    | 188/363 [00:04<00:04, 42.94it/s]
Loading 0:  53%|█████▎    | 193/363 [00:04<00:03, 43.44it/s]
Loading 0:  55%|█████▍    | 199/363 [00:04<00:03, 42.44it/s]
Loading 0:  56%|█████▌    | 204/363 [00:04<00:03, 41.48it/s]
Loading 0:  58%|█████▊    | 210/363 [00:05<00:03, 45.36it/s]
Loading 0:  59%|█████▉    | 215/363 [00:05<00:03, 45.30it/s]
Loading 0:  61%|██████    | 221/363 [00:05<00:02, 48.43it/s]
Loading 0:  62%|██████▏   | 226/363 [00:05<00:04, 29.06it/s]
Loading 0:  63%|██████▎   | 230/363 [00:05<00:04, 29.69it/s]
Loading 0:  65%|██████▌   | 237/363 [00:05<00:03, 36.84it/s]
Loading 0:  67%|██████▋   | 242/363 [00:05<00:03, 37.80it/s]
Loading 0:  68%|██████▊   | 247/363 [00:06<00:02, 39.54it/s]
Loading 0:  70%|██████▉   | 253/363 [00:06<00:02, 38.40it/s]
Loading 0:  71%|███████   | 258/363 [00:06<00:02, 37.99it/s]
Loading 0:  73%|███████▎  | 264/363 [00:06<00:02, 42.80it/s]
Loading 0:  74%|███████▍  | 269/363 [00:06<00:02, 43.35it/s]
Loading 0:  75%|███████▌  | 274/363 [00:06<00:02, 43.31it/s]
Loading 0:  77%|███████▋  | 279/363 [00:06<00:01, 44.27it/s]
Loading 0:  78%|███████▊  | 284/363 [00:06<00:02, 37.26it/s]
Loading 0:  80%|████████  | 291/363 [00:07<00:01, 43.96it/s]
Loading 0:  82%|████████▏ | 296/363 [00:07<00:01, 43.73it/s]
Loading 0:  83%|████████▎ | 301/363 [00:07<00:01, 44.83it/s]
Loading 0:  84%|████████▍ | 306/363 [00:14<00:23,  2.45it/s]
Loading 0:  85%|████████▌ | 310/363 [00:14<00:16,  3.17it/s]
Loading 0:  87%|████████▋ | 314/363 [00:14<00:11,  4.17it/s]
Loading 0:  88%|████████▊ | 320/363 [00:14<00:06,  6.22it/s]
Loading 0:  90%|████████▉ | 326/363 [00:14<00:04,  8.71it/s]
Loading 0:  91%|█████████ | 331/363 [00:14<00:02, 11.27it/s]
Loading 0:  93%|█████████▎| 338/363 [00:14<00:01, 16.07it/s]
Loading 0:  95%|█████████▍| 344/363 [00:15<00:00, 19.77it/s]
Loading 0:  96%|█████████▌| 349/363 [00:15<00:00, 23.03it/s]
Loading 0:  98%|█████████▊| 355/363 [00:15<00:00, 28.58it/s]
Loading 0:  99%|█████████▉| 360/363 [00:15<00:00, 31.92it/s]
                                                            
Job gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer completed after 170.43s with status: succeeded
Stopping job with name gryphe-pantheon-rp-1-6-1-6536-v1-mkmlizer
Pipeline stage MKMLizer completed in 171.45s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.09s
Running pipeline stage ISVCDeployer
Creating inference service gryphe-pantheon-rp-1-6-1-6536-v1
Waiting for inference service gryphe-pantheon-rp-1-6-1-6536-v1 to be ready
Failed to get response for submission blend_koran_2024-08-16: ('http://zonemercy-lexical-nemo-1518-v18-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission blend_dedat_2024-08-16: ('http://zonemercy-lexical-nemo-1518-v18-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Failed to get response for submission blend_berib_2024-08-16: ('http://zonemercy-lexical-nemo-1518-v18-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Inference service gryphe-pantheon-rp-1-6-1-6536-v1 ready after 243.0510437488556s
Pipeline stage ISVCDeployer completed in 244.60s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.4040777683258057s
Received healthy response to inference request in 2.2774717807769775s
Received healthy response to inference request in 2.1108784675598145s
Received healthy response to inference request in 1.643404483795166s
Received healthy response to inference request in 1.6996889114379883s
5 requests
0 failed requests
5th percentile: 1.6546613693237304
10th percentile: 1.6659182548522948
20th percentile: 1.6884320259094239
30th percentile: 1.7819268226623535
40th percentile: 1.946402645111084
50th percentile: 2.1108784675598145
60th percentile: 2.1775157928466795
70th percentile: 2.244153118133545
80th percentile: 2.3027929782867433
90th percentile: 2.3534353733062745
95th percentile: 2.37875657081604
99th percentile: 2.3990135288238523
mean time: 2.0271042823791503
Pipeline stage StressChecker completed in 10.84s
gryphe-pantheon-rp-1-6-1_6536_v1 status is now deployed due to DeploymentManager action
gryphe-pantheon-rp-1-6-1_6536_v1 status is now inactive due to auto deactivation removed underperforming models
Running pipeline stage MKMLModelDeleter
gryphe-pantheon-rp-1-6-1_6536_v1 status is now torndown due to DeploymentManager action