cycy233-l3-ss-e-v0-c2

developer_uid: shiroe40
submission_id: cycy233-l3-ss-e-v0-c2_v1
model_name: auto
model_group: cycy233/L3-ss-e-v0-c2
status: torndown
timestamp: 2024-08-13T06:46:21+00:00
num_battles: 7330
num_wins: 3223
celo_rating: 1169.67
family_friendly_score: 0.0
submission_type: basic
model_repo: cycy233/L3-ss-e-v0-c2
model_architecture: LlamaForCausalLM
reward_repo: Jellywibble/gpt2_xl_pairwise_89m_step_347634
model_num_parameters: 8030261248.0
best_of: 16
max_input_tokens: 512
max_output_tokens: 64
display_name: auto
is_internal_developer: False
language_model: cycy233/L3-ss-e-v0-c2
model_size: 8B
ranking_group: single
us_pacific_date: 2024-08-12
win_ratio: 0.439699863574352
generation_params: {'temperature': 1.0, 'top_p': 0.9, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['<|end_header_id|>', '<|eot_id|>'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64, 'reward_max_token_input': 256}
formatter: {'memory_template': "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n{bot_name}'s Persona: {memory}\n\n", 'prompt_template': '{prompt}<|eot_id|>', 'bot_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}: {message}<|eot_id|>', 'user_template': '<|start_header_id|>user<|end_header_id|>\n\n{user_name}: {message}<|eot_id|>', 'response_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}:', 'truncate_by_message': False}
reward_formatter: {'bot_template': '{bot_name}: {message}\n', 'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}\n'}
Resubmit model
Running pipeline stage MKMLizer
Starting job with name cycy233-l3-ss-e-v0-c2-v1-mkmlizer
Waiting for job on cycy233-l3-ss-e-v0-c2-v1-mkmlizer to finish
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:57166->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'dial tcp 127.0.0.1:8080: connect: connection refused\n')
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: ║     _____            __           __                                ║
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: ║       /___/                                                         ║
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: ║                                                                     ║
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: ║  Version: 0.9.9                                                     ║
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: ║  https://mk1.ai                                                     ║
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: ║                                                                     ║
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: ║  belonging to:                                                      ║
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: ║                                                                     ║
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: ║  Chai Research Corp.                                                ║
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: ║                                                                     ║
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: Downloaded to shared memory in 30.511s
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp58pyzwbr, device:0
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:35774->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:35676->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:36062->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'dial tcp 127.0.0.1:8080: connect: connection refused\n')
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: quantized model in 27.233s
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: Processed model cycy233/L3-ss-e-v0-c2 in 57.744s
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: creating bucket guanaco-mkml-models
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/cycy233-l3-ss-e-v0-c2-v1
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/cycy233-l3-ss-e-v0-c2-v1/config.json
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/cycy233-l3-ss-e-v0-c2-v1/special_tokens_map.json
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/cycy233-l3-ss-e-v0-c2-v1/tokenizer_config.json
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/cycy233-l3-ss-e-v0-c2-v1/tokenizer.json
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/cycy233-l3-ss-e-v0-c2-v1/flywheel_model.0.safetensors
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: loading reward model from Jellywibble/gpt2_xl_pairwise_89m_step_347634
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 5/291 [00:00<00:08, 35.46it/s]
Loading 0:   4%|▍         | 13/291 [00:00<00:05, 54.65it/s]
Loading 0:   7%|▋         | 19/291 [00:00<00:05, 46.45it/s]
Loading 0:   8%|▊         | 24/291 [00:00<00:05, 45.06it/s]
Loading 0:  10%|█         | 30/291 [00:00<00:05, 48.85it/s]
Loading 0:  12%|█▏        | 36/291 [00:00<00:05, 46.98it/s]
Loading 0:  14%|█▍        | 41/291 [00:00<00:06, 38.52it/s]
Loading 0:  17%|█▋        | 49/291 [00:01<00:05, 46.95it/s]
Loading 0:  19%|█▉        | 55/291 [00:01<00:05, 44.07it/s]
Loading 0:  21%|██        | 60/291 [00:01<00:05, 42.59it/s]
Loading 0:  23%|██▎       | 67/291 [00:01<00:04, 47.22it/s]
Loading 0:  25%|██▍       | 72/291 [00:01<00:04, 47.75it/s]
Loading 0:  26%|██▋       | 77/291 [00:01<00:05, 39.56it/s]
Loading 0:  29%|██▊       | 83/291 [00:02<00:06, 30.82it/s]
Loading 0:  30%|██▉       | 87/291 [00:02<00:06, 31.07it/s]
Loading 0:  32%|███▏      | 94/291 [00:02<00:05, 38.18it/s]
Loading 0:  34%|███▍      | 100/291 [00:02<00:04, 38.50it/s]
Loading 0:  36%|███▌      | 105/291 [00:02<00:04, 39.88it/s]
Loading 0:  38%|███▊      | 112/291 [00:02<00:03, 45.30it/s]
Loading 0:  41%|████      | 118/291 [00:02<00:04, 43.09it/s]
Loading 0:  42%|████▏     | 123/291 [00:02<00:03, 43.12it/s]
Loading 0:  44%|████▍     | 129/291 [00:03<00:03, 47.06it/s]
Loading 0:  46%|████▌     | 134/291 [00:03<00:03, 45.52it/s]
Loading 0:  48%|████▊     | 139/291 [00:03<00:03, 45.32it/s]
Loading 0:  50%|████▉     | 145/291 [00:03<00:03, 43.63it/s]
Loading 0:  52%|█████▏    | 150/291 [00:03<00:03, 43.67it/s]
Loading 0:  54%|█████▍    | 157/291 [00:03<00:02, 49.62it/s]
Loading 0:  56%|█████▌    | 163/291 [00:03<00:02, 46.78it/s]
Loading 0:  58%|█████▊    | 168/291 [00:03<00:02, 45.67it/s]
Loading 0:  60%|██████    | 175/291 [00:03<00:02, 51.47it/s]
Loading 0:  62%|██████▏   | 181/291 [00:04<00:02, 43.57it/s]
Loading 0:  64%|██████▍   | 187/291 [00:04<00:03, 34.23it/s]
Loading 0:  66%|██████▌   | 191/291 [00:04<00:02, 35.27it/s]
Loading 0:  67%|██████▋   | 195/291 [00:04<00:02, 34.91it/s]
Loading 0:  69%|██████▉   | 201/291 [00:04<00:02, 39.77it/s]
Loading 0:  71%|███████   | 206/291 [00:04<00:02, 41.85it/s]
Loading 0:  73%|███████▎  | 211/291 [00:04<00:01, 43.51it/s]
Loading 0:  75%|███████▍  | 217/291 [00:05<00:01, 41.80it/s]
Loading 0:  76%|███████▋  | 222/291 [00:05<00:01, 41.61it/s]
Loading 0:  79%|███████▊  | 229/291 [00:05<00:01, 47.31it/s]
Loading 0:  81%|████████  | 235/291 [00:05<00:01, 44.41it/s]
Loading 0:  82%|████████▏ | 240/291 [00:05<00:01, 43.83it/s]
Loading 0:  85%|████████▍ | 247/291 [00:05<00:00, 48.86it/s]
Loading 0:  87%|████████▋ | 252/291 [00:05<00:00, 49.07it/s]
Loading 0:  88%|████████▊ | 257/291 [00:06<00:00, 40.66it/s]
Loading 0:  91%|█████████ | 265/291 [00:06<00:00, 48.27it/s]
Loading 0:  93%|█████████▎| 271/291 [00:06<00:00, 45.49it/s]
Loading 0:  95%|█████████▍| 276/291 [00:06<00:00, 44.79it/s]
Loading 0:  97%|█████████▋| 281/291 [00:06<00:00, 46.02it/s]
Loading 0:  98%|█████████▊| 286/291 [00:06<00:00, 41.48it/s]
Loading 0: 100%|██████████| 291/291 [00:12<00:00,  3.05it/s]
                                                            
/opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:957: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
cycy233-l3-ss-e-v0-c2-v1-mkmlizer:   warnings.warn(
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:785: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
cycy233-l3-ss-e-v0-c2-v1-mkmlizer:   warnings.warn(
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
cycy233-l3-ss-e-v0-c2-v1-mkmlizer:   warnings.warn(
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: Saving duration: 1.518s
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: Processed model Jellywibble/gpt2_xl_pairwise_89m_step_347634 in 10.985s
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: creating bucket guanaco-reward-models
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: Bucket 's3://guanaco-reward-models/' created
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/cycy233-l3-ss-e-v0-c2-v1_reward
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/cycy233-l3-ss-e-v0-c2-v1_reward/config.json
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/cycy233-l3-ss-e-v0-c2-v1_reward/special_tokens_map.json
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/cycy233-l3-ss-e-v0-c2-v1_reward/tokenizer_config.json
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/cycy233-l3-ss-e-v0-c2-v1_reward/merges.txt
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/cycy233-l3-ss-e-v0-c2-v1_reward/vocab.json
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/cycy233-l3-ss-e-v0-c2-v1_reward/tokenizer.json
cycy233-l3-ss-e-v0-c2-v1-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/cycy233-l3-ss-e-v0-c2-v1_reward/reward.tensors
Job cycy233-l3-ss-e-v0-c2-v1-mkmlizer completed after 94.63s with status: succeeded
Stopping job with name cycy233-l3-ss-e-v0-c2-v1-mkmlizer
Pipeline stage MKMLizer completed in 95.91s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.11s
Running pipeline stage ISVCDeployer
Creating inference service cycy233-l3-ss-e-v0-c2-v1
Waiting for inference service cycy233-l3-ss-e-v0-c2-v1 to be ready
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:52980->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'dial tcp 127.0.0.1:8080: connect: connection refused\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:43146->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:43428->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'dial tcp 127.0.0.1:8080: connect: connection refused\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
Inference service cycy233-l3-ss-e-v0-c2-v1 ready after 211.40604996681213s
Pipeline stage ISVCDeployer completed in 213.70s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.497854232788086s
Received healthy response to inference request in 1.5802278518676758s
Received healthy response to inference request in 1.5711002349853516s
Received healthy response to inference request in 1.7592833042144775s
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Received healthy response to inference request in 1.5466790199279785s
5 requests
0 failed requests
5th percentile: 1.5515632629394531
10th percentile: 1.5564475059509277
20th percentile: 1.566215991973877
30th percentile: 1.5729257583618164
40th percentile: 1.5765768051147462
50th percentile: 1.5802278518676758
60th percentile: 1.6518500328063965
70th percentile: 1.723472213745117
80th percentile: 1.9069974899291993
90th percentile: 2.2024258613586425
95th percentile: 2.350140047073364
99th percentile: 2.4683113956451415
mean time: 1.7910289287567138
Pipeline stage StressChecker completed in 10.20s
cycy233-l3-ss-e-v0-c2_v1 status is now deployed due to DeploymentManager action
cycy233-l3-ss-e-v0-c2_v1 status is now inactive due to auto deactivation removed underperforming models
cycy233-l3-ss-e-v0-c2_v1 status is now torndown due to DeploymentManager action