trace2333-fd-llama3-v2-f_2628

developer_uid: Trace2333
submission_id: trace2333-fd-llama3-v2-f_2628_v2
model_name: trace2333-fd-llama3-v2-f_2628_v2
model_group: Trace2333/fd_llama3_v2_f
status: torndown
timestamp: 2024-08-13T06:17:10+00:00
num_battles: 7131
num_wins: 3456
celo_rating: 1200.46
family_friendly_score: 0.0
submission_type: basic
model_repo: Trace2333/fd_llama3_v2_fdall_r32a16_5w
model_architecture: LlamaForCausalLM
reward_repo: Jellywibble/gpt2_xl_pairwise_89m_step_347634
model_num_parameters: 8030261248.0
best_of: 16
max_input_tokens: 512
max_output_tokens: 64
display_name: trace2333-fd-llama3-v2-f_2628_v2
is_internal_developer: False
language_model: Trace2333/fd_llama3_v2_fdall_r32a16_5w
model_size: 8B
ranking_group: single
us_pacific_date: 2024-08-12
win_ratio: 0.48464450988641145
generation_params: {'temperature': 1.15, 'top_p': 1.0, 'min_p': 0.06, 'top_k': 250, 'presence_penalty': 0.0, 'frequency_penalty': 0.1, 'stopping_words': ['\n', '<|eot_id|>'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64, 'reward_max_token_input': 256}
formatter: {'memory_template': "<|start_header_id|>system<|end_header_id|>\n\n{bot_name}'s Persona: {memory}\n\n", 'prompt_template': '{prompt}<|eot_id|>', 'bot_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}: {message}<|eot_id|>', 'user_template': '<|start_header_id|>user<|end_header_id|>\n\nYou: {message}<|eot_id|>', 'response_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}:', 'truncate_by_message': False}
reward_formatter: {'bot_template': '{bot_name}: {message}\n', 'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}\n'}
Resubmit model
Running pipeline stage MKMLizer
Starting job with name trace2333-fd-llama3-v2-f-2628-v2-mkmlizer
Waiting for job on trace2333-fd-llama3-v2-f-2628-v2-mkmlizer to finish
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ║     _____            __           __                                ║
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ║       /___/                                                         ║
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ║                                                                     ║
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ║  Version: 0.9.9                                                     ║
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ║  https://mk1.ai                                                     ║
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ║                                                                     ║
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ║  belonging to:                                                      ║
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ║                                                                     ║
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ║  Chai Research Corp.                                                ║
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ║                                                                     ║
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'dial tcp 127.0.0.1:8080: connect: connection refused\n')
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: Downloaded to shared memory in 38.736s
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpgg4f4b_9, device:0
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:49716->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:34730->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: quantized model in 28.347s
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: Processed model Trace2333/fd_llama3_v2_fdall_r32a16_5w in 67.083s
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: creating bucket guanaco-mkml-models
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/trace2333-fd-llama3-v2-f-2628-v2
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/trace2333-fd-llama3-v2-f-2628-v2/config.json
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/trace2333-fd-llama3-v2-f-2628-v2/tokenizer_config.json
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/trace2333-fd-llama3-v2-f-2628-v2/special_tokens_map.json
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/trace2333-fd-llama3-v2-f-2628-v2/tokenizer.json
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/trace2333-fd-llama3-v2-f-2628-v2/flywheel_model.0.safetensors
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: loading reward model from Jellywibble/gpt2_xl_pairwise_89m_step_347634
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 5/291 [00:00<00:10, 27.24it/s]
Loading 0:   4%|▍         | 12/291 [00:00<00:07, 35.80it/s]
Loading 0:   5%|▌         | 16/291 [00:00<00:08, 33.23it/s]
Loading 0:   7%|▋         | 21/291 [00:00<00:07, 36.37it/s]
Loading 0:   9%|▊         | 25/291 [00:00<00:07, 34.75it/s]
Loading 0:  10%|█         | 30/291 [00:00<00:06, 37.91it/s]
Loading 0:  12%|█▏        | 34/291 [00:01<00:09, 26.10it/s]
Loading 0:  13%|█▎        | 38/291 [00:01<00:09, 27.85it/s]
Loading 0:  14%|█▍        | 42/291 [00:01<00:09, 26.91it/s]
Loading 0:  16%|█▋        | 48/291 [00:01<00:07, 32.93it/s]
Loading 0:  18%|█▊        | 52/291 [00:01<00:07, 32.52it/s]
Loading 0:  20%|█▉        | 57/291 [00:01<00:06, 35.72it/s]
Loading 0:  21%|██        | 61/291 [00:01<00:06, 34.59it/s]
Loading 0:  23%|██▎       | 66/291 [00:01<00:06, 37.50it/s]
Loading 0:  24%|██▍       | 70/291 [00:02<00:06, 35.72it/s]
Loading 0:  25%|██▌       | 74/291 [00:02<00:06, 35.25it/s]
Loading 0:  27%|██▋       | 78/291 [00:02<00:06, 35.38it/s]
Loading 0:  28%|██▊       | 82/291 [00:02<00:08, 24.59it/s]
Loading 0:  29%|██▉       | 85/291 [00:02<00:08, 25.64it/s]
Loading 0:  31%|███       | 90/291 [00:02<00:06, 30.28it/s]
Loading 0:  32%|███▏      | 94/291 [00:02<00:06, 30.08it/s]
Loading 0:  34%|███▍      | 99/291 [00:03<00:05, 33.86it/s]
Loading 0:  35%|███▌      | 103/291 [00:03<00:05, 33.53it/s]
Loading 0:  37%|███▋      | 108/291 [00:03<00:04, 36.70it/s]
Loading 0:  38%|███▊      | 112/291 [00:03<00:05, 34.59it/s]
Loading 0:  40%|███▉      | 116/291 [00:03<00:04, 35.04it/s]
Loading 0:  42%|████▏     | 122/291 [00:03<00:04, 39.84it/s]
Loading 0:  44%|████▎     | 127/291 [00:03<00:04, 37.24it/s]
Loading 0:  46%|████▌     | 133/291 [00:04<00:05, 31.55it/s]
Loading 0:  47%|████▋     | 137/291 [00:04<00:04, 32.05it/s]
Loading 0:  48%|████▊     | 141/291 [00:04<00:05, 29.77it/s]
Loading 0:  51%|█████     | 147/291 [00:04<00:04, 34.62it/s]
Loading 0:  52%|█████▏    | 151/291 [00:04<00:04, 32.95it/s]
Loading 0:  54%|█████▎    | 156/291 [00:04<00:03, 35.50it/s]
Loading 0:  55%|█████▍    | 160/291 [00:04<00:03, 34.31it/s]
Loading 0:  57%|█████▋    | 165/291 [00:04<00:03, 36.87it/s]
Loading 0:  58%|█████▊    | 169/291 [00:05<00:03, 35.33it/s]
Loading 0:  60%|█████▉    | 174/291 [00:05<00:03, 37.54it/s]
Loading 0:  61%|██████    | 178/291 [00:05<00:03, 35.87it/s]
Loading 0:  63%|██████▎   | 184/291 [00:05<00:02, 41.98it/s]
Loading 0:  65%|██████▍   | 189/291 [00:05<00:04, 24.93it/s]
Loading 0:  67%|██████▋   | 194/291 [00:05<00:03, 26.37it/s]
Loading 0:  69%|██████▉   | 201/291 [00:06<00:02, 32.48it/s]
Loading 0:  70%|███████   | 205/291 [00:06<00:02, 32.16it/s]
Loading 0:  72%|███████▏  | 210/291 [00:06<00:02, 34.85it/s]
Loading 0:  74%|███████▎  | 214/291 [00:06<00:02, 32.82it/s]
Loading 0:  75%|███████▌  | 219/291 [00:06<00:02, 35.57it/s]
Loading 0:  77%|███████▋  | 223/291 [00:06<00:01, 34.40it/s]
Loading 0:  78%|███████▊  | 227/291 [00:06<00:01, 34.51it/s]
Loading 0:  79%|███████▉  | 231/291 [00:06<00:01, 34.64it/s]
Loading 0:  81%|████████  | 235/291 [00:07<00:02, 25.05it/s]
Loading 0:  82%|████████▏ | 239/291 [00:07<00:02, 25.35it/s]
Loading 0:  85%|████████▍ | 246/291 [00:07<00:01, 33.13it/s]
Loading 0:  86%|████████▌ | 250/291 [00:07<00:01, 32.84it/s]
Loading 0:  88%|████████▊ | 255/291 [00:07<00:01, 35.80it/s]
Loading 0:  89%|████████▉ | 259/291 [00:07<00:00, 34.44it/s]
Loading 0:  91%|█████████ | 264/291 [00:08<00:00, 37.14it/s]
Loading 0:  92%|█████████▏| 268/291 [00:08<00:00, 35.57it/s]
Loading 0:  94%|█████████▍| 273/291 [00:08<00:00, 38.26it/s]
Loading 0:  95%|█████████▌| 277/291 [00:08<00:00, 35.69it/s]
Loading 0:  97%|█████████▋| 281/291 [00:08<00:00, 35.33it/s]
Loading 0:  98%|█████████▊| 286/291 [00:13<00:01,  2.61it/s]
Loading 0:  99%|█████████▉| 289/291 [00:14<00:00,  3.25it/s]
                                                            
/opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:957: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer:   warnings.warn(
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer:   warnings.warn(
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: Saving duration: 1.355s
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: Processed model Jellywibble/gpt2_xl_pairwise_89m_step_347634 in 10.870s
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: creating bucket guanaco-reward-models
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: Bucket 's3://guanaco-reward-models/' created
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/trace2333-fd-llama3-v2-f-2628-v2_reward
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/trace2333-fd-llama3-v2-f-2628-v2_reward/config.json
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/trace2333-fd-llama3-v2-f-2628-v2_reward/special_tokens_map.json
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/trace2333-fd-llama3-v2-f-2628-v2_reward/tokenizer_config.json
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/trace2333-fd-llama3-v2-f-2628-v2_reward/merges.txt
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/trace2333-fd-llama3-v2-f-2628-v2_reward/vocab.json
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/trace2333-fd-llama3-v2-f-2628-v2_reward/tokenizer.json
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/trace2333-fd-llama3-v2-f-2628-v2_reward/reward.tensors
Job trace2333-fd-llama3-v2-f-2628-v2-mkmlizer completed after 104.92s with status: succeeded
Stopping job with name trace2333-fd-llama3-v2-f-2628-v2-mkmlizer
Pipeline stage MKMLizer completed in 105.87s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.10s
Running pipeline stage ISVCDeployer
Creating inference service trace2333-fd-llama3-v2-f-2628-v2
Waiting for inference service trace2333-fd-llama3-v2-f-2628-v2 to be ready
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:33866->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:33544->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:32780->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:59800->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:58432->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:34692->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:47888->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:48502->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:48624->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:37374->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:35824->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:37492->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'dial tcp 127.0.0.1:8080: connect: connection refused\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Inference service trace2333-fd-llama3-v2-f-2628-v2 ready after 211.55208468437195s
Pipeline stage ISVCDeployer completed in 213.17s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.3319308757781982s
Received healthy response to inference request in 1.5398287773132324s
Received healthy response to inference request in 1.481647253036499s
Received healthy response to inference request in 1.4613711833953857s
Received healthy response to inference request in 1.422942876815796s
5 requests
0 failed requests
5th percentile: 1.4306285381317139
10th percentile: 1.4383141994476318
20th percentile: 1.4536855220794678
30th percentile: 1.4654263973236084
40th percentile: 1.4735368251800538
50th percentile: 1.481647253036499
60th percentile: 1.5049198627471925
70th percentile: 1.5281924724578857
80th percentile: 1.6982491970062257
90th percentile: 2.0150900363922117
95th percentile: 2.173510456085205
99th percentile: 2.3002467918395997
mean time: 1.6475441932678223
Pipeline stage StressChecker completed in 9.30s
trace2333-fd-llama3-v2-f_2628_v2 status is now deployed due to DeploymentManager action
trace2333-fd-llama3-v2-f_2628_v2 status is now inactive due to auto deactivation removed underperforming models
trace2333-fd-llama3-v2-f_2628_v2 status is now torndown due to DeploymentManager action