submission_id: trace2333-fd-llama3-v2-f_2628_v2
developer_uid: Trace2333
alignment_samples: 7132
alignment_score: 1.6336844890075721
best_of: 16
celo_rating: 1200.51
display_name: trace2333-fd-llama3-v2-f_2628_v2
formatter: {'memory_template': "<|start_header_id|>system<|end_header_id|>\n\n{bot_name}'s Persona: {memory}\n\n", 'prompt_template': '{prompt}<|eot_id|>', 'bot_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}: {message}<|eot_id|>', 'user_template': '<|start_header_id|>user<|end_header_id|>\n\nYou: {message}<|eot_id|>', 'response_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.15, 'top_p': 1.0, 'min_p': 0.06, 'top_k': 250, 'presence_penalty': 0.0, 'frequency_penalty': 0.1, 'stopping_words': ['\n', '<|eot_id|>'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64, 'reward_max_token_input': 256}
is_internal_developer: False
language_model: Trace2333/fd_llama3_v2_fdall_r32a16_5w
max_input_tokens: 512
max_output_tokens: 64
model_architecture: LlamaForCausalLM
model_group: Trace2333/fd_llama3_v2_f
model_name: trace2333-fd-llama3-v2-f_2628_v2
model_num_parameters: 8030261248.0
model_repo: Trace2333/fd_llama3_v2_fdall_r32a16_5w
model_size: 8B
num_battles: 7131
num_wins: 3456
propriety_score: 0.7235079171741778
propriety_total_count: 821.0
ranking_group: single
reward_formatter: {'bot_template': '{bot_name}: {message}\n', 'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}\n'}
reward_repo: Jellywibble/gpt2_xl_pairwise_89m_step_347634
status: torndown
submission_type: basic
timestamp: 2024-08-13T06:17:10+00:00
us_pacific_date: 2024-08-12
win_ratio: 0.48464450988641145
Download Preference Data
Resubmit model
Running pipeline stage MKMLizer
Starting job with name trace2333-fd-llama3-v2-f-2628-v2-mkmlizer
Waiting for job on trace2333-fd-llama3-v2-f-2628-v2-mkmlizer to finish
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ║ _____ __ __ ║
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ║ /___/ ║
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ║ ║
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ║ Version: 0.9.9 ║
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ║ https://mk1.ai ║
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ║ ║
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ║ The license key for the current software has been verified as ║
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ║ belonging to: ║
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ║ ║
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ║ Chai Research Corp. ║
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ║ ║
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'dial tcp 127.0.0.1:8080: connect: connection refused\n')
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: Downloaded to shared memory in 38.736s
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpgg4f4b_9, device:0
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:49716->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:34730->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: quantized model in 28.347s
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: Processed model Trace2333/fd_llama3_v2_fdall_r32a16_5w in 67.083s
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: creating bucket guanaco-mkml-models
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/trace2333-fd-llama3-v2-f-2628-v2
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/trace2333-fd-llama3-v2-f-2628-v2/config.json
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/trace2333-fd-llama3-v2-f-2628-v2/tokenizer_config.json
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/trace2333-fd-llama3-v2-f-2628-v2/special_tokens_map.json
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/trace2333-fd-llama3-v2-f-2628-v2/tokenizer.json
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/trace2333-fd-llama3-v2-f-2628-v2/flywheel_model.0.safetensors
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: loading reward model from Jellywibble/gpt2_xl_pairwise_89m_step_347634
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: Loading 0: 0%| | 0/291 [00:00<?, ?it/s] Loading 0: 2%|▏ | 5/291 [00:00<00:10, 27.24it/s] Loading 0: 4%|▍ | 12/291 [00:00<00:07, 35.80it/s] Loading 0: 5%|▌ | 16/291 [00:00<00:08, 33.23it/s] Loading 0: 7%|▋ | 21/291 [00:00<00:07, 36.37it/s] Loading 0: 9%|▊ | 25/291 [00:00<00:07, 34.75it/s] Loading 0: 10%|█ | 30/291 [00:00<00:06, 37.91it/s] Loading 0: 12%|█▏ | 34/291 [00:01<00:09, 26.10it/s] Loading 0: 13%|█▎ | 38/291 [00:01<00:09, 27.85it/s] Loading 0: 14%|█▍ | 42/291 [00:01<00:09, 26.91it/s] Loading 0: 16%|█▋ | 48/291 [00:01<00:07, 32.93it/s] Loading 0: 18%|█▊ | 52/291 [00:01<00:07, 32.52it/s] Loading 0: 20%|█▉ | 57/291 [00:01<00:06, 35.72it/s] Loading 0: 21%|██ | 61/291 [00:01<00:06, 34.59it/s] Loading 0: 23%|██▎ | 66/291 [00:01<00:06, 37.50it/s] Loading 0: 24%|██▍ | 70/291 [00:02<00:06, 35.72it/s] Loading 0: 25%|██▌ | 74/291 [00:02<00:06, 35.25it/s] Loading 0: 27%|██▋ | 78/291 [00:02<00:06, 35.38it/s] Loading 0: 28%|██▊ | 82/291 [00:02<00:08, 24.59it/s] Loading 0: 29%|██▉ | 85/291 [00:02<00:08, 25.64it/s] Loading 0: 31%|███ | 90/291 [00:02<00:06, 30.28it/s] Loading 0: 32%|███▏ | 94/291 [00:02<00:06, 30.08it/s] Loading 0: 34%|███▍ | 99/291 [00:03<00:05, 33.86it/s] Loading 0: 35%|███▌ | 103/291 [00:03<00:05, 33.53it/s] Loading 0: 37%|███▋ | 108/291 [00:03<00:04, 36.70it/s] Loading 0: 38%|███▊ | 112/291 [00:03<00:05, 34.59it/s] Loading 0: 40%|███▉ | 116/291 [00:03<00:04, 35.04it/s] Loading 0: 42%|████▏ | 122/291 [00:03<00:04, 39.84it/s] Loading 0: 44%|████▎ | 127/291 [00:03<00:04, 37.24it/s] Loading 0: 46%|████▌ | 133/291 [00:04<00:05, 31.55it/s] Loading 0: 47%|████▋ | 137/291 [00:04<00:04, 32.05it/s] Loading 0: 48%|████▊ | 141/291 [00:04<00:05, 29.77it/s] Loading 0: 51%|█████ | 147/291 [00:04<00:04, 34.62it/s] Loading 0: 52%|█████▏ | 151/291 [00:04<00:04, 32.95it/s] Loading 0: 54%|█████▎ | 156/291 [00:04<00:03, 35.50it/s] Loading 0: 55%|█████▍ | 160/291 [00:04<00:03, 34.31it/s] Loading 0: 57%|█████▋ | 165/291 [00:04<00:03, 36.87it/s] Loading 0: 58%|█████▊ | 169/291 [00:05<00:03, 35.33it/s] Loading 0: 60%|█████▉ | 174/291 [00:05<00:03, 37.54it/s] Loading 0: 61%|██████ | 178/291 [00:05<00:03, 35.87it/s] Loading 0: 63%|██████▎ | 184/291 [00:05<00:02, 41.98it/s] Loading 0: 65%|██████▍ | 189/291 [00:05<00:04, 24.93it/s] Loading 0: 67%|██████▋ | 194/291 [00:05<00:03, 26.37it/s] Loading 0: 69%|██████▉ | 201/291 [00:06<00:02, 32.48it/s] Loading 0: 70%|███████ | 205/291 [00:06<00:02, 32.16it/s] Loading 0: 72%|███████▏ | 210/291 [00:06<00:02, 34.85it/s] Loading 0: 74%|███████▎ | 214/291 [00:06<00:02, 32.82it/s] Loading 0: 75%|███████▌ | 219/291 [00:06<00:02, 35.57it/s] Loading 0: 77%|███████▋ | 223/291 [00:06<00:01, 34.40it/s] Loading 0: 78%|███████▊ | 227/291 [00:06<00:01, 34.51it/s] Loading 0: 79%|███████▉ | 231/291 [00:06<00:01, 34.64it/s] Loading 0: 81%|████████ | 235/291 [00:07<00:02, 25.05it/s] Loading 0: 82%|████████▏ | 239/291 [00:07<00:02, 25.35it/s] Loading 0: 85%|████████▍ | 246/291 [00:07<00:01, 33.13it/s] Loading 0: 86%|████████▌ | 250/291 [00:07<00:01, 32.84it/s] Loading 0: 88%|████████▊ | 255/291 [00:07<00:01, 35.80it/s] Loading 0: 89%|████████▉ | 259/291 [00:07<00:00, 34.44it/s] Loading 0: 91%|█████████ | 264/291 [00:08<00:00, 37.14it/s] Loading 0: 92%|█████████▏| 268/291 [00:08<00:00, 35.57it/s] Loading 0: 94%|█████████▍| 273/291 [00:08<00:00, 38.26it/s] Loading 0: 95%|█████████▌| 277/291 [00:08<00:00, 35.69it/s] Loading 0: 97%|█████████▋| 281/291 [00:08<00:00, 35.33it/s] Loading 0: 98%|█████████▊| 286/291 [00:13<00:01, 2.61it/s] Loading 0: 99%|█████████▉| 289/291 [00:14<00:00, 3.25it/s] /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:957: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: warnings.warn(
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: warnings.warn(
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: Saving duration: 1.355s
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: Processed model Jellywibble/gpt2_xl_pairwise_89m_step_347634 in 10.870s
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: creating bucket guanaco-reward-models
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: Bucket 's3://guanaco-reward-models/' created
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/trace2333-fd-llama3-v2-f-2628-v2_reward
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/trace2333-fd-llama3-v2-f-2628-v2_reward/config.json
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/trace2333-fd-llama3-v2-f-2628-v2_reward/special_tokens_map.json
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/trace2333-fd-llama3-v2-f-2628-v2_reward/tokenizer_config.json
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/trace2333-fd-llama3-v2-f-2628-v2_reward/merges.txt
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/trace2333-fd-llama3-v2-f-2628-v2_reward/vocab.json
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/trace2333-fd-llama3-v2-f-2628-v2_reward/tokenizer.json
trace2333-fd-llama3-v2-f-2628-v2-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/trace2333-fd-llama3-v2-f-2628-v2_reward/reward.tensors
Job trace2333-fd-llama3-v2-f-2628-v2-mkmlizer completed after 104.92s with status: succeeded
Stopping job with name trace2333-fd-llama3-v2-f-2628-v2-mkmlizer
Pipeline stage MKMLizer completed in 105.87s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.10s
Running pipeline stage ISVCDeployer
Creating inference service trace2333-fd-llama3-v2-f-2628-v2
Waiting for inference service trace2333-fd-llama3-v2-f-2628-v2 to be ready
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:33866->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:33544->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:32780->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:59800->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:58432->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:34692->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:47888->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:48502->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:48624->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:37374->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:35824->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:37492->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v36: ('http://mistralai-mistral-nemo-9330-v36-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'dial tcp 127.0.0.1:8080: connect: connection refused\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission mistralai-mistral-nemo-_9330_v42: ('http://mistralai-mistral-nemo-9330-v42-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Inference service trace2333-fd-llama3-v2-f-2628-v2 ready after 211.55208468437195s
Pipeline stage ISVCDeployer completed in 213.17s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.3319308757781982s
Received healthy response to inference request in 1.5398287773132324s
Received healthy response to inference request in 1.481647253036499s
Received healthy response to inference request in 1.4613711833953857s
Received healthy response to inference request in 1.422942876815796s
5 requests
0 failed requests
5th percentile: 1.4306285381317139
10th percentile: 1.4383141994476318
20th percentile: 1.4536855220794678
30th percentile: 1.4654263973236084
40th percentile: 1.4735368251800538
50th percentile: 1.481647253036499
60th percentile: 1.5049198627471925
70th percentile: 1.5281924724578857
80th percentile: 1.6982491970062257
90th percentile: 2.0150900363922117
95th percentile: 2.173510456085205
99th percentile: 2.3002467918395997
mean time: 1.6475441932678223
Pipeline stage StressChecker completed in 9.30s
trace2333-fd-llama3-v2-f_2628_v2 status is now deployed due to DeploymentManager action
trace2333-fd-llama3-v2-f_2628_v2 status is now inactive due to auto deactivation removed underperforming models
trace2333-fd-llama3-v2-f_2628_v2 status is now torndown due to DeploymentManager action

Usage Metrics

Latency Metrics