developer_uid: zonemercy
submission_id: chaiml-bol-kimid-v8-lr5_74286_v3
model_name: chaiml-bol-kimid-v8-lr5_74286_v3
model_group: ChaiML/bol-kimid-v8-lr5e
status: torndown
timestamp: 2026-02-09T06:30:52+00:00
num_battles: 11267
num_wins: 6518
celo_rating: 1358.26
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/bol-kimid-v8-lr5e6ep2r64g4b02
model_architecture: Qwen3MoeForCausalLM
model_num_parameters: 18790207488.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 80
reward_model: default
display_name: chaiml-bol-kimid-v8-lr5_74286_v3
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/bol-kimid-v8-lr5e6ep2r64g4b02
model_size: 19B
ranking_group: single
us_pacific_date: 2026-02-05
win_ratio: 0.578503594568208
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['<|im_end|>', '</s>', '<|assistant|>', '<|user|>', '####', '</think>'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': "<|im_start|>system\n{bot_name}'s persona: {memory}<|im_end|>\n", 'prompt_template': '', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-bol-kimid-v8-lr5-74286-v3-uploader
Waiting for job on chaiml-bol-kimid-v8-lr5-74286-v3-uploader to finish
Retrying (%r) after connection broken by '%r': %s
Retrying (%r) after connection broken by '%r': %s
Retrying (%r) after connection broken by '%r': %s
Retrying (%r) after connection broken by '%r': %s
Retrying (%r) after connection broken by '%r': %s
Retrying (%r) after connection broken by '%r': %s
Retrying (%r) after connection broken by '%r': %s
Retrying (%r) after connection broken by '%r': %s
Retrying (%r) after connection broken by '%r': %s
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: 
Fetching 39 files:   0%|          | 0/39 [00:00<?, ?it/s]
Fetching 39 files:   3%|▎         | 1/39 [00:00<00:09,  4.10it/s]
Fetching 39 files:  18%|█▊        | 7/39 [00:46<03:43,  7.00s/it]
Fetching 39 files:  21%|██        | 8/39 [01:04<04:34,  8.86s/it]
Fetching 39 files:  38%|███▊      | 15/39 [01:26<02:05,  5.24s/it]
Fetching 39 files:  41%|████      | 16/39 [01:33<02:03,  5.37s/it]
Fetching 39 files:  46%|████▌     | 18/39 [01:38<01:38,  4.71s/it]
Fetching 39 files:  51%|█████▏    | 20/39 [01:38<01:06,  3.51s/it]
Fetching 39 files:  54%|█████▍    | 21/39 [01:49<01:22,  4.58s/it]
Fetching 39 files:  59%|█████▉    | 23/39 [02:36<02:48, 10.53s/it]
Fetching 39 files:  62%|██████▏   | 24/39 [03:07<03:33, 14.24s/it]
Fetching 39 files: 100%|██████████| 39/39 [03:07<00:00,  4.81s/it]
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: Downloaded in 187.779s
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: Processed model ChaiML/bol-kimid-v8-lr5e6ep2r64g4b02 in 188.308s
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: creating bucket guanaco-vllm-models
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-bol-kimid-v8-lr5-74286-v3-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-bol-kimid-v8-lr5-74286-v3-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-bol-kimid-v8-lr5-74286-v3-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-bol-kimid-v8-lr5-74286-v3-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-bol-kimid-v8-lr5-74286-v3-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-bol-kimid-v8-lr5-74286-v3-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-bol-kimid-v8-lr5-74286-v3-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-bol-kimid-v8-lr5-74286-v3-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/.gitattributes
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/added_tokens.json s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/added_tokens.json
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/tokenizer_config.json
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/chat_template.jinja
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/generation_config.json
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/special_tokens_map.json
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/config.json
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/quantization_config.json s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/quantization_config.json
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/merges.txt s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/merges.txt
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/vocab.json s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/vocab.json
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model.safetensors.index.json
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/tokenizer.json
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: DEBUG retryable error: ServiceUnavailable: Service Unavailable
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: 	status code: 503, request id: , host id: 
Failed to get response for submission chaiml-mistral-24b-2048_25909_v3: ('http://chaiml-mistral-24b-2048-25909-v3-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:40452->127.0.0.1:8080: read: connection reset by peer\n')
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: DEBUG retryable error: ServiceUnavailable: Service Unavailable
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: 	status code: 503, request id: , host id: 
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: DEBUG retryable error: ServiceUnavailable: Service Unavailable
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: 	status code: 503, request id: , host id: 
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: DEBUG retryable error: ServiceUnavailable: Service Unavailable
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: 	status code: 503, request id: , host id: 
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: DEBUG retryable error: ServiceUnavailable: Service Unavailable
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: 	status code: 503, request id: , host id: 
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: ERROR "cp /dev/shm/model_output/model-00013-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00013-of-00027.safetensors": MultipartUpload: upload multipart failed upload id: 2~RFNwuEAcS6nRJ58ctDq5C4f5TfvHEdv caused by: SignatureDoesNotMatch:  status code: 403, request id: tx00000c8b1b9fdf5901bc5-00698555f3-14b31daae8-default, host id:
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: DEBUG retryable error: ServiceUnavailable: Service Unavailable
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: 	status code: 503, request id: , host id: 
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: DEBUG retryable error: ServiceUnavailable: Service Unavailable
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: 	status code: 503, request id: , host id: 
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: DEBUG retryable error: ServiceUnavailable: Service Unavailable
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: 	status code: 503, request id: , host id: 
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: DEBUG retryable error: ServiceUnavailable: Service Unavailable
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: 	status code: 503, request id: , host id: 
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: DEBUG retryable error: ServiceUnavailable: Service Unavailable
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: 	status code: 503, request id: , host id: 
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: DEBUG retryable error: ServiceUnavailable: Service Unavailable
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: 	status code: 503, request id: , host id: 
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: DEBUG retryable error: ServiceUnavailable: Service Unavailable
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: 	status code: 503, request id: , host id: 
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: DEBUG retryable error: ServiceUnavailable: Service Unavailable
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: 	status code: 503, request id: , host id: 
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: ERROR "cp /dev/shm/model_output/model-00017-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00017-of-00027.safetensors": MultipartUpload: upload multipart failed upload id: 2~pncK5izzoF2Ba3M-t4FDI87z6fgxhFN caused by: SignatureDoesNotMatch:  status code: 403, request id: tx000004c2cbb3068b94240-00698555e5-14af544255-default, host id:
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: ERROR "cp /dev/shm/model_output/model-00005-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00005-of-00027.safetensors": MultipartUpload: upload multipart failed upload id: 2~xcPHrC3fI5b4K5VdHwv_v0JRImfaSko caused by: SignatureDoesNotMatch:  status code: 403, request id: tx00000e67ebe597c0e90ef-00698555eb-149d0ff886-default, host id:
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: ERROR "cp /dev/shm/model_output/model-00007-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00007-of-00027.safetensors": MultipartUpload: upload multipart failed upload id: 2~FvdEWIg-23bPbxMGgr-8Klwd8u7eZym caused by: SignatureDoesNotMatch:  status code: 403, request id: tx00000507abfa8138fb7b5-0069855616-14a26732ad-default, host id:
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: ERROR "cp /dev/shm/model_output/model-00027-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00027-of-00027.safetensors": MultipartUpload: upload multipart failed upload id: 2~2Jbq67b4YitZuHlvCFLhHb6XTm0dZ2C caused by: SignatureDoesNotMatch:  status code: 403, request id: tx0000000f8d0909aa67dea-0069855626-14b31daae8-default, host id:
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: ERROR "cp /dev/shm/model_output/model-00001-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00001-of-00027.safetensors": MultipartUpload: upload multipart failed upload id: 2~sTLnGurqg_ktpjgvsSJehDdigdA_IIH caused by: SignatureDoesNotMatch:  status code: 403, request id: tx000005974a7020d2faeee-0069855648-14a24de22e-default, host id:
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: ERROR "cp /dev/shm/model_output/model-00020-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00020-of-00027.safetensors": MultipartUpload: upload multipart failed upload id: 2~x-i_trjxvJcCbk3mJwbkMDB3S6eafuN caused by: SignatureDoesNotMatch:  status code: 403, request id: tx00000cc31f9879925d99b-006985563f-14af544255-default, host id:
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: ERROR "cp /dev/shm/model_output/model-00012-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00012-of-00027.safetensors": MultipartUpload: upload multipart failed upload id: 2~LMp12GAn7coOJwd1Gn2Q951LK2d83CK caused by: SignatureDoesNotMatch:  status code: 403, request id: tx00000d6c99632be24f08c-00698555e6-14b33dd27d-default, host id:
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: ERROR "cp /dev/shm/model_output/model-00021-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00021-of-00027.safetensors": MultipartUpload: upload multipart failed upload id: 2~RdQ95lUqeWHhxw5n8NMDiVpDJgBWR2w caused by: SignatureDoesNotMatch:  status code: 403, request id: tx00000396222054dc7643f-0069855628-14b31daae8-default, host id:
Retrying (%r) after connection broken by '%r': %s
Retrying (%r) after connection broken by '%r': %s
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: ERROR "cp /dev/shm/model_output/model-00022-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00022-of-00027.safetensors": MultipartUpload: upload multipart failed upload id: 2~m63tj0W5_0fjEOduUmMd0MmUiqYbwcI caused by: SignatureDoesNotMatch:  status code: 403, request id: tx000000abdd124bce46eb7-0069855648-14a19326cd-default, host id:
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: ERROR "cp /dev/shm/model_output/model-00009-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00009-of-00027.safetensors": MultipartUpload: upload multipart failed upload id: 2~CBHNCLiiKcOOq8Wq9zaPaIx6pGkO-Cv caused by: SignatureDoesNotMatch:  status code: 403, request id: tx0000016cbc1ff5b2c2b63-0069855699-14b33dd27d-default, host id:
Retrying (%r) after connection broken by '%r': %s
Retrying (%r) after connection broken by '%r': %s
HTTP Request: %s %s "%s %d %s"
HTTP Request: %s %s "%s %d %s"
Failed to get response for submission chaiml-mistral-24b-2048_25909_v3: ('http://chaiml-mistral-24b-2048-25909-v3-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:40354->127.0.0.1:8080: read: connection reset by peer\n')
Unable to record family friendly update due to error: Invalid JSON input: Expecting value: line 1 column 1 (char 0)
HTTP Request: %s %s "%s %d %s"
HTTP Request: %s %s "%s %d %s"
HTTP Request: %s %s "%s %d %s"
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/model-00018-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00018-of-00027.safetensors
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/model-00025-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00025-of-00027.safetensors
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/model-00019-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00019-of-00027.safetensors
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/model-00023-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00023-of-00027.safetensors
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/model-00026-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00026-of-00027.safetensors
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/model-00016-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00016-of-00027.safetensors
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/model-00008-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00008-of-00027.safetensors
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/model-00010-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00010-of-00027.safetensors
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/model-00006-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00006-of-00027.safetensors
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/model-00015-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00015-of-00027.safetensors
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/model-00002-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00002-of-00027.safetensors
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/model-00011-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00011-of-00027.safetensors
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/model-00024-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00024-of-00027.safetensors
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/model-00004-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00004-of-00027.safetensors
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/model-00003-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00003-of-00027.safetensors
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/model-00014-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00014-of-00027.safetensors
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: Retry 1/5 exited 1, retrying in 2 seconds...
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/model-00027-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00027-of-00027.safetensors
HTTP Request: %s %s "%s %d %s"
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/model-00012-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00012-of-00027.safetensors
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/model-00013-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00013-of-00027.safetensors
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/model-00009-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00009-of-00027.safetensors
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/model-00021-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00021-of-00027.safetensors
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/model-00022-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00022-of-00027.safetensors
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/model-00001-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00001-of-00027.safetensors
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/model-00017-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00017-of-00027.safetensors
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/model-00005-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00005-of-00027.safetensors
chaiml-bol-kimid-v8-lr5-74286-v3-uploader: cp /dev/shm/model_output/model-00020-of-00027.safetensors s3://guanaco-vllm-models/chaiml-bol-kimid-v8-lr5-74286-v3/model-00020-of-00027.safetensors
Job chaiml-bol-kimid-v8-lr5-74286-v3-uploader completed after 5333.92s with status: succeeded
Stopping job with name chaiml-bol-kimid-v8-lr5-74286-v3-uploader
Pipeline stage VLLMUploader completed in 5334.50s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.16s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-bol-kimid-v8-lr5-74286-v3
Waiting for inference service chaiml-bol-kimid-v8-lr5-74286-v3 to be ready
Inference service chaiml-bol-kimid-v8-lr5-74286-v3 ready after 561.4128818511963s
Pipeline stage VLLMDeployer completed in 561.99s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.086937665939331s
Received healthy response to inference request in 1.9540975093841553s
Received healthy response to inference request in 2.4777848720550537s
Received healthy response to inference request in 2.0269081592559814s
Received healthy response to inference request in 1.949002981185913s
Received healthy response to inference request in 2.029883623123169s
Received healthy response to inference request in 2.1049792766571045s
Received healthy response to inference request in 2.2284090518951416s
Received healthy response to inference request in 1.8816163539886475s
Received healthy response to inference request in 2.126173496246338s
Received healthy response to inference request in 2.0550498962402344s
Received healthy response to inference request in 2.4089627265930176s
Received healthy response to inference request in 1.922494649887085s
Received healthy response to inference request in 1.9921417236328125s
Received healthy response to inference request in 2.166569948196411s
Received healthy response to inference request in 1.9508986473083496s
Received healthy response to inference request in 2.039799213409424s
Received healthy response to inference request in 1.9437341690063477s
Received healthy response to inference request in 2.1790637969970703s
Received healthy response to inference request in 1.889723539352417s
Received healthy response to inference request in 2.2923600673675537s
Received healthy response to inference request in 1.9491853713989258s
Received healthy response to inference request in 2.009762763977051s
Received healthy response to inference request in 2.1022815704345703s
Received healthy response to inference request in 2.1771717071533203s
Received healthy response to inference request in 2.2822957038879395s
Received healthy response to inference request in 2.098217725753784s
Received healthy response to inference request in 2.022778034210205s
Received healthy response to inference request in 2.0828988552093506s
Received healthy response to inference request in 2.1632766723632812s
30 requests
0 failed requests
5th percentile: 1.9044705390930177
10th percentile: 1.9416102170944214
20th percentile: 1.9505559921264648
30th percentile: 2.004476451873779
40th percentile: 2.028693437576294
50th percentile: 2.0689743757247925
60th percentile: 2.0998432636260986
70th percentile: 2.137304449081421
80th percentile: 2.17755012512207
90th percentile: 2.283302140235901
95th percentile: 2.3564915299415583
99th percentile: 2.4578264498710634
mean time: 2.0864819924036664
Pipeline stage StressChecker completed in 67.54s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.67s
Shutdown handler de-registered
chaiml-bol-kimid-v8-lr5_74286_v3 status is now deployed due to DeploymentManager action
chaiml-bol-kimid-v8-lr5_74286_v3 status is now inactive due to auto deactivation removed underperforming models
chaiml-bol-kimid-v8-lr5_74286_v3 status is now torndown due to DeploymentManager action