developer_uid: acehao-chai
submission_id: chaiml-kimid-v9-opusdv_23365_v23
model_name: chaiml-kimid-v9-opusdv_23365_v23
model_group: ChaiML/kimid-v9-opusdv1-
status: inactive
timestamp: 2026-02-14T04:27:12+00:00
num_battles: 11740
num_wins: 6429
celo_rating: 1333.55
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/kimid-v9-opusdv1-lr5e6ep2r64g4b01
model_architecture: Qwen3MoeForCausalLM
model_num_parameters: 18790207488.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 80
reward_model: default
display_name: chaiml-kimid-v9-opusdv_23365_v23
ineligible_reason: max_output_tokens!=64
is_internal_developer: False
language_model: ChaiML/kimid-v9-opusdv1-lr5e6ep2r64g4b01
model_size: 19B
ranking_group: single
us_pacific_date: 2026-02-13
win_ratio: 0.5476149914821125
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['<|im_end|>', '####', '<|user|>', '</think>', '</s>', '<|assistant|>'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': "<|im_start|>system\n{bot_name}'s persona: {memory}<|im_end|>\n", 'prompt_template': '', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-kimid-v9-opusdv-23365-v23-uploader
Waiting for job on chaiml-kimid-v9-opusdv-23365-v23-uploader to finish
chaiml-kimid-v9-opusdv-23365-v23-uploader: Using quantization_mode: w4a16
chaiml-kimid-v9-opusdv-23365-v23-uploader: Checking if ChaiML/kimid-v9-opusdv1-lr5e6ep2r64g4b01-W4A16 already exists in ChaiML
chaiml-kimid-v9-opusdv-23365-v23-uploader: Model already exists. Downloading to /dev/shm/model_output...
chaiml-kimid-v9-opusdv-23365-v23-uploader: Downloading snapshot of ChaiML/kimid-v9-opusdv1-lr5e6ep2r64g4b01-W4A16...
HTTP Request: %s %s "%s %d %s"
Failed to get response for submission chaiml-mistral-24b-2048_15988_v1: ('http://chaiml-mistral-24b-2048-15988-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-mistral-24b-2048_15988_v1: ('http://chaiml-mistral-24b-2048-15988-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
HTTP Request: %s %s "%s %d %s"
chaiml-kimid-v9-opusdv-23365-v23-uploader: Downloaded in 151.105s
chaiml-kimid-v9-opusdv-23365-v23-uploader: Processed model ChaiML/kimid-v9-opusdv1-lr5e6ep2r64g4b01 in 151.655s
chaiml-kimid-v9-opusdv-23365-v23-uploader: creating bucket guanaco-vllm-models
chaiml-kimid-v9-opusdv-23365-v23-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-kimid-v9-opusdv-23365-v23-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-kimid-v9-opusdv-23365-v23-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-kimid-v9-opusdv-23365-v23-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-kimid-v9-opusdv-23365-v23-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-kimid-v9-opusdv-23365-v23-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-kimid-v9-opusdv-23365-v23-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-kimid-v9-opusdv-23365-v23-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-kimid-v9-opusdv-23365-v23-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-kimid-v9-opusdv-23365-v23-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-kimid-v9-opusdv-23365-v23-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-kimid-v9-opusdv-23365-v23-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-kimid-v9-opusdv-23365-v23-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-kimid-v9-opusdv-23365-v23-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-kimid-v9-opusdv-23365-v23-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-kimid-v9-opusdv-23365-v23-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-kimid-v9-opusdv-23365-v23-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-kimid-v9-opusdv-23365-v23-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/.gitattributes
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/chat_template.jinja
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/tokenizer_config.json
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/added_tokens.json s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/added_tokens.json
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/generation_config.json
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/special_tokens_map.json
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/config.json
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/quantization_config.json s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/quantization_config.json
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/merges.txt s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/merges.txt
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/vocab.json s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/vocab.json
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model.safetensors.index.json
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/tokenizer.json
Failed to get response for submission chaiml-mistral-24b-2048-_2678_v3: ('http://chaiml-mistral-24b-2048-2678-v3-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG retryable error: RequestError: send request failed
chaiml-kimid-v9-opusdv-23365-v23-uploader: caused by: Put "https://object.ord1.coreweave.com/guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00002-of-00027.safetensors?partNumber=23&uploadId=2~RSYxl3ehjVKUuur_TtkgGvLiwh_ayaH": write tcp 10.0.17.197:56748->216.153.53.63:443: write: connection reset by peer
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/model-00027-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00027-of-00027.safetensors
Failed to get response for submission chaiml-mistral-24b-2048_54327_v6: ('http://chaiml-mistral-24b-2048-54327-v6-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/model-00023-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00023-of-00027.safetensors
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/model-00004-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00004-of-00027.safetensors
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/model-00010-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00010-of-00027.safetensors
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/model-00015-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00015-of-00027.safetensors
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/model-00013-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00013-of-00027.safetensors
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/model-00012-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00012-of-00027.safetensors
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/model-00005-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00005-of-00027.safetensors
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/model-00003-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00003-of-00027.safetensors
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/model-00025-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00025-of-00027.safetensors
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/model-00016-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00016-of-00027.safetensors
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/model-00014-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00014-of-00027.safetensors
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/model-00019-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00019-of-00027.safetensors
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/model-00008-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00008-of-00027.safetensors
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/model-00006-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00006-of-00027.safetensors
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/model-00021-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00021-of-00027.safetensors
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/model-00022-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00022-of-00027.safetensors
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/model-00007-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00007-of-00027.safetensors
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/model-00017-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00017-of-00027.safetensors
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/model-00018-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00018-of-00027.safetensors
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/model-00024-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00024-of-00027.safetensors
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/model-00020-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00020-of-00027.safetensors
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/model-00026-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00026-of-00027.safetensors
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/model-00011-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00011-of-00027.safetensors
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/model-00001-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00001-of-00027.safetensors
chaiml-kimid-v9-opusdv-23365-v23-uploader: Retry 1/5 exited 1, retrying in 2 seconds...
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/.gitattributes": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/added_tokens.json s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/added_tokens.json": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/chat_template.jinja": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/config.json": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/generation_config.json s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/generation_config.json": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/merges.txt s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/merges.txt": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/model-00001-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00001-of-00027.safetensors": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/model-00003-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00003-of-00027.safetensors": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/model-00004-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00004-of-00027.safetensors": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/model-00005-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00005-of-00027.safetensors": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/model-00006-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00006-of-00027.safetensors": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/model-00007-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00007-of-00027.safetensors": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/model-00008-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00008-of-00027.safetensors": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/model-00009-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00009-of-00027.safetensors": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/model-00010-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00010-of-00027.safetensors": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/model-00011-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00011-of-00027.safetensors": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/model-00012-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00012-of-00027.safetensors": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/model-00013-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00013-of-00027.safetensors": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/model-00014-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00014-of-00027.safetensors": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/model-00015-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00015-of-00027.safetensors": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/model-00016-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00016-of-00027.safetensors": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/model-00017-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00017-of-00027.safetensors": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/model-00018-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00018-of-00027.safetensors": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/model-00019-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00019-of-00027.safetensors": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/model-00020-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00020-of-00027.safetensors": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/model-00021-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00021-of-00027.safetensors": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/model-00022-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00022-of-00027.safetensors": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/model-00023-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00023-of-00027.safetensors": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/model-00024-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00024-of-00027.safetensors": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/model-00025-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00025-of-00027.safetensors": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/model-00026-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00026-of-00027.safetensors": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/model-00027-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00027-of-00027.safetensors": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model.safetensors.index.json": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/quantization_config.json s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/quantization_config.json": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/special_tokens_map.json": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/tokenizer.json": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/tokenizer_config.json": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: DEBUG "sync /dev/shm/model_output/vocab.json s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/vocab.json": object size matches
chaiml-kimid-v9-opusdv-23365-v23-uploader: cp /dev/shm/model_output/model-00002-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v9-opusdv-23365-v23/default/model-00002-of-00027.safetensors
Failed to get response for submission chaiml-mistral-24b-2048-_2678_v3: ('http://chaiml-mistral-24b-2048-2678-v3-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Job chaiml-kimid-v9-opusdv-23365-v23-uploader completed after 432.42s with status: succeeded
Stopping job with name chaiml-kimid-v9-opusdv-23365-v23-uploader
Pipeline stage VLLMUploader completed in 433.80s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.16s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-kimid-v9-opusdv-23365-v23
Waiting for inference service chaiml-kimid-v9-opusdv-23365-v23 to be ready
Failed to get response for submission chaiml-mistral-24b-2048_54327_v6: ('http://chaiml-mistral-24b-2048-54327-v6-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-mistral-24b-2048_15988_v1: ('http://chaiml-mistral-24b-2048-15988-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-mistral-24b-2048_15988_v1: ('http://chaiml-mistral-24b-2048-15988-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-mistral-24b-2048_15988_v1: ('http://chaiml-mistral-24b-2048-15988-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-mistral-24b-2048_15988_v1: ('http://chaiml-mistral-24b-2048-15988-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Inference service chaiml-kimid-v9-opusdv-23365-v23 ready after 673.3414137363434s
Pipeline stage VLLMDeployer completed in 673.96s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.326188325881958s
Received healthy response to inference request in 2.123025417327881s
Received healthy response to inference request in 1.9541523456573486s
Received healthy response to inference request in 2.0738697052001953s
Received healthy response to inference request in 1.9592869281768799s
Received healthy response to inference request in 2.030238151550293s
Received healthy response to inference request in 2.131096124649048s
Received healthy response to inference request in 1.9055020809173584s
Received healthy response to inference request in 1.9318046569824219s
Received healthy response to inference request in 1.9011859893798828s
Received healthy response to inference request in 1.9633512496948242s
Received healthy response to inference request in 1.997333288192749s
Received healthy response to inference request in 1.9897687435150146s
Received healthy response to inference request in 1.926438808441162s
Received healthy response to inference request in 1.9533321857452393s
Received healthy response to inference request in 1.9571666717529297s
Received healthy response to inference request in 1.8866760730743408s
Received healthy response to inference request in 1.9031765460968018s
Received healthy response to inference request in 1.9630756378173828s
Received healthy response to inference request in 1.981475591659546s
Received healthy response to inference request in 2.3705687522888184s
Received healthy response to inference request in 2.59281325340271s
Received healthy response to inference request in 2.329885244369507s
Received healthy response to inference request in 2.01045823097229s
Received healthy response to inference request in 2.0270707607269287s
Received healthy response to inference request in 2.027416944503784s
Received healthy response to inference request in 2.0766429901123047s
Received healthy response to inference request in 1.9638075828552246s
Received healthy response to inference request in 2.34503436088562s
Received healthy response to inference request in 1.988041639328003s
30 requests
0 failed requests
5th percentile: 1.9020817399024963
10th percentile: 1.9052695274353026
20th percentile: 1.9490266799926759
30th percentile: 1.9586508512496947
40th percentile: 1.9636250495910645
50th percentile: 1.9889051914215088
60th percentile: 2.0171032428741453
70th percentile: 2.0433276176452635
80th percentile: 2.124639558792114
90th percentile: 2.331400156021118
95th percentile: 2.359078276157379
99th percentile: 2.5283623480796815
mean time: 2.0529961427052816
Pipeline stage StressChecker completed in 74.06s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.72s
Shutdown handler de-registered
chaiml-kimid-v9-opusdv_23365_v23 status is now deployed due to DeploymentManager action
chaiml-kimid-v9-opusdv_23365_v23 status is now inactive due to auto deactivation removed underperforming models