developer_uid: rirv938
submission_id: chaiml-reward-dpo-d9b1-_68526_v1
model_name: chaiml-reward-dpo-d9b1-_68526_v1
model_group: ChaiML/reward-dpo-d9b1-c
status: torndown
timestamp: 2026-03-06T03:42:19+00:00
num_battles: 10849
num_wins: 5554
celo_rating: 1302.89
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/reward-dpo-d9b1-chaiml-glm-air-4-5-sft-_90753_v1
model_architecture: Glm4MoeForCausalLM
model_num_parameters: 9073971200.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 80
reward_model: default
display_name: chaiml-reward-dpo-d9b1-_68526_v1
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/reward-dpo-d9b1-chaiml-glm-air-4-5-sft-_90753_v1
model_size: 9B
ranking_group: single
us_pacific_date: 2026-03-02
win_ratio: 0.5119365840169601
generation_params: {'temperature': 1.0, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 60, 'presence_penalty': 0.1, 'frequency_penalty': 0.0, 'stopping_words': ['<|im_end|>', 'You:', '</s>', '###', '<|im_start|>'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': '[gMASK]<sop><|system|>\nRespond as a high quality storyteller.<|user|>\n', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '<|assistant|>\n<think></think>\n{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-reward-dpo-d9b1-68526-v1-uploader
Waiting for job on chaiml-reward-dpo-d9b1-68526-v1-uploader to finish
chaiml-reward-dpo-1044-39258-v1-uploader: Using quantization_mode: none
chaiml-reward-dpo-1044-39258-v1-uploader: Downloading snapshot of ChaiML/reward-dpo-1044-chaiml-glm-air-4-5-sft-_90753_v1...
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-reward-dpo-b0ee-82897-v1-uploader
Waiting for job on chaiml-reward-dpo-b0ee-82897-v1-uploader to finish
chaiml-reward-dpo-d9b1-68526-v1-uploader: Using quantization_mode: none
chaiml-reward-dpo-d9b1-68526-v1-uploader: Downloading snapshot of ChaiML/reward-dpo-d9b1-chaiml-glm-air-4-5-sft-_90753_v1...
chaiml-reward-dpo-b0ee-82897-v1-uploader: Using quantization_mode: none
chaiml-reward-dpo-b0ee-82897-v1-uploader: Downloading snapshot of ChaiML/reward-dpo-b0ee-chaiml-glm-air-4-5-sft-_90753_v1...
chaiml-reward-dpo-1044-39258-v1-uploader: Downloaded in 76.525s
chaiml-reward-dpo-d9b1-68526-v1-uploader: Downloaded in 89.698s
chaiml-reward-dpo-b0ee-82897-v1-uploader: Downloaded in 77.252s
chaiml-reward-dpo-1044-39258-v1-uploader: Processed model ChaiML/reward-dpo-1044-chaiml-glm-air-4-5-sft-_90753_v1 in 152.533s
chaiml-reward-dpo-b0ee-82897-v1-uploader: Processed model ChaiML/reward-dpo-b0ee-chaiml-glm-air-4-5-sft-_90753_v1 in 152.050s
chaiml-reward-dpo-b0ee-82897-v1-uploader: creating bucket guanaco-vllm-models
chaiml-reward-dpo-b0ee-82897-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-b0ee-82897-v1-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-reward-dpo-b0ee-82897-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-reward-dpo-b0ee-82897-v1-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00043-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00043-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-b0ee-82897-v1-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-reward-dpo-b0ee-82897-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-b0ee-82897-v1-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-reward-dpo-b0ee-82897-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-b0ee-82897-v1-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-reward-dpo-b0ee-82897-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-b0ee-82897-v1-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-reward-dpo-b0ee-82897-v1-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-reward-dpo-b0ee-82897-v1-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-reward-dpo-b0ee-82897-v1-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00009-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00009-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00032-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00032-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00039-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00039-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00020-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00020-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/README.md s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/README.md
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00001-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00001-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/args.json s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/args.json
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00007-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00007-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/chat_template.jinja
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00022-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00022-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/.gitattributes
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00010-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00010-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/model.safetensors.index.json
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00002-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00002-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/config.json
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00012-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00012-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/special_tokens_map.json
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00014-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00014-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/tokenizer_config.json
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00018-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00018-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/tokenizer.json
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00038-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00038-of-00043.safetensors
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00024-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00024-of-00043.safetensors
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00015-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00015-of-00043.safetensors
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00026-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00026-of-00043.safetensors
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00030-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00030-of-00043.safetensors
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00040-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00040-of-00043.safetensors
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00005-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00005-of-00043.safetensors
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00003-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00003-of-00043.safetensors
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00017-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00017-of-00043.safetensors
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00019-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00019-of-00043.safetensors
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00028-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00028-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/model-00005-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/model-00005-of-00043.safetensors
chaiml-reward-dpo-d9b1-68526-v1-uploader: Processed model ChaiML/reward-dpo-d9b1-chaiml-glm-air-4-5-sft-_90753_v1 in 214.873s
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00004-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00004-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/model-00023-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/model-00023-of-00043.safetensors
chaiml-reward-dpo-d9b1-68526-v1-uploader: creating bucket guanaco-vllm-models
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00034-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00034-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/model-00001-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/model-00001-of-00043.safetensors
chaiml-reward-dpo-d9b1-68526-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00027-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00027-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/model-00004-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/model-00004-of-00043.safetensors
chaiml-reward-dpo-d9b1-68526-v1-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00042-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00042-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/model-00036-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/model-00036-of-00043.safetensors
chaiml-reward-dpo-d9b1-68526-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00041-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00041-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/model-00022-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/model-00022-of-00043.safetensors
chaiml-reward-dpo-d9b1-68526-v1-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00031-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00031-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/model-00021-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/model-00021-of-00043.safetensors
chaiml-reward-dpo-d9b1-68526-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00025-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00025-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/model-00012-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/model-00012-of-00043.safetensors
chaiml-reward-dpo-d9b1-68526-v1-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00021-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00021-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/model-00042-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/model-00042-of-00043.safetensors
chaiml-reward-dpo-d9b1-68526-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00013-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00013-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/model-00014-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/model-00014-of-00043.safetensors
chaiml-reward-dpo-d9b1-68526-v1-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00037-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00037-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/model-00025-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/model-00025-of-00043.safetensors
chaiml-reward-dpo-d9b1-68526-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00033-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00033-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/model-00029-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/model-00029-of-00043.safetensors
chaiml-reward-dpo-d9b1-68526-v1-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00029-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00029-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/model-00028-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/model-00028-of-00043.safetensors
chaiml-reward-dpo-d9b1-68526-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00011-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00011-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/model-00010-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/model-00010-of-00043.safetensors
chaiml-reward-dpo-d9b1-68526-v1-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00006-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00006-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/model-00006-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/model-00006-of-00043.safetensors
chaiml-reward-dpo-d9b1-68526-v1-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00016-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00016-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/model-00027-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/model-00027-of-00043.safetensors
chaiml-reward-dpo-d9b1-68526-v1-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-reward-dpo-1044-39258-v1-uploader: cp /dev/shm/model_output/model-00036-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-1044-39258-v1/default/model-00036-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/model-00018-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/model-00018-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/model-00024-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/model-00024-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/model-00041-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/model-00041-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/model-00019-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/model-00019-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/model-00016-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/model-00016-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/model-00035-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/model-00035-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/model-00040-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/model-00040-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/model-00031-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/model-00031-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/model-00030-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/model-00030-of-00043.safetensors
chaiml-reward-dpo-d9b1-68526-v1-uploader: cp /dev/shm/model_output/model-00043-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-d9b1-68526-v1/default/model-00043-of-00043.safetensors
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/model-00009-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/model-00009-of-00043.safetensors
Job chaiml-reward-dpo-1044-39258-v1-uploader completed after 274.2s with status: succeeded
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/model-00015-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/model-00015-of-00043.safetensors
Stopping job with name chaiml-reward-dpo-1044-39258-v1-uploader
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/model-00020-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/model-00020-of-00043.safetensors
Pipeline stage VLLMUploader completed in 281.29s
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/model-00038-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/model-00038-of-00043.safetensors
run pipeline stage %s
chaiml-reward-dpo-b0ee-82897-v1-uploader: cp /dev/shm/model_output/model-00003-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-b0ee-82897-v1/default/model-00003-of-00043.safetensors
Running pipeline stage VLLMTemplater
chaiml-reward-dpo-d9b1-68526-v1-uploader: cp /dev/shm/model_output/model-00020-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-d9b1-68526-v1/default/model-00020-of-00043.safetensors
Pipeline stage VLLMTemplater completed in 3.16s
chaiml-reward-dpo-d9b1-68526-v1-uploader: cp /dev/shm/model_output/model-00022-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-d9b1-68526-v1/default/model-00022-of-00043.safetensors
run pipeline stage %s
chaiml-reward-dpo-d9b1-68526-v1-uploader: cp /dev/shm/model_output/model-00026-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-d9b1-68526-v1/default/model-00026-of-00043.safetensors
Running pipeline stage VLLMDeployer
chaiml-reward-dpo-d9b1-68526-v1-uploader: cp /dev/shm/model_output/model-00013-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-d9b1-68526-v1/default/model-00013-of-00043.safetensors
Creating inference service chaiml-reward-dpo-1044-39258-v1
chaiml-reward-dpo-d9b1-68526-v1-uploader: cp /dev/shm/model_output/model-00034-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-d9b1-68526-v1/default/model-00034-of-00043.safetensors
chaiml-reward-dpo-d9b1-68526-v1-uploader: cp /dev/shm/model_output/model-00042-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-d9b1-68526-v1/default/model-00042-of-00043.safetensors
Waiting for inference service chaiml-reward-dpo-1044-39258-v1 to be ready
Job chaiml-reward-dpo-b0ee-82897-v1-uploader completed after 272.84s with status: succeeded
Stopping job with name chaiml-reward-dpo-b0ee-82897-v1-uploader
Pipeline stage VLLMUploader completed in 280.33s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 2.57s
Job chaiml-reward-dpo-d9b1-68526-v1-uploader completed after 295.28s with status: succeeded
run pipeline stage %s
Stopping job with name chaiml-reward-dpo-d9b1-68526-v1-uploader
Running pipeline stage VLLMDeployer
Pipeline stage VLLMUploader completed in 300.15s
Creating inference service chaiml-reward-dpo-b0ee-82897-v1
run pipeline stage %s
Waiting for inference service chaiml-reward-dpo-b0ee-82897-v1 to be ready
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 2.81s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-reward-dpo-d9b1-68526-v1
Waiting for inference service chaiml-reward-dpo-d9b1-68526-v1 to be ready
Inference service chaiml-reward-dpo-1044-39258-v1 ready after 582.1946132183075s
Pipeline stage VLLMDeployer completed in 589.60s
run pipeline stage %s
Running pipeline stage StressChecker
Inference service chaiml-reward-dpo-b0ee-82897-v1 ready after 570.4972398281097s
Pipeline stage VLLMDeployer completed in 578.07s
run pipeline stage %s
Received healthy response to inference request in 6.508826732635498s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.170912742614746s
Received healthy response to inference request in 4.745554208755493s
Received healthy response to inference request in 4.230443954467773s
Received healthy response to inference request in 4.857105255126953s
Received healthy response to inference request in 5.038939952850342s
Received healthy response to inference request in 3.3851490020751953s
Received healthy response to inference request in 3.1830456256866455s
Received healthy response to inference request in 4.083965301513672s
Received healthy response to inference request in 6.014947891235352s
Received healthy response to inference request in 3.533442497253418s
Received healthy response to inference request in 2.8944873809814453s
Received healthy response to inference request in 3.352712869644165s
Received healthy response to inference request in 9.138283252716064s
Received healthy response to inference request in 7.425544738769531s
Received healthy response to inference request in 3.566768169403076s
Received healthy response to inference request in 3.6807243824005127s
Received healthy response to inference request in 4.67350172996521s
Received healthy response to inference request in 3.1271443367004395s
Received healthy response to inference request in 4.728521823883057s
Received healthy response to inference request in 3.836097240447998s
Received healthy response to inference request in 3.1165616512298584s
Received healthy response to inference request in 2.7975921630859375s
Received healthy response to inference request in 4.659898042678833s
Received healthy response to inference request in 3.96530818939209s
Received healthy response to inference request in 3.32319712638855s
Received healthy response to inference request in 3.1055357456207275s
Received healthy response to inference request in 4.636388063430786s
Received healthy response to inference request in 3.783607006072998s
Received healthy response to inference request in 2.802870750427246s
Received healthy response to inference request in 3.0747034549713135s
Received healthy response to inference request in 6.986799955368042s
Received healthy response to inference request in 13.652459144592285s
Received healthy response to inference request in 15.262922763824463s
Received healthy response to inference request in 10.893314123153687s
Received healthy response to inference request in 5.451234340667725s
Received healthy response to inference request in 3.621187210083008s
Received healthy response to inference request in 4.79976487159729s
Received healthy response to inference request in 4.26019811630249s
Received healthy response to inference request in 4.6345274448394775s
Received healthy response to inference request in 4.09852933883667s
Received healthy response to inference request in 2.956503391265869s
Received healthy response to inference request in 3.2062251567840576s
Received healthy response to inference request in 3.7041525840759277s
Received healthy response to inference request in 4.289891004562378s
Received healthy response to inference request in 3.565707206726074s
Received healthy response to inference request in 4.717375040054321s
Received healthy response to inference request in 3.30077862739563s
Received healthy response to inference request in 3.002894639968872s
Received healthy response to inference request in 5.362760782241821s
Received healthy response to inference request in 4.291159629821777s
Received healthy response to inference request in 3.567808151245117s
Received healthy response to inference request in 3.0580384731292725s
Received healthy response to inference request in 5.2096381187438965s
Received healthy response to inference request in 4.668217182159424s
Received healthy response to inference request in 2.915046453475952s
Received healthy response to inference request in 8.438023567199707s
Received healthy response to inference request in 3.549656629562378s
Received healthy response to inference request in 4.9222412109375s
Received healthy response to inference request in 5.902876853942871s
30 requests
30 requests
0 failed requests
0 failed requests
5th percentile: 2.9944208025932313
5th percentile: 2.9337020754814147
10th percentile: 3.1654776334762573
10th percentile: 2.9982555150985717
20th percentile: 3.2818679332733156
20th percentile: 3.0993692874908447
30th percentile: 3.5664498805999756
30th percentile: 3.3754181623458863
40th percentile: 4.266112184524536
40th percentile: 3.5925749778747558
50th percentile: 4.64814305305481
50th percentile: 3.743879795074463
60th percentile: 4.75701904296875
60th percentile: 4.012771034240722
70th percentile: 5.090149402618407
70th percentile: 4.269486570358277
80th percentile: 5.563977050781252
80th percentile: 4.723010873794555
90th percentile: 7.1319223165512105
90th percentile: 6.05514364242554
95th percentile: 8.823166394233702
95th percentile: 9.332817900180807
99th percentile: 13.486777305603033
99th percentile: 12.852307088375094
mean time: 5.0228639364242555
mean time: 4.468193173408508
Pipeline stage StressChecker completed in 255.21s
Pipeline stage StressChecker completed in 247.06s
run pipeline stage %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 10.38s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 10.75s
Shutdown handler de-registered
Shutdown handler de-registered
chaiml-reward-dpo-1044-_39258_v1 status is now deployed due to DeploymentManager action
chaiml-reward-dpo-b0ee-_82897_v1 status is now deployed due to DeploymentManager action
Inference service chaiml-reward-dpo-d9b1-68526-v1 ready after 1200.7217338085175s
Pipeline stage VLLMDeployer completed in 1206.57s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 9.082847833633423s
Received healthy response to inference request in 3.6210734844207764s
Received healthy response to inference request in 6.087874412536621s
Received healthy response to inference request in 3.2009353637695312s
Received healthy response to inference request in 4.510427713394165s
Received healthy response to inference request in 4.187309265136719s
Received healthy response to inference request in 4.306625127792358s
Received healthy response to inference request in 3.433605670928955s
Received healthy response to inference request in 4.387970447540283s
Received healthy response to inference request in 2.8854024410247803s
Received healthy response to inference request in 3.216348648071289s
Received healthy response to inference request in 3.3508083820343018s
Received healthy response to inference request in 4.053387880325317s
Received healthy response to inference request in 5.02363395690918s
Received healthy response to inference request in 4.252445936203003s
Received healthy response to inference request in 3.7111976146698s
Received healthy response to inference request in 3.6732990741729736s
Received healthy response to inference request in 9.28880524635315s
Received healthy response to inference request in 7.2511467933654785s
Received healthy response to inference request in 3.328641891479492s
Received healthy response to inference request in 4.18003511428833s
Received healthy response to inference request in 3.8177897930145264s
Received healthy response to inference request in 3.4337427616119385s
Received healthy response to inference request in 3.1277382373809814s
Received healthy response to inference request in 4.5154359340667725s
Received healthy response to inference request in 5.147414445877075s
Received healthy response to inference request in 4.1542134284973145s
Received healthy response to inference request in 2.801900863647461s
Received healthy response to inference request in 4.490008354187012s
Received healthy response to inference request in 4.445825815200806s
30 requests
0 failed requests
5th percentile: 2.994453549385071
10th percentile: 3.1936156511306764
20th percentile: 3.34637508392334
30th percentile: 3.5648742675781246
40th percentile: 3.7751529216766357
50th percentile: 4.167124271392822
60th percentile: 4.2741176128387455
70th percentile: 4.4590805768966675
80th percentile: 4.617075538635255
90th percentile: 6.204201650619509
95th percentile: 8.258582365512844
99th percentile: 9.229077596664428
mean time: 4.43226306438446
Pipeline stage StressChecker completed in 168.05s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 3.04s
Shutdown handler de-registered
chaiml-reward-dpo-d9b1-_68526_v1 status is now deployed due to DeploymentManager action
chaiml-reward-dpo-d9b1-_68526_v1 status is now inactive due to auto deactivation removed underperforming models
chaiml-reward-dpo-d9b1-_68526_v1 status is now torndown due to DeploymentManager action