chaiml-reward-dpo-922e-_45044

developer_uid: rirv938
submission_id: chaiml-reward-dpo-922e-_45044_v1
model_name: chaiml-reward-dpo-922e-_45044_v1
model_group: ChaiML/reward-dpo-922e-c
status: torndown
timestamp: 2026-03-08T01:06:43+00:00
num_battles: 11345
num_wins: 6353
celo_rating: 1331.94
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/reward-dpo-922e-chaiml-glm-air-4-5-sft_93550_v11
model_architecture: Glm4MoeForCausalLM
model_num_parameters: 9073971200.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 80
reward_model: default
display_name: chaiml-reward-dpo-922e-_45044_v1
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/reward-dpo-922e-chaiml-glm-air-4-5-sft_93550_v11
model_size: 9B
ranking_group: single
us_pacific_date: 2026-03-03
win_ratio: 0.5599823710885853
generation_params: {'temperature': 1.0, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 60, 'presence_penalty': 0.1, 'frequency_penalty': 0.0, 'stopping_words': ['<|im_end|>', 'You:', '</s>', '###', '<|im_start|>'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': '[gMASK]<sop><|system|>\nRespond as a high quality storyteller.<|user|>\n', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '<|assistant|>\n<think></think>\n{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-reward-dpo-922e-45044-v1-uploader
Waiting for job on chaiml-reward-dpo-922e-45044-v1-uploader to finish
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-reward-dpo-5807-c-9042-v1-uploader
Waiting for job on chaiml-reward-dpo-5807-c-9042-v1-uploader to finish
chaiml-reward-dpo-922e-45044-v1-uploader: Using quantization_mode: none
chaiml-reward-dpo-922e-45044-v1-uploader: Downloading snapshot of ChaiML/reward-dpo-922e-chaiml-glm-air-4-5-sft_93550_v11...
chaiml-reward-dpo-5807-c-9042-v1-uploader: Using quantization_mode: none
chaiml-reward-dpo-5807-c-9042-v1-uploader: Downloading snapshot of ChaiML/reward-dpo-5807-chaiml-glm-air-4-5-sft_93550_v11...
chaiml-reward-dpo-922e-45044-v1-uploader: Downloaded in 72.251s
chaiml-reward-dpo-5807-c-9042-v1-uploader: Downloaded in 72.623s
chaiml-reward-dpo-5807-c-9042-v1-uploader: Processed model ChaiML/reward-dpo-5807-chaiml-glm-air-4-5-sft_93550_v11 in 147.916s
chaiml-reward-dpo-5807-c-9042-v1-uploader: creating bucket guanaco-vllm-models
chaiml-reward-dpo-5807-c-9042-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-5807-c-9042-v1-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-reward-dpo-5807-c-9042-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-reward-dpo-5807-c-9042-v1-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-reward-dpo-5807-c-9042-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-5807-c-9042-v1-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-reward-dpo-5807-c-9042-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-5807-c-9042-v1-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-reward-dpo-5807-c-9042-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-5807-c-9042-v1-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-reward-dpo-5807-c-9042-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-5807-c-9042-v1-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-reward-dpo-5807-c-9042-v1-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-reward-dpo-5807-c-9042-v1-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-reward-dpo-5807-c-9042-v1-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-reward-dpo-5807-c-9042-v1-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-reward-dpo-5807-c-9042-v1-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-reward-dpo-5807-c-9042-v1-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/.gitattributes
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/args.json s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/args.json
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/config.json
chaiml-reward-dpo-922e-45044-v1-uploader: Processed model ChaiML/reward-dpo-922e-chaiml-glm-air-4-5-sft_93550_v11 in 152.839s
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/README.md s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/README.md
chaiml-reward-dpo-922e-45044-v1-uploader: creating bucket guanaco-vllm-models
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/special_tokens_map.json
chaiml-reward-dpo-922e-45044-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/tokenizer_config.json
chaiml-reward-dpo-922e-45044-v1-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/chat_template.jinja
chaiml-reward-dpo-922e-45044-v1-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/model.safetensors.index.json
chaiml-reward-dpo-922e-45044-v1-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/tokenizer.json
chaiml-reward-dpo-922e-45044-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-922e-45044-v1-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-reward-dpo-922e-45044-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-922e-45044-v1-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-reward-dpo-922e-45044-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-922e-45044-v1-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-reward-dpo-922e-45044-v1-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-922e-45044-v1-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-reward-dpo-922e-45044-v1-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-reward-dpo-922e-45044-v1-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-reward-dpo-922e-45044-v1-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-reward-dpo-922e-45044-v1-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-reward-dpo-922e-45044-v1-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-reward-dpo-922e-45044-v1-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/.gitattributes
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/args.json s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/args.json
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/chat_template.jinja
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/model.safetensors.index.json
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/config.json
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/tokenizer_config.json
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/README.md s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/README.md
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/special_tokens_map.json
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/tokenizer.json
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/model-00043-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/model-00043-of-00043.safetensors
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/model-00036-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/model-00036-of-00043.safetensors
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/model-00016-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/model-00016-of-00043.safetensors
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/model-00015-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/model-00015-of-00043.safetensors
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/model-00030-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/model-00030-of-00043.safetensors
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/model-00024-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/model-00024-of-00043.safetensors
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/model-00005-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/model-00005-of-00043.safetensors
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/model-00008-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/model-00008-of-00043.safetensors
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/model-00033-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/model-00033-of-00043.safetensors
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/model-00003-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/model-00003-of-00043.safetensors
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/model-00042-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/model-00042-of-00043.safetensors
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/model-00011-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/model-00011-of-00043.safetensors
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/model-00022-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/model-00022-of-00043.safetensors
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/model-00031-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/model-00031-of-00043.safetensors
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/model-00009-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/model-00009-of-00043.safetensors
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/model-00018-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/model-00018-of-00043.safetensors
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/model-00034-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/model-00034-of-00043.safetensors
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/model-00037-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/model-00037-of-00043.safetensors
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/model-00035-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/model-00035-of-00043.safetensors
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/model-00017-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/model-00017-of-00043.safetensors
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/model-00038-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/model-00038-of-00043.safetensors
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/model-00019-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/model-00019-of-00043.safetensors
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/model-00020-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/model-00020-of-00043.safetensors
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/model-00041-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/model-00041-of-00043.safetensors
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/model-00002-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/model-00002-of-00043.safetensors
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/model-00032-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/model-00032-of-00043.safetensors
chaiml-reward-dpo-922e-45044-v1-uploader: cp /dev/shm/model_output/model-00029-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-922e-45044-v1/default/model-00029-of-00043.safetensors
Job chaiml-reward-dpo-922e-45044-v1-uploader completed after 231.88s with status: succeeded
Stopping job with name chaiml-reward-dpo-922e-45044-v1-uploader
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/model-00043-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/model-00043-of-00043.safetensors
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/model-00033-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/model-00033-of-00043.safetensors
Pipeline stage VLLMUploader completed in 233.22s
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/model-00009-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/model-00009-of-00043.safetensors
run pipeline stage %s
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/model-00017-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/model-00017-of-00043.safetensors
Running pipeline stage VLLMTemplater
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/model-00036-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/model-00036-of-00043.safetensors
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/model-00016-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/model-00016-of-00043.safetensors
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/model-00042-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/model-00042-of-00043.safetensors
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/model-00014-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/model-00014-of-00043.safetensors
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/model-00038-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/model-00038-of-00043.safetensors
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/model-00010-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/model-00010-of-00043.safetensors
Pipeline stage VLLMTemplater completed in 2.81s
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/model-00006-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/model-00006-of-00043.safetensors
run pipeline stage %s
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/model-00023-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/model-00023-of-00043.safetensors
Running pipeline stage VLLMDeployer
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/model-00027-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/model-00027-of-00043.safetensors
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/model-00032-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/model-00032-of-00043.safetensors
Creating inference service chaiml-reward-dpo-922e-45044-v1
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/model-00041-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/model-00041-of-00043.safetensors
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/model-00011-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/model-00011-of-00043.safetensors
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/model-00030-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/model-00030-of-00043.safetensors
Waiting for inference service chaiml-reward-dpo-922e-45044-v1 to be ready
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/model-00007-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/model-00007-of-00043.safetensors
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/model-00013-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/model-00013-of-00043.safetensors
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/model-00004-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/model-00004-of-00043.safetensors
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/model-00020-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/model-00020-of-00043.safetensors
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/model-00008-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/model-00008-of-00043.safetensors
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/model-00037-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/model-00037-of-00043.safetensors
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/model-00026-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/model-00026-of-00043.safetensors
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/model-00005-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/model-00005-of-00043.safetensors
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/model-00040-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/model-00040-of-00043.safetensors
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/model-00034-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/model-00034-of-00043.safetensors
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/model-00022-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/model-00022-of-00043.safetensors
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/model-00002-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/model-00002-of-00043.safetensors
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/model-00001-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/model-00001-of-00043.safetensors
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/model-00019-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/model-00019-of-00043.safetensors
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/model-00003-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/model-00003-of-00043.safetensors
chaiml-reward-dpo-5807-c-9042-v1-uploader: cp /dev/shm/model_output/model-00018-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-5807-c-9042-v1/default/model-00018-of-00043.safetensors
Job chaiml-reward-dpo-5807-c-9042-v1-uploader completed after 246.86s with status: succeeded
Stopping job with name chaiml-reward-dpo-5807-c-9042-v1-uploader
Pipeline stage VLLMUploader completed in 248.52s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.83s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-reward-dpo-5807-c-9042-v1
Waiting for inference service chaiml-reward-dpo-5807-c-9042-v1 to be ready
Inference service chaiml-reward-dpo-922e-45044-v1 ready after 567.2933096885681s
Pipeline stage VLLMDeployer completed in 569.21s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.160264015197754s
Received healthy response to inference request in 2.4254674911499023s
Inference service chaiml-reward-dpo-5807-c-9042-v1 ready after 557.5018117427826s
Pipeline stage VLLMDeployer completed in 559.08s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.42284893989563s
Received healthy response to inference request in 2.5840868949890137s
Received healthy response to inference request in 3.200526237487793s
Received healthy response to inference request in 2.3908984661102295s
Received healthy response to inference request in 2.197808027267456s
Received healthy response to inference request in 2.240811824798584s
Received healthy response to inference request in 2.1452465057373047s
Received healthy response to inference request in 2.306300640106201s
Received healthy response to inference request in 3.0038719177246094s
Received healthy response to inference request in 2.228785991668701s
Received healthy response to inference request in 2.3034815788269043s
Received healthy response to inference request in 2.37441086769104s
Received healthy response to inference request in 2.545063018798828s
Received healthy response to inference request in 2.3318111896514893s
Received healthy response to inference request in 2.278939962387085s
Received healthy response to inference request in 2.281283378601074s
Received healthy response to inference request in 2.7416117191314697s
Received healthy response to inference request in 2.309504985809326s
Received healthy response to inference request in 2.2444238662719727s
Received healthy response to inference request in 2.4115614891052246s
Received healthy response to inference request in 2.1499369144439697s
Received healthy response to inference request in 2.3675098419189453s
Received healthy response to inference request in 2.1497175693511963s
Received healthy response to inference request in 2.305903673171997s
Received healthy response to inference request in 2.1399519443511963s
Received healthy response to inference request in 2.271395683288574s
Received healthy response to inference request in 2.8216466903686523s
Received healthy response to inference request in 2.123253583908081s
Received healthy response to inference request in 2.2099385261535645s
Received healthy response to inference request in 2.2142081260681152s
Received healthy response to inference request in 2.4421205520629883s
Received healthy response to inference request in 2.290030002593994s
Received healthy response to inference request in 2.508002281188965s
Received healthy response to inference request in 3.531592607498169s
Received healthy response to inference request in 2.1596059799194336s
Received healthy response to inference request in 2.1831467151641846s
Received healthy response to inference request in 2.178469657897949s
Received healthy response to inference request in 2.1750223636627197s
Received healthy response to inference request in 2.1669559478759766s
Received healthy response to inference request in 2.2242913246154785s
Received healthy response to inference request in 2.263880968093872s
Received healthy response to inference request in 2.15372633934021s
Received healthy response to inference request in 2.856275796890259s
Received healthy response to inference request in 2.444445848464966s
Received healthy response to inference request in 2.2244629859924316s
Received healthy response to inference request in 2.249692678451538s
Received healthy response to inference request in 2.2087697982788086s
Received healthy response to inference request in 2.2966079711914062s
Received healthy response to inference request in 2.2323391437530518s
Received healthy response to inference request in 2.0790722370147705s
Received healthy response to inference request in 2.2417891025543213s
Received healthy response to inference request in 2.1251707077026367s
Received healthy response to inference request in 2.327160120010376s
Received healthy response to inference request in 2.1324164867401123s
Received healthy response to inference request in 2.150400400161743s
30 requests
0 failed requests
5th percentile: 2.147258484363556
10th percentile: 2.1499149799346924
20th percentile: 2.165485954284668
Received healthy response to inference request in 2.1193864345550537s
30th percentile: 2.205481266975403
40th percentile: 2.2291886806488037
50th percentile: 2.2541524171829224
60th percentile: 2.312952995300293
70th percentile: 2.430463409423828
80th percentile: 2.584372758865357
90th percentile: 2.871035408973694
95th percentile: 3.0898875713348386
99th percentile: 3.188850193023682
mean time: 2.4033659219741823
Received healthy response to inference request in 2.4921011924743652s
Pipeline stage StressChecker completed in 86.63s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Received healthy response to inference request in 2.2304201126098633s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 1.99s
30 requests
Shutdown handler de-registered
0 failed requests
chaiml-reward-dpo-922e-_45044_v1 status is now deployed due to DeploymentManager action
5th percentile: 2.121126651763916
chaiml-reward-dpo-922e-_45044_v1 status is now inactive due to auto deactivation removed underperforming models
chaiml-reward-dpo-922e-_45044_v1 status is now torndown due to DeploymentManager action