chaiml-reward-dpo-7a6f-_69008

developer_uid: rirv938
submission_id: chaiml-reward-dpo-7a6f-_69008_v2
model_name: chaiml-reward-dpo-7a6f-_69008_v2
model_group: ChaiML/reward-dpo-7a6f-c
status: torndown
timestamp: 2026-03-06T05:21:39+00:00
num_battles: 12578
num_wins: 7183
celo_rating: 1343.02
family_friendly_score: 0.0
family_friendly_standard_error: 0.0
submission_type: basic
model_repo: ChaiML/reward-dpo-7a6f-chaiml-glm-air-4-5-sft_93550_v11
model_architecture: Glm4MoeForCausalLM
model_num_parameters: 9073971200.0
best_of: 8
max_input_tokens: 2048
max_output_tokens: 80
reward_model: default
display_name: chaiml-reward-dpo-7a6f-_69008_v2
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/reward-dpo-7a6f-chaiml-glm-air-4-5-sft_93550_v11
model_size: 9B
ranking_group: single
us_pacific_date: 2026-03-02
win_ratio: 0.5710764827476547
generation_params: {'temperature': 1.0, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 60, 'presence_penalty': 0.1, 'frequency_penalty': 0.0, 'stopping_words': ['<|im_end|>', 'You:', '</s>', '###', '<|im_start|>'], 'max_input_tokens': 2048, 'best_of': 8, 'max_output_tokens': 80}
formatter: {'memory_template': '[gMASK]<sop><|system|>\nRespond as a high quality storyteller.<|user|>\n', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '<|assistant|>\n<think></think>\n{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-reward-dpo-7a6f-69008-v2-uploader
Waiting for job on chaiml-reward-dpo-7a6f-69008-v2-uploader to finish
chaiml-reward-dpo-f477-83868-v2-uploader: Using quantization_mode: none
chaiml-reward-dpo-f477-83868-v2-uploader: Downloading snapshot of ChaiML/reward-dpo-f477-chaiml-glm-air-4-5-sft_93550_v11...
chaiml-reward-dpo-53f2-22066-v2-uploader: Using quantization_mode: none
chaiml-reward-dpo-53f2-22066-v2-uploader: Downloading snapshot of ChaiML/reward-dpo-53f2-chaiml-glm-air-4-5-sft_93550_v11...
chaiml-reward-dpo-7a6f-69008-v2-uploader: Using quantization_mode: none
chaiml-reward-dpo-7a6f-69008-v2-uploader: Downloading snapshot of ChaiML/reward-dpo-7a6f-chaiml-glm-air-4-5-sft_93550_v11...
chaiml-reward-dpo-f477-83868-v2-uploader: Downloaded in 70.319s
chaiml-reward-dpo-53f2-22066-v2-uploader: Downloaded in 70.261s
chaiml-reward-dpo-f477-83868-v2-uploader: Processed model ChaiML/reward-dpo-f477-chaiml-glm-air-4-5-sft_93550_v11 in 152.364s
chaiml-reward-dpo-f477-83868-v2-uploader: creating bucket guanaco-vllm-models
chaiml-reward-dpo-f477-83868-v2-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-f477-83868-v2-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-reward-dpo-f477-83868-v2-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-reward-dpo-f477-83868-v2-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-reward-dpo-f477-83868-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-f477-83868-v2-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-reward-dpo-f477-83868-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-f477-83868-v2-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-reward-dpo-f477-83868-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-f477-83868-v2-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-reward-dpo-f477-83868-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-f477-83868-v2-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-reward-dpo-f477-83868-v2-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-reward-dpo-f477-83868-v2-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-reward-dpo-7a6f-69008-v2-uploader: Processed model ChaiML/reward-dpo-7a6f-chaiml-glm-air-4-5-sft_93550_v11 in 148.421s
chaiml-reward-dpo-7a6f-69008-v2-uploader: creating bucket guanaco-vllm-models
chaiml-reward-dpo-7a6f-69008-v2-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-7a6f-69008-v2-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-reward-dpo-7a6f-69008-v2-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-reward-dpo-7a6f-69008-v2-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-reward-dpo-7a6f-69008-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-7a6f-69008-v2-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-reward-dpo-7a6f-69008-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-7a6f-69008-v2-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-reward-dpo-7a6f-69008-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-7a6f-69008-v2-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-reward-dpo-7a6f-69008-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-7a6f-69008-v2-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-reward-dpo-7a6f-69008-v2-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-reward-dpo-7a6f-69008-v2-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-reward-dpo-7a6f-69008-v2-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-reward-dpo-7a6f-69008-v2-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-reward-dpo-7a6f-69008-v2-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-reward-dpo-7a6f-69008-v2-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/.gitattributes
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/config.json
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/args.json s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/args.json
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/README.md s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/README.md
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/special_tokens_map.json
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/tokenizer_config.json
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/chat_template.jinja
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model.safetensors.index.json
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/tokenizer.json
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00043-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00043-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00043-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00043-of-00043.safetensors
chaiml-reward-dpo-53f2-22066-v2-uploader: Processed model ChaiML/reward-dpo-53f2-chaiml-glm-air-4-5-sft_93550_v11 in 304.951s
chaiml-reward-dpo-53f2-22066-v2-uploader: creating bucket guanaco-vllm-models
chaiml-reward-dpo-53f2-22066-v2-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:56: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-53f2-22066-v2-uploader:   RE_S3_DATESTRING = re.compile('\.[0-9]*(?:[Z\\-\\+]*?)')
chaiml-reward-dpo-53f2-22066-v2-uploader: /usr/lib/python3/dist-packages/S3/BaseUtils.py:57: SyntaxWarning: invalid escape sequence '\s'
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00001-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00001-of-00043.safetensors
chaiml-reward-dpo-53f2-22066-v2-uploader:   RE_XML_NAMESPACE = re.compile(b'^(<?[^>]+?>\s*|\s*)(<\w+) xmlns=[\'"](https?://[^\'"]+)[\'"]', re.MULTILINE)
chaiml-reward-dpo-53f2-22066-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:240: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-53f2-22066-v2-uploader:   invalid = re.search("([^a-z0-9\.-])", bucket, re.UNICODE)
chaiml-reward-dpo-53f2-22066-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:244: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-53f2-22066-v2-uploader:   invalid = re.search("([^A-Za-z0-9\._-])", bucket, re.UNICODE)
chaiml-reward-dpo-53f2-22066-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:255: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-53f2-22066-v2-uploader:   if re.search("-\.", bucket, re.UNICODE):
chaiml-reward-dpo-53f2-22066-v2-uploader: /usr/lib/python3/dist-packages/S3/Utils.py:257: SyntaxWarning: invalid escape sequence '\.'
chaiml-reward-dpo-53f2-22066-v2-uploader:   if re.search("\.\.", bucket, re.UNICODE):
chaiml-reward-dpo-53f2-22066-v2-uploader: /usr/lib/python3/dist-packages/S3/S3Uri.py:155: SyntaxWarning: invalid escape sequence '\w'
chaiml-reward-dpo-53f2-22066-v2-uploader:   _re = re.compile("^(\w+://)?(.*)", re.UNICODE)
chaiml-reward-dpo-53f2-22066-v2-uploader: /usr/lib/python3/dist-packages/S3/FileLists.py:480: SyntaxWarning: invalid escape sequence '\*'
chaiml-reward-dpo-53f2-22066-v2-uploader:   wildcard_split_result = re.split("\*|\?", uri_str, maxsplit=1)
chaiml-reward-dpo-53f2-22066-v2-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-reward-dpo-53f2-22066-v2-uploader: uploading /dev/shm/model_output to s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/chat_template.jinja s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/chat_template.jinja
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/args.json s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/args.json
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/config.json s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/config.json
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/.gitattributes s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/.gitattributes
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/tokenizer_config.json s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/tokenizer_config.json
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/special_tokens_map.json s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/special_tokens_map.json
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/README.md s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/README.md
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model.safetensors.index.json
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/tokenizer.json s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/tokenizer.json
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00025-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00025-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00036-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00036-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00028-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00028-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00032-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00032-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00036-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00036-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00041-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00041-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00023-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00023-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00040-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00040-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00009-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00009-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00006-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00006-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00018-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00018-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00014-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00014-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00026-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00026-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00011-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00011-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00008-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00008-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00002-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00002-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00041-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00041-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00033-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00033-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00031-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00031-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00039-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00039-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00024-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00024-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00038-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00038-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00032-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00032-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00031-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00031-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00002-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00002-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00005-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00005-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00006-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00006-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00022-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00022-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00013-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00013-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00021-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00021-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00019-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00019-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00004-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00004-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00003-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00003-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00018-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00018-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00029-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00029-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00003-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00003-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00027-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00027-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00030-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00030-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00004-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00004-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00013-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00013-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00020-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00020-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00025-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00025-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00034-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00034-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00008-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00008-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00037-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00037-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00016-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00016-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00033-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00033-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00035-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00035-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00042-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00042-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00019-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00019-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00039-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00039-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00042-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00042-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00022-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00022-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00001-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00001-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00011-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00011-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00026-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00026-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00005-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00005-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00009-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00009-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00040-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00040-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00029-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00029-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00010-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00010-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00007-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00007-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00007-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00007-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00034-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00034-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00015-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00015-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00017-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00017-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00038-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00038-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00028-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00028-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00017-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00017-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00010-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00010-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00035-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00035-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00037-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00037-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00014-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00014-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00023-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00023-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00016-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00016-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00024-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00024-of-00043.safetensors
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00043-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00043-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00012-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00012-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00015-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00015-of-00043.safetensors
chaiml-reward-dpo-f477-83868-v2-uploader: cp /dev/shm/model_output/model-00021-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-f477-83868-v2/default/model-00021-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00020-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00020-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00012-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00012-of-00043.safetensors
chaiml-reward-dpo-7a6f-69008-v2-uploader: cp /dev/shm/model_output/model-00027-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-7a6f-69008-v2/default/model-00027-of-00043.safetensors
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00028-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00028-of-00043.safetensors
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00008-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00008-of-00043.safetensors
Job chaiml-reward-dpo-f477-83868-v2-uploader completed after 369.83s with status: succeeded
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00021-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00021-of-00043.safetensors
Stopping job with name chaiml-reward-dpo-f477-83868-v2-uploader
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00015-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00015-of-00043.safetensors
Job chaiml-reward-dpo-7a6f-69008-v2-uploader completed after 366.55s with status: succeeded
Pipeline stage VLLMUploader completed in 374.53s
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00037-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00037-of-00043.safetensors
Stopping job with name chaiml-reward-dpo-7a6f-69008-v2-uploader
run pipeline stage %s
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00030-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00030-of-00043.safetensors
Pipeline stage VLLMUploader completed in 370.44s
Running pipeline stage VLLMTemplater
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00018-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00018-of-00043.safetensors
run pipeline stage %s
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00017-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00017-of-00043.safetensors
Running pipeline stage VLLMTemplater
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00036-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00036-of-00043.safetensors
Pipeline stage VLLMTemplater completed in 1.81s
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00012-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00012-of-00043.safetensors
Pipeline stage VLLMTemplater completed in 1.03s
run pipeline stage %s
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00042-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00042-of-00043.safetensors
run pipeline stage %s
Running pipeline stage VLLMDeployer
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00027-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00027-of-00043.safetensors
Running pipeline stage VLLMDeployer
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00011-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00011-of-00043.safetensors
Creating inference service chaiml-reward-dpo-f477-83868-v2
Creating inference service chaiml-reward-dpo-7a6f-69008-v2
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00023-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00023-of-00043.safetensors
Waiting for inference service chaiml-reward-dpo-f477-83868-v2 to be ready
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00033-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00033-of-00043.safetensors
Waiting for inference service chaiml-reward-dpo-7a6f-69008-v2 to be ready
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00040-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00040-of-00043.safetensors
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00022-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00022-of-00043.safetensors
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00007-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00007-of-00043.safetensors
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00013-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00013-of-00043.safetensors
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00001-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00001-of-00043.safetensors
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00041-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00041-of-00043.safetensors
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00039-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00039-of-00043.safetensors
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00002-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00002-of-00043.safetensors
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00035-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00035-of-00043.safetensors
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00005-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00005-of-00043.safetensors
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00003-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00003-of-00043.safetensors
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00004-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00004-of-00043.safetensors
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00038-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00038-of-00043.safetensors
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00019-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00019-of-00043.safetensors
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00024-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00024-of-00043.safetensors
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00006-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00006-of-00043.safetensors
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00009-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00009-of-00043.safetensors
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00029-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00029-of-00043.safetensors
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00010-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00010-of-00043.safetensors
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00032-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00032-of-00043.safetensors
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00034-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00034-of-00043.safetensors
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00020-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00020-of-00043.safetensors
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00025-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00025-of-00043.safetensors
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00026-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00026-of-00043.safetensors
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00014-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00014-of-00043.safetensors
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00031-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00031-of-00043.safetensors
chaiml-reward-dpo-53f2-22066-v2-uploader: cp /dev/shm/model_output/model-00016-of-00043.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-53f2-22066-v2/default/model-00016-of-00043.safetensors
Job chaiml-reward-dpo-53f2-22066-v2-uploader completed after 412.45s with status: succeeded
Stopping job with name chaiml-reward-dpo-53f2-22066-v2-uploader
Pipeline stage VLLMUploader completed in 415.21s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 1.36s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-reward-dpo-53f2-22066-v2
Waiting for inference service chaiml-reward-dpo-53f2-22066-v2 to be ready
Inference service chaiml-reward-dpo-f477-83868-v2 ready after 833.632892370224s
Inference service chaiml-reward-dpo-7a6f-69008-v2 ready after 833.1648700237274s
Pipeline stage VLLMDeployer completed in 838.56s
Pipeline stage VLLMDeployer completed in 837.22s
run pipeline stage %s
run pipeline stage %s
Running pipeline stage StressChecker
Running pipeline stage StressChecker
Received healthy response to inference request in 2.894890308380127s
Received healthy response to inference request in 3.290431499481201s
Received healthy response to inference request in 2.5226826667785645s
Received healthy response to inference request in 2.458979845046997s
Received healthy response to inference request in 2.2971320152282715s
Received healthy response to inference request in 2.4586069583892822s
Received healthy response to inference request in 2.41748046875s
Received healthy response to inference request in 2.148237943649292s
Received healthy response to inference request in 2.6897451877593994s
Received healthy response to inference request in 2.4179255962371826s
Received healthy response to inference request in 2.1638710498809814s
Received healthy response to inference request in 2.3150341510772705s
Received healthy response to inference request in 2.594219207763672s
Received healthy response to inference request in 2.4696643352508545s
Received healthy response to inference request in 2.3308911323547363s
Received healthy response to inference request in 2.321413040161133s
Received healthy response to inference request in 2.2964162826538086s
Received healthy response to inference request in 2.363767147064209s
Received healthy response to inference request in 2.288029193878174s
Received healthy response to inference request in 2.315779447555542s
Received healthy response to inference request in 2.6348118782043457s
Received healthy response to inference request in 2.3185207843780518s
Received healthy response to inference request in 2.1534600257873535s
Received healthy response to inference request in 2.38031005859375s
Received healthy response to inference request in 2.3589413166046143s
Received healthy response to inference request in 2.2695555686950684s
Received healthy response to inference request in 2.2383124828338623s
Received healthy response to inference request in 2.1891655921936035s
Received healthy response to inference request in 2.662076234817505s
Received healthy response to inference request in 2.4630415439605713s
Inference service chaiml-reward-dpo-53f2-22066-v2 ready after 852.8487062454224s
Received healthy response to inference request in 2.643120288848877s
Pipeline stage VLLMDeployer completed in 858.28s
Received healthy response to inference request in 3.494055986404419s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.5473968982696533s
Received healthy response to inference request in 2.2524592876434326s
Received healthy response to inference request in 2.6748580932617188s
Received healthy response to inference request in 2.4542059898376465s
Received healthy response to inference request in 2.881242036819458s
Received healthy response to inference request in 2.5060362815856934s
Received healthy response to inference request in 2.8251326084136963s
Received healthy response to inference request in 3.02397084236145s
Received healthy response to inference request in 2.7854177951812744s
Received healthy response to inference request in 2.1408209800720215s
Received healthy response to inference request in 2.217219829559326s
Received healthy response to inference request in 2.218118906021118s
Received healthy response to inference request in 2.2237279415130615s
Received healthy response to inference request in 2.308720111846924s
Received healthy response to inference request in 2.39054536819458s
Received healthy response to inference request in 2.2923097610473633s
Received healthy response to inference request in 2.4694759845733643s
Received healthy response to inference request in 2.300701856613159s
Received healthy response to inference request in 2.203230857849121s
Received healthy response to inference request in 2.350571632385254s
Received healthy response to inference request in 2.346956491470337s
Received healthy response to inference request in 3.0842626094818115s
Received healthy response to inference request in 2.363219976425171s
Received healthy response to inference request in 2.4910905361175537s
Received healthy response to inference request in 2.615004539489746s
Received healthy response to inference request in 2.656187057495117s
Received healthy response to inference request in 2.315225124359131s
Received healthy response to inference request in 2.341360092163086s
Received healthy response to inference request in 2.7248992919921875s
Received healthy response to inference request in 2.386289119720459s
Received healthy response to inference request in 2.819695234298706s
Received healthy response to inference request in 2.31620454788208s
Received healthy response to inference request in 2.378946304321289s
Received healthy response to inference request in 2.272387742996216s
Received healthy response to inference request in 2.2115490436553955s
Received healthy response to inference request in 2.4819722175598145s
Received healthy response to inference request in 3.9098758697509766s
Received healthy response to inference request in 3.1348893642425537s
Received healthy response to inference request in 2.8957231044769287s
Received healthy response to inference request in 3.477189540863037s
30 requests
Received healthy response to inference request in 2.518240213394165s
Received healthy response to inference request in 2.382915735244751s
0 failed requests
30 requests
5th percentile: 2.158144986629486
0 failed requests
10th percentile: 2.1992948770523073
5th percentile: 2.19923814535141
20th percentile: 2.265572690963745
Received healthy response to inference request in 2.677605390548706s
10th percentile: 2.216652750968933
30th percentile: 2.295184326171875
20th percentile: 2.300887203216553
40th percentile: 2.337172508239746
30th percentile: 2.3160770177841186
50th percentile: 2.4358432292938232
40th percentile: 2.3389081954956055
60th percentile: 2.5661258220672605
50th percentile: 2.3720386028289795
Received healthy response to inference request in 3.8927230834960938s
70th percentile: 2.637304401397705
60th percentile: 2.458756113052368
80th percentile: 2.715735197067261
70th percentile: 2.4695324897766113
90th percentile: 2.913827538490296
80th percentile: 2.6699295043945312
95th percentile: 3.3003724217414847
Received healthy response to inference request in 2.7397544384002686s
90th percentile: 3.035062694549561
99th percentile: 3.7843968343734744
95th percentile: 3.2204375386238095
mean time: 2.5464226802190146
99th percentile: 3.435004885196686
Pipeline stage StressChecker completed in 125.53s
mean time: 2.5034446239471437
Received healthy response to inference request in 2.373479127883911s
run pipeline stage %s
Pipeline stage StressChecker completed in 128.54s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run pipeline stage %s
run_pipeline:run_in_cloud %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
starting trigger_guanaco_pipeline args=%s
Received healthy response to inference request in 2.7235958576202393s
run_pipeline:run_in_cloud %s
triggered trigger_guanaco_pipeline args=%s
starting trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 5.41s
triggered trigger_guanaco_pipeline args=%s
Shutdown handler de-registered
Received healthy response to inference request in 2.6272830963134766s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 5.27s
chaiml-reward-dpo-7a6f-_69008_v2 status is now deployed due to DeploymentManager action
chaiml-reward-dpo-7a6f-_69008_v2 status is now inactive due to auto deactivation removed underperforming models
chaiml-reward-dpo-7a6f-_69008_v2 status is now torndown due to DeploymentManager action