developer_uid: rirv938
submission_id: chaiml-reward-dpo-a6ba-_75525_v1
model_name: chaiml-reward-dpo-a6ba-_75525_v1
model_group: ChaiML/reward-dpo-a6ba-c
status: torndown
timestamp: 2026-02-02T09:48:27+00:00
num_battles: 11064
num_wins: 6513
celo_rating: 1367.49
family_friendly_score: 0.5094000000000001
family_friendly_standard_error: 0.007069818102327669
submission_type: basic
model_repo: ChaiML/reward-dpo-a6ba-chaiml-kimid-v8b-kimid_63800_v12-int4-mixed
model_architecture: Qwen3MoeForCausalLM
model_num_parameters: 18790207488.0
best_of: 4
max_input_tokens: 2048
max_output_tokens: 80
reward_model: default
display_name: chaiml-reward-dpo-a6ba-_75525_v1
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: ChaiML/reward-dpo-a6ba-chaiml-kimid-v8b-kimid_63800_v12-int4-mixed
model_size: 19B
ranking_group: single
us_pacific_date: 2026-01-30
win_ratio: 0.5886659436008677
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['</think>', '<|assistant|>', '</s>', '<|im_end|>', '<|user|>', '####'], 'max_input_tokens': 2048, 'best_of': 4, 'max_output_tokens': 80}
formatter: {'memory_template': "<|im_start|>system\n{bot_name}'s persona: {memory}<|im_end|>\n", 'prompt_template': '', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-reward-dpo-a6ba-75525-v1-uploader
Waiting for job on chaiml-reward-dpo-a6ba-75525-v1-uploader to finish
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-reward-dpo-6b53-25279-v1-uploader
Waiting for job on chaiml-reward-dpo-6b53-25279-v1-uploader to finish
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-reward-dpo-4670-64599-v1-uploader
Waiting for job on chaiml-reward-dpo-4670-64599-v1-uploader to finish
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMUploader
Starting job with name chaiml-kimid-v8b-kimid-63800-v27-uploader
Waiting for job on chaiml-kimid-v8b-kimid-63800-v27-uploader to finish
chaiml-reward-dpo-a6ba-75525-v1-uploader: /root/miniconda3/envs/nvidia/lib/python3.11/site-packages/mk1/__init__.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
chaiml-reward-dpo-a6ba-75525-v1-uploader:   __import__('pkg_resources').declare_namespace(__name__)
chaiml-reward-dpo-a6ba-75525-v1-uploader: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║                                                                     ║
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║   ██████   ██████  █████   ████  ████                               ║
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║   ░███░█████░███   ░███  ███     ░███                               ║
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║   ░███░░███ ░███   ░███████      ░███                               ║
chaiml-reward-dpo-6b53-25279-v1-uploader: /root/miniconda3/envs/nvidia/lib/python3.11/site-packages/mk1/__init__.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
chaiml-reward-dpo-6b53-25279-v1-uploader:   __import__('pkg_resources').declare_namespace(__name__)
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║   ░███      ░███   ░███ ░░███    ░███                               ║
chaiml-reward-dpo-6b53-25279-v1-uploader: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║   █████     █████  █████ ░░████  █████                              ║
chaiml-reward-dpo-6b53-25279-v1-uploader: ║                                                                     ║
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
chaiml-reward-dpo-6b53-25279-v1-uploader: ║   ██████   ██████  █████   ████  ████                               ║
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║                                                                     ║
chaiml-reward-dpo-6b53-25279-v1-uploader: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║  Version: 0.30.6+torch280                                           ║
chaiml-reward-dpo-6b53-25279-v1-uploader: ║   ░███░█████░███   ░███  ███     ░███                               ║
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║  Features: FLYWHEEL, CUDA                                           ║
chaiml-reward-dpo-4670-64599-v1-uploader: /root/miniconda3/envs/nvidia/lib/python3.11/site-packages/mk1/__init__.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
chaiml-reward-dpo-6b53-25279-v1-uploader: ║   ░███░░███ ░███   ░███████      ░███                               ║
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
chaiml-reward-dpo-4670-64599-v1-uploader:   __import__('pkg_resources').declare_namespace(__name__)
chaiml-reward-dpo-6b53-25279-v1-uploader: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║  https://mk1.ai                                                     ║
chaiml-reward-dpo-4670-64599-v1-uploader: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-reward-dpo-6b53-25279-v1-uploader: ║   ░███      ░███   ░███ ░░███    ░███                               ║
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║                                                                     ║
chaiml-reward-dpo-4670-64599-v1-uploader: ║                                                                     ║
chaiml-reward-dpo-6b53-25279-v1-uploader: ║   █████     █████  █████ ░░████  █████                              ║
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║  The license key for the current software has been verified as      ║
chaiml-reward-dpo-4670-64599-v1-uploader: ║   ██████   ██████  █████   ████  ████                               ║
chaiml-reward-dpo-6b53-25279-v1-uploader: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║  belonging to:                                                      ║
chaiml-reward-dpo-4670-64599-v1-uploader: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
chaiml-reward-dpo-6b53-25279-v1-uploader: ║                                                                     ║
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║                                                                     ║
chaiml-reward-dpo-4670-64599-v1-uploader: ║   ░███░█████░███   ░███  ███     ░███                               ║
chaiml-reward-dpo-6b53-25279-v1-uploader: ║  Version: 0.30.6+torch280                                           ║
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║  Chai Research Corp.                                                ║
chaiml-reward-dpo-4670-64599-v1-uploader: ║   ░███░░███ ░███   ░███████      ░███                               ║
chaiml-reward-dpo-6b53-25279-v1-uploader: ║  Features: FLYWHEEL, CUDA                                           ║
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-reward-dpo-4670-64599-v1-uploader: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
chaiml-reward-dpo-6b53-25279-v1-uploader: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║  Expiration: 2028-03-31 23:59:59                                    ║
chaiml-reward-dpo-4670-64599-v1-uploader: ║   ░███      ░███   ░███ ░░███    ░███                               ║
chaiml-reward-dpo-6b53-25279-v1-uploader: ║  https://mk1.ai                                                     ║
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║                                                                     ║
chaiml-reward-dpo-4670-64599-v1-uploader: ║   █████     █████  █████ ░░████  █████                              ║
chaiml-reward-dpo-6b53-25279-v1-uploader: ║                                                                     ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: /root/miniconda3/envs/nvidia/lib/python3.11/site-packages/mk1/__init__.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
chaiml-reward-dpo-a6ba-75525-v1-uploader: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-reward-dpo-4670-64599-v1-uploader: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
chaiml-reward-dpo-6b53-25279-v1-uploader: ║  The license key for the current software has been verified as      ║
chaiml-kimid-v8b-kimid-63800-v27-uploader:   __import__('pkg_resources').declare_namespace(__name__)
chaiml-reward-dpo-4670-64599-v1-uploader: ║                                                                     ║
chaiml-reward-dpo-6b53-25279-v1-uploader: ║  belonging to:                                                      ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-reward-dpo-4670-64599-v1-uploader: ║  Version: 0.30.6+torch280                                           ║
chaiml-reward-dpo-6b53-25279-v1-uploader: ║                                                                     ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: ║                                                                     ║
chaiml-reward-dpo-4670-64599-v1-uploader: ║  Features: FLYWHEEL, CUDA                                           ║
chaiml-reward-dpo-6b53-25279-v1-uploader: ║  Chai Research Corp.                                                ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: ║   ██████   ██████  █████   ████  ████                               ║
chaiml-reward-dpo-4670-64599-v1-uploader: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
chaiml-reward-dpo-6b53-25279-v1-uploader: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
chaiml-reward-dpo-4670-64599-v1-uploader: ║  https://mk1.ai                                                     ║
chaiml-reward-dpo-6b53-25279-v1-uploader: ║  Expiration: 2028-03-31 23:59:59                                    ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: ║   ░███░█████░███   ░███  ███     ░███                               ║
chaiml-reward-dpo-4670-64599-v1-uploader: ║                                                                     ║
chaiml-reward-dpo-6b53-25279-v1-uploader: ║                                                                     ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: ║   ░███░░███ ░███   ░███████      ░███                               ║
chaiml-reward-dpo-4670-64599-v1-uploader: ║  The license key for the current software has been verified as      ║
chaiml-reward-dpo-6b53-25279-v1-uploader: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-kimid-v8b-kimid-63800-v27-uploader: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
chaiml-reward-dpo-4670-64599-v1-uploader: ║  belonging to:                                                      ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: ║   ░███      ░███   ░███ ░░███    ░███                               ║
chaiml-reward-dpo-4670-64599-v1-uploader: ║                                                                     ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: ║   █████     █████  █████ ░░████  █████                              ║
chaiml-reward-dpo-4670-64599-v1-uploader: ║  Chai Research Corp.                                                ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
chaiml-reward-dpo-4670-64599-v1-uploader: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: ║                                                                     ║
chaiml-reward-dpo-4670-64599-v1-uploader: ║  Expiration: 2028-03-31 23:59:59                                    ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: ║  Version: 0.30.6+torch280                                           ║
chaiml-reward-dpo-4670-64599-v1-uploader: ║                                                                     ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: ║  Features: FLYWHEEL, CUDA                                           ║
chaiml-reward-dpo-4670-64599-v1-uploader: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-kimid-v8b-kimid-63800-v27-uploader: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: ║  https://mk1.ai                                                     ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: ║                                                                     ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: ║  The license key for the current software has been verified as      ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: ║  belonging to:                                                      ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: ║                                                                     ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: ║  Chai Research Corp.                                                ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: ║  Expiration: 2028-03-31 23:59:59                                    ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: ║                                                                     ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-reward-dpo-a6ba-75525-v1-uploader: Downloaded to shared memory in 124.268s
chaiml-reward-dpo-6b53-25279-v1-uploader: Downloaded to shared memory in 120.346s
chaiml-reward-dpo-4670-64599-v1-uploader: Downloaded to shared memory in 128.130s
chaiml-kimid-v8b-kimid-63800-v27-uploader: Downloaded to shared memory in 135.893s
chaiml-reward-dpo-6b53-25279-v1-uploader: Processed model ChaiML/reward-dpo-6b53-chaiml-kimid-v8b-kimid_63800_v12-int4-mixed in 179.668s
chaiml-reward-dpo-6b53-25279-v1-uploader: creating bucket guanaco-vllm-models
chaiml-reward-dpo-a6ba-75525-v1-uploader: Processed model ChaiML/reward-dpo-a6ba-chaiml-kimid-v8b-kimid_63800_v12-int4-mixed in 182.111s
chaiml-reward-dpo-a6ba-75525-v1-uploader: creating bucket guanaco-vllm-models
chaiml-reward-dpo-a6ba-75525-v1-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-reward-dpo-a6ba-75525-v1-uploader: uploading /dev/shm/model_cache to s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/.gitattributes s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/.gitattributes
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/chat_template.jinja s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/chat_template.jinja
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/added_tokens.json s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/added_tokens.json
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/generation_config.json s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/generation_config.json
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/tokenizer_config.json
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/special_tokens_map.json
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/config.json s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/config.json
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/quantization_config.json s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/quantization_config.json
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/merges.txt s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/merges.txt
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/vocab.json s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/vocab.json
chaiml-reward-dpo-4670-64599-v1-uploader: Processed model ChaiML/reward-dpo-4670-chaiml-kimid-v8b-kimid_63800_v12-int4-mixed in 185.726s
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/tokenizer.json
chaiml-reward-dpo-4670-64599-v1-uploader: creating bucket guanaco-vllm-models
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model.safetensors.index.json
chaiml-reward-dpo-4670-64599-v1-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-reward-dpo-4670-64599-v1-uploader: uploading /dev/shm/model_cache to s3://guanaco-vllm-models/chaiml-reward-dpo-4670-64599-v1
chaiml-reward-dpo-4670-64599-v1-uploader: cp /dev/shm/model_cache/added_tokens.json s3://guanaco-vllm-models/chaiml-reward-dpo-4670-64599-v1/added_tokens.json
chaiml-reward-dpo-4670-64599-v1-uploader: cp /dev/shm/model_cache/generation_config.json s3://guanaco-vllm-models/chaiml-reward-dpo-4670-64599-v1/generation_config.json
chaiml-reward-dpo-4670-64599-v1-uploader: cp /dev/shm/model_cache/chat_template.jinja s3://guanaco-vllm-models/chaiml-reward-dpo-4670-64599-v1/chat_template.jinja
chaiml-reward-dpo-4670-64599-v1-uploader: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-vllm-models/chaiml-reward-dpo-4670-64599-v1/tokenizer_config.json
chaiml-reward-dpo-4670-64599-v1-uploader: cp /dev/shm/model_cache/.gitattributes s3://guanaco-vllm-models/chaiml-reward-dpo-4670-64599-v1/.gitattributes
chaiml-kimid-v8b-kimid-63800-v27-uploader: Processed model ChaiML/kimid-v8b-kimidv5a-lr5e6ep2r64g4b01-int4-mixed in 189.283s
chaiml-reward-dpo-4670-64599-v1-uploader: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-vllm-models/chaiml-reward-dpo-4670-64599-v1/special_tokens_map.json
chaiml-kimid-v8b-kimid-63800-v27-uploader: creating bucket guanaco-vllm-models
chaiml-reward-dpo-4670-64599-v1-uploader: cp /dev/shm/model_cache/quantization_config.json s3://guanaco-vllm-models/chaiml-reward-dpo-4670-64599-v1/quantization_config.json
chaiml-reward-dpo-4670-64599-v1-uploader: cp /dev/shm/model_cache/config.json s3://guanaco-vllm-models/chaiml-reward-dpo-4670-64599-v1/config.json
chaiml-reward-dpo-4670-64599-v1-uploader: cp /dev/shm/model_cache/merges.txt s3://guanaco-vllm-models/chaiml-reward-dpo-4670-64599-v1/merges.txt
chaiml-reward-dpo-4670-64599-v1-uploader: cp /dev/shm/model_cache/vocab.json s3://guanaco-vllm-models/chaiml-reward-dpo-4670-64599-v1/vocab.json
chaiml-kimid-v8b-kimid-63800-v27-uploader: cp /dev/shm/model_cache/merges.txt s3://guanaco-vllm-models/chaiml-kimid-v8b-kimid-63800-v27/merges.txt
chaiml-kimid-v8b-kimid-63800-v27-uploader: cp /dev/shm/model_cache/vocab.json s3://guanaco-vllm-models/chaiml-kimid-v8b-kimid-63800-v27/vocab.json
chaiml-kimid-v8b-kimid-63800-v27-uploader: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-vllm-models/chaiml-kimid-v8b-kimid-63800-v27/tokenizer.json
chaiml-kimid-v8b-kimid-63800-v27-uploader: cp /dev/shm/model_cache/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-kimid-v8b-kimid-63800-v27/model.safetensors.index.json
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG retryable error: RequestError: send request failed
chaiml-reward-dpo-a6ba-75525-v1-uploader: caused by: Put "https://object.ord1.coreweave.com/guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00005-of-00027.safetensors?partNumber=3&uploadId=2~b1vdznOhs_vir0FoTuBIbhog0nqvFwQ": write tcp 10.0.5.197:36052->216.153.53.63:443: write: connection reset by peer
chaiml-reward-dpo-a6ba-75525-v1-uploader: ERROR "cp /dev/shm/model_cache/model-00005-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00005-of-00027.safetensors": MultipartUpload: upload multipart failed upload id: 2~b1vdznOhs_vir0FoTuBIbhog0nqvFwQ caused by: SignatureDoesNotMatch:  status code: 403, request id: tx00000ec493965c58d4c57-00697c5b4d-14a26732ad-default, host id:
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/model-00027-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00027-of-00027.safetensors
chaiml-reward-dpo-6b53-25279-v1-uploader: cp /dev/shm/model_cache/model-00027-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-6b53-25279-v1/model-00027-of-00027.safetensors
chaiml-reward-dpo-4670-64599-v1-uploader: cp /dev/shm/model_cache/model-00027-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-4670-64599-v1/model-00027-of-00027.safetensors
chaiml-kimid-v8b-kimid-63800-v27-uploader: cp /dev/shm/model_cache/model-00027-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimid-63800-v27/model-00027-of-00027.safetensors
chaiml-reward-dpo-4670-64599-v1-uploader: cp /dev/shm/model_cache/model-00012-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-4670-64599-v1/model-00012-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/model-00018-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00018-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/model-00011-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00011-of-00027.safetensors
chaiml-reward-dpo-4670-64599-v1-uploader: cp /dev/shm/model_cache/model-00018-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-4670-64599-v1/model-00018-of-00027.safetensors
chaiml-reward-dpo-4670-64599-v1-uploader: cp /dev/shm/model_cache/model-00019-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-4670-64599-v1/model-00019-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/model-00003-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00003-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/model-00001-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00001-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/model-00002-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00002-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/model-00014-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00014-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/model-00006-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00006-of-00027.safetensors
chaiml-reward-dpo-4670-64599-v1-uploader: cp /dev/shm/model_cache/model-00004-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-4670-64599-v1/model-00004-of-00027.safetensors
chaiml-reward-dpo-4670-64599-v1-uploader: cp /dev/shm/model_cache/model-00015-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-4670-64599-v1/model-00015-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/model-00008-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00008-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/model-00020-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00020-of-00027.safetensors
chaiml-reward-dpo-4670-64599-v1-uploader: cp /dev/shm/model_cache/model-00010-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-4670-64599-v1/model-00010-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/model-00017-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00017-of-00027.safetensors
chaiml-reward-dpo-6b53-25279-v1-uploader: cp /dev/shm/model_cache/model-00015-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-6b53-25279-v1/model-00015-of-00027.safetensors
chaiml-reward-dpo-4670-64599-v1-uploader: cp /dev/shm/model_cache/model-00017-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-4670-64599-v1/model-00017-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/model-00019-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00019-of-00027.safetensors
chaiml-reward-dpo-6b53-25279-v1-uploader: cp /dev/shm/model_cache/model-00014-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-6b53-25279-v1/model-00014-of-00027.safetensors
chaiml-reward-dpo-4670-64599-v1-uploader: cp /dev/shm/model_cache/model-00022-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-4670-64599-v1/model-00022-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/model-00007-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00007-of-00027.safetensors
chaiml-reward-dpo-4670-64599-v1-uploader: cp /dev/shm/model_cache/model-00014-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-4670-64599-v1/model-00014-of-00027.safetensors
chaiml-reward-dpo-6b53-25279-v1-uploader: cp /dev/shm/model_cache/model-00020-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-6b53-25279-v1/model-00020-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/model-00023-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00023-of-00027.safetensors
chaiml-reward-dpo-6b53-25279-v1-uploader: cp /dev/shm/model_cache/model-00025-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-6b53-25279-v1/model-00025-of-00027.safetensors
chaiml-reward-dpo-4670-64599-v1-uploader: cp /dev/shm/model_cache/model-00011-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-4670-64599-v1/model-00011-of-00027.safetensors
chaiml-reward-dpo-6b53-25279-v1-uploader: cp /dev/shm/model_cache/model-00017-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-6b53-25279-v1/model-00017-of-00027.safetensors
chaiml-reward-dpo-4670-64599-v1-uploader: cp /dev/shm/model_cache/model-00021-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-4670-64599-v1/model-00021-of-00027.safetensors
chaiml-reward-dpo-6b53-25279-v1-uploader: cp /dev/shm/model_cache/model-00008-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-6b53-25279-v1/model-00008-of-00027.safetensors
chaiml-reward-dpo-4670-64599-v1-uploader: cp /dev/shm/model_cache/model-00016-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-4670-64599-v1/model-00016-of-00027.safetensors
chaiml-reward-dpo-4670-64599-v1-uploader: cp /dev/shm/model_cache/model-00002-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-4670-64599-v1/model-00002-of-00027.safetensors
chaiml-reward-dpo-4670-64599-v1-uploader: cp /dev/shm/model_cache/model-00013-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-4670-64599-v1/model-00013-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/model-00022-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00022-of-00027.safetensors
chaiml-reward-dpo-6b53-25279-v1-uploader: cp /dev/shm/model_cache/model-00016-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-6b53-25279-v1/model-00016-of-00027.safetensors
chaiml-reward-dpo-6b53-25279-v1-uploader: cp /dev/shm/model_cache/model-00018-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-6b53-25279-v1/model-00018-of-00027.safetensors
chaiml-reward-dpo-4670-64599-v1-uploader: cp /dev/shm/model_cache/model-00025-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-4670-64599-v1/model-00025-of-00027.safetensors
chaiml-reward-dpo-4670-64599-v1-uploader: cp /dev/shm/model_cache/model-00020-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-4670-64599-v1/model-00020-of-00027.safetensors
chaiml-reward-dpo-4670-64599-v1-uploader: cp /dev/shm/model_cache/model-00005-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-4670-64599-v1/model-00005-of-00027.safetensors
chaiml-reward-dpo-4670-64599-v1-uploader: cp /dev/shm/model_cache/model-00009-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-4670-64599-v1/model-00009-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/model-00004-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00004-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/model-00015-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00015-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/model-00021-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00021-of-00027.safetensors
chaiml-reward-dpo-6b53-25279-v1-uploader: cp /dev/shm/model_cache/model-00022-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-6b53-25279-v1/model-00022-of-00027.safetensors
chaiml-reward-dpo-6b53-25279-v1-uploader: cp /dev/shm/model_cache/model-00004-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-6b53-25279-v1/model-00004-of-00027.safetensors
chaiml-reward-dpo-6b53-25279-v1-uploader: cp /dev/shm/model_cache/model-00005-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-6b53-25279-v1/model-00005-of-00027.safetensors
chaiml-reward-dpo-6b53-25279-v1-uploader: cp /dev/shm/model_cache/model-00019-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-6b53-25279-v1/model-00019-of-00027.safetensors
chaiml-reward-dpo-6b53-25279-v1-uploader: cp /dev/shm/model_cache/model-00007-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-6b53-25279-v1/model-00007-of-00027.safetensors
chaiml-reward-dpo-4670-64599-v1-uploader: cp /dev/shm/model_cache/model-00008-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-4670-64599-v1/model-00008-of-00027.safetensors
chaiml-reward-dpo-6b53-25279-v1-uploader: cp /dev/shm/model_cache/model-00003-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-6b53-25279-v1/model-00003-of-00027.safetensors
chaiml-reward-dpo-6b53-25279-v1-uploader: cp /dev/shm/model_cache/model-00011-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-6b53-25279-v1/model-00011-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/model-00016-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00016-of-00027.safetensors
chaiml-reward-dpo-4670-64599-v1-uploader: cp /dev/shm/model_cache/model-00023-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-4670-64599-v1/model-00023-of-00027.safetensors
chaiml-reward-dpo-4670-64599-v1-uploader: cp /dev/shm/model_cache/model-00026-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-4670-64599-v1/model-00026-of-00027.safetensors
chaiml-reward-dpo-6b53-25279-v1-uploader: cp /dev/shm/model_cache/model-00026-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-6b53-25279-v1/model-00026-of-00027.safetensors
chaiml-reward-dpo-6b53-25279-v1-uploader: cp /dev/shm/model_cache/model-00010-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-6b53-25279-v1/model-00010-of-00027.safetensors
chaiml-reward-dpo-6b53-25279-v1-uploader: cp /dev/shm/model_cache/model-00012-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-6b53-25279-v1/model-00012-of-00027.safetensors
chaiml-reward-dpo-6b53-25279-v1-uploader: cp /dev/shm/model_cache/model-00013-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-6b53-25279-v1/model-00013-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/model-00013-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00013-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/model-00012-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00012-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/model-00026-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00026-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/model-00010-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00010-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/model-00024-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00024-of-00027.safetensors
chaiml-reward-dpo-4670-64599-v1-uploader: cp /dev/shm/model_cache/model-00024-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-4670-64599-v1/model-00024-of-00027.safetensors
chaiml-kimid-v8b-kimid-63800-v27-uploader: cp /dev/shm/model_cache/model-00014-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimid-63800-v27/model-00014-of-00027.safetensors
chaiml-kimid-v8b-kimid-63800-v27-uploader: cp /dev/shm/model_cache/model-00016-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimid-63800-v27/model-00016-of-00027.safetensors
chaiml-reward-dpo-6b53-25279-v1-uploader: cp /dev/shm/model_cache/model-00024-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-6b53-25279-v1/model-00024-of-00027.safetensors
chaiml-reward-dpo-6b53-25279-v1-uploader: cp /dev/shm/model_cache/model-00021-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-6b53-25279-v1/model-00021-of-00027.safetensors
chaiml-reward-dpo-6b53-25279-v1-uploader: cp /dev/shm/model_cache/model-00001-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-6b53-25279-v1/model-00001-of-00027.safetensors
chaiml-reward-dpo-6b53-25279-v1-uploader: cp /dev/shm/model_cache/model-00002-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-6b53-25279-v1/model-00002-of-00027.safetensors
chaiml-reward-dpo-6b53-25279-v1-uploader: cp /dev/shm/model_cache/model-00006-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-6b53-25279-v1/model-00006-of-00027.safetensors
chaiml-reward-dpo-6b53-25279-v1-uploader: cp /dev/shm/model_cache/model-00009-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-6b53-25279-v1/model-00009-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: bash: cannot set terminal process group (-1): Inappropriate ioctl for device
chaiml-reward-dpo-a6ba-75525-v1-uploader: bash: no job control in this shell
chaiml-reward-dpo-a6ba-75525-v1-uploader: /root/miniconda3/envs/nvidia/lib/python3.11/site-packages/mk1/__init__.py:1: UserWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html. The pkg_resources package is slated for removal as early as 2025-11-30. Refrain from using this package or pin to Setuptools<81.
chaiml-reward-dpo-a6ba-75525-v1-uploader:   __import__('pkg_resources').declare_namespace(__name__)
chaiml-reward-dpo-4670-64599-v1-uploader: cp /dev/shm/model_cache/model-00006-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-4670-64599-v1/model-00006-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║                                                                     ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: cp /dev/shm/model_cache/model-00019-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimid-63800-v27/model-00019-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║   ██████   ██████  █████   ████  ████                               ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: cp /dev/shm/model_cache/model-00013-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimid-63800-v27/model-00013-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: cp /dev/shm/model_cache/model-00009-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimid-63800-v27/model-00009-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║   ░███░█████░███   ░███  ███     ░███                               ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: cp /dev/shm/model_cache/model-00008-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimid-63800-v27/model-00008-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║   ░███░░███ ░███   ░███████      ░███                               ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: cp /dev/shm/model_cache/model-00020-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimid-63800-v27/model-00020-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
chaiml-reward-dpo-6b53-25279-v1-uploader: cp /dev/shm/model_cache/model-00023-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-6b53-25279-v1/model-00023-of-00027.safetensors
chaiml-kimid-v8b-kimid-63800-v27-uploader: cp /dev/shm/model_cache/model-00022-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimid-63800-v27/model-00022-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║   ░███      ░███   ░███ ░░███    ░███                               ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: cp /dev/shm/model_cache/model-00026-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimid-63800-v27/model-00026-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║   █████     █████  █████ ░░████  █████                              ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: cp /dev/shm/model_cache/model-00012-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimid-63800-v27/model-00012-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: cp /dev/shm/model_cache/model-00003-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimid-63800-v27/model-00003-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║                                                                     ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: cp /dev/shm/model_cache/model-00017-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimid-63800-v27/model-00017-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║  Version: 0.30.6+torch280                                           ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: cp /dev/shm/model_cache/model-00015-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimid-63800-v27/model-00015-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║  Features: FLYWHEEL, CUDA                                           ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: cp /dev/shm/model_cache/model-00024-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimid-63800-v27/model-00024-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: cp /dev/shm/model_cache/model-00025-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimid-63800-v27/model-00025-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║  https://mk1.ai                                                     ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: cp /dev/shm/model_cache/model-00004-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimid-63800-v27/model-00004-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║                                                                     ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: cp /dev/shm/model_cache/model-00010-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimid-63800-v27/model-00010-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║  The license key for the current software has been verified as      ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: cp /dev/shm/model_cache/model-00006-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimid-63800-v27/model-00006-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║  belonging to:                                                      ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: cp /dev/shm/model_cache/model-00001-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimid-63800-v27/model-00001-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║                                                                     ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: cp /dev/shm/model_cache/model-00018-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimid-63800-v27/model-00018-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║  Chai Research Corp.                                                ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: cp /dev/shm/model_cache/model-00002-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimid-63800-v27/model-00002-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: cp /dev/shm/model_cache/model-00023-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimid-63800-v27/model-00023-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║  Expiration: 2028-03-31 23:59:59                                    ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: cp /dev/shm/model_cache/model-00021-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimid-63800-v27/model-00021-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: ║                                                                     ║
chaiml-kimid-v8b-kimid-63800-v27-uploader: cp /dev/shm/model_cache/model-00011-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimid-63800-v27/model-00011-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-kimid-v8b-kimid-63800-v27-uploader: cp /dev/shm/model_cache/model-00005-of-00027.safetensors s3://guanaco-vllm-models/chaiml-kimid-v8b-kimid-63800-v27/model-00005-of-00027.safetensors
Job chaiml-reward-dpo-a6ba-75525-v1-uploader completed after 858.84s with status: failed
Stopping job with name chaiml-reward-dpo-a6ba-75525-v1-uploader
%s, retrying in %s seconds...
Starting job with name chaiml-reward-dpo-a6ba-75525-v1-uploader
Waiting for job on chaiml-reward-dpo-a6ba-75525-v1-uploader to finish
Job chaiml-reward-dpo-4670-64599-v1-uploader completed after 862.85s with status: succeeded
Stopping job with name chaiml-reward-dpo-4670-64599-v1-uploader
Pipeline stage VLLMUploader completed in 865.16s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Job chaiml-kimid-v8b-kimid-63800-v27-uploader completed after 862.19s with status: succeeded
Pipeline stage VLLMTemplater completed in 0.69s
Stopping job with name chaiml-kimid-v8b-kimid-63800-v27-uploader
run pipeline stage %s
Running pipeline stage VLLMDeployer
Job chaiml-reward-dpo-6b53-25279-v1-uploader completed after 869.01s with status: succeeded
Pipeline stage VLLMUploader completed in 865.03s
Creating inference service chaiml-reward-dpo-4670-64599-v1
Stopping job with name chaiml-reward-dpo-6b53-25279-v1-uploader
run pipeline stage %s
Waiting for inference service chaiml-reward-dpo-4670-64599-v1 to be ready
Pipeline stage VLLMUploader completed in 871.61s
Running pipeline stage VLLMTemplater
run pipeline stage %s
Pipeline stage VLLMTemplater completed in 1.07s
Running pipeline stage VLLMTemplater
run pipeline stage %s
Pipeline stage VLLMTemplater completed in 0.68s
Running pipeline stage VLLMDeployer
run pipeline stage %s
Creating inference service chaiml-kimid-v8b-kimid-63800-v27
Running pipeline stage VLLMDeployer
Waiting for inference service chaiml-kimid-v8b-kimid-63800-v27 to be ready
Creating inference service chaiml-reward-dpo-6b53-25279-v1
Waiting for inference service chaiml-reward-dpo-6b53-25279-v1 to be ready
chaiml-reward-dpo-a6ba-75525-v1-uploader: Downloaded to shared memory in 112.405s
chaiml-reward-dpo-a6ba-75525-v1-uploader: Processed model ChaiML/reward-dpo-a6ba-chaiml-kimid-v8b-kimid_63800_v12-int4-mixed in 172.697s
chaiml-reward-dpo-a6ba-75525-v1-uploader: creating bucket guanaco-vllm-models
chaiml-reward-dpo-a6ba-75525-v1-uploader: Bucket 's3://guanaco-vllm-models/' created
chaiml-reward-dpo-a6ba-75525-v1-uploader: uploading /dev/shm/model_cache to s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/.gitattributes s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/.gitattributes": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/added_tokens.json s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/added_tokens.json": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/chat_template.jinja s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/chat_template.jinja": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/config.json s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/config.json": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/generation_config.json s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/generation_config.json": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/merges.txt s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/merges.txt": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/model-00001-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00001-of-00027.safetensors": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/model-00002-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00002-of-00027.safetensors": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/model-00003-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00003-of-00027.safetensors": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/model-00004-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00004-of-00027.safetensors": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/model-00006-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00006-of-00027.safetensors": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/model-00007-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00007-of-00027.safetensors": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/model-00008-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00008-of-00027.safetensors": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/model-00009-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00009-of-00027.safetensors": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/model-00010-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00010-of-00027.safetensors": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/model-00011-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00011-of-00027.safetensors": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/model-00012-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00012-of-00027.safetensors": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/model-00013-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00013-of-00027.safetensors": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/model-00014-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00014-of-00027.safetensors": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/model-00015-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00015-of-00027.safetensors": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/model-00016-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00016-of-00027.safetensors": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/model-00017-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00017-of-00027.safetensors": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/model-00018-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00018-of-00027.safetensors": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/model-00019-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00019-of-00027.safetensors": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/model-00020-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00020-of-00027.safetensors": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/model-00021-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00021-of-00027.safetensors": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/model-00022-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00022-of-00027.safetensors": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/model-00023-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00023-of-00027.safetensors": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/model-00024-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00024-of-00027.safetensors": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/model-00026-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00026-of-00027.safetensors": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/model-00027-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00027-of-00027.safetensors": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/model.safetensors.index.json s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model.safetensors.index.json": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/quantization_config.json s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/quantization_config.json": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/special_tokens_map.json s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/special_tokens_map.json": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/tokenizer.json s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/tokenizer.json": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/tokenizer_config.json s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/tokenizer_config.json": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: DEBUG "sync /dev/shm/model_cache/vocab.json s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/vocab.json": object size matches
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/model-00005-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00005-of-00027.safetensors
chaiml-reward-dpo-a6ba-75525-v1-uploader: cp /dev/shm/model_cache/model-00025-of-00027.safetensors s3://guanaco-vllm-models/chaiml-reward-dpo-a6ba-75525-v1/model-00025-of-00027.safetensors
Job chaiml-reward-dpo-a6ba-75525-v1-uploader completed after 242.29s with status: succeeded
Stopping job with name chaiml-reward-dpo-a6ba-75525-v1-uploader
Pipeline stage VLLMUploader completed in 1106.21s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.72s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-reward-dpo-a6ba-75525-v1
Waiting for inference service chaiml-reward-dpo-a6ba-75525-v1 to be ready
Inference service chaiml-reward-dpo-4670-64599-v1 ready after 588.1797688007355s
Pipeline stage VLLMDeployer completed in 591.04s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.805509090423584s
Received healthy response to inference request in 2.262218713760376s
Received healthy response to inference request in 2.2196145057678223s
Received healthy response to inference request in 2.2807090282440186s
Received healthy response to inference request in 2.3828115463256836s
Received healthy response to inference request in 2.3218295574188232s
Inference service chaiml-kimid-v8b-kimid-63800-v27 ready after 608.4902970790863s
Inference service chaiml-reward-dpo-6b53-25279-v1 ready after 608.3236565589905s
Pipeline stage VLLMDeployer completed in 611.19s
Pipeline stage VLLMDeployer completed in 611.95s
run pipeline stage %s
run pipeline stage %s
Running pipeline stage StressChecker
Running pipeline stage StressChecker
Received healthy response to inference request in 8.39267873764038s
Received healthy response to inference request in 2.617608070373535s
Received healthy response to inference request in 2.4935197830200195s
Received healthy response to inference request in 2.3355448246002197s
Received healthy response to inference request in 2.4723105430603027s
Received healthy response to inference request in 2.2751638889312744s
Received healthy response to inference request in 2.145624876022339s
Received healthy response to inference request in 2.1326987743377686s
Received healthy response to inference request in 2.703773260116577s
Received healthy response to inference request in 2.293989896774292s
Received healthy response to inference request in 2.6163554191589355s
Received healthy response to inference request in 2.312918186187744s
Received healthy response to inference request in 2.324244737625122s
Received healthy response to inference request in 2.3087949752807617s
Received healthy response to inference request in 2.214679718017578s
Received healthy response to inference request in 2.3168790340423584s
Received healthy response to inference request in 2.198876142501831s
Received healthy response to inference request in 2.3826229572296143s
Received healthy response to inference request in 2.2107768058776855s
Received healthy response to inference request in 2.348483085632324s
Received healthy response to inference request in 2.2884066104888916s
Received healthy response to inference request in 2.3398263454437256s
Received healthy response to inference request in 2.942676067352295s
Received healthy response to inference request in 2.714766025543213s
Received healthy response to inference request in 2.4928386211395264s
Received healthy response to inference request in 2.239535093307495s
Received healthy response to inference request in 2.544346332550049s
Received healthy response to inference request in 2.3510565757751465s
Received healthy response to inference request in 2.392852306365967s
Received healthy response to inference request in 2.2388439178466797s
Received healthy response to inference request in 2.360380172729492s
Received healthy response to inference request in 2.204347610473633s
Received healthy response to inference request in 2.2029507160186768s
Received healthy response to inference request in 2.294111967086792s
Received healthy response to inference request in 2.371666193008423s
Received healthy response to inference request in 2.281301736831665s
Received healthy response to inference request in 2.9072651863098145s
Received healthy response to inference request in 2.173752546310425s
Received healthy response to inference request in 2.0659139156341553s
Received healthy response to inference request in 2.1516218185424805s
Received healthy response to inference request in 2.4409542083740234s
Received healthy response to inference request in 2.400339126586914s
Received healthy response to inference request in 2.3671841621398926s
Received healthy response to inference request in 2.6967358589172363s
Received healthy response to inference request in 2.528903007507324s
Received healthy response to inference request in 2.165403366088867s
Received healthy response to inference request in 2.1463418006896973s
Received healthy response to inference request in 2.182648181915283s
Received healthy response to inference request in 2.207146167755127s
Received healthy response to inference request in 2.3566842079162598s
Received healthy response to inference request in 2.2280209064483643s
Received healthy response to inference request in 2.4318597316741943s
Received healthy response to inference request in 2.240870952606201s
Received healthy response to inference request in 2.1637847423553467s
Received healthy response to inference request in 2.1225671768188477s
Received healthy response to inference request in 2.186901569366455s
Received healthy response to inference request in 2.1699867248535156s
Received healthy response to inference request in 2.284458637237549s
Received healthy response to inference request in 2.209665536880493s
Received healthy response to inference request in 2.1958811283111572s
Received healthy response to inference request in 2.808636426925659s
Received healthy response to inference request in 2.3458175659179688s
Received healthy response to inference request in 2.089926242828369s
Received healthy response to inference request in 2.1643226146698s
Received healthy response to inference request in 2.2051608562469482s
Received healthy response to inference request in 2.911195755004883s
Received healthy response to inference request in 2.518831968307495s
Received healthy response to inference request in 2.630655527114868s
Received healthy response to inference request in 2.211989641189575s
Received healthy response to inference request in 2.3259329795837402s
30 requests
0 failed requests
5th percentile: 2.1329431414604185
Received healthy response to inference request in 2.128941774368286s
10th percentile: 2.151022124290466
20th percentile: 2.204893159866333
Received healthy response to inference request in 2.342273712158203s
30th percentile: 2.217327046394348
40th percentile: 2.2829587936401365
Received healthy response to inference request in 2.112535238265991s
50th percentile: 2.305495500564575
60th percentile: 2.328764772415161
Received healthy response to inference request in 2.2052040100097656s
70th percentile: 2.3538536548614504
80th percentile: 2.392621183395386
Received healthy response to inference request in 2.478806495666504s
90th percentile: 2.8156847000122074
95th percentile: 2.9094269990921022
Received healthy response to inference request in 2.1287729740142822s
99th percentile: 6.803048672676091
mean time: 2.539238135019938
Received healthy response to inference request in 2.340813159942627s
Pipeline stage StressChecker completed in 108.43s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
Received healthy response to inference request in 2.2956345081329346s
run_pipeline:run_in_cloud %s
Received healthy response to inference request in 2.0723202228546143s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Received healthy response to inference request in 2.184652090072632s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 3.22s
Received healthy response to inference request in 2.236638307571411s
Shutdown handler de-registered
chaiml-reward-dpo-4670-_64599_v1 status is now deployed due to DeploymentManager action
Received healthy response to inference request in 2.200474739074707s
Received healthy response to inference request in 2.180067777633667s
30 requests
0 failed requests
5th percentile: 2.119918179512024
10th percentile: 2.1446017980575562
Received healthy response to inference request in 2.237600803375244s
20th percentile: 2.1855348110198975
30 requests
30th percentile: 2.20831413269043
0 failed requests
40th percentile: 2.2383763790130615
5th percentile: 2.130539584159851
50th percentile: 2.300662398338318
10th percentile: 2.160676145553589
60th percentile: 2.350164222717285
20th percentile: 2.18011589050293
70th percentile: 2.38569176197052
30th percentile: 2.1999951601028442
80th percentile: 2.473609733581543
40th percentile: 2.2188941478729247
90th percentile: 2.637263560295105
50th percentile: 2.257003903388977
95th percentile: 2.7006064295768737
60th percentile: 2.3008986949920653
99th percentile: 2.7115781235694887
70th percentile: 2.3441365242004393
mean time: 2.336638339360555
80th percentile: 2.4985822200775147
Pipeline stage StressChecker completed in 99.59s
90th percentile: 2.5515472412109377
run pipeline stage %s
95th percentile: 2.722109973430633
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
99th percentile: 2.9038045716285708
run_pipeline:run_in_cloud %s
mean time: 2.3245314757029214
starting trigger_guanaco_pipeline args=%s
Pipeline stage StressChecker completed in 101.85s
run pipeline stage %s
triggered trigger_guanaco_pipeline args=%s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.80s
run_pipeline:run_in_cloud %s
Shutdown handler de-registered
starting trigger_guanaco_pipeline args=%s
chaiml-kimid-v8b-kimid_63800_v27 status is now deployed due to DeploymentManager action
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.56s
Shutdown handler de-registered
chaiml-reward-dpo-6b53-_25279_v1 status is now deployed due to DeploymentManager action
Inference service chaiml-reward-dpo-a6ba-75525-v1 ready after 517.448935508728s
Pipeline stage VLLMDeployer completed in 519.62s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 7.6563193798065186s
Received healthy response to inference request in 2.3510701656341553s
Received healthy response to inference request in 2.2995412349700928s
Received healthy response to inference request in 2.1910247802734375s
Received healthy response to inference request in 2.543935537338257s
Received healthy response to inference request in 7.0671961307525635s
Received healthy response to inference request in 2.392864465713501s
Received healthy response to inference request in 2.04722261428833s
Received healthy response to inference request in 2.0853240489959717s
Received healthy response to inference request in 2.2114522457122803s
Received healthy response to inference request in 2.35878849029541s
Received healthy response to inference request in 2.112840414047241s
Received healthy response to inference request in 2.2643802165985107s
Received healthy response to inference request in 2.3186826705932617s
Received healthy response to inference request in 2.160353422164917s
Received healthy response to inference request in 2.2804782390594482s
Received healthy response to inference request in 2.4217121601104736s
Received healthy response to inference request in 2.2999179363250732s
Received healthy response to inference request in 2.1191961765289307s
Received healthy response to inference request in 2.6227004528045654s
Received healthy response to inference request in 2.496288776397705s
Received healthy response to inference request in 2.4441018104553223s
Received healthy response to inference request in 2.726039409637451s
Received healthy response to inference request in 2.548447608947754s
Received healthy response to inference request in 2.582798719406128s
Received healthy response to inference request in 2.247596263885498s
Received healthy response to inference request in 2.434319019317627s
Received healthy response to inference request in 2.557506561279297s
Received healthy response to inference request in 2.3846166133880615s
Received healthy response to inference request in 2.1490352153778076s
30 requests
0 failed requests
5th percentile: 2.097706413269043
10th percentile: 2.1185606002807615
20th percentile: 2.1848905086517334
30th percentile: 2.259345030784607
40th percentile: 2.299767255783081
50th percentile: 2.3549293279647827
60th percentile: 2.40440354347229
70th percentile: 2.4597579002380368
80th percentile: 2.5502593994140623
90th percentile: 2.633034348487854
95th percentile: 5.113675606250751
99th percentile: 7.485473637580872
mean time: 2.6791916926701864
Pipeline stage StressChecker completed in 91.70s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 1.37s
Shutdown handler de-registered
chaiml-reward-dpo-a6ba-_75525_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Generating Leaderboard row for %s
Generated Leaderboard row for %s
Pipeline stage OfflineFamilyFriendlyScorer completed in 2694.95s
Shutdown handler de-registered
chaiml-reward-dpo-a6ba-_75525_v1 status is now inactive due to auto deactivation removed underperforming models
chaiml-reward-dpo-a6ba-_75525_v1 status is now torndown due to DeploymentManager action