Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.14s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-opusd-v1-q235-lr-57831-v2
Waiting for inference service chaiml-opusd-v1-q235-lr-57831-v2 to be ready
Failed to get response for submission chaiml-kimid-v8a-kimidv_94399_v1: HTTPConnectionPool(host='chaiml-kimid-v8a-kimidv-94399-v1-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Inference service chaiml-opusd-v1-q235-lr-57831-v2 ready after 221.68874168395996s
Pipeline stage VLLMDeployer completed in 222.20s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.9963874816894531s
Received healthy response to inference request in 1.8575420379638672s
Received healthy response to inference request in 1.9151287078857422s
Received healthy response to inference request in 1.9587757587432861s
Received healthy response to inference request in 2.8113982677459717s
Received healthy response to inference request in 2.1820902824401855s
Received healthy response to inference request in 2.078636884689331s
Received healthy response to inference request in 2.0799646377563477s
Received healthy response to inference request in 2.0475358963012695s
Received healthy response to inference request in 1.896024227142334s
Received healthy response to inference request in 2.056825876235962s
Received healthy response to inference request in 2.5984437465667725s
Received healthy response to inference request in 1.8031644821166992s
Received healthy response to inference request in 1.8362822532653809s
Received healthy response to inference request in 1.8975656032562256s
Received healthy response to inference request in 1.9766972064971924s
Received healthy response to inference request in 2.0948686599731445s
Received healthy response to inference request in 1.8278999328613281s
Received healthy response to inference request in 2.08790922164917s
Received healthy response to inference request in 1.9073565006256104s
Received healthy response to inference request in 2.0860393047332764s
Received healthy response to inference request in 1.996286392211914s
Received healthy response to inference request in 1.7513563632965088s
Received healthy response to inference request in 1.9731271266937256s
Received healthy response to inference request in 1.9089961051940918s
Received healthy response to inference request in 2.2648303508758545s
Received healthy response to inference request in 1.7784473896026611s
Received healthy response to inference request in 1.9038968086242676s
Received healthy response to inference request in 1.926689863204956s
Received healthy response to inference request in 1.9099230766296387s
30 requests
0 failed requests
5th percentile: 1.7895700812339783
10th percentile: 1.8254263877868653
20th percentile: 1.8883277893066406
30th percentile: 1.9063185930252076
40th percentile: 1.9130464553833009
50th percentile: 1.9659514427185059
60th percentile: 1.9963268280029296
70th percentile: 2.0633691787719726
80th percentile: 2.086413288116455
90th percentile: 2.1903642892837527
95th percentile: 2.448317718505858
99th percentile: 2.749641456604004
mean time: 2.0136696815490724
Pipeline stage StressChecker completed in 62.85s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.61s
Shutdown handler de-registered
chaiml-opusd-v1-q235-lr_57831_v2 status is now deployed due to DeploymentManager action
chaiml-opusd-v1-q235-lr_57831_v2 status is now inactive due to auto deactivation removed underperforming models
chaiml-opusd-v1-q235-lr_57831_v2 status is now torndown due to DeploymentManager action