Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.13s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-opusd-v1-noname-45151-v1
Waiting for inference service chaiml-opusd-v1-noname-45151-v1 to be ready
Inference service chaiml-opusd-v1-noname-45151-v1 ready after 523.446813583374s
Pipeline stage VLLMDeployer completed in 524.11s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.023329973220825s
Received healthy response to inference request in 1.854494571685791s
Received healthy response to inference request in 1.9042243957519531s
Received healthy response to inference request in 2.110337257385254s
Received healthy response to inference request in 1.9518773555755615s
Received healthy response to inference request in 1.8960158824920654s
Received healthy response to inference request in 1.764235019683838s
Received healthy response to inference request in 2.0860323905944824s
Received healthy response to inference request in 1.9825022220611572s
Received healthy response to inference request in 2.1633496284484863s
Received healthy response to inference request in 2.004115104675293s
Received healthy response to inference request in 1.7343087196350098s
Received healthy response to inference request in 1.9701414108276367s
Received healthy response to inference request in 1.8871159553527832s
Received healthy response to inference request in 1.7475695610046387s
Received healthy response to inference request in 1.9484379291534424s
Received healthy response to inference request in 1.903846263885498s
Received healthy response to inference request in 2.31018328666687s
Received healthy response to inference request in 2.1625990867614746s
Received healthy response to inference request in 1.8814585208892822s
Received healthy response to inference request in 2.0812325477600098s
Received healthy response to inference request in 1.9562807083129883s
Received healthy response to inference request in 1.8894219398498535s
Received healthy response to inference request in 1.9052619934082031s
Received healthy response to inference request in 1.887516736984253s
Received healthy response to inference request in 1.8699312210083008s
Received healthy response to inference request in 2.277430534362793s
Received healthy response to inference request in 1.8358542919158936s
Received healthy response to inference request in 2.384046792984009s
Received healthy response to inference request in 1.8493995666503906s
30 requests
0 failed requests
5th percentile: 1.7550690174102783
10th percentile: 1.828692364692688
20th percentile: 1.8668438911437988
30th percentile: 1.887396502494812
40th percentile: 1.900714111328125
50th percentile: 1.9268499612808228
60th percentile: 1.9618249893188477
70th percentile: 2.0098795652389527
80th percentile: 2.090893363952637
90th percentile: 2.174757719039917
95th percentile: 2.2954445481300354
99th percentile: 2.3626263761520385
mean time: 1.974085028966268
Pipeline stage StressChecker completed in 61.94s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.64s
Shutdown handler de-registered
chaiml-opusd-v1-noname-_45151_v1 status is now deployed due to DeploymentManager action
chaiml-opusd-v1-noname-_45151_v1 status is now inactive due to auto deactivation removed underperforming models
chaiml-opusd-v1-noname-_45151_v1 status is now torndown due to DeploymentManager action