Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.13s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-opusd-v2b-opusd-33883-v14
Waiting for inference service chaiml-opusd-v2b-opusd-33883-v14 to be ready
Inference service chaiml-opusd-v2b-opusd-33883-v14 ready after 723.9907286167145s
Pipeline stage VLLMDeployer completed in 724.46s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.449763536453247s
Received healthy response to inference request in 1.897148609161377s
Received healthy response to inference request in 1.9187629222869873s
Received healthy response to inference request in 2.238859176635742s
Received healthy response to inference request in 1.8846447467803955s
Received healthy response to inference request in 2.1111347675323486s
Received healthy response to inference request in 1.883805274963379s
Received healthy response to inference request in 1.9133403301239014s
Received healthy response to inference request in 1.9844443798065186s
Received healthy response to inference request in 2.0194497108459473s
Received healthy response to inference request in 2.599405527114868s
Received healthy response to inference request in 1.935328722000122s
Received healthy response to inference request in 1.881723403930664s
Received healthy response to inference request in 1.9522643089294434s
Received healthy response to inference request in 2.1155166625976562s
Received healthy response to inference request in 1.9633476734161377s
Received healthy response to inference request in 1.9592623710632324s
Received healthy response to inference request in 2.0502145290374756s
Received healthy response to inference request in 1.953275203704834s
Received healthy response to inference request in 1.9681527614593506s
Received healthy response to inference request in 1.9485313892364502s
Received healthy response to inference request in 1.92775559425354s
Received healthy response to inference request in 1.9033939838409424s
Received healthy response to inference request in 2.016270875930786s
Received healthy response to inference request in 1.9796500205993652s
Received healthy response to inference request in 1.9813308715820312s
Received healthy response to inference request in 2.018249750137329s
Received healthy response to inference request in 1.9587604999542236s
Received healthy response to inference request in 1.9499986171722412s
Received healthy response to inference request in 1.9813122749328613s
30 requests
0 failed requests
5th percentile: 1.8841830372810364
10th percentile: 1.8958982229232788
20th percentile: 1.9176784038543702
30th percentile: 1.9445705890655518
40th percentile: 1.9528708457946777
50th percentile: 1.961305022239685
60th percentile: 1.9803149223327636
70th percentile: 1.9939923286437988
80th percentile: 2.025602674484253
90th percentile: 2.127850914001465
95th percentile: 2.3548565745353693
99th percentile: 2.556009349822998
mean time: 2.01150328318278
Pipeline stage StressChecker completed in 63.20s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.58s
Shutdown handler de-registered
chaiml-opusd-v2b-opusd_33883_v14 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
Generating Leaderboard row for %s
Generated Leaderboard row for %s
Pipeline stage OfflineFamilyFriendlyScorer completed in 3716.73s
Shutdown handler de-registered
chaiml-opusd-v2b-opusd_33883_v14 status is now inactive due to auto deactivation removed underperforming models
chaiml-opusd-v2b-opusd_33883_v14 status is now torndown due to DeploymentManager action