Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.14s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-opusd-v2b-opusd-33883-v11
Waiting for inference service chaiml-opusd-v2b-opusd-33883-v11 to be ready
Inference service chaiml-opusd-v2b-opusd-33883-v11 ready after 553.0464718341827s
Pipeline stage VLLMDeployer completed in 553.61s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.153599500656128s
Received healthy response to inference request in 1.9005885124206543s
Received healthy response to inference request in 1.8899269104003906s
Received healthy response to inference request in 2.259661912918091s
Received healthy response to inference request in 2.0668840408325195s
Received healthy response to inference request in 1.987520456314087s
Received healthy response to inference request in 1.719794750213623s
Received healthy response to inference request in 1.8510079383850098s
Received healthy response to inference request in 1.9011180400848389s
Received healthy response to inference request in 2.0113558769226074s
Received healthy response to inference request in 2.449017286300659s
Received healthy response to inference request in 1.8204467296600342s
Received healthy response to inference request in 1.7849905490875244s
Received healthy response to inference request in 2.019186496734619s
Received healthy response to inference request in 1.6893632411956787s
Received healthy response to inference request in 2.1117312908172607s
Received healthy response to inference request in 1.6702547073364258s
Received healthy response to inference request in 1.954843282699585s
Received healthy response to inference request in 1.8181936740875244s
Received healthy response to inference request in 2.1156792640686035s
Received healthy response to inference request in 2.1086931228637695s
Received healthy response to inference request in 1.941908836364746s
Received healthy response to inference request in 2.031191349029541s
Received healthy response to inference request in 2.3503966331481934s
Received healthy response to inference request in 1.721696138381958s
Received healthy response to inference request in 2.249260902404785s
Received healthy response to inference request in 1.7815587520599365s
Received healthy response to inference request in 1.869553804397583s
Received healthy response to inference request in 2.1087777614593506s
Received healthy response to inference request in 1.7707948684692383s
30 requests
0 failed requests
5th percentile: 1.7030574202537536
10th percentile: 1.7215059995651245
20th percentile: 1.7843041896820069
30th percentile: 1.841839575767517
40th percentile: 1.8963238716125488
50th percentile: 1.9483760595321655
60th percentile: 2.014488124847412
70th percentile: 2.0794267654418945
80th percentile: 2.1125208854675295
90th percentile: 2.2503010034561157
95th percentile: 2.309566009044647
99th percentile: 2.4204172968864444
mean time: 1.9702998876571656
Pipeline stage StressChecker completed in 61.71s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.90s
Shutdown handler de-registered
chaiml-opusd-v2b-opusd_33883_v11 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Generating Leaderboard row for %s
Generated Leaderboard row for %s
Pipeline stage OfflineFamilyFriendlyScorer completed in 3095.95s
Shutdown handler de-registered
chaiml-opusd-v2b-opusd_33883_v11 status is now inactive due to auto deactivation removed underperforming models
chaiml-opusd-v2b-opusd_33883_v11 status is now torndown due to DeploymentManager action