Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.14s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-4d70-fd43-linear-w01-v25
Waiting for inference service chaiml-4d70-fd43-linear-w01-v25 to be ready
Inference service chaiml-4d70-fd43-linear-w01-v25 ready after 150.70362448692322s
Pipeline stage VLLMDeployer completed in 151.24s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.4864544868469238s
Received healthy response to inference request in 1.9637506008148193s
Received healthy response to inference request in 1.5999386310577393s
Received healthy response to inference request in 1.431464433670044s
Received healthy response to inference request in 1.4774434566497803s
Received healthy response to inference request in 1.456857681274414s
Received healthy response to inference request in 1.6123766899108887s
Received healthy response to inference request in 1.7756130695343018s
Received healthy response to inference request in 1.6014695167541504s
Received healthy response to inference request in 1.4878182411193848s
Received healthy response to inference request in 1.5109548568725586s
Received healthy response to inference request in 1.4462363719940186s
Received healthy response to inference request in 1.5137872695922852s
Received healthy response to inference request in 1.453312873840332s
Received healthy response to inference request in 1.4884319305419922s
Received healthy response to inference request in 1.548414707183838s
Received healthy response to inference request in 1.6222009658813477s
Received healthy response to inference request in 1.5028960704803467s
Received healthy response to inference request in 1.6837022304534912s
Received healthy response to inference request in 1.5862188339233398s
Received healthy response to inference request in 1.457716464996338s
Received healthy response to inference request in 1.4543402194976807s
Received healthy response to inference request in 1.4966418743133545s
Received healthy response to inference request in 1.464900016784668s
Received healthy response to inference request in 1.4561076164245605s
Received healthy response to inference request in 1.8807110786437988s
Received healthy response to inference request in 1.6502182483673096s
Received healthy response to inference request in 1.6611359119415283s
Received healthy response to inference request in 1.491959810256958s
Received healthy response to inference request in 1.4913458824157715s
30 requests
0 failed requests
5th percentile: 1.4494207978248597
10th percentile: 1.4542374849319457
20th percentile: 1.457544708251953
30th percentile: 1.4837511777877808
40th percentile: 1.4901803016662598
50th percentile: 1.4997689723968506
60th percentile: 1.5276382446289063
70th percentile: 1.6003978967666626
80th percentile: 1.6278044223785402
90th percentile: 1.6928933143615723
95th percentile: 1.8334169745445248
99th percentile: 1.9396691393852235
mean time: 1.5584806680679322
Pipeline stage StressChecker completed in 48.88s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.59s
Shutdown handler de-registered
chaiml-4d70-fd43-linear-w01_v25 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Generating Leaderboard row for %s
Generated Leaderboard row for %s
Pipeline stage OfflineFamilyFriendlyScorer completed in 2210.63s
Shutdown handler de-registered
chaiml-4d70-fd43-linear-w01_v25 status is now torndown due to DeploymentManager action