Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.25s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-7b07-69d4-linear-87172-v2
Waiting for inference service chaiml-7b07-69d4-linear-87172-v2 to be ready
Inference service chaiml-7b07-69d4-linear-87172-v2 ready after 150.9857678413391s
Pipeline stage VLLMDeployer completed in 151.45s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 0.9466197490692139s
Received healthy response to inference request in 0.9666242599487305s
Received healthy response to inference request in 1.0447030067443848s
Received healthy response to inference request in 0.9302232265472412s
Received healthy response to inference request in 1.0988011360168457s
Received healthy response to inference request in 0.8515040874481201s
Received healthy response to inference request in 0.8458473682403564s
Received healthy response to inference request in 1.040964126586914s
Received healthy response to inference request in 1.1446318626403809s
Received healthy response to inference request in 0.9046416282653809s
Received healthy response to inference request in 0.9603488445281982s
Received healthy response to inference request in 0.8421111106872559s
Received healthy response to inference request in 1.0497634410858154s
Received healthy response to inference request in 0.9445147514343262s
Received healthy response to inference request in 1.078054666519165s
Received healthy response to inference request in 0.9703660011291504s
Received healthy response to inference request in 1.5075349807739258s
Received healthy response to inference request in 0.952423095703125s
Received healthy response to inference request in 1.0313849449157715s
Received healthy response to inference request in 0.9065322875976562s
Received healthy response to inference request in 0.9183411598205566s
Received healthy response to inference request in 0.8902804851531982s
Received healthy response to inference request in 0.8536829948425293s
Received healthy response to inference request in 1.072622299194336s
Received healthy response to inference request in 0.8752899169921875s
Received healthy response to inference request in 0.8874344825744629s
Received healthy response to inference request in 1.1808228492736816s
Received healthy response to inference request in 0.9430286884307861s
Received healthy response to inference request in 0.8666336536407471s
Received healthy response to inference request in 0.8749082088470459s
30 requests
0 failed requests
5th percentile: 0.8483928918838501
10th percentile: 0.8534651041030884
20th percentile: 0.8752135753631591
30th percentile: 0.9003332853317261
40th percentile: 0.9254703998565674
50th percentile: 0.94556725025177
60th percentile: 0.9628590106964111
70th percentile: 1.0342586994171141
80th percentile: 1.0543352127075196
90th percentile: 1.1033842086791992
95th percentile: 1.1645369052886962
99th percentile: 1.4127884626388552
mean time: 0.9793546438217163
Pipeline stage StressChecker completed in 32.25s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.61s
Shutdown handler de-registered
chaiml-7b07-69d4-linear_87172_v2 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Generating Leaderboard row for %s
Generated Leaderboard row for %s
Pipeline stage OfflineFamilyFriendlyScorer completed in 2109.54s
Shutdown handler de-registered
chaiml-7b07-69d4-linear_87172_v2 status is now protected due to ABTestQueueItem