Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.14s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-kimid-v8a-kimidv-94399-v6
Waiting for inference service chaiml-kimid-v8a-kimidv-94399-v6 to be ready
Inference service chaiml-kimid-v8a-kimidv-94399-v6 ready after 321.6551375389099s
Pipeline stage VLLMDeployer completed in 322.20s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.7537109851837158s
Received healthy response to inference request in 1.4975316524505615s
Received healthy response to inference request in 1.7964966297149658s
Received healthy response to inference request in 1.6416938304901123s
Received healthy response to inference request in 1.770735740661621s
Received healthy response to inference request in 1.8247926235198975s
Received healthy response to inference request in 1.5769765377044678s
Received healthy response to inference request in 1.6335301399230957s
Received healthy response to inference request in 1.6848819255828857s
Received healthy response to inference request in 1.490370750427246s
Received healthy response to inference request in 1.874035120010376s
Received healthy response to inference request in 1.6458752155303955s
Received healthy response to inference request in 1.7559678554534912s
Received healthy response to inference request in 1.7160139083862305s
Received healthy response to inference request in 1.6785378456115723s
Received healthy response to inference request in 1.6611104011535645s
Received healthy response to inference request in 1.5205333232879639s
Received healthy response to inference request in 1.636425495147705s
Received healthy response to inference request in 1.827162504196167s
Received healthy response to inference request in 1.8254597187042236s
Received healthy response to inference request in 1.6409521102905273s
Received healthy response to inference request in 1.5881035327911377s
Received healthy response to inference request in 1.526177167892456s
Received healthy response to inference request in 1.6235754489898682s
Received healthy response to inference request in 1.6604270935058594s
Received healthy response to inference request in 1.7026317119598389s
Received healthy response to inference request in 1.6893441677093506s
Received healthy response to inference request in 1.5285053253173828s
Received healthy response to inference request in 1.7995922565460205s
Received healthy response to inference request in 1.7386157512664795s
30 requests
0 failed requests
5th percentile: 1.5078824043273926
10th percentile: 1.5256127834320068
20th percentile: 1.5858781337738037
30th percentile: 1.6355568885803222
40th percentile: 1.6442026615142822
50th percentile: 1.6698241233825684
60th percentile: 1.6946591854095459
70th percentile: 1.7431443214416504
80th percentile: 1.7758879184722902
90th percentile: 1.82485933303833
95th percentile: 1.8263962507247924
99th percentile: 1.8604420614242554
mean time: 1.6769922256469727
Pipeline stage StressChecker completed in 52.75s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.59s
Shutdown handler de-registered
chaiml-kimid-v8a-kimidv_94399_v6 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Generating Leaderboard row for %s
Generated Leaderboard row for %s
Pipeline stage OfflineFamilyFriendlyScorer completed in 2158.01s
Shutdown handler de-registered
chaiml-kimid-v8a-kimidv_94399_v6 status is now torndown due to DeploymentManager action