Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.64s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-kimid-v8b-kimidv-63800-v6
Waiting for inference service chaiml-kimid-v8b-kimidv-63800-v6 to be ready
HTTP Request: %s %s "%s %d %s"
HTTP Request: %s %s "%s %d %s"
HTTP Request: %s %s "%s %d %s"
Inference service chaiml-kimid-v8b-kimidv-63800-v6 ready after 350.24382758140564s
Pipeline stage VLLMDeployer completed in 350.61s
run pipeline stage %s
Running pipeline stage StressChecker
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.8961901664733887s
Received healthy response to inference request in 2.0644545555114746s
Received healthy response to inference request in 1.8877911567687988s
Received healthy response to inference request in 1.8857765197753906s
Received healthy response to inference request in 2.325627326965332s
Received healthy response to inference request in 1.8663384914398193s
Received healthy response to inference request in 1.6821835041046143s
Received healthy response to inference request in 1.8339314460754395s
Received healthy response to inference request in 1.921649694442749s
Received healthy response to inference request in 2.409177541732788s
Received healthy response to inference request in 1.7534270286560059s
Received healthy response to inference request in 2.194232702255249s
Received healthy response to inference request in 2.489866018295288s
Received healthy response to inference request in 1.7583515644073486s
Received healthy response to inference request in 1.836566686630249s
Received healthy response to inference request in 2.0420339107513428s
Received healthy response to inference request in 3.404188394546509s
Received healthy response to inference request in 1.758979082107544s
Received healthy response to inference request in 2.815452814102173s
Received healthy response to inference request in 1.8217980861663818s
Received healthy response to inference request in 1.883896827697754s
Received healthy response to inference request in 1.9370057582855225s
Received healthy response to inference request in 2.0909862518310547s
Received healthy response to inference request in 1.7085397243499756s
Received healthy response to inference request in 1.8310778141021729s
Received healthy response to inference request in 2.5047965049743652s
Received healthy response to inference request in 1.8479561805725098s
Received healthy response to inference request in 2.0401391983032227s
Received healthy response to inference request in 1.89457106590271s
Received healthy response to inference request in 1.6919755935668945s
30 requests
0 failed requests
5th percentile: 1.699429452419281
10th percentile: 1.748938298225403
20th percentile: 1.8092342853546144
30th percentile: 1.835776114463806
40th percentile: 1.87687349319458
50th percentile: 1.8911811113357544
60th percentile: 1.9277921199798584
70th percentile: 2.048760104179382
80th percentile: 2.220511627197266
90th percentile: 2.491359066963196
95th percentile: 2.6756574749946584
99th percentile: 3.233455076217652
mean time: 2.035965387026469
Pipeline stage StressChecker completed in 64.24s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.62s
Shutdown handler de-registered
chaiml-kimid-v8b-kimidv_63800_v6 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
Generating Leaderboard row for %s
Generated Leaderboard row for %s
Pipeline stage OfflineFamilyFriendlyScorer completed in 5400.60s
Shutdown handler de-registered
chaiml-kimid-v8b-kimidv_63800_v6 status is now torndown due to DeploymentManager action