Shutdown handler not registered because Python interpreter is not running in the main thread
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
run pipeline %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Connection pool is full, discarding connection: %s. Connection pool size: %s
Pipeline stage VLLMTemplater completed in 1.11s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-kimid-v5a-q235-l-60352-v1
Waiting for inference service chaiml-kimid-v5a-q235-l-60352-v1 to be ready
Inference service chaiml-kimid-v5a-q235-l-60352-v1 ready after 523.3214812278748s
Pipeline stage VLLMDeployer completed in 523.92s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.7614216804504395s
Received healthy response to inference request in 1.9364423751831055s
Received healthy response to inference request in 1.843259334564209s
Received healthy response to inference request in 1.938645362854004s
Received healthy response to inference request in 1.9599273204803467s
Received healthy response to inference request in 1.6871178150177002s
Received healthy response to inference request in 1.8000962734222412s
Received healthy response to inference request in 1.819784164428711s
Received healthy response to inference request in 1.3983988761901855s
Received healthy response to inference request in 1.6974167823791504s
Received healthy response to inference request in 2.1618189811706543s
Received healthy response to inference request in 1.6927368640899658s
Received healthy response to inference request in 1.8975629806518555s
Received healthy response to inference request in 2.002349615097046s
Received healthy response to inference request in 1.8010914325714111s
Received healthy response to inference request in 2.017246723175049s
Received healthy response to inference request in 1.8417026996612549s
Received healthy response to inference request in 1.7435381412506104s
Received healthy response to inference request in 2.009894847869873s
Received healthy response to inference request in 2.045823335647583s
Received healthy response to inference request in 1.978407382965088s
Received healthy response to inference request in 1.7172231674194336s
Received healthy response to inference request in 2.0975565910339355s
Received healthy response to inference request in 2.22243332862854s
Received healthy response to inference request in 2.0181987285614014s
Received healthy response to inference request in 2.0514578819274902s
Received healthy response to inference request in 1.926011085510254s
Received healthy response to inference request in 2.0577101707458496s
Received healthy response to inference request in 1.6207489967346191s
Received healthy response to inference request in 2.0944061279296875s
30 requests
0 failed requests
5th percentile: 1.6506149649620057
10th percentile: 1.6921749591827393
20th percentile: 1.738275146484375
30th percentile: 1.8007928848266601
40th percentile: 1.8426366806030274
50th percentile: 1.9312267303466797
60th percentile: 1.967319345474243
70th percentile: 2.012100410461426
80th percentile: 2.0469502449035644
90th percentile: 2.0947211742401124
95th percentile: 2.1329009056091306
99th percentile: 2.204855167865753
mean time: 1.8946809689203898
Pipeline stage StressChecker completed in 59.41s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.64s
Shutdown handler de-registered
chaiml-kimid-v5a-q235-l_60352_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Generating Leaderboard row for %s
Generated Leaderboard row for %s
Pipeline stage OfflineFamilyFriendlyScorer completed in 2063.90s
Shutdown handler de-registered
chaiml-kimid-v5a-q235-l_60352_v1 status is now inactive due to admin request
chaiml-kimid-v5a-q235-l_60352_v1 status is now torndown due to DeploymentManager action