Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.14s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-kimid-v10-noname-50547-v7
Waiting for inference service chaiml-kimid-v10-noname-50547-v7 to be ready
Inference service chaiml-kimid-v10-noname-50547-v7 ready after 211.0626060962677s
Pipeline stage VLLMDeployer completed in 211.58s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.1051411628723145s
Received healthy response to inference request in 2.3824167251586914s
Received healthy response to inference request in 2.2441954612731934s
Received healthy response to inference request in 2.1422383785247803s
Received healthy response to inference request in 2.0319128036499023s
Received healthy response to inference request in 2.151916027069092s
Received healthy response to inference request in 2.1211636066436768s
Received healthy response to inference request in 1.883875846862793s
Received healthy response to inference request in 1.9152634143829346s
Received healthy response to inference request in 2.1887753009796143s
Received healthy response to inference request in 2.1415882110595703s
Received healthy response to inference request in 2.3540284633636475s
Received healthy response to inference request in 1.996821641921997s
Received healthy response to inference request in 1.911987543106079s
Received healthy response to inference request in 2.2053258419036865s
Received healthy response to inference request in 2.0316426753997803s
Received healthy response to inference request in 2.179605722427368s
Received healthy response to inference request in 1.902674913406372s
Received healthy response to inference request in 2.197922945022583s
Received healthy response to inference request in 2.2698261737823486s
Received healthy response to inference request in 1.8892793655395508s
Received healthy response to inference request in 2.301729440689087s
Received healthy response to inference request in 2.1728355884552s
Received healthy response to inference request in 2.1150825023651123s
Received healthy response to inference request in 2.2832226753234863s
Received healthy response to inference request in 2.470076560974121s
Received healthy response to inference request in 1.9038875102996826s
Received healthy response to inference request in 2.0916388034820557s
Received healthy response to inference request in 2.0719640254974365s
Received healthy response to inference request in 2.0592706203460693s
30 requests
0 failed requests
5th percentile: 1.8953073620796204
10th percentile: 1.9037662506103517
20th percentile: 1.9805099964141846
30th percentile: 2.0510632753372193
40th percentile: 2.099740219116211
50th percentile: 2.1313759088516235
60th percentile: 2.160283851623535
70th percentile: 2.191519594192505
80th percentile: 2.2493216037750243
90th percentile: 2.306959342956543
95th percentile: 2.3696420073509215
99th percentile: 2.4446552085876467
mean time: 2.123910331726074
Pipeline stage StressChecker completed in 65.97s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.79s
Shutdown handler de-registered
chaiml-kimid-v10-noname_50547_v7 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
Generating Leaderboard row for %s
Generated Leaderboard row for %s
Pipeline stage OfflineFamilyFriendlyScorer completed in 3499.52s
Shutdown handler de-registered
chaiml-kimid-v10-noname_50547_v7 status is now torndown due to DeploymentManager action