Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.14s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-kimid-v10-nonam-50547-v16
Waiting for inference service chaiml-kimid-v10-nonam-50547-v16 to be ready
Inference service chaiml-kimid-v10-nonam-50547-v16 ready after 460.4850060939789s
Pipeline stage VLLMDeployer completed in 460.87s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.612992763519287s
Received healthy response to inference request in 2.1601243019104004s
Received healthy response to inference request in 2.214823007583618s
Received healthy response to inference request in 2.0145065784454346s
Received healthy response to inference request in 2.1154890060424805s
Received healthy response to inference request in 1.9655754566192627s
Received healthy response to inference request in 1.9988279342651367s
Received healthy response to inference request in 1.8738114833831787s
Received healthy response to inference request in 2.5132906436920166s
Received healthy response to inference request in 2.5395658016204834s
Received healthy response to inference request in 2.3651280403137207s
Received healthy response to inference request in 1.9214799404144287s
Received healthy response to inference request in 1.9362502098083496s
Received healthy response to inference request in 2.025977373123169s
Received healthy response to inference request in 2.1276931762695312s
Received healthy response to inference request in 1.915388584136963s
Received healthy response to inference request in 2.3318636417388916s
Received healthy response to inference request in 1.8815991878509521s
Received healthy response to inference request in 2.140345573425293s
Received healthy response to inference request in 2.2977092266082764s
Received healthy response to inference request in 2.122605800628662s
Received healthy response to inference request in 2.2718632221221924s
Received healthy response to inference request in 2.171616792678833s
Received healthy response to inference request in 2.144341230392456s
Received healthy response to inference request in 2.3411269187927246s
Received healthy response to inference request in 2.0985422134399414s
Received healthy response to inference request in 1.9329049587249756s
Received healthy response to inference request in 1.9252169132232666s
Received healthy response to inference request in 2.2309093475341797s
Received healthy response to inference request in 3.4092578887939453s
30 requests
0 failed requests
5th percentile: 1.896804416179657
10th percentile: 1.920870804786682
20th percentile: 1.9355811595916748
30th percentile: 2.009802985191345
40th percentile: 2.108710289001465
50th percentile: 2.134019374847412
60th percentile: 2.1647212982177733
70th percentile: 2.2431955099105836
80th percentile: 2.333716297149658
90th percentile: 2.515918159484863
95th percentile: 2.5799506306648254
99th percentile: 3.178341002464295
mean time: 2.1866942405700684
Pipeline stage StressChecker completed in 68.09s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.63s
Shutdown handler de-registered
chaiml-kimid-v10-nonam_50547_v16 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
Generating Leaderboard row for %s
Generated Leaderboard row for %s
Pipeline stage OfflineFamilyFriendlyScorer completed in 3756.63s
Shutdown handler de-registered
chaiml-kimid-v10-nonam_50547_v16 status is now torndown due to DeploymentManager action