Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.13s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-goodconvo-v1-dpo-90455-v4
Waiting for inference service chaiml-goodconvo-v1-dpo-90455-v4 to be ready
Inference service chaiml-goodconvo-v1-dpo-90455-v4 ready after 200.71306610107422s
Pipeline stage VLLMDeployer completed in 201.16s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.966886281967163s
Received healthy response to inference request in 2.083681583404541s
Received healthy response to inference request in 1.9254672527313232s
Received healthy response to inference request in 2.1028149127960205s
Received healthy response to inference request in 1.9566137790679932s
Received healthy response to inference request in 1.8328595161437988s
Received healthy response to inference request in 2.133528232574463s
Received healthy response to inference request in 1.8898308277130127s
Received healthy response to inference request in 1.9930419921875s
Received healthy response to inference request in 1.975553035736084s
Received healthy response to inference request in 2.392148494720459s
Received healthy response to inference request in 1.9755034446716309s
Received healthy response to inference request in 2.54301118850708s
Received healthy response to inference request in 2.134507894515991s
Received healthy response to inference request in 1.9635376930236816s
Received healthy response to inference request in 1.9896633625030518s
Received healthy response to inference request in 1.929131031036377s
Received healthy response to inference request in 2.1482555866241455s
Received healthy response to inference request in 2.0055840015411377s
Received healthy response to inference request in 2.115131378173828s
Received healthy response to inference request in 1.8669042587280273s
Received healthy response to inference request in 1.8715007305145264s
Received healthy response to inference request in 2.0028486251831055s
Received healthy response to inference request in 2.298175096511841s
Received healthy response to inference request in 2.0171661376953125s
Received healthy response to inference request in 1.9177212715148926s
Received healthy response to inference request in 2.035083770751953s
Received healthy response to inference request in 2.190437078475952s
Received healthy response to inference request in 1.898085117340088s
Received healthy response to inference request in 1.9873955249786377s
30 requests
0 failed requests
5th percentile: 1.8689726710319519
10th percentile: 1.887997817993164
20th percentile: 1.9239180564880372
30th percentile: 1.9614605188369751
40th percentile: 1.9755331993103027
50th percentile: 1.9913526773452759
60th percentile: 2.0102168560028075
70th percentile: 2.0894215822219846
80th percentile: 2.1337241649627687
90th percentile: 2.2012108802795414
95th percentile: 2.3498604655265805
99th percentile: 2.4992610073089603
mean time: 2.038068970044454
Pipeline stage StressChecker completed in 63.46s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.84s
Shutdown handler de-registered
chaiml-goodconvo-v1-dpo_90455_v4 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
Generating Leaderboard row for %s
Generated Leaderboard row for %s
Pipeline stage OfflineFamilyFriendlyScorer completed in 6113.25s
Shutdown handler de-registered
chaiml-goodconvo-v1-dpo_90455_v4 status is now torndown due to DeploymentManager action