Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Running pipeline stage VLLMTemplater
Connection pool is full, discarding connection: %s. Connection pool size: %s
Pipeline stage VLLMTemplater completed in 0.19s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-kimid-v10-nonam-50547-v10
Waiting for inference service chaiml-kimid-v10-nonam-50547-v10 to be ready
Failed to get response for submission chaiml-kimid-v10-noname_50547_v7: HTTPConnectionPool(host='chaiml-kimid-v10-noname-50547-v7-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission chaiml-kimid-v10-noname_50547_v9: HTTPConnectionPool(host='chaiml-kimid-v10-noname-50547-v9-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission chaiml-kimid-v10-noname_50547_v7: HTTPConnectionPool(host='chaiml-kimid-v10-noname-50547-v7-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission chaiml-kimid-v10-noname_50547_v7: HTTPConnectionPool(host='chaiml-kimid-v10-noname-50547-v7-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Inference service chaiml-kimid-v10-nonam-50547-v10 ready after 241.06279921531677s
Pipeline stage VLLMDeployer completed in 241.63s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.007983446121216s
Received healthy response to inference request in 2.062211513519287s
Received healthy response to inference request in 1.7246708869934082s
Received healthy response to inference request in 1.8176310062408447s
Received healthy response to inference request in 1.727595329284668s
Received healthy response to inference request in 1.8012633323669434s
Received healthy response to inference request in 1.9613964557647705s
Received healthy response to inference request in 2.073864459991455s
Received healthy response to inference request in 1.700406551361084s
Received healthy response to inference request in 1.8442628383636475s
Received healthy response to inference request in 1.8817625045776367s
Received healthy response to inference request in 1.7332265377044678s
Received healthy response to inference request in 1.9698529243469238s
Received healthy response to inference request in 1.923560380935669s
Received healthy response to inference request in 2.0247764587402344s
Received healthy response to inference request in 1.7641363143920898s
Received healthy response to inference request in 2.025496006011963s
Received healthy response to inference request in 1.774775505065918s
Received healthy response to inference request in 2.49944806098938s
Received healthy response to inference request in 1.773589849472046s
Received healthy response to inference request in 1.7169475555419922s
Received healthy response to inference request in 2.0886600017547607s
Received healthy response to inference request in 2.2395710945129395s
Received healthy response to inference request in 1.9403576850891113s
Received healthy response to inference request in 1.7792012691497803s
Received healthy response to inference request in 1.8455753326416016s
Received healthy response to inference request in 1.7335340976715088s
Received healthy response to inference request in 1.76904296875s
Received healthy response to inference request in 2.3665473461151123s
Received healthy response to inference request in 1.871523380279541s
30 requests
0 failed requests
5th percentile: 1.7204230546951294
10th percentile: 1.727302885055542
20th percentile: 1.7580158710479736
30th percentile: 1.7744198083877563
40th percentile: 1.8110839366912843
50th percentile: 1.8585493564605713
60th percentile: 1.930279302597046
70th percentile: 1.9812920808792114
80th percentile: 2.032839107513428
90th percentile: 2.103751111030579
95th percentile: 2.309408032894134
99th percentile: 2.4609068536758425
mean time: 1.9147623697916667
Pipeline stage StressChecker completed in 60.06s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.62s
Shutdown handler de-registered
chaiml-kimid-v10-nonam_50547_v10 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
Generating Leaderboard row for %s
Generated Leaderboard row for %s
Pipeline stage OfflineFamilyFriendlyScorer completed in 4695.98s
Shutdown handler de-registered
chaiml-kimid-v10-nonam_50547_v10 status is now torndown due to DeploymentManager action