Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage VLLMTemplater
Pipeline stage VLLMTemplater completed in 0.13s
run pipeline stage %s
Running pipeline stage VLLMDeployer
Creating inference service chaiml-opusd-v2b-opusd-33883-v16
Waiting for inference service chaiml-opusd-v2b-opusd-33883-v16 to be ready
Failed to get response for submission chaiml-kimid-v10-nonam_50547_v12: HTTPConnectionPool(host='chaiml-kimid-v10-nonam-50547-v12-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission chaiml-kimid-v10-nonam_50547_v11: HTTPConnectionPool(host='chaiml-kimid-v10-nonam-50547-v11-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Inference service chaiml-opusd-v2b-opusd-33883-v16 ready after 501.55593514442444s
Pipeline stage VLLMDeployer completed in 502.01s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.0783565044403076s
Received healthy response to inference request in 2.1388728618621826s
Received healthy response to inference request in 1.9566526412963867s
Received healthy response to inference request in 2.2373013496398926s
Received healthy response to inference request in 2.352099657058716s
Received healthy response to inference request in 2.240959882736206s
Received healthy response to inference request in 2.187925100326538s
Received healthy response to inference request in 2.1649999618530273s
Received healthy response to inference request in 1.982600450515747s
Received healthy response to inference request in 2.290348529815674s
Received healthy response to inference request in 2.5540428161621094s
Received healthy response to inference request in 2.106267213821411s
Received healthy response to inference request in 2.1202971935272217s
Received healthy response to inference request in 2.1652872562408447s
Received healthy response to inference request in 1.8721916675567627s
Received healthy response to inference request in 1.9819533824920654s
Received healthy response to inference request in 1.9065275192260742s
Received healthy response to inference request in 2.039841413497925s
Received healthy response to inference request in 2.382855176925659s
Received healthy response to inference request in 1.9904899597167969s
Received healthy response to inference request in 2.4132237434387207s
Received healthy response to inference request in 2.2098090648651123s
Received healthy response to inference request in 1.8609461784362793s
Received healthy response to inference request in 1.991211175918579s
Received healthy response to inference request in 2.3470230102539062s
Failed to get response for submission chaiml-kimid-v10-nonam_50547_v12: HTTPConnectionPool(host='chaiml-kimid-v10-nonam-50547-v12-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Received healthy response to inference request in 2.256044864654541s
Received healthy response to inference request in 2.229017972946167s
Received healthy response to inference request in 2.262376546859741s
Received healthy response to inference request in 1.9933850765228271s
Received healthy response to inference request in 2.30954647064209s
30 requests
0 failed requests
5th percentile: 1.8876428008079529
10th percentile: 1.9516401290893555
20th percentile: 1.9889120578765869
30th percentile: 2.0259045124053956
40th percentile: 2.1146852016448974
50th percentile: 2.165143609046936
60th percentile: 2.217492628097534
70th percentile: 2.2454853773117067
80th percentile: 2.294188117980957
90th percentile: 2.3551752090454103
95th percentile: 2.399557888507843
99th percentile: 2.5132052850723268
mean time: 2.1540818214416504
Pipeline stage StressChecker completed in 67.01s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.59s
Shutdown handler de-registered
chaiml-opusd-v2b-opusd_33883_v16 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
Generating Leaderboard row for %s
Generated Leaderboard row for %s
Pipeline stage OfflineFamilyFriendlyScorer completed in 5533.09s
Shutdown handler de-registered
chaiml-opusd-v2b-opusd_33883_v16 status is now torndown due to DeploymentManager action