Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.6868605613708496s
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 8.2692551612854s
Received healthy response to inference request in 4.610949754714966s
Received healthy response to inference request in 3.8772947788238525s
Received healthy response to inference request in 8.53609299659729s
Received healthy response to inference request in 2.9310462474823s
Received healthy response to inference request in 4.47443699836731s
Received healthy response to inference request in 3.1583547592163086s
Received healthy response to inference request in 2.746699571609497s
10 requests
1 failed requests
5th percentile: 2.829655575752258
10th percentile: 2.9126115798950196
20th percentile: 3.112893056869507
30th percentile: 3.528308820724487
40th percentile: 3.8011210918426515
50th percentile: 4.175865888595581
60th percentile: 4.529042100906372
70th percentile: 5.708441376686095
80th percentile: 8.322622728347778
90th percentile: 9.71215324401855
95th percentile: 15.004424357414234
99th percentile: 19.2382412481308
mean time: 6.258768630027771
%s, retrying in %s seconds...
Received healthy response to inference request in 3.002225875854492s
Received healthy response to inference request in 3.555162191390991s
Received healthy response to inference request in 2.3769209384918213s
Failed to get response for submission chaiml-muster-v3a-kakit-_3784_v4: HTTPConnectionPool(host='chaiml-muster-v3a-kakit-3784-v4-predictor.tenant-chaiml-guanaco.k2.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Received healthy response to inference request in 4.271798133850098s
Received healthy response to inference request in 5.448518991470337s
Received healthy response to inference request in 3.0344886779785156s
Received healthy response to inference request in 4.721208095550537s
Received healthy response to inference request in 5.867665529251099s
Received healthy response to inference request in 6.45136022567749s
Received healthy response to inference request in 4.578413963317871s
10 requests
0 failed requests
5th percentile: 2.6583081603050234
10th percentile: 2.939695382118225
20th percentile: 3.028036117553711
30th percentile: 3.3989601373672484
40th percentile: 3.985143756866455
50th percentile: 4.425106048583984
60th percentile: 4.635531616210938
70th percentile: 4.939401364326477
80th percentile: 5.53234829902649
90th percentile: 5.926034998893738
95th percentile: 6.188697612285614
99th percentile: 6.398827702999115
mean time: 4.330776262283325
Pipeline stage StressChecker completed in 108.62s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.66s
Shutdown handler de-registered
function_damak_2026-02-15 status is now deployed due to DeploymentManager action
function_damak_2026-02-15 status is now inactive due to auto deactivation removed underperforming models