Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 4.0784385204315186s
Received healthy response to inference request in 3.8186686038970947s
Received healthy response to inference request in 4.458700180053711s
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.8517863750457764s
Received healthy response to inference request in 4.758614778518677s
Received healthy response to inference request in 10.794280529022217s
Received healthy response to inference request in 4.134451627731323s
Received healthy response to inference request in 6.035650253295898s
10 requests
2 failed requests
5th percentile: 3.2868833780288695
10th percentile: 3.7219803810119627
20th percentile: 4.026484537124634
30th percentile: 4.117647695541382
40th percentile: 4.329000759124756
50th percentile: 4.608657479286194
60th percentile: 5.2694289684295645
70th percentile: 7.463239336013793
80th percentile: 12.654161405563356
90th percentile: 20.095520758628844
95th percentile: 20.103782069683074
99th percentile: 20.110391118526458
mean time: 8.113631916046142
%s, retrying in %s seconds...
Received healthy response to inference request in 4.727750778198242s
Received healthy response to inference request in 3.671384811401367s
Received healthy response to inference request in 5.942685842514038s
Received healthy response to inference request in 5.246973514556885s
Received healthy response to inference request in 7.632884502410889s
Received healthy response to inference request in 4.257896184921265s
Received healthy response to inference request in 5.185067176818848s
Received healthy response to inference request in 7.203845977783203s
Received healthy response to inference request in 3.803234815597534s
Received healthy response to inference request in 4.833113431930542s
10 requests
0 failed requests
5th percentile: 3.7307173132896425
10th percentile: 3.7900498151779174
20th percentile: 4.166963911056518
30th percentile: 4.586794400215148
40th percentile: 4.790968370437622
50th percentile: 5.009090304374695
60th percentile: 5.209829711914063
70th percentile: 5.455687212944031
80th percentile: 6.194917869567871
90th percentile: 7.246749830245972
95th percentile: 7.43981716632843
99th percentile: 7.594271035194397
mean time: 5.250483703613281
Pipeline stage StressChecker completed in 139.37s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
Failed to get response for submission chaiml-mistral-24b-2048-_2678_v3: ('http://chaiml-mistral-24b-2048-2678-v3-predictor.tenant-chaiml-guanaco.kchai-coreweave-us-east-04a.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.62s
Shutdown handler de-registered
function_paror_2026-02-16 status is now deployed due to DeploymentManager action
function_paror_2026-02-16 status is now inactive due to auto deactivation removed underperforming models
function_paror_2026-02-16 status is now torndown due to DeploymentManager action