Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage StressChecker
Failed to get response for submission nischaydnk-mistral24b-s_27493_v1: HTTPConnectionPool(host='nischaydnk-mistral24b-s-27493-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission nischaydnk-mistral24b-s_33506_v1: HTTPConnectionPool(host='nischaydnk-mistral24b-s-33506-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 17.26715087890625s
Received healthy response to inference request in 10.337964534759521s
Failed to get response for submission nischaydnk-mistral24b-s_27493_v1: HTTPConnectionPool(host='nischaydnk-mistral24b-s-27493-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 3.1992363929748535s
5 requests
2 failed requests
5th percentile: 4.626982021331787
10th percentile: 6.05472764968872
20th percentile: 8.910218906402589
30th percentile: 11.723801803588866
40th percentile: 14.49547634124756
50th percentile: 17.26715087890625
60th percentile: 18.403566074371337
70th percentile: 19.539981269836424
80th percentile: 20.109823989868165
90th percentile: 20.11309423446655
95th percentile: 20.114729356765746
99th percentile: 20.116037454605102
mean time: 14.205781030654908
%s, retrying in %s seconds...
Received healthy response to inference request in 1.4602844715118408s
Received healthy response to inference request in 2.9497320652008057s
Received healthy response to inference request in 2.180544137954712s
Received healthy response to inference request in 0.8488671779632568s
Received healthy response to inference request in 2.735139846801758s
5 requests
0 failed requests
5th percentile: 0.9711506366729736
10th percentile: 1.0934340953826904
20th percentile: 1.338001012802124
30th percentile: 1.604336404800415
40th percentile: 1.8924402713775637
50th percentile: 2.180544137954712
60th percentile: 2.40238242149353
70th percentile: 2.6242207050323487
80th percentile: 2.7780582904815674
90th percentile: 2.8638951778411865
95th percentile: 2.906813621520996
99th percentile: 2.941148376464844
mean time: 2.0349135398864746
Pipeline stage StressChecker completed in 83.57s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.64s
Shutdown handler de-registered
function_sileb_2025-02-24 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 4276.38s
Shutdown handler de-registered
function_sileb_2025-02-24 status is now inactive due to auto deactivation removed underperforming models
function_sileb_2025-02-24 status is now torndown due to DeploymentManager action