alexdaoud-trainer-bagir_29250

developer_uid: chai_backend_admin
submission_id: alexdaoud-trainer-bagir_29250_v2
model_name: alexdaoud-trainer-bagir_29250_v2
model_group: alexdaoud/trainer_bagir_
status: torndown
timestamp: 2025-01-24T01:08:28+00:00
num_battles: 6326
num_wins: 3514
celo_rating: 1298.83
family_friendly_score: 0.5680000000000001
family_friendly_standard_error: 0.007005369369276684
submission_type: basic
model_repo: alexdaoud/trainer_bagir_2024-12-11-checkpoint-105
model_architecture: LlamaForSequenceClassification
model_num_parameters: 8030261248.0
best_of: 1
max_input_tokens: 256
max_output_tokens: 1
reward_model: default
display_name: alexdaoud-trainer-bagir_29250_v2
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: alexdaoud/trainer_bagir_2024-12-11-checkpoint-105
model_size: 8B
ranking_group: single
us_pacific_date: 2025-01-23
win_ratio: 0.5554852987669934
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 256, 'best_of': 1, 'max_output_tokens': 1}
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name alexdaoud-trainer-bagir-29250-v2-mkmlizer
Waiting for job on alexdaoud-trainer-bagir-29250-v2-mkmlizer to finish
alexdaoud-trainer-bagir-29250-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
alexdaoud-trainer-bagir-29250-v2-mkmlizer: ║     _____            __           __                                ║
alexdaoud-trainer-bagir-29250-v2-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
alexdaoud-trainer-bagir-29250-v2-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
alexdaoud-trainer-bagir-29250-v2-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
alexdaoud-trainer-bagir-29250-v2-mkmlizer: ║       /___/                                                         ║
alexdaoud-trainer-bagir-29250-v2-mkmlizer: ║                                                                     ║
alexdaoud-trainer-bagir-29250-v2-mkmlizer: ║  Version: 0.11.12                                                   ║
alexdaoud-trainer-bagir-29250-v2-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
alexdaoud-trainer-bagir-29250-v2-mkmlizer: ║  https://mk1.ai                                                     ║
alexdaoud-trainer-bagir-29250-v2-mkmlizer: ║                                                                     ║
alexdaoud-trainer-bagir-29250-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
alexdaoud-trainer-bagir-29250-v2-mkmlizer: ║  belonging to:                                                      ║
alexdaoud-trainer-bagir-29250-v2-mkmlizer: ║                                                                     ║
alexdaoud-trainer-bagir-29250-v2-mkmlizer: ║  Chai Research Corp.                                                ║
alexdaoud-trainer-bagir-29250-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
alexdaoud-trainer-bagir-29250-v2-mkmlizer: ║  Expiration: 2025-04-15 23:59:59                                    ║
alexdaoud-trainer-bagir-29250-v2-mkmlizer: ║                                                                     ║
alexdaoud-trainer-bagir-29250-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Failed to get response for submission doctor-shotgun-l3-3-70b_70691_v2: HTTPConnectionPool(host='doctor-shotgun-l3-3-70b-70691-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
alexdaoud-trainer-bagir-29250-v2-mkmlizer: Downloaded to shared memory in 38.422s
alexdaoud-trainer-bagir-29250-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:t0, folder:/tmp/tmpmles29h2, device:0
alexdaoud-trainer-bagir-29250-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
alexdaoud-trainer-bagir-29250-v2-mkmlizer: quantized model in 82.720s
alexdaoud-trainer-bagir-29250-v2-mkmlizer: Processed model alexdaoud/trainer_bagir_2024-12-11-checkpoint-105 in 121.142s
alexdaoud-trainer-bagir-29250-v2-mkmlizer: creating bucket guanaco-mkml-models
alexdaoud-trainer-bagir-29250-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
alexdaoud-trainer-bagir-29250-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/alexdaoud-trainer-bagir-29250-v2
alexdaoud-trainer-bagir-29250-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/alexdaoud-trainer-bagir-29250-v2/config.json
alexdaoud-trainer-bagir-29250-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/alexdaoud-trainer-bagir-29250-v2/special_tokens_map.json
alexdaoud-trainer-bagir-29250-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/alexdaoud-trainer-bagir-29250-v2/tokenizer_config.json
alexdaoud-trainer-bagir-29250-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/alexdaoud-trainer-bagir-29250-v2/tokenizer.json
alexdaoud-trainer-bagir-29250-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/alexdaoud-trainer-bagir-29250-v2/flywheel_model.0.safetensors
alexdaoud-trainer-bagir-29250-v2-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   1%|          | 3/291 [00:00<00:55,  5.22it/s]
Loading 0:   1%|▏         | 4/291 [00:01<01:30,  3.18it/s]
Loading 0:   2%|▏         | 5/291 [00:01<01:59,  2.39it/s]
Loading 0:   3%|▎         | 8/291 [00:02<01:01,  4.58it/s]
Loading 0:   3%|▎         | 9/291 [00:02<01:00,  4.63it/s]
Loading 0:   3%|▎         | 10/291 [00:02<00:53,  5.24it/s]
Loading 0:   4%|▍         | 12/291 [00:02<01:03,  4.37it/s]
Loading 0:   4%|▍         | 13/291 [00:03<01:24,  3.29it/s]
Loading 0:   5%|▍         | 14/291 [00:04<01:48,  2.56it/s]
Loading 0:   6%|▌         | 17/291 [00:04<01:02,  4.42it/s]
Loading 0:   6%|▌         | 18/291 [00:04<00:58,  4.66it/s]
Loading 0:   7%|▋         | 19/291 [00:04<00:51,  5.27it/s]
Loading 0:   7%|▋         | 21/291 [00:05<01:01,  4.42it/s]
Loading 0:   8%|▊         | 22/291 [00:05<01:20,  3.34it/s]
Loading 0:   8%|▊         | 23/291 [00:06<01:42,  2.60it/s]
Loading 0:   9%|▉         | 26/291 [00:06<00:59,  4.45it/s]
Loading 0:   9%|▉         | 27/291 [00:06<00:56,  4.65it/s]
Loading 0:  10%|█         | 30/291 [00:07<00:56,  4.59it/s]
Loading 0:  11%|█         | 31/291 [00:07<01:12,  3.59it/s]
Loading 0:  11%|█         | 32/291 [00:08<01:30,  2.87it/s]
Loading 0:  12%|█▏        | 35/291 [00:08<00:55,  4.60it/s]
Loading 0:  12%|█▏        | 36/291 [00:08<00:52,  4.82it/s]
Loading 0:  13%|█▎        | 39/291 [00:09<00:53,  4.69it/s]
Loading 0:  14%|█▎        | 40/291 [00:10<01:08,  3.65it/s]
Loading 0:  14%|█▍        | 41/291 [00:10<01:26,  2.88it/s]
Loading 0:  15%|█▌        | 44/291 [00:11<00:54,  4.57it/s]
Loading 0:  15%|█▌        | 45/291 [00:11<00:51,  4.78it/s]
Loading 0:  16%|█▌        | 46/291 [00:11<00:46,  5.30it/s]
Loading 0:  16%|█▋        | 48/291 [00:11<00:55,  4.39it/s]
Loading 0:  17%|█▋        | 49/291 [00:12<01:12,  3.33it/s]
Loading 0:  17%|█▋        | 50/291 [00:13<01:30,  2.66it/s]
Loading 0:  18%|█▊        | 53/291 [00:13<00:52,  4.52it/s]
Loading 0:  19%|█▊        | 54/291 [00:13<00:49,  4.76it/s]
Loading 0:  20%|█▉        | 57/291 [00:14<00:50,  4.63it/s]
Loading 0:  20%|█▉        | 58/291 [00:14<01:04,  3.60it/s]
Loading 0:  20%|██        | 59/291 [00:15<01:21,  2.85it/s]
Loading 0:  21%|██▏       | 62/291 [00:15<00:50,  4.55it/s]
Loading 0:  22%|██▏       | 63/291 [00:15<00:47,  4.78it/s]
Loading 0:  23%|██▎       | 66/291 [00:16<00:48,  4.66it/s]
Loading 0:  23%|██▎       | 67/291 [00:16<01:01,  3.64it/s]
Loading 0:  23%|██▎       | 68/291 [00:17<01:16,  2.91it/s]
Loading 0:  24%|██▍       | 71/291 [00:17<00:47,  4.65it/s]
Loading 0:  25%|██▍       | 72/291 [00:17<00:45,  4.86it/s]
Loading 0:  26%|██▌       | 75/291 [00:18<00:45,  4.71it/s]
Loading 0:  26%|██▌       | 76/291 [00:19<00:58,  3.66it/s]
Loading 0:  26%|██▋       | 77/291 [00:19<01:13,  2.90it/s]
Loading 0:  27%|██▋       | 80/291 [00:19<00:45,  4.64it/s]
Loading 0:  28%|██▊       | 81/291 [00:20<00:43,  4.85it/s]
Loading 0:  29%|██▊       | 83/291 [00:20<00:38,  5.42it/s]
Loading 0:  29%|██▉       | 84/291 [00:20<00:53,  3.87it/s]
Loading 0:  29%|██▉       | 85/291 [00:21<01:06,  3.08it/s]
Loading 0:  30%|██▉       | 86/291 [00:22<01:22,  2.50it/s]
Loading 0:  31%|███       | 89/291 [00:22<00:46,  4.32it/s]
Loading 0:  31%|███       | 90/291 [00:22<00:43,  4.58it/s]
Loading 0:  31%|███▏      | 91/291 [00:22<00:38,  5.15it/s]
Loading 0:  32%|███▏      | 93/291 [00:23<00:45,  4.38it/s]
Loading 0:  32%|███▏      | 94/291 [00:23<00:59,  3.31it/s]
Loading 0:  33%|███▎      | 95/291 [00:24<01:14,  2.64it/s]
Loading 0:  34%|███▎      | 98/291 [00:24<00:43,  4.48it/s]
Loading 0:  34%|███▍      | 99/291 [00:24<00:40,  4.72it/s]
Loading 0:  34%|███▍      | 100/291 [00:24<00:35,  5.32it/s]
Loading 0:  35%|███▌      | 102/291 [00:25<00:42,  4.47it/s]
Loading 0:  35%|███▌      | 103/291 [00:26<00:55,  3.37it/s]
Loading 0:  36%|███▌      | 104/291 [00:26<01:10,  2.67it/s]
Loading 0:  37%|███▋      | 107/291 [00:26<00:40,  4.58it/s]
Loading 0:  37%|███▋      | 108/291 [00:26<00:37,  4.82it/s]
Loading 0:  38%|███▊      | 111/291 [00:27<00:38,  4.69it/s]
Loading 0:  38%|███▊      | 112/291 [00:28<00:49,  3.63it/s]
Loading 0:  39%|███▉      | 113/291 [00:28<01:01,  2.89it/s]
Loading 0:  40%|███▉      | 116/291 [00:29<00:37,  4.63it/s]
Loading 0:  40%|████      | 117/291 [00:29<00:35,  4.85it/s]
Loading 0:  41%|████      | 120/291 [00:29<00:36,  4.73it/s]
Loading 0:  42%|████▏     | 121/291 [00:30<00:46,  3.69it/s]
Loading 0:  42%|████▏     | 122/291 [00:31<00:57,  2.93it/s]
Loading 0:  43%|████▎     | 125/291 [00:31<00:35,  4.68it/s]
Loading 0:  43%|████▎     | 126/291 [00:31<00:33,  4.89it/s]
Loading 0:  44%|████▍     | 129/291 [00:32<00:34,  4.73it/s]
Loading 0:  45%|████▍     | 130/291 [00:32<00:43,  3.68it/s]
Loading 0:  45%|████▌     | 131/291 [00:33<00:55,  2.90it/s]
Loading 0:  46%|████▌     | 134/291 [00:33<00:33,  4.64it/s]
Loading 0:  46%|████▋     | 135/291 [00:33<00:32,  4.86it/s]
Loading 0:  47%|████▋     | 136/291 [00:33<00:28,  5.38it/s]
Loading 0:  47%|████▋     | 138/291 [00:34<00:33,  4.53it/s]
Loading 0:  48%|████▊     | 139/291 [00:34<00:44,  3.43it/s]
Loading 0:  48%|████▊     | 140/291 [00:35<00:55,  2.71it/s]
Loading 0:  49%|████▉     | 143/291 [00:35<00:32,  4.52it/s]
Loading 0:  49%|████▉     | 144/291 [00:35<00:30,  4.77it/s]
Loading 0:  50%|████▉     | 145/291 [00:35<00:27,  5.34it/s]
Loading 0:  51%|█████     | 147/291 [00:36<00:32,  4.49it/s]
Loading 0:  51%|█████     | 148/291 [00:37<00:42,  3.40it/s]
Loading 0:  51%|█████     | 149/291 [00:37<00:52,  2.69it/s]
Loading 0:  52%|█████▏    | 152/291 [00:37<00:30,  4.55it/s]
Loading 0:  53%|█████▎    | 153/291 [00:38<00:28,  4.80it/s]
Loading 0:  53%|█████▎    | 154/291 [00:38<00:25,  5.40it/s]
Loading 0:  54%|█████▎    | 156/291 [00:38<00:29,  4.53it/s]
Loading 0:  54%|█████▍    | 157/291 [00:39<00:39,  3.41it/s]
Loading 0:  54%|█████▍    | 158/291 [00:39<00:49,  2.67it/s]
Loading 0:  55%|█████▌    | 161/291 [00:40<00:28,  4.52it/s]
Loading 0:  56%|█████▌    | 162/291 [00:40<00:27,  4.77it/s]
Loading 0:  57%|█████▋    | 165/291 [00:40<00:26,  4.68it/s]
Loading 0:  57%|█████▋    | 166/291 [00:41<00:34,  3.62it/s]
Loading 0:  57%|█████▋    | 167/291 [00:42<00:43,  2.84it/s]
Loading 0:  58%|█████▊    | 170/291 [00:42<00:26,  4.59it/s]
Loading 0:  59%|█████▉    | 171/291 [00:42<00:24,  4.82it/s]
Loading 0:  59%|█████▉    | 173/291 [00:43<00:29,  4.03it/s]
Loading 0:  60%|██████    | 175/291 [00:43<00:22,  5.08it/s]
Loading 0:  60%|██████    | 176/291 [00:43<00:21,  5.26it/s]
Loading 0:  61%|██████    | 177/291 [00:43<00:19,  5.79it/s]
Loading 0:  62%|██████▏   | 179/291 [00:44<00:23,  4.68it/s]
Loading 0:  62%|██████▏   | 180/291 [00:44<00:32,  3.46it/s]
Loading 0:  62%|██████▏   | 181/291 [00:45<00:40,  2.73it/s]
Loading 0:  63%|██████▎   | 184/291 [00:45<00:23,  4.62it/s]
Loading 0:  64%|██████▎   | 185/291 [00:45<00:21,  4.86it/s]
Loading 0:  64%|██████▍   | 187/291 [00:46<00:19,  5.46it/s]
Loading 0:  65%|██████▍   | 188/291 [00:46<00:26,  3.87it/s]
Loading 0:  65%|██████▍   | 189/291 [00:47<00:34,  2.95it/s]
Loading 0:  66%|██████▌   | 192/291 [00:47<00:26,  3.80it/s]
Loading 0:  66%|██████▋   | 193/291 [00:48<00:31,  3.15it/s]
Loading 0:  67%|██████▋   | 194/291 [00:48<00:36,  2.62it/s]
Loading 0:  68%|██████▊   | 197/291 [00:49<00:21,  4.38it/s]
Loading 0:  68%|██████▊   | 198/291 [00:49<00:20,  4.62it/s]
Loading 0:  68%|██████▊   | 199/291 [00:49<00:17,  5.14it/s]
Loading 0:  69%|██████▉   | 201/291 [00:49<00:20,  4.42it/s]
Loading 0:  69%|██████▉   | 202/291 [00:50<00:26,  3.38it/s]
Loading 0:  70%|██████▉   | 203/291 [00:51<00:32,  2.71it/s]
Loading 0:  71%|███████   | 206/291 [00:51<00:18,  4.59it/s]
Loading 0:  71%|███████   | 207/291 [00:51<00:17,  4.82it/s]
Loading 0:  72%|███████▏  | 210/291 [00:52<00:17,  4.72it/s]
Loading 0:  73%|███████▎  | 211/291 [00:52<00:21,  3.66it/s]
Loading 0:  73%|███████▎  | 212/291 [00:53<00:27,  2.90it/s]
Loading 0:  74%|███████▍  | 215/291 [00:53<00:16,  4.64it/s]
Loading 0:  74%|███████▍  | 216/291 [00:53<00:15,  4.86it/s]
Loading 0:  75%|███████▍  | 217/291 [00:53<00:13,  5.44it/s]
Loading 0:  75%|███████▌  | 219/291 [00:54<00:15,  4.56it/s]
Loading 0:  76%|███████▌  | 220/291 [00:54<00:20,  3.46it/s]
Loading 0:  76%|███████▌  | 221/291 [00:55<00:25,  2.76it/s]
Loading 0:  77%|███████▋  | 224/291 [00:55<00:14,  4.70it/s]
Loading 0:  77%|███████▋  | 225/291 [00:55<00:13,  4.93it/s]
Loading 0:  78%|███████▊  | 228/291 [00:56<00:13,  4.65it/s]
Loading 0:  79%|███████▊  | 229/291 [00:57<00:17,  3.65it/s]
Loading 0:  79%|███████▉  | 230/291 [00:57<00:20,  2.91it/s]
Loading 0:  80%|████████  | 233/291 [00:57<00:12,  4.64it/s]
Loading 0:  80%|████████  | 234/291 [00:58<00:11,  4.86it/s]
Loading 0:  81%|████████▏ | 237/291 [00:58<00:11,  4.72it/s]
Loading 0:  82%|████████▏ | 238/291 [00:59<00:14,  3.70it/s]
Loading 0:  82%|████████▏ | 239/291 [00:59<00:17,  2.95it/s]
Loading 0:  83%|████████▎ | 242/291 [01:00<00:10,  4.66it/s]
Loading 0:  84%|████████▎ | 243/291 [01:00<00:09,  4.87it/s]
Loading 0:  85%|████████▍ | 246/291 [01:00<00:09,  4.75it/s]
Loading 0:  85%|████████▍ | 247/291 [01:01<00:11,  3.71it/s]
Loading 0:  85%|████████▌ | 248/291 [01:02<00:14,  2.93it/s]
Loading 0:  86%|████████▋ | 251/291 [01:02<00:08,  4.64it/s]
Loading 0:  87%|████████▋ | 252/291 [01:02<00:08,  4.85it/s]
Loading 0:  87%|████████▋ | 253/291 [01:02<00:07,  5.40it/s]
Loading 0:  88%|████████▊ | 255/291 [01:03<00:07,  4.56it/s]
Loading 0:  88%|████████▊ | 256/291 [01:03<00:10,  3.46it/s]
Loading 0:  88%|████████▊ | 257/291 [01:04<00:12,  2.74it/s]
Loading 0:  89%|████████▉ | 260/291 [01:04<00:06,  4.66it/s]
Loading 0:  90%|████████▉ | 261/291 [01:04<00:06,  4.90it/s]
Loading 0:  91%|█████████ | 264/291 [01:05<00:05,  4.76it/s]
Loading 0:  91%|█████████ | 265/291 [01:05<00:07,  3.70it/s]
Loading 0:  91%|█████████▏| 266/291 [01:06<00:08,  2.96it/s]
Loading 0:  92%|█████████▏| 269/291 [01:06<00:04,  4.71it/s]
Loading 0:  93%|█████████▎| 270/291 [01:06<00:04,  4.93it/s]
Loading 0:  94%|█████████▍| 273/291 [01:07<00:03,  4.77it/s]
Loading 0:  94%|█████████▍| 274/291 [01:07<00:04,  3.73it/s]
Loading 0:  95%|█████████▍| 275/291 [01:08<00:05,  2.98it/s]
Loading 0:  96%|█████████▌| 278/291 [01:08<00:02,  4.76it/s]
Loading 0:  96%|█████████▌| 279/291 [01:08<00:02,  5.04it/s]
Loading 0:  97%|█████████▋| 281/291 [01:09<00:02,  4.15it/s]
Loading 0:  97%|█████████▋| 282/291 [01:10<00:02,  3.19it/s]
Loading 0:  98%|█████████▊| 284/291 [01:10<00:01,  4.24it/s]
Loading 0:  98%|█████████▊| 285/291 [01:10<00:01,  4.54it/s]
Loading 0:  99%|█████████▊| 287/291 [01:10<00:00,  5.33it/s]
Loading 0:  99%|█████████▉| 288/291 [01:11<00:00,  3.79it/s]
                                                            
Job alexdaoud-trainer-bagir-29250-v2-mkmlizer completed after 145.15s with status: succeeded
Stopping job with name alexdaoud-trainer-bagir-29250-v2-mkmlizer
Pipeline stage MKMLizer completed in 145.61s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.14s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service alexdaoud-trainer-bagir-29250-v2
Waiting for inference service alexdaoud-trainer-bagir-29250-v2 to be ready
Inference service alexdaoud-trainer-bagir-29250-v2 ready after 150.96651029586792s
Pipeline stage MKMLDeployer completed in 151.91s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.3791344165802s
Received healthy response to inference request in 2.6385340690612793s
Received healthy response to inference request in 3.5380873680114746s
Received healthy response to inference request in 4.037241458892822s
Received healthy response to inference request in 4.7884557247161865s
5 requests
0 failed requests
5th percentile: 2.7866541385650634
10th percentile: 2.9347742080688475
20th percentile: 3.231014347076416
30th percentile: 3.410925006866455
40th percentile: 3.4745061874389647
50th percentile: 3.5380873680114746
60th percentile: 3.7377490043640136
70th percentile: 3.9374106407165526
80th percentile: 4.187484312057495
90th percentile: 4.487970018386841
95th percentile: 4.638212871551514
99th percentile: 4.758407154083252
mean time: 3.6762906074523927
%s, retrying in %s seconds...
Received healthy response to inference request in 4.2671403884887695s
Received healthy response to inference request in 3.7543318271636963s
Received healthy response to inference request in 2.9013874530792236s
Received healthy response to inference request in 4.22029709815979s
Received healthy response to inference request in 3.389268636703491s
5 requests
0 failed requests
5th percentile: 2.9989636898040772
10th percentile: 3.096539926528931
20th percentile: 3.2916923999786376
30th percentile: 3.462281274795532
40th percentile: 3.608306550979614
50th percentile: 3.7543318271636963
60th percentile: 3.940717935562134
70th percentile: 4.127104043960571
80th percentile: 4.229665756225586
90th percentile: 4.248403072357178
95th percentile: 4.257771730422974
99th percentile: 4.26526665687561
mean time: 3.706485080718994
%s, retrying in %s seconds...
Received healthy response to inference request in 3.146211624145508s
Received healthy response to inference request in 4.176082611083984s
Received healthy response to inference request in 2.3083934783935547s
Received healthy response to inference request in 2.7848355770111084s
Received healthy response to inference request in 2.4251787662506104s
5 requests
0 failed requests
5th percentile: 2.3317505359649657
10th percentile: 2.3551075935363768
20th percentile: 2.4018217086791993
30th percentile: 2.49711012840271
40th percentile: 2.640972852706909
50th percentile: 2.7848355770111084
60th percentile: 2.9293859958648683
70th percentile: 3.0739364147186277
80th percentile: 3.352185821533203
90th percentile: 3.764134216308594
95th percentile: 3.970108413696289
99th percentile: 4.134887771606445
mean time: 2.968140411376953
Pipeline stage StressChecker completed in 55.61s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.67s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 1.11s
Shutdown handler de-registered
alexdaoud-trainer-bagir_29250_v2 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 4277.98s
Shutdown handler de-registered
alexdaoud-trainer-bagir_29250_v2 status is now inactive due to auto deactivation removed underperforming models
alexdaoud-trainer-bagir_29250_v2 status is now torndown due to DeploymentManager action