developer_uid: rirv938
submission_id: rirv938-viral-40k-1248-_40308_v1
model_name: rirv938-viral-40k-1248-_40308_v1
model_group: rirv938/viral_40k_1248_b
status: torndown
timestamp: 2025-01-06T04:58:04+00:00
num_battles: 13737
num_wins: 7048
celo_rating: 1270.75
family_friendly_score: 0.5966
family_friendly_standard_error: 0.006937844622071036
submission_type: basic
model_repo: rirv938/viral_40k_1248_bo8_83w_dpo2_lr2_v2
model_architecture: MistralForCausalLM
model_num_parameters: 22247282688.0
best_of: 8
max_input_tokens: 768
max_output_tokens: 64
latencies: [{'batch_size': 1, 'throughput': 0.40270692185598356, 'latency_mean': 2.483101053237915, 'latency_p50': 2.47534441947937, 'latency_p90': 2.752796244621277}, {'batch_size': 3, 'throughput': 0.8397979893621885, 'latency_mean': 3.566678396463394, 'latency_p50': 3.548640489578247, 'latency_p90': 3.924363088607788}, {'batch_size': 5, 'throughput': 1.1119534005676068, 'latency_mean': 4.47482427239418, 'latency_p50': 4.450724363327026, 'latency_p90': 5.032526540756225}, {'batch_size': 6, 'throughput': 1.1891623021444613, 'latency_mean': 5.015175495147705, 'latency_p50': 5.044305801391602, 'latency_p90': 5.56858651638031}, {'batch_size': 10, 'throughput': 1.348993038187191, 'latency_mean': 7.365528769493103, 'latency_p50': 7.347730994224548, 'latency_p90': 8.308813643455505}]
gpu_counts: {'NVIDIA RTX A6000': 1}
display_name: rirv938-viral-40k-1248-_40308_v1
is_internal_developer: True
language_model: rirv938/viral_40k_1248_bo8_83w_dpo2_lr2_v2
model_size: 22B
ranking_group: single
throughput_3p7s: 0.89
us_pacific_date: 2025-01-05
win_ratio: 0.5130668996141807
generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '####', 'Bot:', 'User:', 'You:', '<|im_end|>', '<|eot_id|>'], 'max_input_tokens': 768, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name rirv938-viral-40k-1248-40308-v1-mkmlizer
Waiting for job on rirv938-viral-40k-1248-40308-v1-mkmlizer to finish
rirv938-viral-40k-1248-40308-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
rirv938-viral-40k-1248-40308-v1-mkmlizer: ║     _____            __           __                                ║
rirv938-viral-40k-1248-40308-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
rirv938-viral-40k-1248-40308-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
rirv938-viral-40k-1248-40308-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
rirv938-viral-40k-1248-40308-v1-mkmlizer: ║       /___/                                                         ║
rirv938-viral-40k-1248-40308-v1-mkmlizer: ║                                                                     ║
rirv938-viral-40k-1248-40308-v1-mkmlizer: ║  Version: 0.11.12                                                   ║
rirv938-viral-40k-1248-40308-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
rirv938-viral-40k-1248-40308-v1-mkmlizer: ║  https://mk1.ai                                                     ║
rirv938-viral-40k-1248-40308-v1-mkmlizer: ║                                                                     ║
rirv938-viral-40k-1248-40308-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
rirv938-viral-40k-1248-40308-v1-mkmlizer: ║  belonging to:                                                      ║
rirv938-viral-40k-1248-40308-v1-mkmlizer: ║                                                                     ║
rirv938-viral-40k-1248-40308-v1-mkmlizer: ║  Chai Research Corp.                                                ║
rirv938-viral-40k-1248-40308-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
rirv938-viral-40k-1248-40308-v1-mkmlizer: ║  Expiration: 2025-04-15 23:59:59                                    ║
rirv938-viral-40k-1248-40308-v1-mkmlizer: ║                                                                     ║
rirv938-viral-40k-1248-40308-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Failed to get response for submission nischaydnk-mistralnemo-_74255_v2: HTTPConnectionPool(host='nischaydnk-mistralnemo-74255-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission nischaydnk-mistralnemo-_74255_v2: HTTPConnectionPool(host='nischaydnk-mistralnemo-74255-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
rirv938-viral-40k-1248-40308-v1-mkmlizer: Downloaded to shared memory in 167.784s
rirv938-viral-40k-1248-40308-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp_jvorvgt, device:0
rirv938-viral-40k-1248-40308-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
rirv938-viral-40k-1248-40308-v1-mkmlizer: quantized model in 53.394s
rirv938-viral-40k-1248-40308-v1-mkmlizer: Processed model rirv938/viral_40k_1248_bo8_83w_dpo2_lr2_v2 in 221.179s
rirv938-viral-40k-1248-40308-v1-mkmlizer: creating bucket guanaco-mkml-models
rirv938-viral-40k-1248-40308-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
rirv938-viral-40k-1248-40308-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/rirv938-viral-40k-1248-40308-v1
rirv938-viral-40k-1248-40308-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/rirv938-viral-40k-1248-40308-v1/config.json
rirv938-viral-40k-1248-40308-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/rirv938-viral-40k-1248-40308-v1/special_tokens_map.json
rirv938-viral-40k-1248-40308-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/rirv938-viral-40k-1248-40308-v1/tokenizer_config.json
rirv938-viral-40k-1248-40308-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/rirv938-viral-40k-1248-40308-v1/tokenizer.json
Failed to get response for submission nischaydnk-mistralnemo-_74255_v2: HTTPConnectionPool(host='nischaydnk-mistralnemo-74255-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
rirv938-viral-40k-1248-40308-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/rirv938-viral-40k-1248-40308-v1/flywheel_model.1.safetensors
rirv938-viral-40k-1248-40308-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/rirv938-viral-40k-1248-40308-v1/flywheel_model.0.safetensors
rirv938-viral-40k-1248-40308-v1-mkmlizer: 
Loading 0:   0%|          | 0/507 [00:00<?, ?it/s]
Loading 0:   1%|          | 4/507 [00:00<00:15, 32.21it/s]
Loading 0:   2%|▏         | 8/507 [00:00<00:19, 25.83it/s]
Loading 0:   2%|▏         | 12/507 [00:00<00:19, 25.33it/s]
Loading 0:   3%|▎         | 15/507 [00:00<00:23, 21.25it/s]
Loading 0:   4%|▎         | 19/507 [00:00<00:19, 24.82it/s]
Loading 0:   5%|▍         | 23/507 [00:00<00:17, 27.71it/s]
Loading 0:   5%|▌         | 26/507 [00:01<00:25, 19.07it/s]
Loading 0:   6%|▌         | 29/507 [00:01<00:23, 20.31it/s]
Loading 0:   6%|▋         | 32/507 [00:01<00:27, 17.28it/s]
Loading 0:   7%|▋         | 37/507 [00:01<00:20, 22.89it/s]
Loading 0:   8%|▊         | 40/507 [00:01<00:19, 23.61it/s]
Loading 0:   8%|▊         | 43/507 [00:01<00:19, 23.27it/s]
Loading 0:   9%|▉         | 46/507 [00:02<00:19, 23.49it/s]
Loading 0:  10%|▉         | 50/507 [00:02<00:17, 26.44it/s]
Loading 0:  10%|█         | 53/507 [00:02<00:24, 18.57it/s]
Loading 0:  11%|█         | 56/507 [00:02<00:22, 19.64it/s]
Loading 0:  12%|█▏        | 59/507 [00:02<00:26, 16.75it/s]
Loading 0:  13%|█▎        | 64/507 [00:02<00:20, 21.70it/s]
Loading 0:  13%|█▎        | 67/507 [00:03<00:19, 22.62it/s]
Loading 0:  14%|█▍        | 70/507 [00:03<00:19, 22.18it/s]
Loading 0:  14%|█▍        | 73/507 [00:03<00:18, 22.93it/s]
Loading 0:  15%|█▌        | 77/507 [00:03<00:16, 25.85it/s]
Loading 0:  16%|█▌        | 80/507 [00:03<00:23, 18.37it/s]
Loading 0:  16%|█▋        | 83/507 [00:03<00:24, 17.55it/s]
Loading 0:  17%|█▋        | 87/507 [00:04<00:20, 20.78it/s]
Loading 0:  18%|█▊        | 91/507 [00:04<00:17, 24.33it/s]
Loading 0:  19%|█▊        | 94/507 [00:04<00:18, 22.89it/s]
Loading 0:  19%|█▉        | 97/507 [00:04<00:17, 22.97it/s]
Loading 0:  20%|█▉        | 100/507 [00:04<00:21, 18.63it/s]
Loading 0:  21%|██        | 105/507 [00:04<00:17, 23.42it/s]
Loading 0:  21%|██▏       | 108/507 [00:05<00:24, 16.48it/s]
Loading 0:  22%|██▏       | 112/507 [00:05<00:20, 19.43it/s]
Loading 0:  23%|██▎       | 115/507 [00:05<00:19, 19.67it/s]
Loading 0:  23%|██▎       | 118/507 [00:05<00:18, 20.89it/s]
Loading 0:  24%|██▍       | 121/507 [00:05<00:17, 21.66it/s]
Loading 0:  24%|██▍       | 124/507 [00:05<00:17, 21.37it/s]
Loading 0:  25%|██▌       | 127/507 [00:05<00:16, 22.45it/s]
Loading 0:  26%|██▌       | 131/507 [00:06<00:14, 25.69it/s]
Loading 0:  26%|██▋       | 134/507 [00:06<00:21, 17.47it/s]
Loading 0:  27%|██▋       | 137/507 [00:06<00:19, 18.75it/s]
Loading 0:  28%|██▊       | 140/507 [00:06<00:22, 15.96it/s]
Loading 0:  29%|██▊       | 145/507 [00:06<00:17, 20.89it/s]
Loading 0:  29%|██▉       | 148/507 [00:06<00:16, 21.68it/s]
Loading 0:  30%|██▉       | 151/507 [00:07<00:16, 21.39it/s]
Loading 0:  30%|███       | 154/507 [00:07<00:15, 22.27it/s]
Loading 0:  31%|███       | 158/507 [00:07<00:13, 25.11it/s]
Loading 0:  32%|███▏      | 161/507 [00:07<00:19, 17.60it/s]
Loading 0:  32%|███▏      | 164/507 [00:07<00:18, 18.60it/s]
Loading 0:  33%|███▎      | 167/507 [00:08<00:21, 15.96it/s]
Loading 0:  34%|███▍      | 172/507 [00:08<00:16, 20.67it/s]
Loading 0:  35%|███▍      | 175/507 [00:08<00:15, 21.20it/s]
Loading 0:  35%|███▌      | 178/507 [00:08<00:15, 21.10it/s]
Loading 0:  36%|███▌      | 181/507 [00:08<00:14, 22.19it/s]
Loading 0:  36%|███▋      | 185/507 [00:08<00:12, 25.26it/s]
Loading 0:  37%|███▋      | 188/507 [00:09<00:17, 17.78it/s]
Loading 0:  38%|███▊      | 191/507 [00:09<00:16, 18.88it/s]
Loading 0:  38%|███▊      | 194/507 [00:09<00:19, 16.00it/s]
Loading 0:  39%|███▉      | 199/507 [00:09<00:14, 20.83it/s]
Loading 0:  40%|███▉      | 202/507 [00:09<00:14, 21.53it/s]
Loading 0:  40%|████      | 205/507 [00:09<00:14, 21.43it/s]
Loading 0:  41%|████      | 208/507 [00:09<00:13, 22.58it/s]
Loading 0:  42%|████▏     | 212/507 [00:10<00:11, 25.71it/s]
Loading 0:  42%|████▏     | 215/507 [00:10<00:15, 18.46it/s]
Loading 0:  43%|████▎     | 218/507 [00:10<00:14, 19.39it/s]
Loading 0:  44%|████▎     | 221/507 [00:10<00:17, 16.23it/s]
Loading 0:  45%|████▍     | 226/507 [00:10<00:13, 21.13it/s]
Loading 0:  45%|████▌     | 229/507 [00:10<00:12, 21.93it/s]
Loading 0:  46%|████▌     | 232/507 [00:11<00:12, 21.79it/s]
Loading 0:  46%|████▋     | 235/507 [00:11<00:11, 23.07it/s]
Loading 0:  47%|████▋     | 239/507 [00:11<00:10, 26.53it/s]
Loading 0:  48%|████▊     | 242/507 [00:11<00:14, 18.40it/s]
Loading 0:  48%|████▊     | 245/507 [00:11<00:13, 19.53it/s]
Loading 0:  49%|████▉     | 248/507 [00:12<00:15, 16.54it/s]
Loading 0:  50%|████▉     | 253/507 [00:12<00:11, 21.38it/s]
Loading 0:  50%|█████     | 256/507 [00:12<00:11, 22.14it/s]
Loading 0:  51%|█████     | 259/507 [00:12<00:11, 21.86it/s]
Loading 0:  52%|█████▏    | 262/507 [00:12<00:10, 22.79it/s]
Loading 0:  52%|█████▏    | 266/507 [00:12<00:09, 25.73it/s]
Loading 0:  53%|█████▎    | 269/507 [00:12<00:12, 18.32it/s]
Loading 0:  54%|█████▎    | 272/507 [00:13<00:12, 19.39it/s]
Loading 0:  54%|█████▍    | 275/507 [00:13<00:14, 16.39it/s]
Loading 0:  55%|█████▌    | 280/507 [00:13<00:10, 21.27it/s]
Loading 0:  56%|█████▌    | 283/507 [00:13<00:10, 21.53it/s]
Loading 0:  56%|█████▋    | 286/507 [00:13<00:10, 21.35it/s]
Loading 0:  57%|█████▋    | 289/507 [00:13<00:09, 21.93it/s]
Loading 0:  58%|█████▊    | 293/507 [00:13<00:08, 25.09it/s]
Loading 0:  58%|█████▊    | 296/507 [00:14<00:11, 17.59it/s]
Loading 0:  59%|█████▉    | 299/507 [00:14<00:11, 18.80it/s]
Loading 0:  59%|█████▉    | 299/507 [00:24<00:11, 18.80it/s]
Loading 0:  59%|█████▉    | 300/507 [00:28<06:02,  1.75s/it]
Loading 0:  60%|█████▉    | 302/507 [00:29<04:34,  1.34s/it]
Loading 0:  60%|██████    | 306/507 [00:29<02:40,  1.25it/s]
Loading 0:  61%|██████    | 309/507 [00:29<01:52,  1.76it/s]
Loading 0:  61%|██████▏   | 311/507 [00:29<01:30,  2.18it/s]
Loading 0:  62%|██████▏   | 316/507 [00:29<00:50,  3.78it/s]
Loading 0:  63%|██████▎   | 320/507 [00:29<00:34,  5.43it/s]
Loading 0:  64%|██████▎   | 323/507 [00:30<00:29,  6.26it/s]
Loading 0:  64%|██████▍   | 326/507 [00:30<00:23,  7.83it/s]
Loading 0:  65%|██████▍   | 329/507 [00:30<00:20,  8.67it/s]
Loading 0:  66%|██████▌   | 334/507 [00:30<00:13, 12.64it/s]
Loading 0:  66%|██████▋   | 337/507 [00:30<00:11, 14.47it/s]
Loading 0:  67%|██████▋   | 340/507 [00:30<00:10, 15.93it/s]
Loading 0:  68%|██████▊   | 343/507 [00:31<00:09, 17.73it/s]
Loading 0:  68%|██████▊   | 347/507 [00:31<00:07, 21.40it/s]
Loading 0:  69%|██████▉   | 350/507 [00:31<00:09, 16.71it/s]
Loading 0:  70%|██████▉   | 353/507 [00:31<00:08, 18.17it/s]
Loading 0:  70%|███████   | 356/507 [00:31<00:09, 15.98it/s]
Loading 0:  71%|███████   | 361/507 [00:31<00:06, 20.89it/s]
Loading 0:  72%|███████▏  | 364/507 [00:32<00:06, 21.79it/s]
Loading 0:  72%|███████▏  | 367/507 [00:32<00:06, 21.68it/s]
Loading 0:  73%|███████▎  | 370/507 [00:32<00:06, 22.71it/s]
Loading 0:  74%|███████▍  | 374/507 [00:32<00:05, 26.00it/s]
Loading 0:  74%|███████▍  | 377/507 [00:32<00:06, 18.68it/s]
Loading 0:  75%|███████▍  | 380/507 [00:32<00:06, 19.60it/s]
Loading 0:  76%|███████▌  | 383/507 [00:33<00:07, 16.38it/s]
Loading 0:  77%|███████▋  | 388/507 [00:33<00:05, 21.15it/s]
Loading 0:  77%|███████▋  | 391/507 [00:33<00:05, 21.70it/s]
Loading 0:  78%|███████▊  | 394/507 [00:33<00:05, 21.44it/s]
Loading 0:  78%|███████▊  | 397/507 [00:33<00:04, 22.42it/s]
Loading 0:  79%|███████▉  | 401/507 [00:33<00:04, 25.56it/s]
Loading 0:  80%|███████▉  | 404/507 [00:34<00:05, 17.72it/s]
Loading 0:  80%|████████  | 407/507 [00:34<00:05, 18.99it/s]
Loading 0:  81%|████████  | 410/507 [00:34<00:05, 16.24it/s]
Loading 0:  82%|████████▏ | 415/507 [00:34<00:04, 21.09it/s]
Loading 0:  82%|████████▏ | 418/507 [00:34<00:04, 21.73it/s]
Loading 0:  83%|████████▎ | 421/507 [00:34<00:03, 21.56it/s]
Loading 0:  84%|████████▎ | 424/507 [00:35<00:03, 22.62it/s]
Loading 0:  84%|████████▍ | 428/507 [00:35<00:03, 25.65it/s]
Loading 0:  85%|████████▌ | 431/507 [00:35<00:04, 17.85it/s]
Loading 0:  86%|████████▌ | 434/507 [00:35<00:03, 19.07it/s]
Loading 0:  86%|████████▌ | 437/507 [00:35<00:04, 16.17it/s]
Loading 0:  87%|████████▋ | 442/507 [00:35<00:03, 21.11it/s]
Loading 0:  88%|████████▊ | 445/507 [00:36<00:02, 21.88it/s]
Loading 0:  88%|████████▊ | 448/507 [00:36<00:02, 21.59it/s]
Loading 0:  89%|████████▉ | 451/507 [00:36<00:02, 22.35it/s]
Loading 0:  90%|████████▉ | 455/507 [00:36<00:02, 25.62it/s]
Loading 0:  90%|█████████ | 458/507 [00:36<00:02, 18.04it/s]
Loading 0:  91%|█████████ | 461/507 [00:36<00:02, 18.72it/s]
Loading 0:  92%|█████████▏| 464/507 [00:37<00:02, 16.16it/s]
Loading 0:  93%|█████████▎| 469/507 [00:37<00:01, 21.09it/s]
Loading 0:  93%|█████████▎| 472/507 [00:37<00:01, 21.71it/s]
Loading 0:  94%|█████████▎| 475/507 [00:37<00:01, 21.41it/s]
Loading 0:  94%|█████████▍| 478/507 [00:37<00:01, 22.14it/s]
Loading 0:  95%|█████████▌| 482/507 [00:37<00:00, 25.24it/s]
Loading 0:  96%|█████████▌| 485/507 [00:40<00:05,  4.21it/s]
Loading 0:  96%|█████████▌| 487/507 [00:40<00:04,  4.80it/s]
Loading 0:  97%|█████████▋| 491/507 [00:40<00:02,  7.06it/s]
Loading 0:  97%|█████████▋| 494/507 [00:40<00:01,  8.79it/s]
Loading 0:  98%|█████████▊| 497/507 [00:40<00:00, 10.98it/s]
Loading 0:  99%|█████████▊| 500/507 [00:40<00:00, 12.86it/s]
Loading 0:  99%|█████████▉| 503/507 [00:40<00:00, 14.80it/s]
                                                            

Job rirv938-viral-40k-1248-40308-v1-mkmlizer completed after 248.57s with status: succeeded
Stopping job with name rirv938-viral-40k-1248-40308-v1-mkmlizer
Pipeline stage MKMLizer completed in 249.05s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.16s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service rirv938-viral-40k-1248-40308-v1
Waiting for inference service rirv938-viral-40k-1248-40308-v1 to be ready
Failed to get response for submission nischaydnk-mistralnemo-_74255_v2: HTTPConnectionPool(host='nischaydnk-mistralnemo-74255-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission nischaydnk-mistralnemo-_74255_v2: HTTPConnectionPool(host='nischaydnk-mistralnemo-74255-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission nischaydnk-mistralnemo-_74255_v2: HTTPConnectionPool(host='nischaydnk-mistralnemo-74255-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission nischaydnk-mistralnemo-_74255_v2: HTTPConnectionPool(host='nischaydnk-mistralnemo-74255-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Inference service rirv938-viral-40k-1248-40308-v1 ready after 341.6493294239044s
Pipeline stage MKMLDeployer completed in 342.46s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.082063674926758s
Failed to get response for submission nischaydnk-mistralnemo-_74255_v2: HTTPConnectionPool(host='nischaydnk-mistralnemo-74255-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Received healthy response to inference request in 2.544670581817627s
Received healthy response to inference request in 2.4783456325531006s
Failed to get response for submission nischaydnk-mistralnemo-_74255_v2: HTTPConnectionPool(host='nischaydnk-mistralnemo-74255-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Received healthy response to inference request in 2.8939547538757324s
Received healthy response to inference request in 2.6465344429016113s
5 requests
0 failed requests
5th percentile: 2.491610622406006
10th percentile: 2.5048756122589113
20th percentile: 2.5314055919647216
30th percentile: 2.5650433540344237
40th percentile: 2.6057888984680178
50th percentile: 2.6465344429016113
60th percentile: 2.74550256729126
70th percentile: 2.8444706916809084
80th percentile: 2.9315765380859373
90th percentile: 3.0068201065063476
95th percentile: 3.044441890716553
99th percentile: 3.074539318084717
mean time: 2.729113817214966
Pipeline stage StressChecker completed in 15.21s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.67s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.66s
Shutdown handler de-registered
rirv938-viral-40k-1248-_40308_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
clean up pipeline due to error=DeploymentChecksError("('http://rirv938-viral-40k-1248-40308-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')")
Shutdown handler de-registered
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
clean up pipeline due to error=DeploymentChecksError("('http://rirv938-viral-40k-1248-40308-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')")
Shutdown handler de-registered
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 6148.83s
Shutdown handler de-registered
rirv938-viral-40k-1248-_40308_v1 status is now inactive due to auto deactivation removed underperforming models
rirv938-viral-40k-1248-_40308_v1 status is now torndown due to DeploymentManager action
rirv938-viral-40k-1248-_40308_v1 status is now torndown due to DeploymentManager action
rirv938-viral-40k-1248-_40308_v1 status is now torndown due to DeploymentManager action
ChatRequest
Generation Params
Prompt Formatter
Chat History
ChatMessage 1