junhua024-chai-06-full-_95931

developer_uid: junhua024
submission_id: junhua024-chai-06-full-_95931_v1
model_name: junhua024-chai-06-full-_95931_v1
model_group: junhua024/chai_06_full_0
status: torndown
timestamp: 2025-07-18T20:20:54+00:00
num_battles: 5955
num_wins: 2915
celo_rating: 1273.48
family_friendly_score: 0.5571999999999999
family_friendly_standard_error: 0.007024644617345421
submission_type: basic
model_repo: junhua024/chai_06_full_02102_1826
model_architecture: MistralForCausalLM
model_num_parameters: 12772070400.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
reward_model: default
latencies: [{'batch_size': 1, 'throughput': 0.5946885411627613, 'latency_mean': 1.6814268410205842, 'latency_p50': 1.6884007453918457, 'latency_p90': 1.8496347665786743}, {'batch_size': 3, 'throughput': 1.074230284244704, 'latency_mean': 2.786984419822693, 'latency_p50': 2.788718581199646, 'latency_p90': 3.1076530694961546}, {'batch_size': 5, 'throughput': 1.2769610447090782, 'latency_mean': 3.902537763118744, 'latency_p50': 3.922341227531433, 'latency_p90': 4.396473956108093}, {'batch_size': 6, 'throughput': 1.3193644864225758, 'latency_mean': 4.521017107963562, 'latency_p50': 4.54496169090271, 'latency_p90': 4.9925537109375}, {'batch_size': 8, 'throughput': 1.3920830751425999, 'latency_mean': 5.709266401529312, 'latency_p50': 5.762045621871948, 'latency_p90': 6.392054319381714}, {'batch_size': 10, 'throughput': 1.421238788131582, 'latency_mean': 6.98454801440239, 'latency_p50': 6.971498250961304, 'latency_p90': 7.917600917816162}]
gpu_counts: {'NVIDIA RTX A5000': 1}
display_name: junhua024-chai-06-full-_95931_v1
is_internal_developer: False
language_model: junhua024/chai_06_full_02102_1826
model_size: 13B
ranking_group: single
throughput_3p7s: 1.25
us_pacific_date: 2025-07-18
win_ratio: 0.489504617968094
generation_params: {'temperature': 1.0, 'top_p': 0.88, 'min_p': 0.0, 'top_k': 10, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name junhua024-chai-06-full-95931-v1-mkmlizer
Waiting for job on junhua024-chai-06-full-95931-v1-mkmlizer to finish
junhua024-chai-06-full-95931-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
junhua024-chai-06-full-95931-v1-mkmlizer: ║                                                                     ║
junhua024-chai-06-full-95931-v1-mkmlizer: ║   ██████   ██████  █████   ████  ████                               ║
junhua024-chai-06-full-95931-v1-mkmlizer: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
junhua024-chai-06-full-95931-v1-mkmlizer: ║   ░███░█████░███   ░███  ███     ░███                               ║
junhua024-chai-06-full-95931-v1-mkmlizer: ║   ░███░░███ ░███   ░███████      ░███                               ║
junhua024-chai-06-full-95931-v1-mkmlizer: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
junhua024-chai-06-full-95931-v1-mkmlizer: ║   ░███      ░███   ░███ ░░███    ░███                               ║
junhua024-chai-06-full-95931-v1-mkmlizer: ║   █████     █████  █████ ░░████  █████                              ║
junhua024-chai-06-full-95931-v1-mkmlizer: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
junhua024-chai-06-full-95931-v1-mkmlizer: ║                                                                     ║
junhua024-chai-06-full-95931-v1-mkmlizer: ║  Version: 0.29.15                                                   ║
junhua024-chai-06-full-95931-v1-mkmlizer: ║  Features: FLYWHEEL, CUDA                                           ║
junhua024-chai-06-full-95931-v1-mkmlizer: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
junhua024-chai-06-full-95931-v1-mkmlizer: ║  https://mk1.ai                                                     ║
junhua024-chai-06-full-95931-v1-mkmlizer: ║                                                                     ║
junhua024-chai-06-full-95931-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
junhua024-chai-06-full-95931-v1-mkmlizer: ║  belonging to:                                                      ║
junhua024-chai-06-full-95931-v1-mkmlizer: ║                                                                     ║
junhua024-chai-06-full-95931-v1-mkmlizer: ║  Chai Research Corp.                                                ║
junhua024-chai-06-full-95931-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
junhua024-chai-06-full-95931-v1-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
junhua024-chai-06-full-95931-v1-mkmlizer: ║                                                                     ║
junhua024-chai-06-full-95931-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
junhua024-chai-06-full-95931-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-06-full-95931-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-06-full-95931-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-06-full-95931-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-06-full-95931-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-06-full-95931-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-06-full-95931-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-06-full-95931-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-06-full-95931-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-06-full-95931-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-06-full-95931-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-06-full-95931-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-06-full-95931-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
Unable to record family friendly update due to error: Invalid JSON input: JSON must contain 'User Safety' and 'Response Safety' fields
junhua024-chai-06-full-95931-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-06-full-95931-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
Failed to get response for submission chaiml-bat-boys-azeril-_87348_v1: ('http://chaiml-bat-boys-azeril-87348-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
junhua024-chai-06-full-95931-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-06-full-95931-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
Failed to get response for submission junhua024-chai-12-full-_19228_v2: HTTPConnectionPool(host='junhua024-chai-12-full-19228-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
junhua024-chai-06-full-95931-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-06-full-95931-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-06-full-95931-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-06-full-95931-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-06-full-95931-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-06-full-95931-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-06-full-95931-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-06-full-95931-v1-mkmlizer: Xet Storage is enabled for this repo, but the 'hf_xet' package is not installed. Falling back to regular HTTP download. For better performance, install the package with: `pip install huggingface_hub[hf_xet]` or `pip install hf_xet`
junhua024-chai-06-full-95931-v1-mkmlizer: Downloaded to shared memory in 128.796s
junhua024-chai-06-full-95931-v1-mkmlizer: Checking if junhua024/chai_06_full_02102_1826 already exists in ChaiML
junhua024-chai-06-full-95931-v1-mkmlizer: Creating repo ChaiML/chai_06_full_02102_1826 and uploading /tmp/tmpsvr4hnj6 to it
Failed to get response for submission chaiml-daren-god-of-deat_6797_v1: HTTPConnectionPool(host='chaiml-daren-god-of-deat-6797-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission chaiml-gy-exp188-sft-gy_24525_v1: HTTPConnectionPool(host='chaiml-gy-exp188-sft-gy-24525-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission chaiml-simon-ghost-rile_65921_v1: HTTPConnectionPool(host='chaiml-simon-ghost-rile-65921-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
junhua024-chai-06-full-95931-v1-mkmlizer: 
  0%|          | 0/26 [00:00<?, ?it/s]
  4%|▍         | 1/26 [00:02<00:56,  2.24s/it]
  8%|▊         | 2/26 [00:03<00:41,  1.73s/it]
 12%|█▏        | 3/26 [00:04<00:35,  1.54s/it]
 15%|█▌        | 4/26 [00:06<00:35,  1.60s/it]
 19%|█▉        | 5/26 [00:11<00:58,  2.80s/it]
 23%|██▎       | 6/26 [00:12<00:46,  2.33s/it]
 27%|██▋       | 7/26 [00:14<00:38,  2.04s/it]
 31%|███       | 8/26 [00:15<00:32,  1.80s/it]
 35%|███▍      | 9/26 [00:17<00:29,  1.74s/it]
 38%|███▊      | 10/26 [00:19<00:30,  1.89s/it]
 42%|████▏     | 11/26 [00:20<00:26,  1.73s/it]
 46%|████▌     | 12/26 [00:22<00:23,  1.68s/it]
 50%|█████     | 13/26 [00:24<00:21,  1.66s/it]
 54%|█████▍    | 14/26 [00:26<00:20,  1.75s/it]
 58%|█████▊    | 15/26 [00:32<00:33,  3.06s/it]
 62%|██████▏   | 16/26 [00:33<00:25,  2.57s/it]
 65%|██████▌   | 17/26 [00:35<00:20,  2.24s/it]
 69%|██████▉   | 18/26 [00:36<00:15,  1.97s/it]
 73%|███████▎  | 19/26 [00:37<00:12,  1.77s/it]
 77%|███████▋  | 20/26 [00:39<00:10,  1.68s/it]
 81%|████████  | 21/26 [00:41<00:08,  1.74s/it]
 85%|████████▍ | 22/26 [00:47<00:12,  3.06s/it]
 88%|████████▊ | 23/26 [00:48<00:07,  2.56s/it]
 92%|█████████▏| 24/26 [00:49<00:04,  2.20s/it]
 96%|█████████▌| 25/26 [00:51<00:01,  1.94s/it]
100%|██████████| 26/26 [00:52<00:00,  1.70s/it]
100%|██████████| 26/26 [00:52<00:00,  2.02s/it]
junhua024-chai-06-full-95931-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpsvr4hnj6, device:0
junhua024-chai-06-full-95931-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Failed to get response for submission chaiml-daren-god-of-deat_6797_v1: HTTPConnectionPool(host='chaiml-daren-god-of-deat-6797-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission chaiml-simon-ghost-rile_65921_v1: HTTPConnectionPool(host='chaiml-simon-ghost-rile-65921-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission chaiml-gy-exp188-sft-gy_24525_v1: HTTPConnectionPool(host='chaiml-gy-exp188-sft-gy-24525-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
junhua024-chai-06-full-95931-v1-mkmlizer: quantized model in 30.860s
junhua024-chai-06-full-95931-v1-mkmlizer: Processed model junhua024/chai_06_full_02102_1826 in 238.253s
junhua024-chai-06-full-95931-v1-mkmlizer: creating bucket guanaco-mkml-models
junhua024-chai-06-full-95931-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
junhua024-chai-06-full-95931-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/junhua024-chai-06-full-95931-v1/nvidia
junhua024-chai-06-full-95931-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/junhua024-chai-06-full-95931-v1/nvidia/config.json
junhua024-chai-06-full-95931-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/junhua024-chai-06-full-95931-v1/nvidia/special_tokens_map.json
junhua024-chai-06-full-95931-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/junhua024-chai-06-full-95931-v1/nvidia/tokenizer_config.json
junhua024-chai-06-full-95931-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/junhua024-chai-06-full-95931-v1/nvidia/tokenizer.json
junhua024-chai-06-full-95931-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/junhua024-chai-06-full-95931-v1/nvidia/flywheel_model.0.safetensors
junhua024-chai-06-full-95931-v1-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|          | 2/363 [00:00<00:21, 16.91it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:20, 17.61it/s]
Loading 0:   3%|▎         | 12/363 [00:00<00:11, 29.41it/s]
Loading 0:   5%|▍         | 17/363 [00:00<00:11, 31.02it/s]
Loading 0:   6%|▋         | 23/363 [00:00<00:10, 33.10it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:07, 44.04it/s]
Loading 0:  10%|▉         | 36/363 [00:01<00:10, 32.61it/s]
Loading 0:  11%|█▏        | 41/363 [00:01<00:09, 33.48it/s]
Loading 0:  14%|█▍        | 50/363 [00:01<00:07, 41.00it/s]
Loading 0:  15%|█▌        | 55/363 [00:01<00:08, 35.11it/s]
Loading 0:  16%|█▋        | 59/363 [00:01<00:08, 34.54it/s]
Loading 0:  18%|█▊        | 65/363 [00:01<00:08, 35.59it/s]
Loading 0:  19%|█▉        | 69/363 [00:02<00:08, 34.32it/s]
Loading 0:  21%|██        | 75/363 [00:02<00:08, 33.97it/s]
Loading 0:  22%|██▏       | 80/363 [00:02<00:08, 31.83it/s]
Loading 0:  24%|██▎       | 86/363 [00:02<00:07, 37.12it/s]
Loading 0:  25%|██▌       | 91/363 [00:02<00:07, 37.15it/s]
Loading 0:  27%|██▋       | 97/363 [00:02<00:07, 37.61it/s]
Loading 0:  28%|██▊       | 101/363 [00:02<00:07, 37.11it/s]
Loading 0:  29%|██▉       | 105/363 [00:03<00:07, 34.72it/s]
Loading 0:  31%|███       | 113/363 [00:03<00:06, 40.51it/s]
Loading 0:  33%|███▎      | 118/363 [00:03<00:07, 34.38it/s]
Loading 0:  34%|███▎      | 122/363 [00:03<00:07, 33.42it/s]
Loading 0:  35%|███▌      | 128/363 [00:03<00:06, 34.34it/s]
Loading 0:  36%|███▋      | 132/363 [00:03<00:06, 33.78it/s]
Loading 0:  38%|███▊      | 138/363 [00:03<00:06, 33.85it/s]
Loading 0:  39%|███▉      | 143/363 [00:04<00:06, 33.31it/s]
Loading 0:  41%|████      | 149/363 [00:04<00:06, 34.69it/s]
Loading 0:  43%|████▎     | 155/363 [00:04<00:05, 39.60it/s]
Loading 0:  44%|████▍     | 160/363 [00:04<00:05, 37.54it/s]
Loading 0:  45%|████▌     | 164/363 [00:04<00:05, 36.22it/s]
Loading 0:  46%|████▋     | 168/363 [00:04<00:05, 33.40it/s]
Loading 0:  48%|████▊     | 175/363 [00:04<00:04, 41.93it/s]
Loading 0:  50%|████▉     | 180/363 [00:05<00:05, 31.67it/s]
Loading 0:  51%|█████     | 185/363 [00:05<00:05, 33.92it/s]
Loading 0:  53%|█████▎    | 191/363 [00:05<00:04, 35.50it/s]
Loading 0:  54%|█████▎    | 195/363 [00:05<00:04, 34.77it/s]
Loading 0:  55%|█████▌    | 201/363 [00:05<00:04, 34.38it/s]
Loading 0:  57%|█████▋    | 206/363 [00:05<00:04, 33.51it/s]
Loading 0:  58%|█████▊    | 212/363 [00:06<00:04, 34.35it/s]
Loading 0:  60%|██████    | 218/363 [00:06<00:03, 39.06it/s]
Loading 0:  61%|██████▏   | 223/363 [00:06<00:03, 36.48it/s]
Loading 0:  63%|██████▎   | 227/363 [00:06<00:03, 35.98it/s]
Loading 0:  64%|██████▎   | 231/363 [00:06<00:03, 33.91it/s]
Loading 0:  66%|██████▌   | 239/363 [00:06<00:03, 39.43it/s]
Loading 0:  67%|██████▋   | 243/363 [00:06<00:03, 32.49it/s]
Loading 0:  68%|██████▊   | 248/363 [00:07<00:03, 33.79it/s]
Loading 0:  70%|██████▉   | 254/363 [00:07<00:03, 35.32it/s]
Loading 0:  71%|███████   | 258/363 [00:07<00:03, 33.57it/s]
Loading 0:  72%|███████▏  | 263/363 [00:07<00:02, 36.90it/s]
Loading 0:  74%|███████▎  | 267/363 [00:07<00:02, 36.50it/s]
Loading 0:  75%|███████▍  | 271/363 [00:07<00:02, 31.77it/s]
Loading 0:  76%|███████▌  | 275/363 [00:07<00:02, 31.23it/s]
Loading 0:  77%|███████▋  | 281/363 [00:08<00:02, 37.20it/s]
Loading 0:  79%|███████▉  | 286/363 [00:08<00:02, 35.71it/s]
Loading 0:  80%|███████▉  | 290/363 [00:08<00:02, 35.40it/s]
Loading 0:  81%|████████  | 294/363 [00:08<00:02, 32.64it/s]
Loading 0:  83%|████████▎ | 302/363 [00:08<00:01, 38.83it/s]
Loading 0:  84%|████████▍ | 306/363 [00:08<00:01, 31.96it/s]
Loading 0:  86%|████████▌ | 311/363 [00:08<00:01, 33.53it/s]
Loading 0:  87%|████████▋ | 317/363 [00:09<00:01, 34.72it/s]
Loading 0:  88%|████████▊ | 321/363 [00:09<00:01, 33.81it/s]
Loading 0:  90%|████████▉ | 326/363 [00:09<00:00, 37.04it/s]
Loading 0:  91%|█████████ | 330/363 [00:09<00:00, 37.53it/s]
Loading 0:  92%|█████████▏| 334/363 [00:09<00:00, 33.11it/s]
Loading 0:  93%|█████████▎| 338/363 [00:09<00:00, 30.86it/s]
Loading 0:  95%|█████████▍| 344/363 [00:09<00:00, 36.96it/s]
Loading 0:  96%|█████████▌| 349/363 [00:10<00:00, 25.45it/s]
Loading 0:  97%|█████████▋| 353/363 [00:10<00:00, 23.80it/s]
Loading 0:  98%|█████████▊| 357/363 [00:10<00:00, 25.86it/s]
                                                            
Job junhua024-chai-06-full-95931-v1-mkmlizer completed after 264.33s with status: succeeded
Stopping job with name junhua024-chai-06-full-95931-v1-mkmlizer
Pipeline stage MKMLizer completed in 264.88s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.18s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service junhua024-chai-06-full-95931-v1
Waiting for inference service junhua024-chai-06-full-95931-v1 to be ready
Failed to get response for submission chaiml-gy-exp188-sft-gy_24525_v1: HTTPConnectionPool(host='chaiml-gy-exp188-sft-gy-24525-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission chaiml-daren-god-of-deat_6797_v1: HTTPConnectionPool(host='chaiml-daren-god-of-deat-6797-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission chaiml-bat-boys-azeril-_87348_v1: ('http://chaiml-bat-boys-azeril-87348-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission junhua024-chai-12-full-_19228_v2: HTTPConnectionPool(host='junhua024-chai-12-full-19228-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission chaiml-bat-boys-azeril-_87348_v1: ('http://chaiml-bat-boys-azeril-87348-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Failed to get response for submission chaiml-gy-exp188-sft-gy_24525_v1: HTTPConnectionPool(host='chaiml-gy-exp188-sft-gy-24525-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission chaiml-isaac-brown-sylu_53639_v2: HTTPConnectionPool(host='chaiml-isaac-brown-sylu-53639-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission chaiml-isaac-brown-sylu_53639_v2: HTTPConnectionPool(host='chaiml-isaac-brown-sylu-53639-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Unable to record family friendly update due to error: Invalid JSON input: JSON must contain 'User Safety' and 'Response Safety' fields
Failed to get response for submission junhua024-chai-12-full-_19228_v2: HTTPConnectionPool(host='junhua024-chai-12-full-19228-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission chaiml-daren-god-of-deat_6797_v1: HTTPConnectionPool(host='chaiml-daren-god-of-deat-6797-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission chaiml-gy-exp188-sft-gy_24525_v1: HTTPConnectionPool(host='chaiml-gy-exp188-sft-gy-24525-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Inference service junhua024-chai-06-full-95931-v1 ready after 322.0707004070282s
Pipeline stage MKMLDeployer completed in 322.72s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.53659987449646s
Received healthy response to inference request in 1.895979404449463s
Received healthy response to inference request in 1.637969732284546s
Received healthy response to inference request in 1.4963696002960205s
Received healthy response to inference request in 1.6545741558074951s
5 requests
0 failed requests
5th percentile: 1.5246896266937255
10th percentile: 1.5530096530914306
20th percentile: 1.6096497058868409
30th percentile: 1.6412906169891357
40th percentile: 1.6479323863983155
50th percentile: 1.6545741558074951
60th percentile: 1.7511362552642822
70th percentile: 1.8476983547210692
80th percentile: 2.0241034984588624
90th percentile: 2.280351686477661
95th percentile: 2.4084757804870605
99th percentile: 2.5109750556945802
mean time: 1.8442985534667968
Pipeline stage StressChecker completed in 11.70s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.82s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 1.24s
Shutdown handler de-registered
junhua024-chai-06-full-_95931_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage MKMLProfilerDeleter completed in 0.16s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.16s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service junhua024-chai-06-full-95931-v1-profiler
Waiting for inference service junhua024-chai-06-full-95931-v1-profiler to be ready
Inference service junhua024-chai-06-full-95931-v1-profiler ready after 324.1656982898712s
Pipeline stage MKMLProfilerDeployer completed in 325.18s
run pipeline stage %s
Running pipeline stage MKMLProfilerRunner
kubectl cp /code/guanaco/guanaco_inference_services/src/inference_scripts tenant-chaiml-guanaco/junhua024-chai-06-fu7610bf2f083f467df33534b229c7d63a-deplosxsr5:/code/chaiverse_profiler_1752871081 --namespace tenant-chaiml-guanaco
kubectl exec -it junhua024-chai-06-fu7610bf2f083f467df33534b229c7d63a-deplosxsr5 --namespace tenant-chaiml-guanaco -- sh -c 'cd /code/chaiverse_profiler_1752871081 && python profiles.py profile --best_of_n 8 --auto_batch 5 --batches 1,5,10,15,20,25,30,35,40,45,50,55,60,65,70,75,80,85,90,95,100,105,110,115,120,125,130,135,140,145,150,155,160,165,170,175,180,185,190,195 --samples 200 --input_tokens 1024 --output_tokens 64 --summary /code/chaiverse_profiler_1752871081/summary.json'
kubectl exec -it junhua024-chai-06-fu7610bf2f083f467df33534b229c7d63a-deplosxsr5 --namespace tenant-chaiml-guanaco -- bash -c 'cat /code/chaiverse_profiler_1752871081/summary.json'
Pipeline stage MKMLProfilerRunner completed in 1128.49s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Checking if service junhua024-chai-06-full-95931-v1-profiler is running
Tearing down inference service junhua024-chai-06-full-95931-v1-profiler
Service junhua024-chai-06-full-95931-v1-profiler has been torndown
Pipeline stage MKMLProfilerDeleter completed in 4.63s
Shutdown handler de-registered
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3953.38s
Shutdown handler de-registered
junhua024-chai-06-full-_95931_v1 status is now inactive due to auto deactivation removed underperforming models
junhua024-chai-06-full-_95931_v1 status is now torndown due to DeploymentManager action
junhua024-chai-06-full-_95931_v1 status is now torndown due to DeploymentManager action