cloudyu-s1-llama-3-2-3bx4-moe

submission_id: cloudyu-s1-llama-3-2-3bx4-moe_v1
developer_uid: cloudyu
status: failed
model_repo: cloudyu/S1-Llama-3.2-3Bx4-MoE
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
timestamp: 2025-02-07T03:30:27+00:00
model_name: cloudyu-s1-llama-3-2-3bx4-moe_v1
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name cloudyu-s1-llama-3-2-3bx4-moe-v1-mkmlizer
Waiting for job on cloudyu-s1-llama-3-2-3bx4-moe-v1-mkmlizer to finish
cloudyu-s1-llama-3-2-3bx4-moe-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
cloudyu-s1-llama-3-2-3bx4-moe-v1-mkmlizer: ║     _____            __           __                                ║
cloudyu-s1-llama-3-2-3bx4-moe-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
cloudyu-s1-llama-3-2-3bx4-moe-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
cloudyu-s1-llama-3-2-3bx4-moe-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
cloudyu-s1-llama-3-2-3bx4-moe-v1-mkmlizer: ║       /___/                                                         ║
cloudyu-s1-llama-3-2-3bx4-moe-v1-mkmlizer: ║                                                                     ║
cloudyu-s1-llama-3-2-3bx4-moe-v1-mkmlizer: ║  Version: 0.11.12                                                   ║
cloudyu-s1-llama-3-2-3bx4-moe-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
cloudyu-s1-llama-3-2-3bx4-moe-v1-mkmlizer: ║  https://mk1.ai                                                     ║
cloudyu-s1-llama-3-2-3bx4-moe-v1-mkmlizer: ║                                                                     ║
cloudyu-s1-llama-3-2-3bx4-moe-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
cloudyu-s1-llama-3-2-3bx4-moe-v1-mkmlizer: ║  belonging to:                                                      ║
cloudyu-s1-llama-3-2-3bx4-moe-v1-mkmlizer: ║                                                                     ║
cloudyu-s1-llama-3-2-3bx4-moe-v1-mkmlizer: ║  Chai Research Corp.                                                ║
cloudyu-s1-llama-3-2-3bx4-moe-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
cloudyu-s1-llama-3-2-3bx4-moe-v1-mkmlizer: ║  Expiration: 2025-04-15 23:59:59                                    ║
cloudyu-s1-llama-3-2-3bx4-moe-v1-mkmlizer: ║                                                                     ║
cloudyu-s1-llama-3-2-3bx4-moe-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
cloudyu-s1-llama-3-2-3bx4-moe-v1-mkmlizer: Downloaded to shared memory in 72.681s
cloudyu-s1-llama-3-2-3bx4-moe-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp67h3tyvm, device:0
cloudyu-s1-llama-3-2-3bx4-moe-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
cloudyu-s1-llama-3-2-3bx4-moe-v1-mkmlizer: quantized model in 30.443s
cloudyu-s1-llama-3-2-3bx4-moe-v1-mkmlizer: Processed model cloudyu/S1-Llama-3.2-3Bx4-MoE in 103.124s
cloudyu-s1-llama-3-2-3bx4-moe-v1-mkmlizer: creating bucket guanaco-mkml-models
cloudyu-s1-llama-3-2-3bx4-moe-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
cloudyu-s1-llama-3-2-3bx4-moe-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/cloudyu-s1-llama-3-2-3bx4-moe-v1
cloudyu-s1-llama-3-2-3bx4-moe-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/cloudyu-s1-llama-3-2-3bx4-moe-v1/tokenizer.json
cloudyu-s1-llama-3-2-3bx4-moe-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/cloudyu-s1-llama-3-2-3bx4-moe-v1/flywheel_model.0.safetensors
cloudyu-s1-llama-3-2-3bx4-moe-v1-mkmlizer: 
Loading 0:   0%|          | 0/534 [00:00<?, ?it/s]
Loading 0:   1%|          | 4/534 [00:00<00:16, 31.78it/s]
Loading 0:   1%|▏         | 8/534 [00:00<00:19, 27.65it/s]
Loading 0:   2%|▏         | 11/534 [00:00<00:18, 28.15it/s]
Loading 0:   4%|▎         | 19/534 [00:00<00:11, 45.57it/s]
Loading 0:   4%|▍         | 24/534 [00:00<00:13, 37.05it/s]
Loading 0:   5%|▌         | 29/534 [00:00<00:15, 32.97it/s]
Loading 0:   6%|▌         | 33/534 [00:00<00:15, 33.05it/s]
Loading 0:   8%|▊         | 41/534 [00:01<00:11, 43.62it/s]
Loading 0:   9%|▊         | 46/534 [00:01<00:13, 36.19it/s]
Loading 0:  10%|▉         | 52/534 [00:01<00:16, 29.32it/s]
Loading 0:  10%|█         | 56/534 [00:01<00:17, 27.05it/s]
Loading 0:  11%|█▏        | 61/534 [00:01<00:15, 31.10it/s]
Loading 0:  12%|█▏        | 65/534 [00:01<00:15, 30.82it/s]
Loading 0:  13%|█▎        | 69/534 [00:02<00:15, 30.84it/s]
Loading 0:  14%|█▍        | 76/534 [00:02<00:11, 39.68it/s]
Loading 0:  15%|█▌        | 81/534 [00:02<00:13, 33.74it/s]
Loading 0:  16%|█▌        | 85/534 [00:02<00:13, 32.70it/s]
Loading 0:  17%|█▋        | 89/534 [00:02<00:15, 28.68it/s]
Loading 0:  18%|█▊        | 98/534 [00:02<00:10, 40.92it/s]
Loading 0:  19%|█▉        | 103/534 [00:03<00:12, 34.53it/s]
Loading 0:  20%|██        | 108/534 [00:03<00:13, 31.19it/s]
Loading 0:  22%|██▏       | 117/534 [00:03<00:09, 41.86it/s]
Loading 0:  23%|██▎       | 122/534 [00:03<00:09, 43.01it/s]
Loading 0:  24%|██▍       | 127/534 [00:03<00:13, 30.83it/s]
Loading 0:  25%|██▍       | 131/534 [00:03<00:12, 32.23it/s]
Loading 0:  25%|██▌       | 135/534 [00:04<00:13, 28.67it/s]
Loading 0:  26%|██▌       | 139/534 [00:04<00:13, 30.36it/s]
Loading 0:  27%|██▋       | 143/534 [00:04<00:12, 31.84it/s]
Loading 0:  28%|██▊       | 147/534 [00:04<00:12, 31.69it/s]
Loading 0:  28%|██▊       | 151/534 [00:04<00:11, 32.00it/s]
Loading 0:  30%|██▉       | 159/534 [00:04<00:09, 37.71it/s]
Loading 0:  31%|███       | 163/534 [00:04<00:10, 35.78it/s]
Loading 0:  31%|███▏      | 167/534 [00:04<00:10, 35.07it/s]
Loading 0:  32%|███▏      | 171/534 [00:05<00:11, 30.49it/s]
Loading 0:  34%|███▎      | 180/534 [00:05<00:08, 43.05it/s]
Loading 0:  35%|███▍      | 185/534 [00:05<00:09, 36.44it/s]
Loading 0:  36%|███▌      | 190/534 [00:05<00:10, 32.33it/s]
Loading 0:  37%|███▋      | 198/534 [00:05<00:11, 30.37it/s]
Loading 0:  38%|███▊      | 202/534 [00:06<00:10, 30.83it/s]
Loading 0:  39%|███▊      | 206/534 [00:06<00:10, 31.16it/s]
Loading 0:  40%|███▉      | 211/534 [00:06<00:10, 31.23it/s]
Loading 0:  40%|████      | 216/534 [00:06<00:09, 33.08it/s]
Loading 0:  41%|████      | 220/534 [00:06<00:09, 32.91it/s]
Loading 0:  43%|████▎     | 228/534 [00:06<00:07, 43.64it/s]
Loading 0:  44%|████▎     | 233/534 [00:06<00:07, 37.98it/s]
Loading 0:  45%|████▍     | 238/534 [00:07<00:08, 33.62it/s]
Loading 0:  45%|████▌     | 242/534 [00:07<00:08, 34.76it/s]
Loading 0:  47%|████▋     | 249/534 [00:07<00:07, 37.34it/s]
Loading 0:  48%|████▊     | 254/534 [00:07<00:07, 36.05it/s]
Loading 0:  48%|████▊     | 258/534 [00:07<00:07, 34.82it/s]
Loading 0:  50%|████▉     | 265/534 [00:07<00:09, 27.63it/s]
Loading 0:  51%|█████     | 270/534 [00:08<00:08, 31.30it/s]
Loading 0:  51%|█████▏    | 274/534 [00:08<00:08, 31.85it/s]
Loading 0:  52%|█████▏    | 278/534 [00:08<00:08, 31.91it/s]
Loading 0:  54%|█████▎    | 286/534 [00:08<00:06, 37.98it/s]
Loading 0:  54%|█████▍    | 290/534 [00:08<00:06, 37.24it/s]
Loading 0:  55%|█████▌    | 294/534 [00:08<00:06, 36.62it/s]
Loading 0:  56%|█████▌    | 298/534 [00:08<00:07, 31.21it/s]
Loading 0:  57%|█████▋    | 306/534 [00:09<00:06, 37.05it/s]
Loading 0:  58%|█████▊    | 311/534 [00:09<00:06, 36.98it/s]
Loading 0:  59%|█████▉    | 315/534 [00:09<00:06, 35.90it/s]
Loading 0:  60%|██████    | 323/534 [00:09<00:04, 46.18it/s]
Loading 0:  61%|██████▏   | 328/534 [00:09<00:05, 38.43it/s]
Loading 0:  62%|██████▏   | 333/534 [00:09<00:05, 38.94it/s]
Loading 0:  63%|██████▎   | 338/534 [00:10<00:07, 25.23it/s]
Loading 0:  64%|██████▍   | 342/534 [00:10<00:07, 27.35it/s]
Loading 0:  65%|██████▍   | 346/534 [00:10<00:06, 29.36it/s]
Loading 0:  66%|██████▌   | 350/534 [00:10<00:06, 29.93it/s]
Loading 0:  66%|██████▋   | 354/534 [00:10<00:05, 30.84it/s]
Loading 0:  68%|██████▊   | 362/534 [00:10<00:04, 38.20it/s]
Loading 0:  69%|██████▊   | 367/534 [00:10<00:04, 38.04it/s]
Loading 0:  69%|██████▉   | 371/534 [00:10<00:04, 32.63it/s]
Loading 0:  70%|███████   | 375/534 [00:11<00:04, 34.10it/s]
Loading 0:  72%|███████▏  | 382/534 [00:11<00:04, 37.59it/s]
Loading 0:  72%|███████▏  | 387/534 [00:11<00:03, 37.26it/s]
Loading 0:  73%|███████▎  | 391/534 [00:11<00:04, 35.07it/s]
Loading 0:  75%|███████▍  | 399/534 [00:11<00:02, 45.49it/s]
Loading 0:  76%|███████▌  | 404/534 [00:11<00:03, 39.10it/s]
Loading 0:  77%|███████▋  | 411/534 [00:12<00:04, 29.77it/s]
Loading 0:  78%|███████▊  | 415/534 [00:12<00:03, 30.25it/s]
Loading 0:  79%|███████▊  | 420/534 [00:12<00:03, 29.62it/s]
Loading 0:  80%|███████▉  | 425/534 [00:12<00:03, 30.61it/s]
Loading 0:  80%|████████  | 429/534 [00:12<00:03, 30.78it/s]
Loading 0:  82%|████████▏ | 436/534 [00:12<00:02, 38.92it/s]
Loading 0:  83%|████████▎ | 441/534 [00:12<00:02, 34.80it/s]
Loading 0:  83%|████████▎ | 445/534 [00:13<00:02, 34.03it/s]
Loading 0:  84%|████████▍ | 449/534 [00:13<00:02, 33.26it/s]
Loading 0:  86%|████████▌ | 457/534 [00:13<00:01, 38.99it/s]
Loading 0:  86%|████████▋ | 461/534 [00:13<00:01, 36.68it/s]
Loading 0:  87%|████████▋ | 465/534 [00:13<00:02, 34.30it/s]
Loading 0:  88%|████████▊ | 469/534 [00:13<00:02, 29.60it/s]
Loading 0:  89%|████████▉ | 477/534 [00:14<00:01, 36.30it/s]
Loading 0:  91%|█████████ | 484/534 [00:14<00:01, 30.42it/s]
Loading 0:  91%|█████████▏| 488/534 [00:14<00:01, 30.68it/s]
Loading 0:  92%|█████████▏| 492/534 [00:14<00:01, 31.15it/s]
Loading 0:  93%|█████████▎| 496/534 [00:14<00:01, 29.35it/s]
Loading 0:  94%|█████████▍| 501/534 [00:14<00:01, 31.40it/s]
Loading 0:  95%|█████████▍| 505/534 [00:14<00:00, 31.37it/s]
Loading 0:  96%|█████████▌| 511/534 [00:15<00:00, 37.79it/s]
Loading 0:  97%|█████████▋| 516/534 [00:15<00:00, 34.64it/s]
Loading 0:  97%|█████████▋| 520/534 [00:15<00:00, 34.89it/s]
Loading 0:  98%|█████████▊| 524/534 [00:15<00:00, 34.60it/s]
Loading 0:  99%|█████████▉| 531/534 [00:15<00:00, 43.16it/s]
                                                            
Job cloudyu-s1-llama-3-2-3bx4-moe-v1-mkmlizer completed after 124.21s with status: succeeded
Stopping job with name cloudyu-s1-llama-3-2-3bx4-moe-v1-mkmlizer
Pipeline stage MKMLizer completed in 124.74s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service cloudyu-s1-llama-3-2-3bx4-moe-v1
Waiting for inference service cloudyu-s1-llama-3-2-3bx4-moe-v1 to be ready
Inference service cloudyu-s1-llama-3-2-3bx4-moe-v1 ready after 180.648695230484s
Pipeline stage MKMLDeployer completed in 181.15s
run pipeline stage %s
Running pipeline stage StressChecker
{"detail":"HTTPConnectionPool(host='cloudyu-s1-llama-3-2-3bx4-moe-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
{"detail":"HTTPConnectionPool(host='cloudyu-s1-llama-3-2-3bx4-moe-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
{"detail":"HTTPConnectionPool(host='cloudyu-s1-llama-3-2-3bx4-moe-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
{"detail":"HTTPConnectionPool(host='cloudyu-s1-llama-3-2-3bx4-moe-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
{"detail":"HTTPConnectionPool(host='cloudyu-s1-llama-3-2-3bx4-moe-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
5 requests
5 failed requests
5th percentile: 12.206604194641113
10th percentile: 12.207165384292603
20th percentile: 12.208287763595582
30th percentile: 12.210694217681885
40th percentile: 12.214384746551513
50th percentile: 12.218075275421143
60th percentile: 12.229148769378662
70th percentile: 12.240222263336182
80th percentile: 12.255975770950318
90th percentile: 12.27640929222107
95th percentile: 12.286626052856445
99th percentile: 12.294799461364747
mean time: 12.23511381149292
%s, retrying in %s seconds...
{"detail":"HTTPConnectionPool(host='cloudyu-s1-llama-3-2-3bx4-moe-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
{"detail":"HTTPConnectionPool(host='cloudyu-s1-llama-3-2-3bx4-moe-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
{"detail":"HTTPConnectionPool(host='cloudyu-s1-llama-3-2-3bx4-moe-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
{"detail":"HTTPConnectionPool(host='cloudyu-s1-llama-3-2-3bx4-moe-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
{"detail":"HTTPConnectionPool(host='cloudyu-s1-llama-3-2-3bx4-moe-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
5 requests
5 failed requests
5th percentile: 12.207643461227416
10th percentile: 12.208773994445801
20th percentile: 12.211035060882569
30th percentile: 12.213594961166383
40th percentile: 12.216453695297242
50th percentile: 12.2193124294281
60th percentile: 12.219386434555053
70th percentile: 12.219460439682006
80th percentile: 12.222506666183472
90th percentile: 12.228525114059448
95th percentile: 12.231534337997436
99th percentile: 12.233941717147827
mean time: 12.218406391143798
%s, retrying in %s seconds...
{"detail":"HTTPConnectionPool(host='cloudyu-s1-llama-3-2-3bx4-moe-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
{"detail":"HTTPConnectionPool(host='cloudyu-s1-llama-3-2-3bx4-moe-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
{"detail":"HTTPConnectionPool(host='cloudyu-s1-llama-3-2-3bx4-moe-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
{"detail":"HTTPConnectionPool(host='cloudyu-s1-llama-3-2-3bx4-moe-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
{"detail":"HTTPConnectionPool(host='cloudyu-s1-llama-3-2-3bx4-moe-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)"}
Received unhealthy response to inference request!
5 requests
5 failed requests
5th percentile: 12.23935809135437
10th percentile: 12.24023370742798
20th percentile: 12.241984939575195
30th percentile: 12.244833421707153
40th percentile: 12.248779153823852
50th percentile: 12.252724885940552
60th percentile: 12.263536977767945
70th percentile: 12.274349069595337
80th percentile: 12.338612413406372
90th percentile: 12.45632700920105
95th percentile: 12.515184307098389
99th percentile: 12.56227014541626
mean time: 12.317572927474975
clean up pipeline due to error=DeploymentChecksError('Unacceptable number of predict errors: 100.0%')
Shutdown handler de-registered
cloudyu-s1-llama-3-2-3bx4-moe_v1 status is now failed due to DeploymentManager action