Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-rm-pointwise-exp-60494-v4-mkmlizer
Waiting for job on chaiml-rm-pointwise-exp-60494-v4-mkmlizer to finish
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: ║ ║
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: ║ ██████ ██████ █████ ████ ████ ║
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: ║ ░░██████ ██████ ░░███ ███░ ░░███ ║
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: ║ ░███░█████░███ ░███ ███ ░███ ║
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: ║ ░███░░███ ░███ ░███████ ░███ ║
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: ║ ░███ ░░░ ░███ ░███░░███ ░███ ║
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: ║ ░███ ░███ ░███ ░░███ ░███ ║
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: ║ █████ █████ █████ ░░████ █████ ║
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: ║ ░░░░░ ░░░░░ ░░░░░ ░░░░ ░░░░░ ║
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: ║ ║
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: ║ Version: 0.27.1+vampire_v3 ║
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: ║ Features: FLYWHEEL, CUDA ║
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: ║ Copyright 2023-2025 MK ONE TECHNOLOGIES Inc. ║
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: ║ https://mk1.ai ║
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: ║ ║
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: ║ The license key for the current software has been verified as ║
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: ║ belonging to: ║
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: ║ ║
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: ║ Chai Research Corp. ║
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: ║ Expiration: 2028-03-31 23:59:59 ║
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: ║ ║
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: Downloaded to shared memory in 23.350s
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: quantizing model to /dev/shm/model_cache, profile:t0, folder:/tmp/tmpoq_dr7i7, device:0
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: quantized model in 20.491s
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: Processed model ChaiML/rm-pointwise-exp1-llama3-8bins in 43.841s
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: creating bucket guanaco-mkml-models
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-rm-pointwise-exp-60494-v4
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-rm-pointwise-exp-60494-v4/special_tokens_map.json
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-rm-pointwise-exp-60494-v4/config.json
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-rm-pointwise-exp-60494-v4/tokenizer_config.json
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-rm-pointwise-exp-60494-v4/tokenizer.json
chaiml-rm-pointwise-exp-60494-v4-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-rm-pointwise-exp-60494-v4/flywheel_model.0.safetensors
chaiml-rm-pointwise-exp-60494-v4-mkmlizer:
Loading 0: 0%| | 0/291 [00:00<?, ?it/s]
Loading 0: 2%|▏ | 5/291 [00:00<00:08, 35.28it/s]
Loading 0: 4%|▍ | 13/291 [00:00<00:05, 55.32it/s]
Loading 0: 7%|▋ | 19/291 [00:00<00:05, 45.61it/s]
Loading 0: 8%|▊ | 24/291 [00:00<00:06, 43.44it/s]
Loading 0: 11%|█ | 31/291 [00:00<00:05, 49.05it/s]
Loading 0: 13%|█▎ | 37/291 [00:00<00:05, 43.92it/s]
Loading 0: 14%|█▍ | 42/291 [00:00<00:05, 41.75it/s]
Loading 0: 16%|█▋ | 48/291 [00:01<00:05, 45.10it/s]
Loading 0: 18%|█▊ | 53/291 [00:01<00:05, 42.49it/s]
Loading 0: 20%|█▉ | 58/291 [00:01<00:05, 43.37it/s]
Loading 0: 22%|██▏ | 64/291 [00:01<00:05, 41.47it/s]
Loading 0: 24%|██▎ | 69/291 [00:01<00:05, 41.16it/s]
Loading 0: 26%|██▌ | 76/291 [00:01<00:04, 45.97it/s]
Loading 0: 28%|██▊ | 81/291 [00:01<00:04, 44.60it/s]
Loading 0: 30%|██▉ | 86/291 [00:02<00:08, 24.19it/s]
Loading 0: 32%|███▏ | 94/291 [00:02<00:06, 32.43it/s]
Loading 0: 34%|███▍ | 99/291 [00:02<00:05, 35.25it/s]
Loading 0: 36%|███▌ | 104/291 [00:02<00:05, 31.78it/s]
Loading 0: 38%|███▊ | 112/291 [00:02<00:04, 40.16it/s]
Loading 0: 41%|████ | 118/291 [00:02<00:04, 39.89it/s]
Loading 0: 42%|████▏ | 123/291 [00:03<00:04, 39.86it/s]
Loading 0: 45%|████▍ | 130/291 [00:03<00:03, 45.76it/s]
Loading 0: 47%|████▋ | 136/291 [00:03<00:03, 43.77it/s]
Loading 0: 48%|████▊ | 141/291 [00:03<00:03, 42.42it/s]
Loading 0: 51%|█████ | 147/291 [00:03<00:03, 46.45it/s]
Loading 0: 52%|█████▏ | 152/291 [00:03<00:03, 45.32it/s]
Loading 0: 54%|█████▍ | 157/291 [00:03<00:02, 45.94it/s]
Loading 0: 56%|█████▌ | 163/291 [00:03<00:02, 43.54it/s]
Loading 0: 58%|█████▊ | 168/291 [00:04<00:02, 41.45it/s]
Loading 0: 60%|██████ | 176/291 [00:04<00:02, 50.58it/s]
Loading 0: 63%|██████▎ | 182/291 [00:04<00:02, 43.95it/s]
Loading 0: 64%|██████▍ | 187/291 [00:04<00:03, 30.87it/s]
Loading 0: 66%|██████▌ | 191/291 [00:04<00:03, 32.04it/s]
Loading 0: 67%|██████▋ | 195/291 [00:04<00:02, 32.29it/s]
Loading 0: 69%|██████▉ | 202/291 [00:05<00:02, 39.14it/s]
Loading 0: 71%|███████▏ | 208/291 [00:05<00:02, 38.44it/s]
Loading 0: 73%|███████▎ | 213/291 [00:05<00:02, 38.65it/s]
Loading 0: 75%|███████▌ | 219/291 [00:05<00:01, 42.44it/s]
Loading 0: 77%|███████▋ | 224/291 [00:05<00:01, 42.04it/s]
Loading 0: 79%|███████▊ | 229/291 [00:05<00:01, 42.78it/s]
Loading 0: 81%|████████ | 235/291 [00:05<00:01, 41.99it/s]
Loading 0: 82%|████████▏ | 240/291 [00:05<00:01, 41.96it/s]
Loading 0: 85%|████████▍ | 247/291 [00:06<00:00, 47.97it/s]
Loading 0: 87%|████████▋ | 253/291 [00:06<00:00, 46.21it/s]
Loading 0: 89%|████████▊ | 258/291 [00:06<00:00, 45.56it/s]
Loading 0: 91%|█████████ | 265/291 [00:06<00:00, 50.05it/s]
Loading 0: 93%|█████████▎| 271/291 [00:06<00:00, 46.55it/s]
Loading 0: 95%|█████████▍| 276/291 [00:06<00:00, 45.20it/s]
Loading 0: 97%|█████████▋| 282/291 [00:06<00:00, 42.05it/s]
Loading 0: 99%|█████████▊| 287/291 [00:07<00:00, 29.99it/s]
Job chaiml-rm-pointwise-exp-60494-v4-mkmlizer completed after 74.48s with status: succeeded
Stopping job with name chaiml-rm-pointwise-exp-60494-v4-mkmlizer
Pipeline stage MKMLizer completed in 75.31s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.16s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-rm-pointwise-exp-60494-v4
Waiting for inference service chaiml-rm-pointwise-exp-60494-v4 to be ready
Inference service chaiml-rm-pointwise-exp-60494-v4 ready after 100.42149615287781s
Pipeline stage MKMLDeployer completed in 101.16s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.321345090866089s
Received healthy response to inference request in 4.324468612670898s
Received healthy response to inference request in 5.357702732086182s
Received healthy response to inference request in 3.3061330318450928s
Received healthy response to inference request in 3.0509212017059326s
5 requests
0 failed requests
5th percentile: 3.1019635677337645
10th percentile: 3.1530059337615968
20th percentile: 3.255090665817261
30th percentile: 3.309175443649292
40th percentile: 3.3152602672576905
50th percentile: 3.321345090866089
60th percentile: 3.7225944995880127
70th percentile: 4.1238439083099365
80th percentile: 4.531115436553955
90th percentile: 4.9444090843200685
95th percentile: 5.151055908203125
99th percentile: 5.31637336730957
mean time: 3.872114133834839
%s, retrying in %s seconds...
Received healthy response to inference request in 6.396216869354248s
Received healthy response to inference request in 3.0260534286499023s
Received healthy response to inference request in 4.011125326156616s
Received healthy response to inference request in 3.4119062423706055s
Received healthy response to inference request in 4.259352684020996s
5 requests
0 failed requests
5th percentile: 3.103223991394043
10th percentile: 3.1803945541381835
20th percentile: 3.3347356796264647
30th percentile: 3.5317500591278077
40th percentile: 3.7714376926422117
50th percentile: 4.011125326156616
60th percentile: 4.110416269302368
70th percentile: 4.20970721244812
80th percentile: 4.686725521087647
90th percentile: 5.541471195220947
95th percentile: 5.968844032287597
99th percentile: 6.310742301940918
mean time: 4.220930910110473
%s, retrying in %s seconds...
Received healthy response to inference request in 3.2253756523132324s
Received healthy response to inference request in 3.085635185241699s
Received healthy response to inference request in 4.127419471740723s
Received healthy response to inference request in 4.10207724571228s
Received healthy response to inference request in 3.642421245574951s
5 requests
0 failed requests
5th percentile: 3.113583278656006
10th percentile: 3.1415313720703124
20th percentile: 3.1974275588989256
30th percentile: 3.308784770965576
40th percentile: 3.4756030082702636
50th percentile: 3.642421245574951
60th percentile: 3.826283645629883
70th percentile: 4.010146045684815
80th percentile: 4.107145690917969
90th percentile: 4.117282581329346
95th percentile: 4.122351026535034
99th percentile: 4.126405782699585
mean time: 3.636585760116577
clean up pipeline due to error=DeploymentChecksError('Unacceptable 70th percentile latency 4.010146045684815s')
Shutdown handler de-registered
chaiml-rm-pointwise-exp_60494_v4 status is now failed due to DeploymentManager action