chaiml-nis-rm-new-datav1-_436

developer_uid: NischayDnk
submission_id: chaiml-nis-rm-new-datav1-_436_v4
model_name: chaiml-nis-rm-new-datav1-_436_v4
model_group: ChaiML/nis-rm-new-datav1
status: torndown
timestamp: 2025-06-17T16:00:51+00:00
num_battles: 10226
num_wins: 4891
celo_rating: 1272.19
family_friendly_score: 0.5438000000000001
family_friendly_standard_error: 0.0070438847236450434
submission_type: basic
model_repo: ChaiML/nis-rm-new-datav1-400k-llama8b-prmflow-256len-s40
model_architecture: LlamaForSequenceClassification
model_num_parameters: 8030261248.0
best_of: 8
max_input_tokens: 256
max_output_tokens: 64
reward_model: default
display_name: chaiml-nis-rm-new-datav1-_436_v4
is_internal_developer: False
language_model: ChaiML/nis-rm-new-datav1-400k-llama8b-prmflow-256len-s40
model_size: 8B
ranking_group: single
us_pacific_date: 2025-06-17
win_ratio: 0.47829063172305886
generation_params: {'temperature': 0.55, 'top_p': 0.95, 'min_p': 0.025, 'top_k': 60, 'presence_penalty': 0.35, 'frequency_penalty': 0.35, 'stopping_words': ['\n', '<|im_end|>'], 'max_input_tokens': 256, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-nis-rm-new-datav1-436-v4-mkmlizer
Waiting for job on chaiml-nis-rm-new-datav1-436-v4-mkmlizer to finish
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: ║                                                                     ║
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: ║   ██████   ██████  █████   ████  ████                               ║
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: ║   ░███░█████░███   ░███  ███     ░███                               ║
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: ║   ░███░░███ ░███   ░███████      ░███                               ║
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: ║   ░███      ░███   ░███ ░░███    ░███                               ║
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: ║   █████     █████  █████ ░░████  █████                              ║
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: ║                                                                     ║
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: ║  Version: 0.27.1+vampire_v3                                         ║
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: ║  Features: FLYWHEEL, CUDA                                           ║
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: ║                                                                     ║
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: ║  belonging to:                                                      ║
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: ║                                                                     ║
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: ║                                                                     ║
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: Downloaded to shared memory in 18.412s
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: quantizing model to /dev/shm/model_cache, profile:t0, folder:/tmp/tmp_pc980oj, device:0
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: quantized model in 19.601s
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: Processed model ChaiML/nis-rm-new-datav1-400k-llama8b-prmflow-256len-s40 in 38.013s
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: creating bucket guanaco-mkml-models
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-nis-rm-new-datav1-436-v4
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-nis-rm-new-datav1-436-v4/special_tokens_map.json
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-nis-rm-new-datav1-436-v4/config.json
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-nis-rm-new-datav1-436-v4/tokenizer_config.json
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-nis-rm-new-datav1-436-v4/tokenizer.json
chaiml-nis-rm-new-datav1-436-v4-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 5/291 [00:00<00:07, 40.22it/s]
Loading 0:   4%|▍         | 13/291 [00:00<00:04, 58.96it/s]
Loading 0:   7%|▋         | 20/291 [00:00<00:05, 52.52it/s]
Loading 0:   9%|▉         | 26/291 [00:00<00:05, 51.49it/s]
Loading 0:  11%|█         | 32/291 [00:00<00:06, 41.72it/s]
Loading 0:  14%|█▎        | 40/291 [00:00<00:04, 50.49it/s]
Loading 0:  16%|█▌        | 46/291 [00:00<00:05, 46.24it/s]
Loading 0:  18%|█▊        | 51/291 [00:01<00:05, 45.45it/s]
Loading 0:  20%|█▉        | 57/291 [00:01<00:04, 48.38it/s]
Loading 0:  22%|██▏       | 63/291 [00:01<00:04, 50.67it/s]
Loading 0:  24%|██▎       | 69/291 [00:01<00:04, 44.70it/s]
Loading 0:  25%|██▌       | 74/291 [00:01<00:05, 42.27it/s]
Loading 0:  27%|██▋       | 79/291 [00:01<00:05, 40.45it/s]
Loading 0:  29%|██▉       | 84/291 [00:02<00:06, 29.65it/s]
Loading 0:  30%|███       | 88/291 [00:02<00:06, 31.10it/s]
Loading 0:  32%|███▏      | 93/291 [00:02<00:05, 34.98it/s]
Loading 0:  33%|███▎      | 97/291 [00:02<00:05, 35.89it/s]
Loading 0:  35%|███▌      | 103/291 [00:02<00:04, 40.77it/s]
Loading 0:  37%|███▋      | 108/291 [00:02<00:04, 43.10it/s]
Loading 0:  39%|███▉      | 113/291 [00:02<00:04, 35.68it/s]
Loading 0:  42%|████▏     | 121/291 [00:02<00:03, 44.81it/s]
Loading 0:  44%|████▎     | 127/291 [00:03<00:03, 42.79it/s]
Loading 0:  45%|████▌     | 132/291 [00:03<00:03, 41.93it/s]
Loading 0:  47%|████▋     | 138/291 [00:03<00:03, 46.21it/s]
Loading 0:  49%|████▉     | 143/291 [00:03<00:03, 46.88it/s]
Loading 0:  51%|█████     | 149/291 [00:03<00:03, 41.41it/s]
Loading 0:  54%|█████▎    | 156/291 [00:03<00:02, 48.08it/s]
Loading 0:  56%|█████▌    | 162/291 [00:03<00:02, 49.95it/s]
Loading 0:  58%|█████▊    | 168/291 [00:03<00:02, 43.82it/s]
Loading 0:  60%|██████    | 175/291 [00:04<00:02, 49.67it/s]
Loading 0:  62%|██████▏   | 181/291 [00:04<00:02, 42.49it/s]
Loading 0:  64%|██████▍   | 187/291 [00:04<00:03, 34.10it/s]
Loading 0:  66%|██████▌   | 191/291 [00:04<00:02, 34.88it/s]
Loading 0:  67%|██████▋   | 195/291 [00:04<00:02, 34.68it/s]
Loading 0:  69%|██████▉   | 202/291 [00:04<00:02, 41.48it/s]
Loading 0:  71%|███████▏  | 208/291 [00:04<00:02, 40.63it/s]
Loading 0:  73%|███████▎  | 213/291 [00:05<00:01, 41.05it/s]
Loading 0:  76%|███████▌  | 220/291 [00:05<00:01, 46.50it/s]
Loading 0:  78%|███████▊  | 226/291 [00:05<00:01, 43.65it/s]
Loading 0:  79%|███████▉  | 231/291 [00:05<00:01, 43.62it/s]
Loading 0:  82%|████████▏ | 238/291 [00:05<00:01, 48.83it/s]
Loading 0:  84%|████████▍ | 244/291 [00:05<00:01, 43.56it/s]
Loading 0:  86%|████████▌ | 249/291 [00:05<00:00, 43.50it/s]
Loading 0:  88%|████████▊ | 256/291 [00:05<00:00, 48.89it/s]
Loading 0:  90%|█████████ | 262/291 [00:06<00:00, 45.01it/s]
Loading 0:  92%|█████████▏| 267/291 [00:06<00:00, 43.73it/s]
Loading 0:  94%|█████████▍| 274/291 [00:06<00:00, 49.06it/s]
Loading 0:  96%|█████████▌| 280/291 [00:06<00:00, 44.88it/s]
Loading 0:  98%|█████████▊| 285/291 [00:06<00:00, 44.07it/s]
Loading 0: 100%|█████████▉| 290/291 [00:06<00:00, 32.76it/s]
                                                            
Job chaiml-nis-rm-new-datav1-436-v4-mkmlizer completed after 63.91s with status: succeeded
Stopping job with name chaiml-nis-rm-new-datav1-436-v4-mkmlizer
Pipeline stage MKMLizer completed in 64.43s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-nis-rm-new-datav1-436-v4
Waiting for inference service chaiml-nis-rm-new-datav1-436-v4 to be ready
Inference service chaiml-nis-rm-new-datav1-436-v4 ready after 110.4526515007019s
Pipeline stage MKMLDeployer completed in 111.08s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 4.082726716995239s
Received healthy response to inference request in 6.061429977416992s
Received healthy response to inference request in 3.264983654022217s
Received healthy response to inference request in 5.1787896156311035s
Received healthy response to inference request in 3.857590436935425s
5 requests
0 failed requests
5th percentile: 3.3835050106048583
10th percentile: 3.5020263671875
20th percentile: 3.7390690803527833
30th percentile: 3.9026176929473877
40th percentile: 3.9926722049713135
50th percentile: 4.082726716995239
60th percentile: 4.521151876449585
70th percentile: 4.959577035903931
80th percentile: 5.355317687988282
90th percentile: 5.7083738327026365
95th percentile: 5.884901905059814
99th percentile: 6.0261243629455565
mean time: 4.489104080200195
%s, retrying in %s seconds...
Received healthy response to inference request in 4.52907133102417s
Received healthy response to inference request in 7.16600227355957s
Received healthy response to inference request in 3.2388854026794434s
Received healthy response to inference request in 3.740586519241333s
Received healthy response to inference request in 4.5490171909332275s
5 requests
0 failed requests
5th percentile: 3.3392256259918214
10th percentile: 3.4395658493041994
20th percentile: 3.640246295928955
30th percentile: 3.8982834815979004
40th percentile: 4.213677406311035
50th percentile: 4.52907133102417
60th percentile: 4.537049674987793
70th percentile: 4.545028018951416
80th percentile: 5.072414207458497
90th percentile: 6.119208240509034
95th percentile: 6.642605257034301
99th percentile: 7.061322870254516
mean time: 4.6447125434875485
%s, retrying in %s seconds...
Received healthy response to inference request in 3.4942824840545654s
Received healthy response to inference request in 4.088257312774658s
Received healthy response to inference request in 3.101707935333252s
Received healthy response to inference request in 3.691948890686035s
Received healthy response to inference request in 3.045234203338623s
5 requests
0 failed requests
5th percentile: 3.0565289497375487
10th percentile: 3.0678236961364744
20th percentile: 3.0904131889343263
30th percentile: 3.1802228450775147
40th percentile: 3.33725266456604
50th percentile: 3.4942824840545654
60th percentile: 3.573349046707153
70th percentile: 3.6524156093597413
80th percentile: 3.7712105751037597
90th percentile: 3.929733943939209
95th percentile: 4.008995628356933
99th percentile: 4.072404975891113
mean time: 3.484286165237427
Pipeline stage StressChecker completed in 66.91s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.79s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.92s
Shutdown handler de-registered
chaiml-nis-rm-new-datav1-_436_v4 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage MKMLProfilerDeleter completed in 0.16s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.17s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service chaiml-nis-rm-new-datav1-436-v4-profiler
Waiting for inference service chaiml-nis-rm-new-datav1-436-v4-profiler to be ready
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3681.98s
Shutdown handler de-registered
chaiml-nis-rm-new-datav1-_436_v4 status is now inactive due to auto deactivation removed underperforming models
chaiml-nis-rm-new-datav1-_436_v4 status is now torndown due to DeploymentManager action