developer_uid: azuruce
submission_id: cgato-nemo-12b-humaniz_74135_v44
model_name: cgato-nemo-12b-humaniz_74135_v44
model_group: cgato/Nemo-12b-Humanize-
status: torndown
timestamp: 2025-02-04T23:05:36+00:00
num_battles: 5475
num_wins: 2668
celo_rating: 1259.55
family_friendly_score: 0.5674
family_friendly_standard_error: 0.007006528955196004
submission_type: basic
model_repo: cgato/Nemo-12b-Humanize-KTO-Experimental-Latest
model_architecture: MistralForCausalLM
model_num_parameters: 12772111360.0
best_of: 8
max_input_tokens: 768
max_output_tokens: 64
display_name: cgato-nemo-12b-humaniz_74135_v44
is_internal_developer: True
language_model: cgato/Nemo-12b-Humanize-KTO-Experimental-Latest
model_size: 13B
ranking_group: single
us_pacific_date: 2025-02-04
win_ratio: 0.4873059360730594
generation_params: {'temperature': 0.7, 'top_p': 0.9, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.01, 'frequency_penalty': 0.01, 'stopping_words': ['\n', '</s>', '###', '####', 'Bot:', 'User:', 'You:', 'Me:', '<|eot_id|>', '<|end_of_text|>', '<|im_end|>'], 'max_input_tokens': 768, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name cgato-nemo-12b-humaniz-74135-v44-mkmlizer
Waiting for job on cgato-nemo-12b-humaniz-74135-v44-mkmlizer to finish
cgato-nemo-12b-humaniz-74135-v44-mkmlizer: Downloaded to shared memory in 33.527s
cgato-nemo-12b-humaniz-74135-v44-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp6x3s1vwy, device:0
cgato-nemo-12b-humaniz-74135-v44-mkmlizer: Saving flywheel model at /dev/shm/model_cache
cgato-nemo-12b-humaniz-74135-v44-mkmlizer: quantized model in 35.047s
cgato-nemo-12b-humaniz-74135-v44-mkmlizer: Processed model cgato/Nemo-12b-Humanize-KTO-Experimental-Latest in 68.574s
cgato-nemo-12b-humaniz-74135-v44-mkmlizer: creating bucket guanaco-mkml-models
cgato-nemo-12b-humaniz-74135-v44-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
cgato-nemo-12b-humaniz-74135-v44-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/cgato-nemo-12b-humaniz-74135-v44
cgato-nemo-12b-humaniz-74135-v44-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/cgato-nemo-12b-humaniz-74135-v44/config.json
cgato-nemo-12b-humaniz-74135-v44-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/cgato-nemo-12b-humaniz-74135-v44/special_tokens_map.json
cgato-nemo-12b-humaniz-74135-v44-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/cgato-nemo-12b-humaniz-74135-v44/tokenizer_config.json
cgato-nemo-12b-humaniz-74135-v44-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/cgato-nemo-12b-humaniz-74135-v44/flywheel_model.0.safetensors
cgato-nemo-12b-humaniz-74135-v44-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:10, 33.57it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:06, 53.51it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:06, 50.89it/s]
Loading 0:   7%|▋         | 25/363 [00:00<00:06, 52.62it/s]
Loading 0:   9%|▉         | 32/363 [00:00<00:06, 48.48it/s]
Loading 0:  11%|█▏        | 41/363 [00:00<00:06, 50.93it/s]
Loading 0:  14%|█▍        | 50/363 [00:00<00:06, 52.13it/s]
Loading 0:  17%|█▋        | 60/363 [00:01<00:05, 57.99it/s]
Loading 0:  18%|█▊        | 66/363 [00:01<00:08, 35.52it/s]
Loading 0:  20%|█▉        | 72/363 [00:01<00:07, 39.61it/s]
Loading 0:  21%|██▏       | 78/363 [00:01<00:07, 40.09it/s]
Loading 0:  23%|██▎       | 83/363 [00:01<00:06, 41.26it/s]
Loading 0:  25%|██▍       | 90/363 [00:01<00:05, 46.97it/s]
Loading 0:  26%|██▋       | 96/363 [00:02<00:05, 46.77it/s]
Loading 0:  28%|██▊       | 102/363 [00:02<00:05, 47.47it/s]
Loading 0:  30%|███       | 110/363 [00:02<00:05, 50.41it/s]
Loading 0:  32%|███▏      | 116/363 [00:02<00:04, 50.21it/s]
Loading 0:  34%|███▎      | 122/363 [00:02<00:04, 51.54it/s]
Loading 0:  35%|███▌      | 128/363 [00:02<00:05, 45.21it/s]
Loading 0:  37%|███▋      | 135/363 [00:02<00:04, 49.10it/s]
Loading 0:  39%|███▉      | 141/363 [00:03<00:04, 47.76it/s]
Loading 0:  40%|████      | 146/363 [00:03<00:06, 31.01it/s]
Loading 0:  41%|████▏     | 150/363 [00:03<00:06, 31.68it/s]
Loading 0:  43%|████▎     | 157/363 [00:03<00:05, 38.24it/s]
Loading 0:  45%|████▍     | 163/363 [00:03<00:05, 39.62it/s]
Loading 0:  46%|████▋     | 168/363 [00:03<00:04, 39.82it/s]
Loading 0:  48%|████▊     | 174/363 [00:03<00:04, 44.29it/s]
Loading 0:  49%|████▉     | 179/363 [00:04<00:04, 45.21it/s]
Loading 0:  51%|█████     | 185/363 [00:04<00:04, 42.03it/s]
Loading 0:  53%|█████▎    | 193/363 [00:04<00:03, 50.00it/s]
Loading 0:  55%|█████▍    | 199/363 [00:04<00:03, 48.08it/s]
Loading 0:  56%|█████▋    | 205/363 [00:04<00:03, 49.53it/s]
Loading 0:  58%|█████▊    | 211/363 [00:04<00:02, 51.76it/s]
Loading 0:  60%|█████▉    | 217/363 [00:04<00:03, 48.52it/s]
Loading 0:  61%|██████▏   | 223/363 [00:05<00:04, 33.34it/s]
Loading 0:  63%|██████▎   | 228/363 [00:05<00:03, 34.27it/s]
Loading 0:  64%|██████▍   | 232/363 [00:05<00:03, 35.44it/s]
Loading 0:  65%|██████▌   | 237/363 [00:05<00:03, 38.50it/s]
Loading 0:  67%|██████▋   | 242/363 [00:05<00:02, 41.10it/s]
Loading 0:  68%|██████▊   | 248/363 [00:05<00:02, 39.72it/s]
Loading 0:  71%|███████   | 256/363 [00:05<00:02, 47.29it/s]
Loading 0:  72%|███████▏  | 262/363 [00:05<00:02, 46.59it/s]
Loading 0:  74%|███████▎  | 267/363 [00:06<00:02, 46.69it/s]
Loading 0:  75%|███████▌  | 274/363 [00:06<00:01, 52.10it/s]
Loading 0:  77%|███████▋  | 280/363 [00:06<00:01, 49.24it/s]
Loading 0:  79%|███████▉  | 286/363 [00:06<00:01, 50.05it/s]
Loading 0:  80%|████████  | 292/363 [00:06<00:01, 52.57it/s]
Loading 0:  82%|████████▏ | 298/363 [00:06<00:01, 50.49it/s]
Loading 0:  84%|████████▎ | 304/363 [00:13<00:20,  2.84it/s]
Loading 0:  85%|████████▍ | 308/363 [00:13<00:15,  3.57it/s]
Loading 0:  86%|████████▌ | 312/363 [00:13<00:11,  4.48it/s]
Loading 0:  88%|████████▊ | 320/363 [00:13<00:05,  7.25it/s]
Loading 0:  90%|████████▉ | 326/363 [00:13<00:03,  9.71it/s]
Loading 0:  91%|█████████ | 331/363 [00:14<00:02, 12.22it/s]
Loading 0:  93%|█████████▎| 338/363 [00:14<00:01, 16.93it/s]
Loading 0:  95%|█████████▍| 344/363 [00:14<00:00, 20.65it/s]
Loading 0:  96%|█████████▌| 349/363 [00:14<00:00, 24.01it/s]
Loading 0:  98%|█████████▊| 356/363 [00:14<00:00, 30.46it/s]
Loading 0: 100%|█████████▉| 362/363 [00:14<00:00, 33.01it/s]
                                                            

Job cgato-nemo-12b-humaniz-74135-v44-mkmlizer completed after 92.84s with status: succeeded
Stopping job with name cgato-nemo-12b-humaniz-74135-v44-mkmlizer
Pipeline stage MKMLizer completed in 93.39s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.16s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service cgato-nemo-12b-humaniz-74135-v44
Waiting for inference service cgato-nemo-12b-humaniz-74135-v44 to be ready
Inference service cgato-nemo-12b-humaniz-74135-v44 ready after 170.59016847610474s
Pipeline stage MKMLDeployer completed in 171.10s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.0951216220855713s
Received healthy response to inference request in 1.7102751731872559s
Received healthy response to inference request in 1.4071016311645508s
Received healthy response to inference request in 1.4735441207885742s
Received healthy response to inference request in 1.5470328330993652s
5 requests
0 failed requests
5th percentile: 1.4203901290893555
10th percentile: 1.4336786270141602
20th percentile: 1.4602556228637695
30th percentile: 1.4882418632507324
40th percentile: 1.517637348175049
50th percentile: 1.5470328330993652
60th percentile: 1.6123297691345215
70th percentile: 1.6776267051696776
80th percentile: 1.787244462966919
90th percentile: 1.941183042526245
95th percentile: 2.0181523323059083
99th percentile: 2.079727764129639
mean time: 1.6466150760650635
Pipeline stage StressChecker completed in 9.59s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.65s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.65s
Shutdown handler de-registered
cgato-nemo-12b-humaniz_74135_v44 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2659.91s
Shutdown handler de-registered
cgato-nemo-12b-humaniz_74135_v44 status is now inactive due to auto deactivation removed underperforming models
cgato-nemo-12b-humaniz_74135_v44 status is now torndown due to DeploymentManager action
ChatRequest
Generation Params
Prompt Formatter
Chat History
ChatMessage 1