cgato-nemo-12b-humanize_81216

developer_uid: cycy233

submission_id: cgato-nemo-12b-humanize_81216_v2

model_name: 55555

model_group: cgato/Nemo-12b-Humanize-

status: torndown

timestamp: 2025-05-07T03:48:01+00:00

num_battles: 8107

num_wins: 3898

celo_rating: 1268.28

family_friendly_score: 0.5218

family_friendly_standard_error: 0.007064343706247595

submission_type: basic

model_repo: cgato/Nemo-12b-Humanize-KTO-Experimental-2

model_architecture: MistralForCausalLM

model_num_parameters: 12772070400.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

reward_model: default

latencies: [{'batch_size': 1, 'throughput': 0.594920127081161, 'latency_mean': 1.6808394753932954, 'latency_p50': 1.6793742179870605, 'latency_p90': 1.8527750015258788}, {'batch_size': 3, 'throughput': 1.0680755971691855, 'latency_mean': 2.8022540259361266, 'latency_p50': 2.8073025941848755, 'latency_p90': 3.093386101722717}, {'batch_size': 5, 'throughput': 1.266496743747971, 'latency_mean': 3.928818244934082, 'latency_p50': 3.9431873559951782, 'latency_p90': 4.355587840080261}, {'batch_size': 6, 'throughput': 1.3249555396939905, 'latency_mean': 4.502366507053376, 'latency_p50': 4.503959536552429, 'latency_p90': 5.113128709793091}, {'batch_size': 8, 'throughput': 1.3794391085496316, 'latency_mean': 5.758677412271499, 'latency_p50': 5.788568139076233, 'latency_p90': 6.474228763580323}, {'batch_size': 10, 'throughput': 1.4222535910660912, 'latency_mean': 6.976111736297607, 'latency_p50': 6.927115082740784, 'latency_p90': 8.000965070724487}]

gpu_counts: {'NVIDIA RTX A5000': 1}

display_name: 55555

is_internal_developer: False

language_model: cgato/Nemo-12b-Humanize-KTO-Experimental-2

model_size: 13B

ranking_group: single

throughput_3p7s: 1.24

us_pacific_date: 2025-05-06

win_ratio: 0.4808190452695202

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name cgato-nemo-12b-humanize-81216-v2-mkmlizer
Waiting for job on cgato-nemo-12b-humanize-81216-v2-mkmlizer to finish
cgato-nemo-12b-humanize-81216-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
cgato-nemo-12b-humanize-81216-v2-mkmlizer: ║     _____            __           __                                ║
cgato-nemo-12b-humanize-81216-v2-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
cgato-nemo-12b-humanize-81216-v2-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
cgato-nemo-12b-humanize-81216-v2-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
cgato-nemo-12b-humanize-81216-v2-mkmlizer: ║       /___/                                                         ║
cgato-nemo-12b-humanize-81216-v2-mkmlizer: ║                                                                     ║
cgato-nemo-12b-humanize-81216-v2-mkmlizer: ║  Version: 0.12.8                                                    ║
cgato-nemo-12b-humanize-81216-v2-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
cgato-nemo-12b-humanize-81216-v2-mkmlizer: ║  https://mk1.ai                                                     ║
cgato-nemo-12b-humanize-81216-v2-mkmlizer: ║                                                                     ║
cgato-nemo-12b-humanize-81216-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
cgato-nemo-12b-humanize-81216-v2-mkmlizer: ║  belonging to:                                                      ║
cgato-nemo-12b-humanize-81216-v2-mkmlizer: ║                                                                     ║
cgato-nemo-12b-humanize-81216-v2-mkmlizer: ║  Chai Research Corp.                                                ║
cgato-nemo-12b-humanize-81216-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
cgato-nemo-12b-humanize-81216-v2-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
cgato-nemo-12b-humanize-81216-v2-mkmlizer: ║                                                                     ║
cgato-nemo-12b-humanize-81216-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
cgato-nemo-12b-humanize-81216-v2-mkmlizer: Downloaded to shared memory in 29.186s
cgato-nemo-12b-humanize-81216-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp69a083sn, device:0
cgato-nemo-12b-humanize-81216-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
cgato-nemo-12b-humanize-81216-v2-mkmlizer: quantized model in 36.600s
cgato-nemo-12b-humanize-81216-v2-mkmlizer: Processed model cgato/Nemo-12b-Humanize-KTO-Experimental-2 in 65.787s
cgato-nemo-12b-humanize-81216-v2-mkmlizer: creating bucket guanaco-mkml-models
cgato-nemo-12b-humanize-81216-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
cgato-nemo-12b-humanize-81216-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/cgato-nemo-12b-humanize-81216-v2
cgato-nemo-12b-humanize-81216-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/cgato-nemo-12b-humanize-81216-v2/config.json
cgato-nemo-12b-humanize-81216-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/cgato-nemo-12b-humanize-81216-v2/special_tokens_map.json
cgato-nemo-12b-humanize-81216-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/cgato-nemo-12b-humanize-81216-v2/tokenizer_config.json
cgato-nemo-12b-humanize-81216-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/cgato-nemo-12b-humanize-81216-v2/tokenizer.json
cgato-nemo-12b-humanize-81216-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/cgato-nemo-12b-humanize-81216-v2/flywheel_model.0.safetensors
cgato-nemo-12b-humanize-81216-v2-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:11, 32.33it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:06, 51.34it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:07, 44.48it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:07, 42.50it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:06, 47.59it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:07, 44.05it/s]
Loading 0:  12%|█▏        | 42/363 [00:00<00:07, 43.22it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:06, 48.37it/s]
Loading 0:  15%|█▍        | 54/363 [00:01<00:06, 48.21it/s]
Loading 0:  16%|█▋        | 59/363 [00:01<00:06, 47.90it/s]
Loading 0:  18%|█▊        | 64/363 [00:01<00:11, 27.09it/s]
Loading 0:  20%|█▉        | 71/363 [00:01<00:08, 34.72it/s]
Loading 0:  21%|██        | 76/363 [00:01<00:08, 35.27it/s]
Loading 0:  22%|██▏       | 81/363 [00:02<00:07, 37.58it/s]
Loading 0:  24%|██▎       | 86/363 [00:02<00:07, 39.53it/s]
Loading 0:  25%|██▌       | 91/363 [00:02<00:07, 34.67it/s]
Loading 0:  27%|██▋       | 98/363 [00:02<00:06, 41.99it/s]
Loading 0:  28%|██▊       | 103/363 [00:02<00:06, 42.05it/s]
Loading 0:  30%|███       | 109/363 [00:02<00:05, 46.18it/s]
Loading 0:  31%|███▏      | 114/363 [00:02<00:06, 39.62it/s]
Loading 0:  33%|███▎      | 119/363 [00:02<00:06, 37.19it/s]
Loading 0:  34%|███▍      | 125/363 [00:03<00:05, 41.36it/s]
Loading 0:  36%|███▌      | 130/363 [00:03<00:05, 40.98it/s]
Loading 0:  37%|███▋      | 135/363 [00:03<00:05, 41.12it/s]
Loading 0:  39%|███▊      | 140/363 [00:03<00:05, 42.83it/s]
Loading 0:  40%|███▉      | 145/363 [00:03<00:07, 27.28it/s]
Loading 0:  41%|████      | 149/363 [00:03<00:07, 27.68it/s]
Loading 0:  43%|████▎     | 156/363 [00:04<00:05, 35.16it/s]
Loading 0:  44%|████▍     | 161/363 [00:04<00:05, 36.66it/s]
Loading 0:  46%|████▌     | 166/363 [00:04<00:05, 37.47it/s]
Loading 0:  47%|████▋     | 171/363 [00:04<00:04, 39.71it/s]
Loading 0:  48%|████▊     | 176/363 [00:04<00:05, 34.08it/s]
Loading 0:  50%|█████     | 183/363 [00:04<00:04, 41.01it/s]
Loading 0:  52%|█████▏    | 188/363 [00:04<00:04, 41.40it/s]
Loading 0:  53%|█████▎    | 193/363 [00:04<00:04, 41.24it/s]
Loading 0:  55%|█████▍    | 198/363 [00:05<00:03, 42.62it/s]
Loading 0:  56%|█████▌    | 203/363 [00:05<00:04, 35.55it/s]
Loading 0:  58%|█████▊    | 210/363 [00:05<00:03, 42.33it/s]
Loading 0:  59%|█████▉    | 215/363 [00:05<00:03, 41.85it/s]
Loading 0:  61%|██████    | 220/363 [00:05<00:03, 42.12it/s]
Loading 0:  62%|██████▏   | 225/363 [00:05<00:05, 27.27it/s]
Loading 0:  63%|██████▎   | 230/363 [00:06<00:04, 29.60it/s]
Loading 0:  65%|██████▌   | 237/363 [00:06<00:03, 36.72it/s]
Loading 0:  67%|██████▋   | 242/363 [00:06<00:03, 38.43it/s]
Loading 0:  68%|██████▊   | 247/363 [00:06<00:02, 39.65it/s]
Loading 0:  70%|██████▉   | 253/363 [00:06<00:02, 38.67it/s]
Loading 0:  71%|███████   | 258/363 [00:06<00:02, 38.03it/s]
Loading 0:  73%|███████▎  | 264/363 [00:06<00:02, 41.90it/s]
Loading 0:  74%|███████▍  | 269/363 [00:06<00:02, 42.22it/s]
Loading 0:  75%|███████▌  | 274/363 [00:07<00:02, 42.64it/s]
Loading 0:  77%|███████▋  | 279/363 [00:07<00:01, 42.70it/s]
Loading 0:  78%|███████▊  | 284/363 [00:07<00:02, 35.75it/s]
Loading 0:  80%|████████  | 291/363 [00:07<00:01, 42.75it/s]
Loading 0:  82%|████████▏ | 296/363 [00:07<00:01, 42.66it/s]
Loading 0:  83%|████████▎ | 301/363 [00:07<00:01, 43.35it/s]
Loading 0:  84%|████████▍ | 306/363 [00:14<00:22,  2.49it/s]
Loading 0:  85%|████████▌ | 310/363 [00:14<00:16,  3.21it/s]
Loading 0:  87%|████████▋ | 314/363 [00:14<00:11,  4.22it/s]
Loading 0:  88%|████████▊ | 320/363 [00:14<00:06,  6.29it/s]
Loading 0:  90%|████████▉ | 326/363 [00:14<00:04,  8.78it/s]
Loading 0:  91%|█████████ | 330/363 [00:15<00:03, 10.65it/s]
Loading 0:  93%|█████████▎| 338/363 [00:15<00:01, 16.24it/s]
Loading 0:  95%|█████████▍| 344/363 [00:15<00:00, 19.71it/s]
Loading 0:  96%|█████████▌| 349/363 [00:15<00:00, 22.80it/s]
Loading 0:  98%|█████████▊| 355/363 [00:15<00:00, 27.96it/s]
Loading 0:  99%|█████████▉| 360/363 [00:15<00:00, 31.29it/s]
                                                            
Job cgato-nemo-12b-humanize-81216-v2-mkmlizer completed after 94.31s with status: succeeded
Stopping job with name cgato-nemo-12b-humanize-81216-v2-mkmlizer
Pipeline stage MKMLizer completed in 94.81s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.16s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service cgato-nemo-12b-humanize-81216-v2
Waiting for inference service cgato-nemo-12b-humanize-81216-v2 to be ready
Inference service cgato-nemo-12b-humanize-81216-v2 ready after 150.56054711341858s
Pipeline stage MKMLDeployer completed in 151.29s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.4322848320007324s
Received healthy response to inference request in 1.519683837890625s
Received healthy response to inference request in 1.861701250076294s
Received healthy response to inference request in 1.6611433029174805s
Received healthy response to inference request in 1.76304030418396s
5 requests
0 failed requests
5th percentile: 1.547975730895996
10th percentile: 1.576267623901367
20th percentile: 1.6328514099121094
30th percentile: 1.6815227031707765
40th percentile: 1.7222815036773682
50th percentile: 1.76304030418396
60th percentile: 1.8025046825408935
70th percentile: 1.8419690608978272
80th percentile: 1.9758179664611817
90th percentile: 2.204051399230957
95th percentile: 2.3181681156158445
99th percentile: 2.4094614887237547
mean time: 1.8475707054138184
Pipeline stage StressChecker completed in 10.58s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.66s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.63s
Shutdown handler de-registered
cgato-nemo-12b-humanize_81216_v2 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 5011.92s
Shutdown handler de-registered
cgato-nemo-12b-humanize_81216_v2 status is now inactive due to auto deactivation removed underperforming models
cgato-nemo-12b-humanize_81216_v2 status is now torndown due to DeploymentManager action