bogoconic1-nemo-280k-s6_91462

developer_uid: bogoconic1

submission_id: bogoconic1-nemo-280k-s6_91462_v4

model_name: bogoconic1-nemo-280k-s6_91462_v4

model_group: bogoconic1/nemo-280k-s60

status: torndown

timestamp: 2025-05-02T02:34:57+00:00

num_battles: 9444

num_wins: 4486

celo_rating: 1269.82

family_friendly_score: 0.5708

family_friendly_standard_error: 0.006999819426242366

submission_type: basic

model_repo: bogoconic1/nemo-280k-s600-chai-step1350

model_architecture: MistralForCausalLM

model_num_parameters: 12772070400.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

latencies: [{'batch_size': 1, 'throughput': 0.6036369370695155, 'latency_mean': 1.6565581512451173, 'latency_p50': 1.6508831977844238, 'latency_p90': 1.815650510787964}, {'batch_size': 3, 'throughput': 1.1147295854459291, 'latency_mean': 2.689979246854782, 'latency_p50': 2.6672768592834473, 'latency_p90': 2.971963572502136}, {'batch_size': 5, 'throughput': 1.3460771454744858, 'latency_mean': 3.700038826465607, 'latency_p50': 3.7046310901641846, 'latency_p90': 4.1527914047241214}, {'batch_size': 6, 'throughput': 1.4029578379059222, 'latency_mean': 4.2508092367649075, 'latency_p50': 4.262670040130615, 'latency_p90': 4.794099450111389}, {'batch_size': 8, 'throughput': 1.460803779146587, 'latency_mean': 5.436189967393875, 'latency_p50': 5.452843904495239, 'latency_p90': 6.039045357704163}, {'batch_size': 10, 'throughput': 1.4985826834525322, 'latency_mean': 6.625728596448898, 'latency_p50': 6.661508560180664, 'latency_p90': 7.493653631210327}]

gpu_counts: {'NVIDIA RTX A5000': 1}

display_name: bogoconic1-nemo-280k-s6_91462_v4

is_internal_developer: True

language_model: bogoconic1/nemo-280k-s600-chai-step1350

model_size: 13B

ranking_group: single

throughput_3p7s: 1.35

us_pacific_date: 2025-05-01

win_ratio: 0.4750105887335875

generation_params: {'temperature': 0.9, 'top_p': 0.9, 'min_p': 0.0, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': True}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name bogoconic1-nemo-280k-s6-91462-v4-mkmlizer
Waiting for job on bogoconic1-nemo-280k-s6-91462-v4-mkmlizer to finish
bogoconic1-nemo-280k-s6-91462-v4-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
bogoconic1-nemo-280k-s6-91462-v4-mkmlizer: ║     _____            __           __                                ║
bogoconic1-nemo-280k-s6-91462-v4-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
bogoconic1-nemo-280k-s6-91462-v4-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
bogoconic1-nemo-280k-s6-91462-v4-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
bogoconic1-nemo-280k-s6-91462-v4-mkmlizer: ║       /___/                                                         ║
bogoconic1-nemo-280k-s6-91462-v4-mkmlizer: ║                                                                     ║
bogoconic1-nemo-280k-s6-91462-v4-mkmlizer: ║  Version: 0.12.8                                                    ║
bogoconic1-nemo-280k-s6-91462-v4-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
bogoconic1-nemo-280k-s6-91462-v4-mkmlizer: ║  https://mk1.ai                                                     ║
bogoconic1-nemo-280k-s6-91462-v4-mkmlizer: ║                                                                     ║
bogoconic1-nemo-280k-s6-91462-v4-mkmlizer: ║  The license key for the current software has been verified as      ║
bogoconic1-nemo-280k-s6-91462-v4-mkmlizer: ║  belonging to:                                                      ║
bogoconic1-nemo-280k-s6-91462-v4-mkmlizer: ║                                                                     ║
bogoconic1-nemo-280k-s6-91462-v4-mkmlizer: ║  Chai Research Corp.                                                ║
bogoconic1-nemo-280k-s6-91462-v4-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
bogoconic1-nemo-280k-s6-91462-v4-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
bogoconic1-nemo-280k-s6-91462-v4-mkmlizer: ║                                                                     ║
bogoconic1-nemo-280k-s6-91462-v4-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
bogoconic1-nemo-280k-s6-91462-v4-mkmlizer: Downloaded to shared memory in 58.194s
bogoconic1-nemo-280k-s6-91462-v4-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp7gkwdx2l, device:0
bogoconic1-nemo-280k-s6-91462-v4-mkmlizer: Saving flywheel model at /dev/shm/model_cache
bogoconic1-nemo-280k-s6-91462-v4-mkmlizer: quantized model in 36.362s
bogoconic1-nemo-280k-s6-91462-v4-mkmlizer: Processed model bogoconic1/nemo-280k-s600-chai-step1350 in 94.556s
bogoconic1-nemo-280k-s6-91462-v4-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
bogoconic1-nemo-280k-s6-91462-v4-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/bogoconic1-nemo-280k-s6-91462-v4
bogoconic1-nemo-280k-s6-91462-v4-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/bogoconic1-nemo-280k-s6-91462-v4/config.json
bogoconic1-nemo-280k-s6-91462-v4-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/bogoconic1-nemo-280k-s6-91462-v4/special_tokens_map.json
bogoconic1-nemo-280k-s6-91462-v4-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/bogoconic1-nemo-280k-s6-91462-v4/tokenizer_config.json
bogoconic1-nemo-280k-s6-91462-v4-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/bogoconic1-nemo-280k-s6-91462-v4/tokenizer.json
bogoconic1-nemo-280k-s6-91462-v4-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/bogoconic1-nemo-280k-s6-91462-v4/flywheel_model.0.safetensors
bogoconic1-nemo-280k-s6-91462-v4-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:12, 29.64it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:07, 49.47it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:07, 46.76it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:07, 45.54it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:06, 50.65it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:07, 45.39it/s]
Loading 0:  12%|█▏        | 42/363 [00:00<00:07, 43.71it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:06, 49.15it/s]
Loading 0:  15%|█▌        | 55/363 [00:01<00:06, 45.11it/s]
Loading 0:  17%|█▋        | 60/363 [00:01<00:06, 45.68it/s]
Loading 0:  18%|█▊        | 65/363 [00:01<00:09, 29.92it/s]
Loading 0:  20%|█▉        | 72/363 [00:01<00:07, 36.80it/s]
Loading 0:  21%|██        | 77/363 [00:01<00:07, 39.27it/s]
Loading 0:  23%|██▎       | 82/363 [00:02<00:08, 34.67it/s]
Loading 0:  25%|██▍       | 89/363 [00:02<00:06, 42.24it/s]
Loading 0:  26%|██▌       | 94/363 [00:02<00:06, 43.31it/s]
Loading 0:  27%|██▋       | 99/363 [00:02<00:05, 44.56it/s]
Loading 0:  29%|██▊       | 104/363 [00:02<00:05, 45.16it/s]
Loading 0:  30%|███       | 109/363 [00:02<00:05, 44.87it/s]
Loading 0:  31%|███▏      | 114/363 [00:02<00:06, 38.87it/s]
Loading 0:  33%|███▎      | 119/363 [00:02<00:06, 38.57it/s]
Loading 0:  34%|███▍      | 125/363 [00:02<00:05, 43.24it/s]
Loading 0:  36%|███▌      | 130/363 [00:03<00:05, 42.99it/s]
Loading 0:  37%|███▋      | 135/363 [00:03<00:05, 43.91it/s]
Loading 0:  39%|███▉      | 141/363 [00:03<00:05, 42.95it/s]
Loading 0:  40%|████      | 146/363 [00:03<00:07, 30.96it/s]
Loading 0:  41%|████▏     | 150/363 [00:03<00:06, 30.83it/s]
Loading 0:  43%|████▎     | 156/363 [00:03<00:05, 35.91it/s]
Loading 0:  44%|████▍     | 160/363 [00:03<00:05, 36.15it/s]
Loading 0:  46%|████▌     | 166/363 [00:04<00:04, 40.24it/s]
Loading 0:  47%|████▋     | 171/363 [00:04<00:04, 42.34it/s]
Loading 0:  48%|████▊     | 176/363 [00:04<00:05, 35.35it/s]
Loading 0:  50%|█████     | 183/363 [00:04<00:04, 43.13it/s]
Loading 0:  52%|█████▏    | 188/363 [00:04<00:03, 43.76it/s]
Loading 0:  53%|█████▎    | 193/363 [00:04<00:03, 44.01it/s]
Loading 0:  55%|█████▍    | 199/363 [00:04<00:03, 43.28it/s]
Loading 0:  56%|█████▌    | 204/363 [00:04<00:03, 41.77it/s]
Loading 0:  58%|█████▊    | 210/363 [00:05<00:03, 45.30it/s]
Loading 0:  59%|█████▉    | 215/363 [00:05<00:03, 44.73it/s]
Loading 0:  61%|██████    | 222/363 [00:05<00:03, 45.42it/s]
Loading 0:  63%|██████▎   | 227/363 [00:05<00:04, 31.94it/s]
Loading 0:  64%|██████▎   | 231/363 [00:05<00:04, 31.81it/s]
Loading 0:  66%|██████▌   | 238/363 [00:05<00:03, 38.09it/s]
Loading 0:  67%|██████▋   | 244/363 [00:06<00:03, 38.77it/s]
Loading 0:  69%|██████▊   | 249/363 [00:06<00:03, 37.93it/s]
Loading 0:  71%|███████   | 256/363 [00:06<00:02, 43.37it/s]
Loading 0:  72%|███████▏  | 262/363 [00:06<00:02, 42.79it/s]
Loading 0:  74%|███████▎  | 267/363 [00:06<00:02, 40.85it/s]
Loading 0:  75%|███████▌  | 273/363 [00:06<00:02, 43.92it/s]
Loading 0:  77%|███████▋  | 278/363 [00:06<00:01, 44.81it/s]
Loading 0:  78%|███████▊  | 283/363 [00:06<00:01, 45.50it/s]
Loading 0:  80%|███████▉  | 289/363 [00:07<00:01, 43.89it/s]
Loading 0:  81%|████████  | 294/363 [00:07<00:01, 42.18it/s]
Loading 0:  83%|████████▎ | 301/363 [00:07<00:01, 48.44it/s]
Loading 0:  84%|████████▍ | 306/363 [00:14<00:21,  2.63it/s]
Loading 0:  85%|████████▌ | 310/363 [00:14<00:15,  3.35it/s]
Loading 0:  87%|████████▋ | 315/363 [00:14<00:10,  4.61it/s]
Loading 0:  88%|████████▊ | 321/363 [00:14<00:06,  6.51it/s]
Loading 0:  91%|█████████ | 329/363 [00:14<00:03, 10.11it/s]
Loading 0:  92%|█████████▏| 335/363 [00:14<00:02, 13.11it/s]
Loading 0:  94%|█████████▎| 340/363 [00:14<00:01, 16.03it/s]
Loading 0:  96%|█████████▌| 347/363 [00:14<00:00, 21.50it/s]
Loading 0:  97%|█████████▋| 353/363 [00:15<00:00, 24.37it/s]
Loading 0:  99%|█████████▊| 358/363 [00:15<00:00, 27.37it/s]
                                                            
Job bogoconic1-nemo-280k-s6-91462-v4-mkmlizer completed after 124.57s with status: succeeded
Stopping job with name bogoconic1-nemo-280k-s6-91462-v4-mkmlizer
Pipeline stage MKMLizer completed in 125.03s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.18s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service bogoconic1-nemo-280k-s6-91462-v4
Waiting for inference service bogoconic1-nemo-280k-s6-91462-v4 to be ready
Inference service bogoconic1-nemo-280k-s6-91462-v4 ready after 150.66989254951477s
Pipeline stage MKMLDeployer completed in 151.18s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.5694234371185303s
Received healthy response to inference request in 1.105527400970459s
Received healthy response to inference request in 1.6475496292114258s
Received healthy response to inference request in 0.9529170989990234s
Received healthy response to inference request in 1.3129854202270508s
5 requests
0 failed requests
5th percentile: 0.9834391593933105
10th percentile: 1.0139612197875976
20th percentile: 1.075005340576172
30th percentile: 1.1470190048217774
40th percentile: 1.230002212524414
50th percentile: 1.3129854202270508
60th percentile: 1.4155606269836425
70th percentile: 1.5181358337402344
80th percentile: 1.5850486755371094
90th percentile: 1.6162991523742676
95th percentile: 1.6319243907928467
99th percentile: 1.6444245815277099
mean time: 1.3176805973052979
Pipeline stage StressChecker completed in 7.99s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.69s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.73s
Shutdown handler de-registered
bogoconic1-nemo-280k-s6_91462_v4 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 4780.70s
Shutdown handler de-registered
bogoconic1-nemo-280k-s6_91462_v4 status is now inactive due to auto deactivation removed underperforming models
bogoconic1-nemo-280k-s6_91462_v4 status is now torndown due to DeploymentManager action