nitral-ai-captain-eris_45741

developer_uid: Luna1040

submission_id: nitral-ai-captain-eris_45741_v11

model_name: nitral-ai-captain-eris_45741_v11

model_group: Nitral-AI/Captain-Eris_V

status: torndown

timestamp: 2025-02-19T12:08:33+00:00

num_battles: 7409

num_wins: 3707

celo_rating: 1268.99

family_friendly_score: 0.5836

family_friendly_standard_error: 0.0069715283833604235

submission_type: basic

model_repo: Nitral-AI/Captain-Eris_Violet-GRPO-v0.420

model_architecture: MistralForCausalLM

model_num_parameters: 12772070400.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

latencies: [{'batch_size': 1, 'throughput': 0.6020801508449135, 'latency_mean': 1.6608062756061555, 'latency_p50': 1.6625171899795532, 'latency_p90': 1.8349202394485473}, {'batch_size': 3, 'throughput': 1.087236465231853, 'latency_mean': 2.7488949608802797, 'latency_p50': 2.7260245084762573, 'latency_p90': 3.025449109077454}, {'batch_size': 5, 'throughput': 1.3232779845756313, 'latency_mean': 3.774457198381424, 'latency_p50': 3.7694636583328247, 'latency_p90': 4.204969787597656}, {'batch_size': 6, 'throughput': 1.3943570523969613, 'latency_mean': 4.297020063400269, 'latency_p50': 4.242348790168762, 'latency_p90': 4.833849310874939}, {'batch_size': 8, 'throughput': 1.4393638231421386, 'latency_mean': 5.5224302542209625, 'latency_p50': 5.554581880569458, 'latency_p90': 6.1282020807266235}, {'batch_size': 10, 'throughput': 1.4637931878540391, 'latency_mean': 6.777238395214081, 'latency_p50': 6.850613713264465, 'latency_p90': 7.578174304962158}]

gpu_counts: {'NVIDIA RTX A5000': 1}

display_name: nitral-ai-captain-eris_45741_v11

is_internal_developer: False

language_model: Nitral-AI/Captain-Eris_Violet-GRPO-v0.420

model_size: 13B

ranking_group: single

throughput_3p7s: 1.31

us_pacific_date: 2025-02-19

win_ratio: 0.5003374274530976

generation_params: {'temperature': 0.82, 'top_p': 0.92, 'min_p': 0.03, 'top_k': 75, 'presence_penalty': 0.12, 'frequency_penalty': 0.18, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name nitral-ai-captain-eris-45741-v11-mkmlizer
Waiting for job on nitral-ai-captain-eris-45741-v11-mkmlizer to finish
nitral-ai-captain-eris-45741-v11-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
nitral-ai-captain-eris-45741-v11-mkmlizer: ║     _____            __           __                                ║
nitral-ai-captain-eris-45741-v11-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
nitral-ai-captain-eris-45741-v11-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
nitral-ai-captain-eris-45741-v11-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
nitral-ai-captain-eris-45741-v11-mkmlizer: ║       /___/                                                         ║
nitral-ai-captain-eris-45741-v11-mkmlizer: ║                                                                     ║
nitral-ai-captain-eris-45741-v11-mkmlizer: ║  Version: 0.12.8                                                    ║
nitral-ai-captain-eris-45741-v11-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
nitral-ai-captain-eris-45741-v11-mkmlizer: ║  https://mk1.ai                                                     ║
nitral-ai-captain-eris-45741-v11-mkmlizer: ║                                                                     ║
nitral-ai-captain-eris-45741-v11-mkmlizer: ║  The license key for the current software has been verified as      ║
nitral-ai-captain-eris-45741-v11-mkmlizer: ║  belonging to:                                                      ║
nitral-ai-captain-eris-45741-v11-mkmlizer: ║                                                                     ║
nitral-ai-captain-eris-45741-v11-mkmlizer: ║  Chai Research Corp.                                                ║
nitral-ai-captain-eris-45741-v11-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
nitral-ai-captain-eris-45741-v11-mkmlizer: ║  Expiration: 2025-04-15 23:59:59                                    ║
nitral-ai-captain-eris-45741-v11-mkmlizer: ║                                                                     ║
nitral-ai-captain-eris-45741-v11-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
nitral-ai-captain-eris-45741-v11-mkmlizer: Downloaded to shared memory in 31.556s
nitral-ai-captain-eris-45741-v11-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpea17ax3b, device:0
nitral-ai-captain-eris-45741-v11-mkmlizer: Saving flywheel model at /dev/shm/model_cache
nitral-ai-captain-eris-45741-v11-mkmlizer: quantized model in 39.047s
nitral-ai-captain-eris-45741-v11-mkmlizer: Processed model Nitral-AI/Captain-Eris_Violet-GRPO-v0.420 in 70.604s
nitral-ai-captain-eris-45741-v11-mkmlizer: creating bucket guanaco-mkml-models
nitral-ai-captain-eris-45741-v11-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
nitral-ai-captain-eris-45741-v11-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/nitral-ai-captain-eris-45741-v11
nitral-ai-captain-eris-45741-v11-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/nitral-ai-captain-eris-45741-v11/config.json
nitral-ai-captain-eris-45741-v11-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/nitral-ai-captain-eris-45741-v11/special_tokens_map.json
nitral-ai-captain-eris-45741-v11-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/nitral-ai-captain-eris-45741-v11/tokenizer_config.json
nitral-ai-captain-eris-45741-v11-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/nitral-ai-captain-eris-45741-v11/flywheel_model.0.safetensors
nitral-ai-captain-eris-45741-v11-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|          | 2/363 [00:06<18:56,  3.15s/it]
Loading 0:   2%|▏         | 6/363 [00:06<05:03,  1.18it/s]
Loading 0:   3%|▎         | 11/363 [00:06<02:13,  2.64it/s]
Loading 0:   4%|▍         | 15/363 [00:06<01:24,  4.09it/s]
Loading 0:   6%|▌         | 22/363 [00:06<00:44,  7.65it/s]
Loading 0:   7%|▋         | 27/363 [00:07<00:32, 10.34it/s]
Loading 0:   9%|▊         | 31/363 [00:07<00:25, 12.84it/s]
Loading 0:  10%|▉         | 35/363 [00:07<00:21, 15.34it/s]
Loading 0:  11%|█         | 40/363 [00:07<00:21, 14.75it/s]
Loading 0:  12%|█▏        | 43/363 [00:07<00:19, 16.12it/s]
Loading 0:  13%|█▎        | 49/363 [00:07<00:14, 21.72it/s]
Loading 0:  15%|█▍        | 53/363 [00:08<00:13, 23.39it/s]
Loading 0:  16%|█▌        | 58/363 [00:08<00:11, 27.33it/s]
Loading 0:  17%|█▋        | 62/363 [00:08<00:10, 28.19it/s]
Loading 0:  18%|█▊        | 67/363 [00:08<00:09, 31.97it/s]
Loading 0:  20%|█▉        | 71/363 [00:08<00:09, 32.38it/s]
Loading 0:  21%|██        | 76/363 [00:08<00:08, 35.81it/s]
Loading 0:  22%|██▏       | 80/363 [00:08<00:08, 35.16it/s]
Loading 0:  24%|██▎       | 86/363 [00:08<00:07, 39.44it/s]
Loading 0:  25%|██▌       | 91/363 [00:08<00:06, 41.10it/s]
Loading 0:  26%|██▋       | 96/363 [00:09<00:08, 32.07it/s]
Loading 0:  28%|██▊       | 103/363 [00:09<00:06, 39.48it/s]
Loading 0:  30%|██▉       | 108/363 [00:09<00:06, 40.48it/s]
Loading 0:  31%|███       | 113/363 [00:09<00:06, 40.88it/s]
Loading 0:  33%|███▎      | 118/363 [00:09<00:06, 39.41it/s]
Loading 0:  34%|███▍      | 123/363 [00:10<00:10, 22.91it/s]
Loading 0:  36%|███▌      | 130/363 [00:10<00:07, 29.49it/s]
Loading 0:  37%|███▋      | 135/363 [00:10<00:07, 31.12it/s]
Loading 0:  38%|███▊      | 139/363 [00:10<00:07, 31.68it/s]
Loading 0:  39%|███▉      | 143/363 [00:10<00:06, 31.78it/s]
Loading 0:  41%|████      | 148/363 [00:10<00:06, 34.73it/s]
Loading 0:  42%|████▏     | 152/363 [00:10<00:06, 34.11it/s]
Loading 0:  43%|████▎     | 157/363 [00:10<00:05, 36.57it/s]
Loading 0:  44%|████▍     | 161/363 [00:11<00:05, 35.64it/s]
Loading 0:  46%|████▌     | 166/363 [00:11<00:05, 38.97it/s]
Loading 0:  47%|████▋     | 171/363 [00:11<00:04, 38.43it/s]
Loading 0:  48%|████▊     | 175/363 [00:11<00:05, 36.85it/s]
Loading 0:  49%|████▉     | 179/363 [00:11<00:05, 35.76it/s]
Loading 0:  51%|█████     | 184/363 [00:11<00:04, 38.00it/s]
Loading 0:  52%|█████▏    | 188/363 [00:11<00:04, 36.18it/s]
Loading 0:  53%|█████▎    | 193/363 [00:11<00:04, 37.11it/s]
Loading 0:  54%|█████▍    | 197/363 [00:12<00:04, 34.40it/s]
Loading 0:  55%|█████▌    | 201/363 [00:12<00:04, 35.48it/s]
Loading 0:  56%|█████▋    | 205/363 [00:12<00:07, 21.83it/s]
Loading 0:  58%|█████▊    | 211/363 [00:12<00:05, 27.48it/s]
Loading 0:  59%|█████▉    | 215/363 [00:12<00:05, 28.16it/s]
Loading 0:  61%|██████    | 220/363 [00:12<00:04, 31.27it/s]
Loading 0:  62%|██████▏   | 224/363 [00:13<00:04, 30.91it/s]
Loading 0:  63%|██████▎   | 229/363 [00:13<00:03, 33.96it/s]
Loading 0:  64%|██████▍   | 233/363 [00:13<00:03, 33.14it/s]
Loading 0:  65%|██████▌   | 237/363 [00:13<00:03, 34.74it/s]
Loading 0:  66%|██████▋   | 241/363 [00:13<00:03, 31.85it/s]
Loading 0:  67%|██████▋   | 245/363 [00:13<00:03, 33.75it/s]
Loading 0:  69%|██████▊   | 249/363 [00:13<00:03, 31.50it/s]
Loading 0:  71%|███████   | 256/363 [00:13<00:02, 38.16it/s]
Loading 0:  72%|███████▏  | 260/363 [00:14<00:02, 35.27it/s]
Loading 0:  73%|███████▎  | 264/363 [00:14<00:02, 36.34it/s]
Loading 0:  74%|███████▍  | 268/363 [00:14<00:02, 32.48it/s]
Loading 0:  75%|███████▍  | 272/363 [00:14<00:02, 33.52it/s]
Loading 0:  76%|███████▌  | 276/363 [00:14<00:02, 29.24it/s]
Loading 0:  77%|███████▋  | 281/363 [00:14<00:02, 32.85it/s]
Loading 0:  79%|███████▊  | 285/363 [00:15<00:03, 21.77it/s]
Loading 0:  80%|████████  | 292/363 [00:15<00:02, 29.26it/s]
Loading 0:  82%|████████▏ | 296/363 [00:15<00:02, 29.97it/s]
Loading 0:  83%|████████▎ | 301/363 [00:15<00:01, 32.65it/s]
Loading 0:  84%|████████▍ | 305/363 [00:15<00:01, 32.11it/s]
Loading 0:  85%|████████▌ | 310/363 [00:15<00:01, 34.67it/s]
Loading 0:  87%|████████▋ | 314/363 [00:15<00:01, 32.92it/s]
Loading 0:  88%|████████▊ | 318/363 [00:15<00:01, 34.56it/s]
Loading 0:  89%|████████▊ | 322/363 [00:16<00:01, 31.69it/s]
Loading 0:  90%|████████▉ | 326/363 [00:16<00:01, 33.61it/s]
Loading 0:  91%|█████████ | 330/363 [00:16<00:01, 31.29it/s]
Loading 0:  93%|█████████▎| 337/363 [00:16<00:00, 38.23it/s]
Loading 0:  94%|█████████▍| 341/363 [00:16<00:00, 36.17it/s]
Loading 0:  95%|█████████▌| 346/363 [00:16<00:00, 37.86it/s]
Loading 0:  96%|█████████▋| 350/363 [00:16<00:00, 36.79it/s]
Loading 0:  98%|█████████▊| 355/363 [00:16<00:00, 39.24it/s]
Loading 0:  99%|█████████▉| 359/363 [00:17<00:00, 37.25it/s]
Loading 0: 100%|██████████| 363/363 [00:17<00:00, 37.94it/s]
                                                            
Job nitral-ai-captain-eris-45741-v11-mkmlizer completed after 94.06s with status: succeeded
Stopping job with name nitral-ai-captain-eris-45741-v11-mkmlizer
Pipeline stage MKMLizer completed in 94.52s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.17s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service nitral-ai-captain-eris-45741-v11
Waiting for inference service nitral-ai-captain-eris-45741-v11 to be ready
Failed to get response for submission chaoticneutrals-captain_27143_v5: HTTPConnectionPool(host='chaoticneutrals-captain-27143-v5-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission mistralai-mistral-nem_93303_v361: {"detail":[{"loc":["body","query"],"msg":"Search query cannot be empty","type":"value_error"}]}
Inference service nitral-ai-captain-eris-45741-v11 ready after 170.57710909843445s
Pipeline stage MKMLDeployer completed in 171.15s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.410006523132324s
Received healthy response to inference request in 1.5776946544647217s
Received healthy response to inference request in 1.6261613368988037s
Received healthy response to inference request in 1.6910183429718018s
Received healthy response to inference request in 1.6854865550994873s
5 requests
0 failed requests
5th percentile: 1.587387990951538
10th percentile: 1.5970813274383544
20th percentile: 1.6164680004119873
30th percentile: 1.6380263805389403
40th percentile: 1.6617564678192138
50th percentile: 1.6854865550994873
60th percentile: 1.687699270248413
70th percentile: 1.6899119853973388
80th percentile: 1.8348159790039065
90th percentile: 2.1224112510681152
95th percentile: 2.2662088871002197
99th percentile: 2.3812469959259035
mean time: 1.7980734825134277
Pipeline stage StressChecker completed in 10.32s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.76s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.68s
Shutdown handler de-registered
nitral-ai-captain-eris_45741_v11 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2757.30s
Shutdown handler de-registered
nitral-ai-captain-eris_45741_v11 status is now inactive due to auto deactivation removed underperforming models
nitral-ai-captain-eris_45741_v11 status is now torndown due to DeploymentManager action
nitral-ai-captain-eris_45741_v11 status is now torndown due to DeploymentManager action
nitral-ai-captain-eris_45741_v11 status is now torndown due to DeploymentManager action