mistralai-mistral-nemo_9330

submission_id: mistralai-mistral-nemo_9330_v177
developer_uid: chai_backend_admin
best_of: 8
celo_rating: 1247.3
display_name: tempv1-2
family_friendly_score: 0.5768
family_friendly_standard_error: 0.006987156216945489
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '####', 'Bot:', 'User:', 'You:', '<|im_end|>', '<|eot_id|>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
is_internal_developer: True
language_model: mistralai/Mistral-Nemo-Instruct-2407
max_input_tokens: 1024
max_output_tokens: 64
model_architecture: MistralForCausalLM
model_group: mistralai/Mistral-Nemo-I
model_name: tempv1-2
model_num_parameters: 12772070400.0
model_repo: mistralai/Mistral-Nemo-Instruct-2407
model_size: 13B
num_battles: 9573
num_wins: 4948
ranking_group: single
status: inactive
submission_type: basic
timestamp: 2024-10-29T01:33:27+00:00
us_pacific_date: 2024-10-28
win_ratio: 0.5168703645670114
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name mistralai-mistral-nemo-9330-v177-mkmlizer
Waiting for job on mistralai-mistral-nemo-9330-v177-mkmlizer to finish
mistralai-mistral-nemo-9330-v177-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mistralai-mistral-nemo-9330-v177-mkmlizer: ║     _____            __           __                                ║
mistralai-mistral-nemo-9330-v177-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
mistralai-mistral-nemo-9330-v177-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
mistralai-mistral-nemo-9330-v177-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
mistralai-mistral-nemo-9330-v177-mkmlizer: ║       /___/                                                         ║
mistralai-mistral-nemo-9330-v177-mkmlizer: ║                                                                     ║
mistralai-mistral-nemo-9330-v177-mkmlizer: ║  Version: 0.11.12                                                   ║
mistralai-mistral-nemo-9330-v177-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
mistralai-mistral-nemo-9330-v177-mkmlizer: ║  https://mk1.ai                                                     ║
mistralai-mistral-nemo-9330-v177-mkmlizer: ║                                                                     ║
mistralai-mistral-nemo-9330-v177-mkmlizer: ║  The license key for the current software has been verified as      ║
mistralai-mistral-nemo-9330-v177-mkmlizer: ║  belonging to:                                                      ║
mistralai-mistral-nemo-9330-v177-mkmlizer: ║                                                                     ║
mistralai-mistral-nemo-9330-v177-mkmlizer: ║  Chai Research Corp.                                                ║
mistralai-mistral-nemo-9330-v177-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
mistralai-mistral-nemo-9330-v177-mkmlizer: ║  Expiration: 2025-01-15 23:59:59                                    ║
mistralai-mistral-nemo-9330-v177-mkmlizer: ║                                                                     ║
mistralai-mistral-nemo-9330-v177-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
mistralai-mistral-nemo-9330-v177-mkmlizer: Downloaded to shared memory in 58.372s
mistralai-mistral-nemo-9330-v177-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpefkwlr8x, device:0
mistralai-mistral-nemo-9330-v177-mkmlizer: Saving flywheel model at /dev/shm/model_cache
mistralai-mistral-nemo-9330-v177-mkmlizer: quantized model in 38.884s
mistralai-mistral-nemo-9330-v177-mkmlizer: Processed model mistralai/Mistral-Nemo-Instruct-2407 in 97.257s
mistralai-mistral-nemo-9330-v177-mkmlizer: creating bucket guanaco-mkml-models
mistralai-mistral-nemo-9330-v177-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
mistralai-mistral-nemo-9330-v177-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v177
mistralai-mistral-nemo-9330-v177-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v177/config.json
mistralai-mistral-nemo-9330-v177-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v177/special_tokens_map.json
mistralai-mistral-nemo-9330-v177-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v177/tokenizer_config.json
mistralai-mistral-nemo-9330-v177-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v177/tokenizer.json
mistralai-mistral-nemo-9330-v177-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/mistralai-mistral-nemo-9330-v177/flywheel_model.0.safetensors
mistralai-mistral-nemo-9330-v177-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:12, 28.38it/s]
Loading 0:   3%|▎         | 12/363 [00:00<00:07, 44.64it/s]
Loading 0:   5%|▍         | 17/363 [00:00<00:08, 41.42it/s]
Loading 0:   6%|▌         | 22/363 [00:00<00:08, 40.41it/s]
Loading 0:   7%|▋         | 27/363 [00:00<00:08, 41.19it/s]
Loading 0:   9%|▉         | 32/363 [00:00<00:09, 34.64it/s]
Loading 0:  11%|█         | 40/363 [00:00<00:07, 44.70it/s]
Loading 0:  13%|█▎        | 46/363 [00:01<00:07, 43.55it/s]
Loading 0:  14%|█▍        | 51/363 [00:01<00:07, 42.49it/s]
Loading 0:  16%|█▌        | 57/363 [00:01<00:06, 46.75it/s]
Loading 0:  17%|█▋        | 62/363 [00:01<00:09, 33.11it/s]
Loading 0:  18%|█▊        | 66/363 [00:01<00:09, 32.92it/s]
Loading 0:  20%|█▉        | 71/363 [00:01<00:08, 34.86it/s]
Loading 0:  21%|██        | 75/363 [00:02<00:09, 31.79it/s]
Loading 0:  22%|██▏       | 79/363 [00:02<00:08, 32.97it/s]
Loading 0:  23%|██▎       | 83/363 [00:02<00:09, 30.35it/s]
Loading 0:  24%|██▍       | 87/363 [00:02<00:08, 31.82it/s]
Loading 0:  25%|██▌       | 91/363 [00:02<00:09, 29.03it/s]
Loading 0:  26%|██▋       | 96/363 [00:02<00:08, 32.78it/s]
Loading 0:  28%|██▊       | 100/363 [00:02<00:08, 29.84it/s]
Loading 0:  29%|██▉       | 105/363 [00:02<00:07, 33.40it/s]
Loading 0:  30%|███       | 110/363 [00:03<00:07, 34.94it/s]
Loading 0:  31%|███▏      | 114/363 [00:03<00:07, 32.56it/s]
Loading 0:  33%|███▎      | 118/363 [00:03<00:08, 29.04it/s]
Loading 0:  34%|███▍      | 123/363 [00:03<00:07, 32.89it/s]
Loading 0:  35%|███▍      | 127/363 [00:03<00:07, 29.85it/s]
Loading 0:  36%|███▋      | 132/363 [00:03<00:07, 32.79it/s]
Loading 0:  37%|███▋      | 136/363 [00:03<00:07, 29.18it/s]
Loading 0:  39%|███▉      | 141/363 [00:04<00:06, 32.58it/s]
Loading 0:  40%|███▉      | 145/363 [00:04<00:10, 21.42it/s]
Loading 0:  41%|████      | 149/363 [00:04<00:09, 21.66it/s]
Loading 0:  42%|████▏     | 154/363 [00:04<00:07, 26.15it/s]
Loading 0:  44%|████▎     | 158/363 [00:04<00:07, 25.75it/s]
Loading 0:  45%|████▍     | 163/363 [00:05<00:06, 29.44it/s]
Loading 0:  46%|████▌     | 167/363 [00:05<00:07, 27.18it/s]
Loading 0:  47%|████▋     | 172/363 [00:05<00:06, 31.08it/s]
Loading 0:  48%|████▊     | 176/363 [00:05<00:06, 28.58it/s]
Loading 0:  50%|████▉     | 181/363 [00:05<00:05, 31.65it/s]
Loading 0:  51%|█████     | 185/363 [00:05<00:06, 28.91it/s]
Loading 0:  52%|█████▏    | 190/363 [00:05<00:05, 33.01it/s]
Loading 0:  53%|█████▎    | 194/363 [00:06<00:05, 30.71it/s]
Loading 0:  55%|█████▍    | 199/363 [00:06<00:04, 34.59it/s]
Loading 0:  56%|█████▌    | 203/363 [00:06<00:04, 32.21it/s]
Loading 0:  58%|█████▊    | 210/363 [00:06<00:03, 39.16it/s]
Loading 0:  59%|█████▉    | 215/363 [00:06<00:03, 38.62it/s]
Loading 0:  60%|██████    | 219/363 [00:06<00:03, 37.32it/s]
Loading 0:  61%|██████▏   | 223/363 [00:06<00:05, 27.09it/s]
Loading 0:  63%|██████▎   | 227/363 [00:07<00:04, 28.52it/s]
Loading 0:  64%|██████▎   | 231/363 [00:07<00:04, 27.78it/s]
Loading 0:  65%|██████▌   | 237/363 [00:07<00:03, 32.78it/s]
Loading 0:  66%|██████▋   | 241/363 [00:07<00:03, 33.41it/s]
Loading 0:  68%|██████▊   | 246/363 [00:07<00:03, 37.21it/s]
Loading 0:  69%|██████▉   | 250/363 [00:07<00:03, 36.52it/s]
Loading 0:  70%|███████   | 255/363 [00:07<00:02, 38.14it/s]
Loading 0:  71%|███████▏  | 259/363 [00:07<00:02, 36.34it/s]
Loading 0:  73%|███████▎  | 265/363 [00:08<00:02, 40.49it/s]
Loading 0:  74%|███████▍  | 270/363 [00:08<00:02, 42.59it/s]
Loading 0:  76%|███████▌  | 275/363 [00:08<00:02, 34.27it/s]
Loading 0:  78%|███████▊  | 282/363 [00:08<00:01, 40.65it/s]
Loading 0:  79%|███████▉  | 287/363 [00:08<00:01, 40.13it/s]
Loading 0:  80%|████████  | 292/363 [00:08<00:01, 40.88it/s]
Loading 0:  82%|████████▏ | 297/363 [00:08<00:01, 41.62it/s]
Loading 0:  83%|████████▎ | 302/363 [00:08<00:01, 41.80it/s]
Loading 0:  85%|████████▍ | 307/363 [00:15<00:24,  2.32it/s]
Loading 0:  86%|████████▌ | 312/363 [00:16<00:15,  3.20it/s]
Loading 0:  88%|████████▊ | 319/363 [00:16<00:08,  4.95it/s]
Loading 0:  89%|████████▉ | 324/363 [00:16<00:05,  6.52it/s]
Loading 0:  91%|█████████ | 329/363 [00:16<00:03,  8.52it/s]
Loading 0:  92%|█████████▏| 334/363 [00:16<00:02, 11.08it/s]
Loading 0:  93%|█████████▎| 339/363 [00:16<00:01, 13.02it/s]
Loading 0:  95%|█████████▌| 346/363 [00:16<00:00, 18.33it/s]
Loading 0:  97%|█████████▋| 351/363 [00:17<00:00, 21.34it/s]
Loading 0:  98%|█████████▊| 356/363 [00:17<00:00, 24.27it/s]
Loading 0:  99%|█████████▉| 361/363 [00:17<00:00, 27.57it/s]
                                                            
Job mistralai-mistral-nemo-9330-v177-mkmlizer completed after 134.61s with status: succeeded
Stopping job with name mistralai-mistral-nemo-9330-v177-mkmlizer
Pipeline stage MKMLizer completed in 135.19s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.17s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service mistralai-mistral-nemo-9330-v177
Waiting for inference service mistralai-mistral-nemo-9330-v177 to be ready
Failed to get response for submission bbchicago-nana-nemo-12b-v1-0_v8: ('http://bbchicago-nana-nemo-12b-v1-0-v8-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
Inference service mistralai-mistral-nemo-9330-v177 ready after 120.64985203742981s
Pipeline stage MKMLDeployer completed in 121.20s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.751171588897705s
Received healthy response to inference request in 1.0866186618804932s
Received healthy response to inference request in 0.7047128677368164s
Received healthy response to inference request in 1.3116419315338135s
Received healthy response to inference request in 1.6217389106750488s
5 requests
0 failed requests
5th percentile: 0.7810940265655517
10th percentile: 0.8574751853942871
20th percentile: 1.0102375030517579
30th percentile: 1.1316233158111573
40th percentile: 1.2216326236724853
50th percentile: 1.3116419315338135
60th percentile: 1.4356807231903077
70th percentile: 1.5597195148468017
80th percentile: 1.64762544631958
90th percentile: 1.6993985176086426
95th percentile: 1.7252850532531738
99th percentile: 1.7459942817687988
mean time: 1.2951767921447754
Pipeline stage StressChecker completed in 7.93s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
Connection pool is full, discarding connection: %s. Connection pool size: %s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 5.66s
Shutdown handler de-registered
mistralai-mistral-nemo_9330_v177 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2434.63s
Shutdown handler de-registered
mistralai-mistral-nemo_9330_v177 status is now inactive due to auto deactivation removed underperforming models