nbeerbower-mistral-nemo-_9180

developer_uid: azuruce

submission_id: nbeerbower-mistral-nemo-_9180_v2

model_name: nbeerbower-mistral-nemo-_9180_v2

model_group: nbeerbower/mistral-nemo-

status: torndown

timestamp: 2024-10-10T00:56:26+00:00

num_battles: 6593

num_wins: 3292

celo_rating: 1247.13

family_friendly_score: 0.5607935494172123

family_friendly_standard_error: 0.0062387348596167624

submission_type: basic

model_repo: nbeerbower/mistral-nemo-wissenschaft-12B

model_architecture: MistralForCausalLM

model_num_parameters: 12772070400.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

display_name: nbeerbower-mistral-nemo-_9180_v2

is_internal_developer: True

language_model: nbeerbower/mistral-nemo-wissenschaft-12B

model_size: 13B

ranking_group: single

us_pacific_date: 2024-10-09

win_ratio: 0.49931745790990445

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|eot_id|>', '<|end_of_text|>', '\nYou:'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '### Response:\n{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name nbeerbower-mistral-nemo-9180-v2-mkmlizer
Waiting for job on nbeerbower-mistral-nemo-9180-v2-mkmlizer to finish
nbeerbower-mistral-nemo-9180-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
nbeerbower-mistral-nemo-9180-v2-mkmlizer: ║     _____            __           __                                ║
nbeerbower-mistral-nemo-9180-v2-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
nbeerbower-mistral-nemo-9180-v2-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
nbeerbower-mistral-nemo-9180-v2-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
nbeerbower-mistral-nemo-9180-v2-mkmlizer: ║       /___/                                                         ║
nbeerbower-mistral-nemo-9180-v2-mkmlizer: ║                                                                     ║
nbeerbower-mistral-nemo-9180-v2-mkmlizer: ║  Version: 0.11.12                                                   ║
nbeerbower-mistral-nemo-9180-v2-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
nbeerbower-mistral-nemo-9180-v2-mkmlizer: ║  https://mk1.ai                                                     ║
nbeerbower-mistral-nemo-9180-v2-mkmlizer: ║                                                                     ║
nbeerbower-mistral-nemo-9180-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
nbeerbower-mistral-nemo-9180-v2-mkmlizer: ║  belonging to:                                                      ║
nbeerbower-mistral-nemo-9180-v2-mkmlizer: ║                                                                     ║
nbeerbower-mistral-nemo-9180-v2-mkmlizer: ║  Chai Research Corp.                                                ║
nbeerbower-mistral-nemo-9180-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
nbeerbower-mistral-nemo-9180-v2-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
nbeerbower-mistral-nemo-9180-v2-mkmlizer: ║                                                                     ║
nbeerbower-mistral-nemo-9180-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
nbeerbower-mistral-nemo-9180-v2-mkmlizer: Downloaded to shared memory in 28.719s
nbeerbower-mistral-nemo-9180-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpaqwjh8gp, device:0
nbeerbower-mistral-nemo-9180-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
nbeerbower-mistral-nemo-9180-v2-mkmlizer: quantized model in 35.720s
nbeerbower-mistral-nemo-9180-v2-mkmlizer: Processed model nbeerbower/mistral-nemo-wissenschaft-12B in 64.439s
nbeerbower-mistral-nemo-9180-v2-mkmlizer: creating bucket guanaco-mkml-models
nbeerbower-mistral-nemo-9180-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
nbeerbower-mistral-nemo-9180-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/nbeerbower-mistral-nemo-9180-v2
nbeerbower-mistral-nemo-9180-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/nbeerbower-mistral-nemo-9180-v2/config.json
nbeerbower-mistral-nemo-9180-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/nbeerbower-mistral-nemo-9180-v2/special_tokens_map.json
nbeerbower-mistral-nemo-9180-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/nbeerbower-mistral-nemo-9180-v2/tokenizer_config.json
nbeerbower-mistral-nemo-9180-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/nbeerbower-mistral-nemo-9180-v2/tokenizer.json
nbeerbower-mistral-nemo-9180-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/nbeerbower-mistral-nemo-9180-v2/flywheel_model.0.safetensors
nbeerbower-mistral-nemo-9180-v2-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:11, 31.16it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:06, 52.03it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:07, 48.02it/s]
Loading 0:   7%|▋         | 25/363 [00:00<00:06, 48.88it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:06, 51.27it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:06, 48.48it/s]
Loading 0:  12%|█▏        | 42/363 [00:00<00:06, 46.44it/s]
Loading 0:  13%|█▎        | 49/363 [00:00<00:06, 51.65it/s]
Loading 0:  15%|█▌        | 55/363 [00:01<00:06, 48.89it/s]
Loading 0:  17%|█▋        | 61/363 [00:01<00:08, 36.35it/s]
Loading 0:  18%|█▊        | 66/363 [00:01<00:08, 36.83it/s]
Loading 0:  20%|█▉        | 72/363 [00:01<00:07, 41.44it/s]
Loading 0:  21%|██        | 77/363 [00:01<00:06, 43.01it/s]
Loading 0:  23%|██▎       | 82/363 [00:01<00:07, 36.23it/s]
Loading 0:  25%|██▍       | 90/363 [00:02<00:06, 44.44it/s]
Loading 0:  26%|██▋       | 96/363 [00:02<00:06, 43.65it/s]
Loading 0:  28%|██▊       | 101/363 [00:02<00:06, 42.92it/s]
Loading 0:  29%|██▉       | 106/363 [00:02<00:05, 43.66it/s]
Loading 0:  31%|███       | 112/363 [00:02<00:05, 47.11it/s]
Loading 0:  32%|███▏      | 117/363 [00:02<00:05, 43.27it/s]
Loading 0:  34%|███▎      | 122/363 [00:02<00:05, 44.19it/s]
Loading 0:  35%|███▍      | 127/363 [00:02<00:06, 37.40it/s]
Loading 0:  37%|███▋      | 134/363 [00:03<00:05, 45.04it/s]
Loading 0:  38%|███▊      | 139/363 [00:03<00:04, 44.91it/s]
Loading 0:  40%|███▉      | 144/363 [00:03<00:07, 27.90it/s]
Loading 0:  41%|████      | 149/363 [00:03<00:07, 29.72it/s]
Loading 0:  43%|████▎     | 156/363 [00:03<00:05, 36.74it/s]
Loading 0:  44%|████▍     | 161/363 [00:03<00:05, 38.33it/s]
Loading 0:  46%|████▌     | 166/363 [00:04<00:04, 40.12it/s]
Loading 0:  47%|████▋     | 172/363 [00:04<00:04, 40.08it/s]
Loading 0:  49%|████▉     | 177/363 [00:04<00:04, 39.21it/s]
Loading 0:  50%|█████     | 183/363 [00:04<00:04, 43.19it/s]
Loading 0:  52%|█████▏    | 188/363 [00:04<00:04, 42.15it/s]
Loading 0:  53%|█████▎    | 193/363 [00:04<00:03, 42.98it/s]
Loading 0:  55%|█████▍    | 199/363 [00:04<00:03, 41.38it/s]
Loading 0:  56%|█████▌    | 204/363 [00:04<00:04, 39.65it/s]
Loading 0:  58%|█████▊    | 210/363 [00:05<00:03, 44.34it/s]
Loading 0:  59%|█████▉    | 215/363 [00:05<00:03, 44.52it/s]
Loading 0:  61%|██████    | 221/363 [00:05<00:02, 47.80it/s]
Loading 0:  62%|██████▏   | 226/363 [00:05<00:04, 29.39it/s]
Loading 0:  63%|██████▎   | 230/363 [00:05<00:04, 29.38it/s]
Loading 0:  65%|██████▌   | 237/363 [00:05<00:03, 36.97it/s]
Loading 0:  67%|██████▋   | 242/363 [00:05<00:03, 38.43it/s]
Loading 0:  68%|██████▊   | 247/363 [00:06<00:02, 40.43it/s]
Loading 0:  70%|██████▉   | 253/363 [00:06<00:02, 40.42it/s]
Loading 0:  71%|███████   | 258/363 [00:06<00:02, 40.24it/s]
Loading 0:  73%|███████▎  | 264/363 [00:06<00:02, 44.42it/s]
Loading 0:  74%|███████▍  | 269/363 [00:06<00:02, 43.79it/s]
Loading 0:  75%|███████▌  | 274/363 [00:06<00:02, 44.36it/s]
Loading 0:  77%|███████▋  | 280/363 [00:06<00:01, 43.24it/s]
Loading 0:  79%|███████▊  | 285/363 [00:06<00:01, 41.53it/s]
Loading 0:  80%|████████  | 291/363 [00:07<00:01, 45.88it/s]
Loading 0:  82%|████████▏ | 296/363 [00:07<00:01, 46.03it/s]
Loading 0:  83%|████████▎ | 302/363 [00:07<00:01, 49.62it/s]
Loading 0:  85%|████████▍ | 308/363 [00:14<00:20,  2.67it/s]
Loading 0:  86%|████████▌ | 312/363 [00:14<00:15,  3.39it/s]
Loading 0:  88%|████████▊ | 320/363 [00:14<00:07,  5.47it/s]
Loading 0:  90%|████████▉ | 326/363 [00:14<00:05,  7.37it/s]
Loading 0:  91%|█████████ | 331/363 [00:14<00:03,  9.40it/s]
Loading 0:  93%|█████████▎| 338/363 [00:14<00:01, 13.31it/s]
Loading 0:  95%|█████████▍| 344/363 [00:14<00:01, 16.75it/s]
Loading 0:  96%|█████████▌| 349/363 [00:14<00:00, 19.84it/s]
Loading 0:  98%|█████████▊| 356/363 [00:15<00:00, 25.88it/s]
Loading 0: 100%|█████████▉| 362/363 [00:15<00:00, 29.13it/s]
                                                            
Job nbeerbower-mistral-nemo-9180-v2-mkmlizer completed after 94.03s with status: succeeded
Stopping job with name nbeerbower-mistral-nemo-9180-v2-mkmlizer
Pipeline stage MKMLizer completed in 94.61s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.17s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service nbeerbower-mistral-nemo-9180-v2
Waiting for inference service nbeerbower-mistral-nemo-9180-v2 to be ready
Inference service nbeerbower-mistral-nemo-9180-v2 ready after 130.6233148574829s
Pipeline stage MKMLDeployer completed in 131.21s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.967634677886963s
Received healthy response to inference request in 1.7990453243255615s
Received healthy response to inference request in 1.6738882064819336s
Received healthy response to inference request in 1.3038628101348877s
Received healthy response to inference request in 1.4000213146209717s
5 requests
0 failed requests
5th percentile: 1.3230945110321044
10th percentile: 1.3423262119293213
20th percentile: 1.380789613723755
30th percentile: 1.454794692993164
40th percentile: 1.5643414497375487
50th percentile: 1.6738882064819336
60th percentile: 1.7239510536193847
70th percentile: 1.7740139007568358
80th percentile: 1.8327631950378418
90th percentile: 1.9001989364624023
95th percentile: 1.9339168071746826
99th percentile: 1.960891103744507
mean time: 1.6288904666900634
Pipeline stage StressChecker completed in 9.64s
Shutdown handler de-registered
nbeerbower-mistral-nemo-_9180_v2 status is now deployed due to DeploymentManager action
nbeerbower-mistral-nemo-_9180_v2 status is now inactive due to auto deactivation removed underperforming models
nbeerbower-mistral-nemo-_9180_v2 status is now torndown due to DeploymentManager action