vagosolutions-sauerkraut_8298

developer_uid: azuruce

submission_id: vagosolutions-sauerkraut_8298_v1

model_name: vagosolutions-sauerkraut_8298_v1

model_group: VAGOsolutions/Sauerkraut

status: torndown

timestamp: 2024-10-09T21:55:24+00:00

num_battles: 5807

num_wins: 2951

celo_rating: 1250.44

family_friendly_score: 0.5650486596214699

family_friendly_standard_error: 0.006608228609155395

submission_type: basic

model_repo: VAGOsolutions/SauerkrautLM-Nemo-12b-Instruct

model_architecture: MistralForCausalLM

model_num_parameters: 12772070400.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

display_name: vagosolutions-sauerkraut_8298_v1

is_internal_developer: True

language_model: VAGOsolutions/SauerkrautLM-Nemo-12b-Instruct

model_size: 13B

ranking_group: single

us_pacific_date: 2024-10-09

win_ratio: 0.5081797830204925

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|eot_id|>', '<|end_of_text|>', '\nYou:'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '### Response:\n{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name vagosolutions-sauerkraut-8298-v1-mkmlizer
Waiting for job on vagosolutions-sauerkraut-8298-v1-mkmlizer to finish
vagosolutions-sauerkraut-8298-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
vagosolutions-sauerkraut-8298-v1-mkmlizer: ║     _____            __           __                                ║
vagosolutions-sauerkraut-8298-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
vagosolutions-sauerkraut-8298-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
vagosolutions-sauerkraut-8298-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
vagosolutions-sauerkraut-8298-v1-mkmlizer: ║       /___/                                                         ║
vagosolutions-sauerkraut-8298-v1-mkmlizer: ║                                                                     ║
vagosolutions-sauerkraut-8298-v1-mkmlizer: ║  Version: 0.11.12                                                   ║
vagosolutions-sauerkraut-8298-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
vagosolutions-sauerkraut-8298-v1-mkmlizer: ║  https://mk1.ai                                                     ║
vagosolutions-sauerkraut-8298-v1-mkmlizer: ║                                                                     ║
vagosolutions-sauerkraut-8298-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
vagosolutions-sauerkraut-8298-v1-mkmlizer: ║  belonging to:                                                      ║
vagosolutions-sauerkraut-8298-v1-mkmlizer: ║                                                                     ║
vagosolutions-sauerkraut-8298-v1-mkmlizer: ║  Chai Research Corp.                                                ║
vagosolutions-sauerkraut-8298-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
vagosolutions-sauerkraut-8298-v1-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
vagosolutions-sauerkraut-8298-v1-mkmlizer: ║                                                                     ║
vagosolutions-sauerkraut-8298-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
vagosolutions-sauerkraut-8298-v1-mkmlizer: Downloaded to shared memory in 62.226s
vagosolutions-sauerkraut-8298-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp4t__s9as, device:0
vagosolutions-sauerkraut-8298-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
vagosolutions-sauerkraut-8298-v1-mkmlizer: quantized model in 37.441s
vagosolutions-sauerkraut-8298-v1-mkmlizer: Processed model VAGOsolutions/SauerkrautLM-Nemo-12b-Instruct in 99.668s
vagosolutions-sauerkraut-8298-v1-mkmlizer: creating bucket guanaco-mkml-models
vagosolutions-sauerkraut-8298-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
vagosolutions-sauerkraut-8298-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/vagosolutions-sauerkraut-8298-v1
vagosolutions-sauerkraut-8298-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/vagosolutions-sauerkraut-8298-v1/config.json
vagosolutions-sauerkraut-8298-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/vagosolutions-sauerkraut-8298-v1/special_tokens_map.json
vagosolutions-sauerkraut-8298-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/vagosolutions-sauerkraut-8298-v1/tokenizer_config.json
vagosolutions-sauerkraut-8298-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/vagosolutions-sauerkraut-8298-v1/tokenizer.json
vagosolutions-sauerkraut-8298-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/vagosolutions-sauerkraut-8298-v1/flywheel_model.0.safetensors
vagosolutions-sauerkraut-8298-v1-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|          | 2/363 [00:06<18:45,  3.12s/it]
Loading 0:   2%|▏         | 6/363 [00:06<04:59,  1.19it/s]
Loading 0:   3%|▎         | 11/363 [00:06<02:11,  2.68it/s]
Loading 0:   4%|▍         | 15/363 [00:06<01:22,  4.20it/s]
Loading 0:   6%|▋         | 23/363 [00:06<00:40,  8.35it/s]
Loading 0:   8%|▊         | 28/363 [00:06<00:29, 11.30it/s]
Loading 0:   9%|▉         | 33/363 [00:07<00:24, 13.75it/s]
Loading 0:  11%|█         | 40/363 [00:07<00:16, 19.90it/s]
Loading 0:  12%|█▏        | 45/363 [00:07<00:13, 23.37it/s]
Loading 0:  14%|█▍        | 50/363 [00:07<00:11, 27.23it/s]
Loading 0:  15%|█▌        | 56/363 [00:07<00:10, 29.60it/s]
Loading 0:  17%|█▋        | 61/363 [00:07<00:13, 22.40it/s]
Loading 0:  18%|█▊        | 67/363 [00:08<00:10, 27.78it/s]
Loading 0:  20%|█▉        | 72/363 [00:08<00:09, 30.62it/s]
Loading 0:  21%|██        | 77/363 [00:08<00:08, 33.45it/s]
Loading 0:  23%|██▎       | 82/363 [00:08<00:07, 36.27it/s]
Loading 0:  24%|██▍       | 87/363 [00:08<00:08, 31.68it/s]
Loading 0:  26%|██▌       | 94/363 [00:08<00:06, 39.09it/s]
Loading 0:  27%|██▋       | 99/363 [00:08<00:06, 39.66it/s]
Loading 0:  29%|██▊       | 104/363 [00:08<00:06, 40.40it/s]
Loading 0:  30%|███       | 109/363 [00:09<00:05, 42.40it/s]
Loading 0:  31%|███▏      | 114/363 [00:09<00:06, 35.70it/s]
Loading 0:  33%|███▎      | 121/363 [00:09<00:05, 42.36it/s]
Loading 0:  35%|███▍      | 126/363 [00:09<00:05, 41.38it/s]
Loading 0:  36%|███▌      | 131/363 [00:09<00:05, 41.19it/s]
Loading 0:  37%|███▋      | 136/363 [00:09<00:05, 42.17it/s]
Loading 0:  39%|███▉      | 141/363 [00:09<00:06, 34.94it/s]
Loading 0:  41%|████      | 148/363 [00:09<00:05, 41.96it/s]
Loading 0:  42%|████▏     | 153/363 [00:10<00:05, 41.99it/s]
Loading 0:  44%|████▎     | 158/363 [00:10<00:07, 27.37it/s]
Loading 0:  45%|████▍     | 163/363 [00:10<00:06, 31.08it/s]
Loading 0:  46%|████▌     | 167/363 [00:10<00:06, 31.90it/s]
Loading 0:  47%|████▋     | 171/363 [00:10<00:05, 33.61it/s]
Loading 0:  48%|████▊     | 175/363 [00:10<00:05, 34.96it/s]
Loading 0:  49%|████▉     | 179/363 [00:11<00:05, 35.21it/s]
Loading 0:  51%|█████     | 184/363 [00:11<00:04, 38.31it/s]
Loading 0:  52%|█████▏    | 189/363 [00:11<00:04, 39.64it/s]
Loading 0:  53%|█████▎    | 194/363 [00:11<00:04, 40.26it/s]
Loading 0:  55%|█████▍    | 199/363 [00:11<00:03, 42.52it/s]
Loading 0:  56%|█████▌    | 204/363 [00:11<00:04, 34.68it/s]
Loading 0:  58%|█████▊    | 211/363 [00:11<00:03, 41.83it/s]
Loading 0:  60%|█████▉    | 216/363 [00:11<00:03, 40.20it/s]
Loading 0:  61%|██████    | 221/363 [00:12<00:03, 39.78it/s]
Loading 0:  62%|██████▏   | 226/363 [00:12<00:03, 41.60it/s]
Loading 0:  64%|██████▎   | 231/363 [00:12<00:03, 35.16it/s]
Loading 0:  66%|██████▌   | 238/363 [00:12<00:02, 42.08it/s]
Loading 0:  67%|██████▋   | 243/363 [00:12<00:02, 41.94it/s]
Loading 0:  68%|██████▊   | 248/363 [00:12<00:02, 41.96it/s]
Loading 0:  70%|██████▉   | 253/363 [00:12<00:02, 43.87it/s]
Loading 0:  71%|███████   | 258/363 [00:13<00:04, 24.70it/s]
Loading 0:  73%|███████▎  | 265/363 [00:13<00:03, 31.62it/s]
Loading 0:  74%|███████▍  | 270/363 [00:13<00:02, 33.33it/s]
Loading 0:  76%|███████▌  | 275/363 [00:13<00:02, 35.16it/s]
Loading 0:  77%|███████▋  | 280/363 [00:13<00:02, 37.95it/s]
Loading 0:  79%|███████▊  | 285/363 [00:13<00:02, 32.70it/s]
Loading 0:  80%|████████  | 292/363 [00:13<00:01, 39.95it/s]
Loading 0:  82%|████████▏ | 297/363 [00:14<00:01, 40.90it/s]
Loading 0:  83%|████████▎ | 302/363 [00:14<00:01, 41.86it/s]
Loading 0:  85%|████████▍ | 307/363 [00:14<00:01, 43.88it/s]
Loading 0:  86%|████████▌ | 312/363 [00:14<00:01, 35.74it/s]
Loading 0:  88%|████████▊ | 319/363 [00:14<00:01, 42.69it/s]
Loading 0:  89%|████████▉ | 324/363 [00:14<00:00, 42.28it/s]
Loading 0:  91%|█████████ | 329/363 [00:14<00:00, 41.28it/s]
Loading 0:  92%|█████████▏| 334/363 [00:14<00:00, 43.04it/s]
Loading 0:  93%|█████████▎| 339/363 [00:15<00:00, 35.92it/s]
Loading 0:  95%|█████████▌| 346/363 [00:15<00:00, 42.75it/s]
Loading 0:  97%|█████████▋| 351/363 [00:15<00:00, 42.71it/s]
Loading 0:  98%|█████████▊| 356/363 [00:15<00:00, 28.47it/s]
Loading 0:  99%|█████████▉| 360/363 [00:15<00:00, 30.50it/s]
                                                            
Job vagosolutions-sauerkraut-8298-v1-mkmlizer completed after 125.15s with status: succeeded
Stopping job with name vagosolutions-sauerkraut-8298-v1-mkmlizer
Pipeline stage MKMLizer completed in 125.65s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service vagosolutions-sauerkraut-8298-v1
Waiting for inference service vagosolutions-sauerkraut-8298-v1 to be ready
Inference service vagosolutions-sauerkraut-8298-v1 ready after 140.52932357788086s
Pipeline stage MKMLDeployer completed in 141.13s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.1275017261505127s
Received healthy response to inference request in 1.7406458854675293s
Received healthy response to inference request in 1.5439233779907227s
Received healthy response to inference request in 1.4867486953735352s
Received healthy response to inference request in 1.3999624252319336s
5 requests
0 failed requests
5th percentile: 1.4173196792602538
10th percentile: 1.4346769332885743
20th percentile: 1.469391441345215
30th percentile: 1.4981836318969726
40th percentile: 1.5210535049438476
50th percentile: 1.5439233779907227
60th percentile: 1.6226123809814452
70th percentile: 1.701301383972168
80th percentile: 1.818017053604126
90th percentile: 1.9727593898773192
95th percentile: 2.0501305580139157
99th percentile: 2.112027492523193
mean time: 1.6597564220428467
Pipeline stage StressChecker completed in 10.05s
Shutdown handler de-registered
vagosolutions-sauerkraut_8298_v1 status is now deployed due to DeploymentManager action
vagosolutions-sauerkraut_8298_v1 status is now inactive due to auto deactivation removed underperforming models
vagosolutions-sauerkraut_8298_v1 status is now torndown due to DeploymentManager action