ilyagusev-saiga-nemo-12b

developer_uid: fengjiayi
submission_id: ilyagusev-saiga-nemo-12b_v2
model_name: ilyagusev-saiga-nemo-12b_v2
model_group: IlyaGusev/saiga_nemo_12b
status: torndown
timestamp: 2025-01-23T07:44:07+00:00
num_battles: 6712
num_wins: 3015
celo_rating: 1229.34
family_friendly_score: 0.6078
family_friendly_standard_error: 0.006904768786860281
submission_type: basic
model_repo: IlyaGusev/saiga_nemo_12b
model_architecture: MistralForCausalLM
model_num_parameters: 12772070400.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
display_name: ilyagusev-saiga-nemo-12b_v2
is_internal_developer: False
language_model: IlyaGusev/saiga_nemo_12b
model_size: 13B
ranking_group: single
us_pacific_date: 2025-01-22
win_ratio: 0.44919547079856975
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name ilyagusev-saiga-nemo-12b-v2-mkmlizer
Waiting for job on ilyagusev-saiga-nemo-12b-v2-mkmlizer to finish
ilyagusev-saiga-nemo-12b-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
ilyagusev-saiga-nemo-12b-v2-mkmlizer: ║     _____            __           __                                ║
ilyagusev-saiga-nemo-12b-v2-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
ilyagusev-saiga-nemo-12b-v2-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
ilyagusev-saiga-nemo-12b-v2-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
ilyagusev-saiga-nemo-12b-v2-mkmlizer: ║       /___/                                                         ║
ilyagusev-saiga-nemo-12b-v2-mkmlizer: ║                                                                     ║
ilyagusev-saiga-nemo-12b-v2-mkmlizer: ║  Version: 0.11.12                                                   ║
ilyagusev-saiga-nemo-12b-v2-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
ilyagusev-saiga-nemo-12b-v2-mkmlizer: ║  https://mk1.ai                                                     ║
ilyagusev-saiga-nemo-12b-v2-mkmlizer: ║                                                                     ║
ilyagusev-saiga-nemo-12b-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
ilyagusev-saiga-nemo-12b-v2-mkmlizer: ║  belonging to:                                                      ║
ilyagusev-saiga-nemo-12b-v2-mkmlizer: ║                                                                     ║
ilyagusev-saiga-nemo-12b-v2-mkmlizer: ║  Chai Research Corp.                                                ║
ilyagusev-saiga-nemo-12b-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
ilyagusev-saiga-nemo-12b-v2-mkmlizer: ║  Expiration: 2025-04-15 23:59:59                                    ║
ilyagusev-saiga-nemo-12b-v2-mkmlizer: ║                                                                     ║
ilyagusev-saiga-nemo-12b-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
ilyagusev-saiga-nemo-12b-v2-mkmlizer: Downloaded to shared memory in 30.968s
ilyagusev-saiga-nemo-12b-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp9epas131, device:0
ilyagusev-saiga-nemo-12b-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
ilyagusev-saiga-nemo-12b-v2-mkmlizer: quantized model in 36.080s
ilyagusev-saiga-nemo-12b-v2-mkmlizer: Processed model IlyaGusev/saiga_nemo_12b in 67.048s
ilyagusev-saiga-nemo-12b-v2-mkmlizer: creating bucket guanaco-mkml-models
ilyagusev-saiga-nemo-12b-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
ilyagusev-saiga-nemo-12b-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/ilyagusev-saiga-nemo-12b-v2
ilyagusev-saiga-nemo-12b-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/ilyagusev-saiga-nemo-12b-v2/config.json
ilyagusev-saiga-nemo-12b-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/ilyagusev-saiga-nemo-12b-v2/special_tokens_map.json
ilyagusev-saiga-nemo-12b-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/ilyagusev-saiga-nemo-12b-v2/tokenizer_config.json
ilyagusev-saiga-nemo-12b-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/ilyagusev-saiga-nemo-12b-v2/tokenizer.json
ilyagusev-saiga-nemo-12b-v2-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:10, 34.59it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:06, 52.32it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:07, 44.93it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:07, 43.75it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:06, 48.95it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:07, 44.24it/s]
Loading 0:  12%|█▏        | 42/363 [00:00<00:07, 42.75it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:06, 47.28it/s]
Loading 0:  15%|█▌        | 55/363 [00:01<00:07, 43.67it/s]
Loading 0:  17%|█▋        | 60/363 [00:01<00:06, 45.13it/s]
Loading 0:  18%|█▊        | 65/363 [00:01<00:10, 28.58it/s]
Loading 0:  20%|█▉        | 72/363 [00:01<00:08, 35.19it/s]
Loading 0:  21%|██        | 77/363 [00:01<00:07, 38.04it/s]
Loading 0:  23%|██▎       | 82/363 [00:02<00:08, 34.18it/s]
Loading 0:  25%|██▍       | 89/363 [00:02<00:06, 41.61it/s]
Loading 0:  26%|██▌       | 94/363 [00:02<00:06, 42.09it/s]
Loading 0:  27%|██▋       | 99/363 [00:02<00:06, 41.87it/s]
Loading 0:  29%|██▊       | 104/363 [00:02<00:05, 43.48it/s]
Loading 0:  30%|███       | 110/363 [00:02<00:06, 41.57it/s]
Loading 0:  32%|███▏      | 115/363 [00:02<00:05, 42.41it/s]
Loading 0:  33%|███▎      | 120/363 [00:02<00:06, 40.00it/s]
Loading 0:  34%|███▍      | 125/363 [00:03<00:05, 42.39it/s]
Loading 0:  36%|███▌      | 130/363 [00:03<00:05, 42.99it/s]
Loading 0:  37%|███▋      | 135/363 [00:03<00:05, 42.84it/s]
Loading 0:  39%|███▊      | 140/363 [00:03<00:05, 44.49it/s]
Loading 0:  40%|███▉      | 145/363 [00:03<00:08, 25.21it/s]
Loading 0:  41%|████      | 149/363 [00:03<00:08, 26.00it/s]
Loading 0:  43%|████▎     | 156/363 [00:04<00:06, 33.83it/s]
Loading 0:  44%|████▍     | 161/363 [00:04<00:05, 35.75it/s]
Loading 0:  46%|████▌     | 166/363 [00:04<00:05, 37.59it/s]
Loading 0:  47%|████▋     | 171/363 [00:04<00:04, 39.83it/s]
Loading 0:  48%|████▊     | 176/363 [00:04<00:05, 34.53it/s]
Loading 0:  50%|█████     | 183/363 [00:04<00:04, 41.82it/s]
Loading 0:  52%|█████▏    | 188/363 [00:04<00:04, 41.67it/s]
Loading 0:  53%|█████▎    | 193/363 [00:04<00:04, 41.73it/s]
Loading 0:  55%|█████▍    | 198/363 [00:04<00:03, 43.53it/s]
Loading 0:  56%|█████▌    | 203/363 [00:05<00:04, 37.31it/s]
Loading 0:  58%|█████▊    | 210/363 [00:05<00:03, 43.61it/s]
Loading 0:  59%|█████▉    | 215/363 [00:05<00:03, 41.35it/s]
Loading 0:  61%|██████    | 221/363 [00:05<00:03, 45.64it/s]
Loading 0:  62%|██████▏   | 226/363 [00:05<00:05, 26.49it/s]
Loading 0:  63%|██████▎   | 230/363 [00:06<00:04, 27.66it/s]
Loading 0:  65%|██████▌   | 237/363 [00:06<00:03, 35.11it/s]
Loading 0:  67%|██████▋   | 242/363 [00:06<00:03, 36.40it/s]
Loading 0:  68%|██████▊   | 247/363 [00:06<00:03, 37.91it/s]
Loading 0:  70%|██████▉   | 253/363 [00:06<00:02, 37.82it/s]
Loading 0:  71%|███████   | 258/363 [00:06<00:02, 38.06it/s]
Loading 0:  73%|███████▎  | 264/363 [00:06<00:02, 41.85it/s]
Loading 0:  74%|███████▍  | 269/363 [00:06<00:02, 41.38it/s]
Loading 0:  75%|███████▌  | 274/363 [00:07<00:02, 42.27it/s]
Loading 0:  77%|███████▋  | 280/363 [00:07<00:02, 40.05it/s]
Loading 0:  79%|███████▊  | 285/363 [00:07<00:01, 39.10it/s]
Loading 0:  80%|████████  | 291/363 [00:07<00:01, 43.41it/s]
Loading 0:  82%|████████▏ | 296/363 [00:07<00:01, 43.69it/s]
Loading 0:  83%|████████▎ | 301/363 [00:07<00:01, 44.73it/s]
Loading 0:  84%|████████▍ | 306/363 [00:14<00:23,  2.45it/s]
Loading 0:  85%|████████▌ | 310/363 [00:14<00:16,  3.18it/s]
Loading 0:  87%|████████▋ | 314/363 [00:14<00:11,  4.19it/s]
Loading 0:  88%|████████▊ | 320/363 [00:14<00:06,  6.27it/s]
Loading 0:  90%|████████▉ | 326/363 [00:14<00:04,  8.75it/s]
Loading 0:  91%|█████████ | 330/363 [00:15<00:03, 10.68it/s]
Loading 0:  93%|█████████▎| 338/363 [00:15<00:01, 16.41it/s]
Loading 0:  95%|█████████▍| 344/363 [00:15<00:00, 19.94it/s]
Loading 0:  96%|█████████▌| 349/363 [00:15<00:00, 23.25it/s]
Loading 0:  98%|█████████▊| 356/363 [00:15<00:00, 29.45it/s]
Loading 0: 100%|█████████▉| 362/363 [00:15<00:00, 31.61it/s]
                                                            
Job ilyagusev-saiga-nemo-12b-v2-mkmlizer completed after 94.04s with status: succeeded
Stopping job with name ilyagusev-saiga-nemo-12b-v2-mkmlizer
Pipeline stage MKMLizer completed in 94.51s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.17s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service ilyagusev-saiga-nemo-12b-v2
Waiting for inference service ilyagusev-saiga-nemo-12b-v2 to be ready
Inference service ilyagusev-saiga-nemo-12b-v2 ready after 140.6604859828949s
Pipeline stage MKMLDeployer completed in 141.19s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.0676722526550293s
Received healthy response to inference request in 1.6991238594055176s
Received healthy response to inference request in 1.6189391613006592s
Received healthy response to inference request in 1.742690086364746s
Received healthy response to inference request in 1.5818898677825928s
5 requests
0 failed requests
5th percentile: 1.589299726486206
10th percentile: 1.5967095851898194
20th percentile: 1.6115293025970459
30th percentile: 1.6349761009216308
40th percentile: 1.6670499801635743
50th percentile: 1.6991238594055176
60th percentile: 1.716550350189209
70th percentile: 1.7339768409729004
80th percentile: 1.8076865196228027
90th percentile: 1.937679386138916
95th percentile: 2.0026758193969725
99th percentile: 2.0546729660034178
mean time: 1.742063045501709
Pipeline stage StressChecker completed in 9.98s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.61s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.67s
Shutdown handler de-registered
ilyagusev-saiga-nemo-12b_v2 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2540.72s
Shutdown handler de-registered
ilyagusev-saiga-nemo-12b_v2 status is now inactive due to auto deactivation removed underperforming models
ilyagusev-saiga-nemo-12b_v2 status is now torndown due to DeploymentManager action