jic062-nemo-v1-6

developer_uid: chace9580
submission_id: jic062-nemo-v1-6_v6
model_name: jic062-nemo-v1-6_v6
model_group: jic062/Nemo-v1.6
status: torndown
timestamp: 2024-10-31T02:19:56+00:00
num_battles: 9854
num_wins: 5303
celo_rating: 1262.98
family_friendly_score: 0.569
family_friendly_standard_error: 0.007003413453452538
submission_type: basic
model_repo: jic062/Nemo-v1.6
model_architecture: MistralForCausalLM
model_num_parameters: 12772070400.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
display_name: jic062-nemo-v1-6_v6
is_internal_developer: False
language_model: jic062/Nemo-v1.6
model_size: 13B
ranking_group: single
us_pacific_date: 2024-10-30
win_ratio: 0.5381570935660646
generation_params: {'temperature': 1.0, 'top_p': 0.9, 'min_p': 0.05, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '[INST]system\n{memory}[/INST]\n', 'prompt_template': '[INST]user\n{prompt}[/INST]\n', 'bot_template': '[INST]assistant\n{bot_name}: {message}[/INST]\n', 'user_template': '[INST]user\n{user_name}: {message}[/INST]\n', 'response_template': '[INST]assistant\n{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name jic062-nemo-v1-6-v6-mkmlizer
Waiting for job on jic062-nemo-v1-6-v6-mkmlizer to finish
jic062-nemo-v1-6-v6-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
jic062-nemo-v1-6-v6-mkmlizer: ║     _____            __           __                                ║
jic062-nemo-v1-6-v6-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
jic062-nemo-v1-6-v6-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
jic062-nemo-v1-6-v6-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
jic062-nemo-v1-6-v6-mkmlizer: ║       /___/                                                         ║
jic062-nemo-v1-6-v6-mkmlizer: ║                                                                     ║
jic062-nemo-v1-6-v6-mkmlizer: ║  Version: 0.11.12                                                   ║
jic062-nemo-v1-6-v6-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
jic062-nemo-v1-6-v6-mkmlizer: ║  https://mk1.ai                                                     ║
jic062-nemo-v1-6-v6-mkmlizer: ║                                                                     ║
jic062-nemo-v1-6-v6-mkmlizer: ║  The license key for the current software has been verified as      ║
jic062-nemo-v1-6-v6-mkmlizer: ║  belonging to:                                                      ║
jic062-nemo-v1-6-v6-mkmlizer: ║                                                                     ║
jic062-nemo-v1-6-v6-mkmlizer: ║  Chai Research Corp.                                                ║
jic062-nemo-v1-6-v6-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
jic062-nemo-v1-6-v6-mkmlizer: ║  Expiration: 2025-01-15 23:59:59                                    ║
jic062-nemo-v1-6-v6-mkmlizer: ║                                                                     ║
jic062-nemo-v1-6-v6-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
jic062-nemo-v1-6-v6-mkmlizer: Downloaded to shared memory in 109.379s
jic062-nemo-v1-6-v6-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpgxsfz3n1, device:0
jic062-nemo-v1-6-v6-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Connection pool is full, discarding connection: %s. Connection pool size: %s
jic062-nemo-v1-6-v6-mkmlizer: quantized model in 35.621s
jic062-nemo-v1-6-v6-mkmlizer: Processed model jic062/Nemo-v1.6 in 145.000s
jic062-nemo-v1-6-v6-mkmlizer: creating bucket guanaco-mkml-models
jic062-nemo-v1-6-v6-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
jic062-nemo-v1-6-v6-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/jic062-nemo-v1-6-v6
jic062-nemo-v1-6-v6-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/jic062-nemo-v1-6-v6/config.json
jic062-nemo-v1-6-v6-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/jic062-nemo-v1-6-v6/special_tokens_map.json
jic062-nemo-v1-6-v6-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/jic062-nemo-v1-6-v6/tokenizer_config.json
jic062-nemo-v1-6-v6-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/jic062-nemo-v1-6-v6/tokenizer.json
jic062-nemo-v1-6-v6-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/jic062-nemo-v1-6-v6/flywheel_model.0.safetensors
jic062-nemo-v1-6-v6-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:11, 31.85it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:06, 52.93it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:07, 46.68it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:07, 44.31it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:06, 50.40it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:07, 46.30it/s]
Loading 0:  12%|█▏        | 42/363 [00:00<00:07, 43.70it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:06, 48.63it/s]
Loading 0:  15%|█▍        | 54/363 [00:01<00:06, 48.24it/s]
Loading 0:  17%|█▋        | 60/363 [00:01<00:06, 44.66it/s]
Loading 0:  18%|█▊        | 65/363 [00:01<00:09, 30.73it/s]
Loading 0:  20%|█▉        | 72/363 [00:01<00:07, 37.38it/s]
Loading 0:  21%|██        | 77/363 [00:01<00:07, 39.70it/s]
Loading 0:  23%|██▎       | 82/363 [00:01<00:07, 35.69it/s]
Loading 0:  25%|██▍       | 90/363 [00:02<00:06, 43.82it/s]
Loading 0:  26%|██▌       | 95/363 [00:02<00:06, 44.58it/s]
Loading 0:  28%|██▊       | 100/363 [00:02<00:06, 38.43it/s]
Loading 0:  30%|██▉       | 108/363 [00:02<00:05, 47.50it/s]
Loading 0:  31%|███▏      | 114/363 [00:02<00:05, 44.20it/s]
Loading 0:  33%|███▎      | 119/363 [00:02<00:05, 43.12it/s]
Loading 0:  34%|███▍      | 125/363 [00:02<00:05, 47.02it/s]
Loading 0:  36%|███▌      | 130/363 [00:03<00:05, 46.43it/s]
Loading 0:  37%|███▋      | 135/363 [00:03<00:05, 44.11it/s]
Loading 0:  39%|███▉      | 141/363 [00:03<00:05, 42.36it/s]
Loading 0:  40%|████      | 146/363 [00:03<00:07, 30.98it/s]
Loading 0:  41%|████▏     | 150/363 [00:03<00:07, 30.08it/s]
Loading 0:  43%|████▎     | 156/363 [00:03<00:05, 36.16it/s]
Loading 0:  44%|████▍     | 161/363 [00:03<00:05, 37.66it/s]
Loading 0:  46%|████▌     | 166/363 [00:04<00:05, 38.99it/s]
Loading 0:  47%|████▋     | 171/363 [00:04<00:04, 41.05it/s]
Loading 0:  48%|████▊     | 176/363 [00:04<00:05, 36.26it/s]
Loading 0:  51%|█████     | 184/363 [00:04<00:04, 44.74it/s]
Loading 0:  52%|█████▏    | 190/363 [00:04<00:04, 43.06it/s]
Loading 0:  54%|█████▎    | 195/363 [00:04<00:03, 42.44it/s]
Loading 0:  56%|█████▌    | 202/363 [00:04<00:03, 47.22it/s]
Loading 0:  57%|█████▋    | 208/363 [00:04<00:03, 44.46it/s]
Loading 0:  59%|█████▊    | 213/363 [00:05<00:03, 42.42it/s]
Loading 0:  61%|██████    | 220/363 [00:05<00:02, 48.45it/s]
Loading 0:  62%|██████▏   | 226/363 [00:05<00:04, 32.31it/s]
Loading 0:  64%|██████▎   | 231/363 [00:05<00:03, 33.93it/s]
Loading 0:  65%|██████▌   | 237/363 [00:05<00:03, 39.04it/s]
Loading 0:  67%|██████▋   | 242/363 [00:05<00:03, 40.30it/s]
Loading 0:  68%|██████▊   | 247/363 [00:06<00:02, 41.58it/s]
Loading 0:  69%|██████▉   | 252/363 [00:06<00:02, 43.51it/s]
Loading 0:  71%|███████   | 257/363 [00:06<00:02, 37.37it/s]
Loading 0:  73%|███████▎  | 264/363 [00:06<00:02, 44.96it/s]
Loading 0:  74%|███████▍  | 269/363 [00:06<00:02, 44.85it/s]
Loading 0:  75%|███████▌  | 274/363 [00:06<00:01, 45.36it/s]
Loading 0:  77%|███████▋  | 280/363 [00:06<00:01, 43.06it/s]
Loading 0:  79%|███████▊  | 285/363 [00:06<00:01, 40.97it/s]
Loading 0:  80%|████████  | 291/363 [00:07<00:01, 45.49it/s]
Loading 0:  82%|████████▏ | 296/363 [00:07<00:01, 44.65it/s]
Loading 0:  83%|████████▎ | 301/363 [00:07<00:01, 45.12it/s]
Loading 0:  84%|████████▍ | 306/363 [00:13<00:23,  2.46it/s]
Loading 0:  85%|████████▌ | 310/363 [00:14<00:16,  3.21it/s]
Loading 0:  87%|████████▋ | 314/363 [00:14<00:11,  4.20it/s]
Loading 0:  88%|████████▊ | 320/363 [00:14<00:06,  6.27it/s]
Loading 0:  89%|████████▉ | 324/363 [00:14<00:04,  7.98it/s]
Loading 0:  91%|█████████ | 329/363 [00:14<00:03, 10.82it/s]
Loading 0:  92%|█████████▏| 335/363 [00:14<00:01, 14.64it/s]
Loading 0:  94%|█████████▎| 340/363 [00:14<00:01, 18.12it/s]
Loading 0:  96%|█████████▌| 347/363 [00:14<00:00, 24.72it/s]
Loading 0:  97%|█████████▋| 353/363 [00:15<00:00, 28.39it/s]
Loading 0:  99%|█████████▊| 358/363 [00:15<00:00, 31.17it/s]
                                                            
Job jic062-nemo-v1-6-v6-mkmlizer completed after 165.39s with status: succeeded
Stopping job with name jic062-nemo-v1-6-v6-mkmlizer
Pipeline stage MKMLizer completed in 165.96s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.20s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service jic062-nemo-v1-6-v6
Waiting for inference service jic062-nemo-v1-6-v6 to be ready
Inference service jic062-nemo-v1-6-v6 ready after 130.84102487564087s
Pipeline stage MKMLDeployer completed in 131.38s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.3237192630767822s
Received healthy response to inference request in 1.6816489696502686s
Received healthy response to inference request in 1.6740620136260986s
Received healthy response to inference request in 1.97515869140625s
5 requests
1 failed requests
5th percentile: 1.6755794048309327
10th percentile: 1.6770967960357666
20th percentile: 1.6801315784454345
30th percentile: 1.7403509140014648
40th percentile: 1.8577548027038575
50th percentile: 1.97515869140625
60th percentile: 2.114582920074463
70th percentile: 2.2540071487426756
80th percentile: 5.892224025726321
90th percentile: 13.029233551025392
95th percentile: 16.597738313674924
99th percentile: 19.452542123794554
mean time: 5.564166402816772
%s, retrying in %s seconds...
Received healthy response to inference request in 2.0096001625061035s
Received healthy response to inference request in 2.0482635498046875s
Received healthy response to inference request in 1.6742534637451172s
Received healthy response to inference request in 2.0522985458374023s
Received healthy response to inference request in 1.8867449760437012s
5 requests
0 failed requests
5th percentile: 1.716751766204834
10th percentile: 1.7592500686645507
20th percentile: 1.8442466735839844
30th percentile: 1.9113160133361817
40th percentile: 1.9604580879211426
50th percentile: 2.0096001625061035
60th percentile: 2.025065517425537
70th percentile: 2.040530872344971
80th percentile: 2.0490705490112306
90th percentile: 2.0506845474243165
95th percentile: 2.051491546630859
99th percentile: 2.0521371459960935
mean time: 1.9342321395874023
Pipeline stage StressChecker completed in 41.02s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 5.65s
Shutdown handler de-registered
jic062-nemo-v1-6_v6 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2624.94s
Shutdown handler de-registered
jic062-nemo-v1-6_v6 status is now inactive due to auto deactivation removed underperforming models
jic062-nemo-v1-6_v6 status is now torndown due to DeploymentManager action