chaiml-nemo-anthropic-ad_3823

developer_uid: azuruce
submission_id: chaiml-nemo-anthropic-ad_3823_v1
model_name: anthropic-addict-lexical
model_group: ChaiML/nemo_anthropic_ad
status: torndown
timestamp: 2024-11-03T05:20:27+00:00
num_battles: 9650
num_wins: 5554
celo_rating: 1278.45
family_friendly_score: 0.5748
family_friendly_standard_error: 0.006991494260885866
submission_type: basic
model_repo: ChaiML/nemo_anthropic_addict_lexical_1120_run2
model_architecture: MistralForCausalLM
model_num_parameters: 12772070400.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
display_name: anthropic-addict-lexical
is_internal_developer: True
language_model: ChaiML/nemo_anthropic_addict_lexical_1120_run2
model_size: 13B
ranking_group: single
us_pacific_date: 2024-11-02
win_ratio: 0.5755440414507772
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|eot_id|>', '<|end_of_text|>', 'You:'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '### Response:\n{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-nemo-anthropic-ad-3823-v1-mkmlizer
Waiting for job on chaiml-nemo-anthropic-ad-3823-v1-mkmlizer to finish
chaiml-nemo-anthropic-ad-3823-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-nemo-anthropic-ad-3823-v1-mkmlizer: ║     _____            __           __                                ║
chaiml-nemo-anthropic-ad-3823-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
chaiml-nemo-anthropic-ad-3823-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
chaiml-nemo-anthropic-ad-3823-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
chaiml-nemo-anthropic-ad-3823-v1-mkmlizer: ║       /___/                                                         ║
chaiml-nemo-anthropic-ad-3823-v1-mkmlizer: ║                                                                     ║
chaiml-nemo-anthropic-ad-3823-v1-mkmlizer: ║  Version: 0.11.12                                                   ║
chaiml-nemo-anthropic-ad-3823-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
chaiml-nemo-anthropic-ad-3823-v1-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-nemo-anthropic-ad-3823-v1-mkmlizer: ║                                                                     ║
chaiml-nemo-anthropic-ad-3823-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-nemo-anthropic-ad-3823-v1-mkmlizer: ║  belonging to:                                                      ║
chaiml-nemo-anthropic-ad-3823-v1-mkmlizer: ║                                                                     ║
chaiml-nemo-anthropic-ad-3823-v1-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-nemo-anthropic-ad-3823-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-nemo-anthropic-ad-3823-v1-mkmlizer: ║  Expiration: 2025-01-15 23:59:59                                    ║
chaiml-nemo-anthropic-ad-3823-v1-mkmlizer: ║                                                                     ║
chaiml-nemo-anthropic-ad-3823-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-nemo-anthropic-ad-3823-v1-mkmlizer: Downloaded to shared memory in 49.233s
chaiml-nemo-anthropic-ad-3823-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpj9twd25x, device:0
chaiml-nemo-anthropic-ad-3823-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-nemo-anthropic-ad-3823-v1-mkmlizer: quantized model in 36.414s
chaiml-nemo-anthropic-ad-3823-v1-mkmlizer: Processed model ChaiML/nemo_anthropic_addict_lexical_1120_run2 in 85.647s
chaiml-nemo-anthropic-ad-3823-v1-mkmlizer: creating bucket guanaco-mkml-models
chaiml-nemo-anthropic-ad-3823-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-nemo-anthropic-ad-3823-v1/config.json
chaiml-nemo-anthropic-ad-3823-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-nemo-anthropic-ad-3823-v1/special_tokens_map.json
chaiml-nemo-anthropic-ad-3823-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-nemo-anthropic-ad-3823-v1/tokenizer_config.json
chaiml-nemo-anthropic-ad-3823-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-nemo-anthropic-ad-3823-v1/tokenizer.json
chaiml-nemo-anthropic-ad-3823-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-nemo-anthropic-ad-3823-v1/flywheel_model.0.safetensors
chaiml-nemo-anthropic-ad-3823-v1-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:12, 27.93it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:07, 47.90it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:07, 43.95it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:07, 43.77it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:06, 50.64it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:06, 48.87it/s]
Loading 0:  12%|█▏        | 43/363 [00:00<00:06, 49.76it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:06, 52.15it/s]
Loading 0:  15%|█▌        | 55/363 [00:01<00:06, 49.41it/s]
Loading 0:  17%|█▋        | 61/363 [00:01<00:08, 36.63it/s]
Loading 0:  18%|█▊        | 66/363 [00:01<00:08, 36.76it/s]
Loading 0:  20%|█▉        | 71/363 [00:01<00:07, 38.83it/s]
Loading 0:  21%|██        | 76/363 [00:01<00:06, 41.39it/s]
Loading 0:  23%|██▎       | 82/363 [00:01<00:07, 39.43it/s]
Loading 0:  25%|██▍       | 90/363 [00:02<00:05, 47.78it/s]
Loading 0:  26%|██▋       | 96/363 [00:02<00:06, 44.43it/s]
Loading 0:  28%|██▊       | 101/363 [00:02<00:06, 42.95it/s]
Loading 0:  30%|██▉       | 108/363 [00:02<00:05, 49.42it/s]
Loading 0:  31%|███▏      | 114/363 [00:02<00:05, 46.28it/s]
Loading 0:  33%|███▎      | 119/363 [00:02<00:05, 44.61it/s]
Loading 0:  34%|███▍      | 125/363 [00:02<00:05, 47.59it/s]
Loading 0:  36%|███▌      | 130/363 [00:02<00:05, 44.84it/s]
Loading 0:  37%|███▋      | 136/363 [00:03<00:05, 41.48it/s]
Loading 0:  39%|███▉      | 142/363 [00:03<00:06, 35.31it/s]
Loading 0:  40%|████      | 146/363 [00:03<00:06, 35.66it/s]
Loading 0:  41%|████▏     | 150/363 [00:03<00:06, 34.64it/s]
Loading 0:  43%|████▎     | 157/363 [00:03<00:05, 41.04it/s]
Loading 0:  45%|████▍     | 163/363 [00:03<00:04, 41.25it/s]
Loading 0:  46%|████▋     | 168/363 [00:03<00:04, 41.53it/s]
Loading 0:  48%|████▊     | 175/363 [00:04<00:04, 46.76it/s]
Loading 0:  50%|████▉     | 180/363 [00:04<00:03, 47.04it/s]
Loading 0:  51%|█████     | 185/363 [00:04<00:04, 39.28it/s]
Loading 0:  53%|█████▎    | 193/363 [00:04<00:03, 47.37it/s]
Loading 0:  55%|█████▍    | 199/363 [00:04<00:03, 45.26it/s]
Loading 0:  56%|█████▌    | 204/363 [00:04<00:03, 43.57it/s]
Loading 0:  58%|█████▊    | 211/363 [00:04<00:03, 48.09it/s]
Loading 0:  60%|█████▉    | 217/363 [00:05<00:03, 44.56it/s]
Loading 0:  61%|██████    | 222/363 [00:05<00:03, 45.34it/s]
Loading 0:  63%|██████▎   | 227/363 [00:05<00:04, 31.88it/s]
Loading 0:  64%|██████▎   | 231/363 [00:05<00:04, 31.73it/s]
Loading 0:  66%|██████▌   | 238/363 [00:05<00:03, 38.32it/s]
Loading 0:  67%|██████▋   | 243/363 [00:05<00:03, 39.80it/s]
Loading 0:  68%|██████▊   | 248/363 [00:05<00:03, 33.53it/s]
Loading 0:  70%|███████   | 255/363 [00:06<00:02, 41.15it/s]
Loading 0:  72%|███████▏  | 260/363 [00:06<00:02, 41.61it/s]
Loading 0:  73%|███████▎  | 265/363 [00:06<00:02, 41.68it/s]
Loading 0:  74%|███████▍  | 270/363 [00:06<00:02, 43.26it/s]
Loading 0:  76%|███████▌  | 275/363 [00:06<00:02, 36.79it/s]
Loading 0:  78%|███████▊  | 282/363 [00:06<00:01, 43.62it/s]
Loading 0:  79%|███████▉  | 287/363 [00:06<00:01, 42.40it/s]
Loading 0:  80%|████████  | 292/363 [00:06<00:01, 40.24it/s]
Loading 0:  82%|████████▏ | 297/363 [00:07<00:01, 40.80it/s]
Loading 0:  83%|████████▎ | 303/363 [00:07<00:01, 39.66it/s]
Loading 0:  85%|████████▍ | 308/363 [00:14<00:22,  2.49it/s]
Loading 0:  86%|████████▌ | 312/363 [00:14<00:15,  3.22it/s]
Loading 0:  88%|████████▊ | 320/363 [00:14<00:08,  5.30it/s]
Loading 0:  90%|████████▉ | 326/363 [00:14<00:05,  7.25it/s]
Loading 0:  91%|█████████ | 331/363 [00:14<00:03,  9.32it/s]
Loading 0:  93%|█████████▎| 338/363 [00:14<00:01, 13.29it/s]
Loading 0:  95%|█████████▍| 344/363 [00:14<00:01, 16.63it/s]
Loading 0:  96%|█████████▌| 349/363 [00:14<00:00, 19.71it/s]
Loading 0:  98%|█████████▊| 356/363 [00:15<00:00, 25.63it/s]
Loading 0: 100%|█████████▉| 362/363 [00:15<00:00, 28.73it/s]
                                                            
Job chaiml-nemo-anthropic-ad-3823-v1-mkmlizer completed after 114.52s with status: succeeded
Stopping job with name chaiml-nemo-anthropic-ad-3823-v1-mkmlizer
Pipeline stage MKMLizer completed in 115.17s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.18s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-nemo-anthropic-ad-3823-v1
Waiting for inference service chaiml-nemo-anthropic-ad-3823-v1 to be ready
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Inference service chaiml-nemo-anthropic-ad-3823-v1 ready after 151.09731197357178s
Pipeline stage MKMLDeployer completed in 151.72s
run pipeline stage %s
Running pipeline stage StressChecker
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.041743040084839s
Received healthy response to inference request in 1.9110863208770752s
Received healthy response to inference request in 1.852806806564331s
Received healthy response to inference request in 1.6005499362945557s
5 requests
1 failed requests
5th percentile: 1.6510013103485108
10th percentile: 1.7014526844024658
20th percentile: 1.8023554325103759
30th percentile: 1.86446270942688
40th percentile: 1.8877745151519776
50th percentile: 1.9110863208770752
60th percentile: 1.9633490085601806
70th percentile: 2.015611696243286
80th percentile: 5.666841459274295
90th percentile: 12.9170382976532
95th percentile: 16.54213671684265
99th percentile: 19.442215452194212
mean time: 5.514684247970581
%s, retrying in %s seconds...
Received healthy response to inference request in 1.8653538227081299s
Received healthy response to inference request in 1.7839596271514893s
Received healthy response to inference request in 1.693404197692871s
Received healthy response to inference request in 1.6404273509979248s
Received healthy response to inference request in 1.8430254459381104s
5 requests
0 failed requests
5th percentile: 1.651022720336914
10th percentile: 1.6616180896759034
20th percentile: 1.6828088283538818
30th percentile: 1.7115152835845948
40th percentile: 1.747737455368042
50th percentile: 1.7839596271514893
60th percentile: 1.8075859546661377
70th percentile: 1.8312122821807861
80th percentile: 1.8474911212921143
90th percentile: 1.8564224720001221
95th percentile: 1.8608881473541259
99th percentile: 1.864460687637329
mean time: 1.7652340888977052
Pipeline stage StressChecker completed in 40.22s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 3.24s
Shutdown handler de-registered
chaiml-nemo-anthropic-ad_3823_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2553.90s
Shutdown handler de-registered
chaiml-nemo-anthropic-ad_3823_v1 status is now inactive due to auto deactivation removed underperforming models
chaiml-nemo-anthropic-ad_3823_v1 status is now torndown due to DeploymentManager action