cgato-nemo-12b-thespice-_7091

developer_uid: c.gato

submission_id: cgato-nemo-12b-thespice-_7091_v1

model_name: cgato-nemo-12b-thespice-_7091_v1

model_group: cgato/Nemo-12b-TheSpice-

status: torndown

timestamp: 2024-10-28T23:34:14+00:00

num_battles: 9040

num_wins: 4903

celo_rating: 1268.05

family_friendly_score: 0.569

family_friendly_standard_error: 0.007003413453452538

submission_type: basic

model_repo: cgato/Nemo-12b-TheSpice-V0.9-All-v2-KTO-v0.1-E3-392

model_architecture: MistralForCausalLM

model_num_parameters: 12772121600.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

display_name: cgato-nemo-12b-thespice-_7091_v1

is_internal_developer: False

language_model: cgato/Nemo-12b-TheSpice-V0.9-All-v2-KTO-v0.1-E3-392

model_size: 13B

ranking_group: single

us_pacific_date: 2024-10-28

win_ratio: 0.5423672566371681

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': True}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name cgato-nemo-12b-thespice-7091-v1-mkmlizer
Waiting for job on cgato-nemo-12b-thespice-7091-v1-mkmlizer to finish
cgato-nemo-12b-thespice-7091-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
cgato-nemo-12b-thespice-7091-v1-mkmlizer: ║     _____            __           __                                ║
cgato-nemo-12b-thespice-7091-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
cgato-nemo-12b-thespice-7091-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
cgato-nemo-12b-thespice-7091-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
cgato-nemo-12b-thespice-7091-v1-mkmlizer: ║       /___/                                                         ║
cgato-nemo-12b-thespice-7091-v1-mkmlizer: ║                                                                     ║
cgato-nemo-12b-thespice-7091-v1-mkmlizer: ║  Version: 0.11.12                                                   ║
cgato-nemo-12b-thespice-7091-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
cgato-nemo-12b-thespice-7091-v1-mkmlizer: ║  https://mk1.ai                                                     ║
cgato-nemo-12b-thespice-7091-v1-mkmlizer: ║                                                                     ║
cgato-nemo-12b-thespice-7091-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
cgato-nemo-12b-thespice-7091-v1-mkmlizer: ║  belonging to:                                                      ║
cgato-nemo-12b-thespice-7091-v1-mkmlizer: ║                                                                     ║
cgato-nemo-12b-thespice-7091-v1-mkmlizer: ║  Chai Research Corp.                                                ║
cgato-nemo-12b-thespice-7091-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
cgato-nemo-12b-thespice-7091-v1-mkmlizer: ║  Expiration: 2025-01-15 23:59:59                                    ║
cgato-nemo-12b-thespice-7091-v1-mkmlizer: ║                                                                     ║
cgato-nemo-12b-thespice-7091-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Failed to get response for submission rica40325-10-14dpo_v2: ('http://rica40325-10-14dpo-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '')
cgato-nemo-12b-thespice-7091-v1-mkmlizer: Downloaded to shared memory in 55.038s
cgato-nemo-12b-thespice-7091-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp33xoerr0, device:0
cgato-nemo-12b-thespice-7091-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
cgato-nemo-12b-thespice-7091-v1-mkmlizer: quantized model in 35.617s
cgato-nemo-12b-thespice-7091-v1-mkmlizer: Processed model cgato/Nemo-12b-TheSpice-V0.9-All-v2-KTO-v0.1-E3-392 in 90.654s
cgato-nemo-12b-thespice-7091-v1-mkmlizer: creating bucket guanaco-mkml-models
cgato-nemo-12b-thespice-7091-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
cgato-nemo-12b-thespice-7091-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/cgato-nemo-12b-thespice-7091-v1
cgato-nemo-12b-thespice-7091-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/cgato-nemo-12b-thespice-7091-v1/config.json
cgato-nemo-12b-thespice-7091-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/cgato-nemo-12b-thespice-7091-v1/special_tokens_map.json
cgato-nemo-12b-thespice-7091-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/cgato-nemo-12b-thespice-7091-v1/tokenizer_config.json
cgato-nemo-12b-thespice-7091-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/cgato-nemo-12b-thespice-7091-v1/tokenizer.json
cgato-nemo-12b-thespice-7091-v1-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:11, 31.85it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:06, 54.48it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:06, 51.88it/s]
Loading 0:   7%|▋         | 25/363 [00:00<00:06, 53.35it/s]
Loading 0:   9%|▉         | 32/363 [00:00<00:06, 48.50it/s]
Loading 0:  11%|█         | 40/363 [00:00<00:05, 57.13it/s]
Loading 0:  13%|█▎        | 47/363 [00:00<00:05, 56.69it/s]
Loading 0:  15%|█▍        | 53/363 [00:00<00:05, 54.82it/s]
Loading 0:  17%|█▋        | 60/363 [00:01<00:05, 53.99it/s]
Loading 0:  18%|█▊        | 66/363 [00:01<00:07, 37.82it/s]
Loading 0:  20%|█▉        | 71/363 [00:01<00:07, 39.94it/s]
Loading 0:  21%|██        | 76/363 [00:01<00:06, 42.02it/s]
Loading 0:  22%|██▏       | 81/363 [00:01<00:06, 42.88it/s]
Loading 0:  24%|██▍       | 87/363 [00:01<00:06, 42.22it/s]
Loading 0:  25%|██▌       | 92/363 [00:02<00:06, 40.66it/s]
Loading 0:  27%|██▋       | 99/363 [00:02<00:05, 45.60it/s]
Loading 0:  29%|██▉       | 105/363 [00:02<00:05, 43.92it/s]
Loading 0:  31%|███       | 112/363 [00:02<00:05, 48.13it/s]
Loading 0:  32%|███▏      | 117/363 [00:02<00:05, 47.01it/s]
Loading 0:  34%|███▎      | 122/363 [00:02<00:05, 46.23it/s]
Loading 0:  35%|███▍      | 127/363 [00:02<00:06, 35.01it/s]
Loading 0:  37%|███▋      | 134/363 [00:02<00:05, 40.95it/s]
Loading 0:  38%|███▊      | 139/363 [00:03<00:05, 41.05it/s]
Loading 0:  40%|███▉      | 144/363 [00:03<00:07, 28.63it/s]
Loading 0:  41%|████      | 149/363 [00:03<00:06, 31.62it/s]
Loading 0:  43%|████▎     | 157/363 [00:03<00:05, 40.37it/s]
Loading 0:  45%|████▍     | 163/363 [00:03<00:04, 41.45it/s]
Loading 0:  46%|████▋     | 168/363 [00:03<00:04, 41.35it/s]
Loading 0:  48%|████▊     | 175/363 [00:04<00:04, 46.71it/s]
Loading 0:  50%|████▉     | 181/363 [00:04<00:03, 46.25it/s]
Loading 0:  51%|█████     | 186/363 [00:04<00:03, 46.05it/s]
Loading 0:  53%|█████▎    | 193/363 [00:04<00:03, 50.31it/s]
Loading 0:  55%|█████▍    | 199/363 [00:04<00:03, 48.88it/s]
Loading 0:  56%|█████▌    | 204/363 [00:04<00:03, 48.19it/s]
Loading 0:  58%|█████▊    | 211/363 [00:04<00:02, 52.77it/s]
Loading 0:  60%|█████▉    | 217/363 [00:04<00:02, 49.90it/s]
Loading 0:  61%|██████▏   | 223/363 [00:05<00:03, 38.11it/s]
Loading 0:  63%|██████▎   | 228/363 [00:05<00:03, 38.28it/s]
Loading 0:  64%|██████▍   | 233/363 [00:05<00:03, 40.81it/s]
Loading 0:  66%|██████▌   | 238/363 [00:05<00:02, 42.07it/s]
Loading 0:  67%|██████▋   | 244/363 [00:05<00:02, 42.85it/s]
Loading 0:  69%|██████▊   | 249/363 [00:05<00:02, 43.30it/s]
Loading 0:  71%|███████   | 256/363 [00:05<00:02, 48.75it/s]
Loading 0:  72%|███████▏  | 262/363 [00:05<00:02, 47.11it/s]
Loading 0:  74%|███████▎  | 267/363 [00:06<00:02, 45.69it/s]
Loading 0:  75%|███████▌  | 274/363 [00:06<00:01, 51.04it/s]
Loading 0:  77%|███████▋  | 280/363 [00:06<00:01, 49.09it/s]
Loading 0:  79%|███████▊  | 285/363 [00:06<00:01, 48.42it/s]
Loading 0:  80%|████████  | 292/363 [00:06<00:01, 52.10it/s]
Loading 0:  82%|████████▏ | 298/363 [00:06<00:01, 48.97it/s]
Loading 0:  84%|████████▎ | 304/363 [00:13<00:20,  2.84it/s]
Loading 0:  85%|████████▍ | 308/363 [00:13<00:15,  3.56it/s]
Loading 0:  86%|████████▌ | 312/363 [00:13<00:11,  4.46it/s]
Loading 0:  88%|████████▊ | 320/363 [00:13<00:05,  7.21it/s]
Loading 0:  90%|████████▉ | 326/363 [00:14<00:03,  9.67it/s]
Loading 0:  91%|█████████ | 331/363 [00:14<00:02, 12.20it/s]
Loading 0:  93%|█████████▎| 338/363 [00:14<00:01, 16.87it/s]
Loading 0:  95%|█████████▍| 344/363 [00:14<00:00, 20.66it/s]
Loading 0:  96%|█████████▌| 349/363 [00:14<00:00, 24.01it/s]
Loading 0:  98%|█████████▊| 356/363 [00:14<00:00, 30.29it/s]
Loading 0: 100%|█████████▉| 362/363 [00:14<00:00, 33.13it/s]
                                                            
Job cgato-nemo-12b-thespice-7091-v1-mkmlizer completed after 124.55s with status: succeeded
Stopping job with name cgato-nemo-12b-thespice-7091-v1-mkmlizer
Pipeline stage MKMLizer completed in 125.11s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.72s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service cgato-nemo-12b-thespice-7091-v1
Waiting for inference service cgato-nemo-12b-thespice-7091-v1 to be ready
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Inference service cgato-nemo-12b-thespice-7091-v1 ready after 110.61705303192139s
Pipeline stage MKMLDeployer completed in 111.20s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.3932483196258545s
Received healthy response to inference request in 1.7058024406433105s
Received healthy response to inference request in 1.9717185497283936s
Received healthy response to inference request in 1.693838357925415s
Received healthy response to inference request in 1.684704065322876s
5 requests
0 failed requests
5th percentile: 1.6865309238433839
10th percentile: 1.6883577823638916
20th percentile: 1.6920114994049071
30th percentile: 1.6962311744689942
40th percentile: 1.7010168075561523
50th percentile: 1.7058024406433105
60th percentile: 1.8121688842773438
70th percentile: 1.9185353279113768
80th percentile: 2.056024503707886
90th percentile: 2.22463641166687
95th percentile: 2.308942365646362
99th percentile: 2.376387128829956
mean time: 1.88986234664917
Pipeline stage StressChecker completed in 10.89s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.76s
Shutdown handler de-registered
cgato-nemo-12b-thespice-_7091_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2461.23s
Shutdown handler de-registered
cgato-nemo-12b-thespice-_7091_v1 status is now inactive due to auto deactivation removed underperforming models
cgato-nemo-12b-thespice-_7091_v1 status is now torndown due to DeploymentManager action