kaoeiri-hedoneo-l3-1-8b-v1-2

developer_uid: helkero

submission_id: kaoeiri-hedoneo-l3-1-8b-v1-2_v1

model_name: kaoeiri-hedoneo-l3-1-8b-v1-2_v1

model_group: Kaoeiri/Hedoneo-L3.1-8B-

status: torndown

timestamp: 2024-08-06T15:19:57+00:00

num_battles: 13731

num_wins: 6506

celo_rating: 1185.95

family_friendly_score: 0.0

submission_type: basic

model_repo: Kaoeiri/Hedoneo-L3.1-8B-v1.2

model_architecture: LlamaForCausalLM

reward_repo: Jellywibble/gpt2_xl_pairwise_89m_step_347634

model_num_parameters: 8030261248.0

best_of: 16

max_input_tokens: 512

max_output_tokens: 64

display_name: kaoeiri-hedoneo-l3-1-8b-v1-2_v1

is_internal_developer: False

language_model: Kaoeiri/Hedoneo-L3.1-8B-v1.2

model_size: 8B

ranking_group: single

us_pacific_date: 2024-08-06

win_ratio: 0.4738183671983104

generation_params: {'temperature': 1.0, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|eot_id|>'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64, 'reward_max_token_input': 256}

formatter: {'memory_template': "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n{bot_name}'s Persona: {memory}\n\n", 'prompt_template': '{prompt}<|eot_id|>', 'bot_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}: {message}<|eot_id|>', 'user_template': '<|start_header_id|>user<|end_header_id|>\n\n{user_name}: {message}<|eot_id|>', 'response_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}:', 'truncate_by_message': False}

reward_formatter: {'bot_template': '{bot_name}: {message}\n', 'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}\n'}

Resubmit model

Running pipeline stage MKMLizer
Starting job with name kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer
Waiting for job on kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer to finish
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Failed to get response for submission turboderp-cat-llama-3-7_8684_v21: ('http://turboderp-cat-llama-3-7-8684-v21-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: ║     _____            __           __                                ║
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: ║       /___/                                                         ║
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: ║                                                                     ║
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: ║  Version: 0.9.9                                                     ║
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: ║  https://mk1.ai                                                     ║
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: ║                                                                     ║
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: ║  belonging to:                                                      ║
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: ║                                                                     ║
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: ║  Chai Research Corp.                                                ║
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: ║                                                                     ║
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Failed to get response for submission turboderp-cat-llama-3-7_8684_v21: ('http://turboderp-cat-llama-3-7-8684-v21-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: Downloaded to shared memory in 26.742s
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpu4okyxes, device:0
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Connection pool is full, discarding connection: %s. Connection pool size: %s
Failed to get response for submission turboderp-cat-llama-3-7_8684_v21: ('http://turboderp-cat-llama-3-7-8684-v21-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'activator request timeout')
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: quantized model in 26.528s
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: Processed model Kaoeiri/Hedoneo-L3.1-8B-v1.2 in 53.271s
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: creating bucket guanaco-mkml-models
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/kaoeiri-hedoneo-l3-1-8b-v1-2-v1
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/kaoeiri-hedoneo-l3-1-8b-v1-2-v1/config.json
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/kaoeiri-hedoneo-l3-1-8b-v1-2-v1/special_tokens_map.json
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/kaoeiri-hedoneo-l3-1-8b-v1-2-v1/tokenizer_config.json
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/kaoeiri-hedoneo-l3-1-8b-v1-2-v1/tokenizer.json
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/kaoeiri-hedoneo-l3-1-8b-v1-2-v1/flywheel_model.0.safetensors
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: loading reward model from Jellywibble/gpt2_xl_pairwise_89m_step_347634
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   1%|          | 2/291 [00:04<11:43,  2.43s/it]
Loading 0:   2%|▏         | 6/291 [00:05<03:07,  1.52it/s]
Loading 0:   4%|▍         | 13/291 [00:05<01:06,  4.16it/s]
Loading 0:   7%|▋         | 19/291 [00:05<00:38,  7.07it/s]
Loading 0:   8%|▊         | 24/291 [00:05<00:27,  9.63it/s]
Loading 0:  11%|█         | 32/291 [00:05<00:16, 15.70it/s]
Loading 0:  13%|█▎        | 38/291 [00:05<00:13, 19.37it/s]
Loading 0:  15%|█▍        | 43/291 [00:05<00:10, 22.96it/s]
Loading 0:  17%|█▋        | 50/291 [00:05<00:08, 29.88it/s]
Loading 0:  19%|█▉        | 56/291 [00:06<00:07, 31.97it/s]
Loading 0:  21%|██        | 61/291 [00:06<00:06, 33.64it/s]
Loading 0:  23%|██▎       | 68/291 [00:06<00:05, 40.10it/s]
Loading 0:  25%|██▌       | 74/291 [00:06<00:05, 40.80it/s]
Loading 0:  27%|██▋       | 79/291 [00:06<00:05, 41.89it/s]
Loading 0:  30%|██▉       | 86/291 [00:06<00:04, 47.92it/s]
Loading 0:  32%|███▏      | 92/291 [00:06<00:04, 46.48it/s]
Loading 0:  34%|███▎      | 98/291 [00:06<00:04, 47.89it/s]
Loading 0:  36%|███▌      | 104/291 [00:06<00:03, 48.61it/s]
Loading 0:  38%|███▊      | 110/291 [00:07<00:03, 45.70it/s]
Loading 0:  40%|███▉      | 115/291 [00:07<00:03, 45.08it/s]
Loading 0:  42%|████▏     | 122/291 [00:07<00:06, 28.14it/s]
Loading 0:  44%|████▍     | 128/291 [00:07<00:05, 30.86it/s]
Loading 0:  45%|████▌     | 132/291 [00:07<00:05, 31.49it/s]
Loading 0:  48%|████▊     | 140/291 [00:08<00:03, 40.49it/s]
Loading 0:  50%|█████     | 146/291 [00:08<00:03, 39.65it/s]
Loading 0:  52%|█████▏    | 151/291 [00:08<00:03, 40.26it/s]
Loading 0:  54%|█████▍    | 157/291 [00:08<00:03, 43.92it/s]
Loading 0:  56%|█████▌    | 162/291 [00:08<00:02, 43.95it/s]
Loading 0:  57%|█████▋    | 167/291 [00:08<00:02, 45.18it/s]
Loading 0:  59%|█████▉    | 173/291 [00:08<00:02, 43.62it/s]
Loading 0:  61%|██████    | 178/291 [00:08<00:02, 43.12it/s]
Loading 0:  64%|██████▎   | 185/291 [00:09<00:02, 48.66it/s]
Loading 0:  66%|██████▌   | 191/291 [00:09<00:02, 47.02it/s]
Loading 0:  67%|██████▋   | 196/291 [00:09<00:02, 46.16it/s]
Loading 0:  69%|██████▉   | 202/291 [00:09<00:01, 49.59it/s]
Loading 0:  71%|███████▏  | 208/291 [00:09<00:01, 50.54it/s]
Loading 0:  74%|███████▎  | 214/291 [00:09<00:01, 45.50it/s]
Loading 0:  76%|███████▌  | 221/291 [00:09<00:01, 51.16it/s]
Loading 0:  78%|███████▊  | 227/291 [00:09<00:01, 48.51it/s]
Loading 0:  80%|████████  | 233/291 [00:10<00:01, 49.08it/s]
Loading 0:  82%|████████▏ | 240/291 [00:10<00:01, 46.54it/s]
Loading 0:  85%|████████▌ | 248/291 [00:10<00:00, 53.76it/s]
Loading 0:  87%|████████▋ | 254/291 [00:10<00:00, 50.46it/s]
Loading 0:  89%|████████▉ | 260/291 [00:10<00:00, 48.53it/s]
Loading 0:  91%|█████████▏| 266/291 [00:10<00:00, 49.98it/s]
Loading 0:  93%|█████████▎| 272/291 [00:10<00:00, 46.73it/s]
Loading 0:  95%|█████████▌| 277/291 [00:10<00:00, 44.96it/s]
Loading 0:  98%|█████████▊| 284/291 [00:11<00:00, 50.60it/s]
Loading 0: 100%|█████████▉| 290/291 [00:11<00:00, 27.36it/s]
                                                            
/opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:957: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer:   warnings.warn(
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer:   warnings.warn(
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: creating bucket guanaco-reward-models
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: Bucket 's3://guanaco-reward-models/' created
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/kaoeiri-hedoneo-l3-1-8b-v1-2-v1_reward
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/kaoeiri-hedoneo-l3-1-8b-v1-2-v1_reward/config.json
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/kaoeiri-hedoneo-l3-1-8b-v1-2-v1_reward/special_tokens_map.json
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/kaoeiri-hedoneo-l3-1-8b-v1-2-v1_reward/tokenizer_config.json
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/kaoeiri-hedoneo-l3-1-8b-v1-2-v1_reward/merges.txt
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/kaoeiri-hedoneo-l3-1-8b-v1-2-v1_reward/vocab.json
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/kaoeiri-hedoneo-l3-1-8b-v1-2-v1_reward/tokenizer.json
kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/kaoeiri-hedoneo-l3-1-8b-v1-2-v1_reward/reward.tensors
Job kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer completed after 95.02s with status: succeeded
Stopping job with name kaoeiri-hedoneo-l3-1-8b-v1-2-v1-mkmlizer
Pipeline stage MKMLizer completed in 96.31s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.09s
Running pipeline stage ISVCDeployer
Creating inference service kaoeiri-hedoneo-l3-1-8b-v1-2-v1
Waiting for inference service kaoeiri-hedoneo-l3-1-8b-v1-2-v1 to be ready
Connection pool is full, discarding connection: %s. Connection pool size: %s
Inference service kaoeiri-hedoneo-l3-1-8b-v1-2-v1 ready after 191.4844310283661s
Pipeline stage ISVCDeployer completed in 193.84s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.1845879554748535s
Received healthy response to inference request in 1.4679639339447021s
Received healthy response to inference request in 1.4321184158325195s
Received healthy response to inference request in 1.3392257690429688s
Received healthy response to inference request in 1.43630051612854s
5 requests
0 failed requests
5th percentile: 1.357804298400879
10th percentile: 1.376382827758789
20th percentile: 1.4135398864746094
30th percentile: 1.4329548358917237
40th percentile: 1.4346276760101317
50th percentile: 1.43630051612854
60th percentile: 1.448965883255005
70th percentile: 1.4616312503814697
80th percentile: 1.6112887382507326
90th percentile: 1.897938346862793
95th percentile: 2.041263151168823
99th percentile: 2.1559229946136473
mean time: 1.5720393180847168
Pipeline stage StressChecker completed in 8.60s
kaoeiri-hedoneo-l3-1-8b-v1-2_v1 status is now deployed due to DeploymentManager action
kaoeiri-hedoneo-l3-1-8b-v1-2_v1 status is now inactive due to auto deactivation removed underperforming models
kaoeiri-hedoneo-l3-1-8b-v1-2_v1 status is now torndown due to DeploymentManager action