nothingiisreal-l3-1-8b-d_6951

submission_id: nothingiisreal-l3-1-8b-d_6951_v1

developer_uid: aaronday3

best_of: 16

celo_rating: 1183.88

display_name: nothingiisreal-l3-8b-celeste-V1

family_friendly_score: 0.0

formatter: {'memory_template': '<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n', 'prompt_template': '{prompt}<|eot_id|>', 'bot_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}: {message}<|eot_id|>', 'user_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{user_name}: {message}<|eot_id|>', 'response_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}:', 'truncate_by_message': False}

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.1, 'top_k': 60, 'presence_penalty': 0.15, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|eot_id|>', '<|end_header_id|>'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64, 'reward_max_token_input': 256}

is_internal_developer: False

language_model: nothingiisreal/L3.1-8B-double-lr-3epochs-2ndepoch

max_input_tokens: 512

max_output_tokens: 64

model_architecture: LlamaForCausalLM

model_group: nothingiisreal/L3.1-8B-d

model_name: nothingiisreal-l3-8b-celeste-V1

model_num_parameters: 8030261248.0

model_repo: nothingiisreal/L3.1-8B-double-lr-3epochs-2ndepoch

model_size: 8B

num_battles: 15009

num_wins: 7156

ranking_group: single

reward_formatter: {'bot_template': '{bot_name}: {message}\\n', 'memory_template': "{bot_name}'s Persona: {memory}\\n####\\n", 'prompt_template': '{prompt}\\n<START>\\n', 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}\\n'}

reward_repo: ChaiML/reward_gpt2_medium_preference_24m_e2

status: torndown

submission_type: basic

timestamp: 2024-07-26T21:51:06+00:00

us_pacific_date: 2024-07-26

win_ratio: 0.47678059830768205

Download Preference Data

Resubmit model

Running pipeline stage MKMLizer
Starting job with name nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer
Waiting for job on nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer to finish
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: ║     _____            __           __                                ║
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: ║       /___/                                                         ║
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: ║                                                                     ║
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: ║  Version: 0.9.7                                                     ║
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: ║  https://mk1.ai                                                     ║
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: ║                                                                     ║
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: ║  belonging to:                                                      ║
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: ║                                                                     ║
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: ║  Chai Research Corp.                                                ║
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: ║                                                                     ║
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: Downloaded to shared memory in 32.170s
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpghhirjsf, device:0
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: quantized model in 26.260s
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: Processed model nothingiisreal/L3.1-8B-double-lr-3epochs-2ndepoch in 58.430s
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: creating bucket guanaco-mkml-models
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/nothingiisreal-l3-1-8b-d-6951-v1
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/nothingiisreal-l3-1-8b-d-6951-v1/config.json
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/nothingiisreal-l3-1-8b-d-6951-v1/special_tokens_map.json
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/nothingiisreal-l3-1-8b-d-6951-v1/tokenizer_config.json
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/nothingiisreal-l3-1-8b-d-6951-v1/tokenizer.json
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/nothingiisreal-l3-1-8b-d-6951-v1/flywheel_model.0.safetensors
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: loading reward model from ChaiML/reward_gpt2_medium_preference_24m_e2
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 5/291 [00:00<00:08, 35.59it/s]
Loading 0:   5%|▍         | 14/291 [00:00<00:05, 46.94it/s]
Loading 0:   8%|▊         | 22/291 [00:00<00:04, 57.63it/s]
Loading 0:  10%|▉         | 29/291 [00:00<00:04, 55.13it/s]
Loading 0:  12%|█▏        | 35/291 [00:00<00:04, 54.18it/s]
Loading 0:  14%|█▍        | 41/291 [00:00<00:05, 47.80it/s]
Loading 0:  17%|█▋        | 49/291 [00:00<00:04, 55.85it/s]
Loading 0:  19%|█▉        | 55/291 [00:01<00:04, 51.96it/s]
Loading 0:  21%|██        | 61/291 [00:01<00:04, 52.27it/s]
Loading 0:  23%|██▎       | 67/291 [00:01<00:04, 54.15it/s]
Loading 0:  25%|██▌       | 73/291 [00:01<00:04, 50.25it/s]
Loading 0:  27%|██▋       | 79/291 [00:01<00:04, 51.24it/s]
Loading 0:  29%|██▉       | 85/291 [00:01<00:05, 36.05it/s]
Loading 0:  31%|███▏      | 91/291 [00:01<00:05, 37.56it/s]
Loading 0:  33%|███▎      | 96/291 [00:02<00:04, 39.52it/s]
Loading 0:  35%|███▌      | 103/291 [00:02<00:04, 45.37it/s]
Loading 0:  37%|███▋      | 109/291 [00:02<00:04, 45.39it/s]
Loading 0:  39%|███▉      | 114/291 [00:02<00:03, 44.74it/s]
Loading 0:  42%|████▏     | 121/291 [00:02<00:03, 50.39it/s]
Loading 0:  44%|████▎     | 127/291 [00:02<00:03, 46.16it/s]
Loading 0:  45%|████▌     | 132/291 [00:02<00:03, 44.68it/s]
Loading 0:  48%|████▊     | 139/291 [00:02<00:03, 49.34it/s]
Loading 0:  50%|████▉     | 145/291 [00:03<00:03, 46.99it/s]
Loading 0:  52%|█████▏    | 150/291 [00:03<00:03, 46.62it/s]
Loading 0:  54%|█████▍    | 157/291 [00:03<00:02, 52.44it/s]
Loading 0:  56%|█████▌    | 163/291 [00:03<00:02, 49.42it/s]
Loading 0:  58%|█████▊    | 169/291 [00:03<00:02, 49.75it/s]
Loading 0:  61%|██████    | 177/291 [00:03<00:02, 50.57it/s]
Loading 0:  63%|██████▎   | 183/291 [00:03<00:02, 50.18it/s]
Loading 0:  65%|██████▍   | 189/291 [00:04<00:03, 32.85it/s]
Loading 0:  67%|██████▋   | 194/291 [00:04<00:02, 35.24it/s]
Loading 0:  69%|██████▉   | 202/291 [00:04<00:02, 43.60it/s]
Loading 0:  71%|███████▏  | 208/291 [00:04<00:01, 42.18it/s]
Loading 0:  73%|███████▎  | 213/291 [00:04<00:01, 42.54it/s]
Loading 0:  76%|███████▌  | 220/291 [00:04<00:01, 48.19it/s]
Loading 0:  78%|███████▊  | 226/291 [00:04<00:01, 46.45it/s]
Loading 0:  79%|███████▉  | 231/291 [00:04<00:01, 45.60it/s]
Loading 0:  82%|████████▏ | 238/291 [00:05<00:01, 50.49it/s]
Loading 0:  84%|████████▍ | 244/291 [00:05<00:00, 47.79it/s]
Loading 0:  86%|████████▌ | 249/291 [00:05<00:00, 46.67it/s]
Loading 0:  88%|████████▊ | 256/291 [00:05<00:00, 51.80it/s]
Loading 0:  90%|█████████ | 262/291 [00:05<00:00, 49.02it/s]
Loading 0:  92%|█████████▏| 268/291 [00:05<00:00, 49.34it/s]
Loading 0:  94%|█████████▍| 274/291 [00:05<00:00, 51.05it/s]
Loading 0:  96%|█████████▌| 280/291 [00:05<00:00, 48.73it/s]
Loading 0:  98%|█████████▊| 285/291 [00:06<00:00, 47.98it/s]
Loading 0: 100%|█████████▉| 290/291 [00:11<00:00,  3.21it/s]
                                                            
/opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:957: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer:   warnings.warn(
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:785: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer:   warnings.warn(
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer:   warnings.warn(
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: Saving duration: 0.329s
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: Processed model ChaiML/reward_gpt2_medium_preference_24m_e2 in 6.232s
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: creating bucket guanaco-reward-models
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: Bucket 's3://guanaco-reward-models/' created
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/nothingiisreal-l3-1-8b-d-6951-v1_reward
nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/nothingiisreal-l3-1-8b-d-6951-v1_reward/reward.tensors
Job nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer completed after 94.42s with status: succeeded
Stopping job with name nothingiisreal-l3-1-8b-d-6951-v1-mkmlizer
Pipeline stage MKMLizer completed in 95.40s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.10s
Running pipeline stage ISVCDeployer
Creating inference service nothingiisreal-l3-1-8b-d-6951-v1
Waiting for inference service nothingiisreal-l3-1-8b-d-6951-v1 to be ready
Inference service nothingiisreal-l3-1-8b-d-6951-v1 ready after 90.63806676864624s
Pipeline stage ISVCDeployer completed in 92.09s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.0003979206085205s
Received healthy response to inference request in 1.286832332611084s
Received healthy response to inference request in 1.221039056777954s
Received healthy response to inference request in 1.21126127243042s
Received healthy response to inference request in 1.2583937644958496s
5 requests
0 failed requests
5th percentile: 1.2132168292999268
10th percentile: 1.2151723861694337
20th percentile: 1.2190834999084472
30th percentile: 1.2285099983215333
40th percentile: 1.2434518814086915
50th percentile: 1.2583937644958496
60th percentile: 1.2697691917419434
70th percentile: 1.281144618988037
80th percentile: 1.4295454502105713
90th percentile: 1.714971685409546
95th percentile: 1.8576848030090332
99th percentile: 1.971855297088623
mean time: 1.3955848693847657
Pipeline stage StressChecker completed in 7.73s
nothingiisreal-l3-1-8b-d_6951_v1 status is now deployed due to DeploymentManager action
nothingiisreal-l3-1-8b-d_6951_v1 status is now inactive due to auto deactivation removed underperforming models
admin requested tearing down of nothingiisreal-l3-1-8b-d_6951_v1
Running pipeline stage ISVCDeleter
Checking if service nothingiisreal-l3-1-8b-d-6951-v1 is running
Tearing down inference service nothingiisreal-l3-1-8b-d-6951-v1
Service nothingiisreal-l3-1-8b-d-6951-v1 has been torndown
Pipeline stage ISVCDeleter completed in 4.23s
Running pipeline stage MKMLModelDeleter
Cleaning model data from S3
Cleaning model data from model cache
Deleting key nothingiisreal-l3-1-8b-d-6951-v1/config.json from bucket guanaco-mkml-models
Deleting key nothingiisreal-l3-1-8b-d-6951-v1/flywheel_model.0.safetensors from bucket guanaco-mkml-models
Deleting key nothingiisreal-l3-1-8b-d-6951-v1/special_tokens_map.json from bucket guanaco-mkml-models
Deleting key nothingiisreal-l3-1-8b-d-6951-v1/tokenizer.json from bucket guanaco-mkml-models
Deleting key nothingiisreal-l3-1-8b-d-6951-v1/tokenizer_config.json from bucket guanaco-mkml-models
Cleaning model data from model cache
Deleting key nothingiisreal-l3-1-8b-d-6951-v1_reward/config.json from bucket guanaco-reward-models
Deleting key nothingiisreal-l3-1-8b-d-6951-v1_reward/merges.txt from bucket guanaco-reward-models
Deleting key nothingiisreal-l3-1-8b-d-6951-v1_reward/reward.tensors from bucket guanaco-reward-models
Deleting key nothingiisreal-l3-1-8b-d-6951-v1_reward/special_tokens_map.json from bucket guanaco-reward-models
Deleting key nothingiisreal-l3-1-8b-d-6951-v1_reward/tokenizer.json from bucket guanaco-reward-models
Deleting key nothingiisreal-l3-1-8b-d-6951-v1_reward/tokenizer_config.json from bucket guanaco-reward-models
Deleting key nothingiisreal-l3-1-8b-d-6951-v1_reward/vocab.json from bucket guanaco-reward-models
Pipeline stage MKMLModelDeleter completed in 5.68s
nothingiisreal-l3-1-8b-d_6951_v1 status is now torndown due to DeploymentManager action