submission_id: nothingiisreal-celeste-v1-9-1_v1
developer_uid: aaronday3
alignment_samples: 0
best_of: 8
celo_rating: 1211.15
display_name: Celeste-v_1_9_1
formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': True}
generation_params: {'temperature': 0.8, 'top_p': 1.0, 'min_p': 0.15, 'top_k': 50, 'presence_penalty': 0.15, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '\n\n', '\nYou:', '</s>', '<|', '<|eot_id|>'], 'max_input_tokens': 512, 'best_of': 8, 'max_output_tokens': 64, 'reward_max_token_input': 512}
is_internal_developer: False
language_model: nothingiisreal/Celeste-V1.9.1
max_input_tokens: 512
max_output_tokens: 64
model_architecture: MistralForCausalLM
model_group: nothingiisreal/Celeste-V
model_name: Celeste-v_1_9_1
model_num_parameters: 12772070400.0
model_repo: nothingiisreal/Celeste-V1.9.1
model_size: 13B
num_battles: 10734
num_wins: 5303
propriety_score: 0.6912751677852349
propriety_total_count: 894.0
ranking_group: single
reward_formatter: {'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': True, 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n'}
reward_repo: ChaiML/gpt2_xl_pairwise_89m_step_347634
status: torndown
submission_type: basic
timestamp: 2024-08-06T04:02:56+00:00
us_pacific_date: 2024-08-05
win_ratio: 0.4940376374138252
Download Preference Data
Resubmit model
Running pipeline stage MKMLizer
Starting job with name nothingiisreal-celeste-v1-9-1-v1-mkmlizer
Waiting for job on nothingiisreal-celeste-v1-9-1-v1-mkmlizer to finish
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: ║ _____ __ __ ║
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: ║ /___/ ║
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: ║ ║
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: ║ Version: 0.9.9 ║
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: ║ https://mk1.ai ║
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: ║ ║
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: ║ The license key for the current software has been verified as ║
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: ║ belonging to: ║
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: ║ ║
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: ║ Chai Research Corp. ║
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: ║ ║
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: Downloaded to shared memory in 47.583s
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpfg40k1_7, device:0
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: quantized model in 36.197s
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: Processed model nothingiisreal/Celeste-V1.9.1 in 83.781s
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: creating bucket guanaco-mkml-models
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/nothingiisreal-celeste-v1-9-1-v1
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/nothingiisreal-celeste-v1-9-1-v1/config.json
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/nothingiisreal-celeste-v1-9-1-v1/special_tokens_map.json
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/nothingiisreal-celeste-v1-9-1-v1/tokenizer_config.json
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: cp /dev/shm/model_cache/vocab.json s3://guanaco-mkml-models/nothingiisreal-celeste-v1-9-1-v1/vocab.json
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: cp /dev/shm/model_cache/merges.txt s3://guanaco-mkml-models/nothingiisreal-celeste-v1-9-1-v1/merges.txt
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/nothingiisreal-celeste-v1-9-1-v1/tokenizer.json
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/nothingiisreal-celeste-v1-9-1-v1/flywheel_model.0.safetensors
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: loading reward model from ChaiML/gpt2_xl_pairwise_89m_step_347634
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: Loading 0: 0%| | 0/363 [00:00<?, ?it/s] Loading 0: 1%|▏ | 5/363 [00:00<00:10, 33.23it/s] Loading 0: 4%|▎ | 13/363 [00:00<00:06, 51.43it/s] Loading 0: 5%|▌ | 19/363 [00:00<00:07, 44.71it/s] Loading 0: 7%|▋ | 24/363 [00:00<00:07, 43.17it/s] Loading 0: 9%|▊ | 31/363 [00:00<00:06, 48.79it/s] Loading 0: 10%|█ | 37/363 [00:00<00:07, 44.63it/s] Loading 0: 12%|█▏ | 42/363 [00:00<00:07, 43.77it/s] Loading 0: 13%|█▎ | 49/363 [00:01<00:06, 48.80it/s] Loading 0: 15%|█▌ | 55/363 [00:01<00:06, 44.82it/s] Loading 0: 17%|█▋ | 61/363 [00:01<00:08, 33.60it/s] Loading 0: 18%|█▊ | 65/363 [00:01<00:09, 32.86it/s] Loading 0: 20%|█▉ | 72/363 [00:01<00:07, 39.41it/s] Loading 0: 21%|██ | 77/363 [00:01<00:06, 41.77it/s] Loading 0: 23%|██▎ | 82/363 [00:02<00:07, 36.13it/s] Loading 0: 25%|██▍ | 89/363 [00:02<00:06, 43.31it/s] Loading 0: 26%|██▌ | 94/363 [00:02<00:06, 42.18it/s] Loading 0: 27%|██▋ | 99/363 [00:02<00:06, 42.48it/s] Loading 0: 29%|██▊ | 104/363 [00:02<00:05, 43.91it/s] Loading 0: 30%|███ | 110/363 [00:02<00:06, 41.45it/s] Loading 0: 32%|███▏ | 115/363 [00:02<00:05, 41.91it/s] Loading 0: 33%|███▎ | 120/363 [00:02<00:06, 40.02it/s] Loading 0: 35%|███▍ | 126/363 [00:03<00:05, 42.85it/s] Loading 0: 36%|███▌ | 131/363 [00:03<00:05, 44.06it/s] Loading 0: 37%|███▋ | 136/363 [00:03<00:06, 37.16it/s] Loading 0: 39%|███▉ | 142/363 [00:03<00:06, 31.80it/s] Loading 0: 40%|████ | 146/363 [00:03<00:06, 33.40it/s] Loading 0: 41%|████▏ | 150/363 [00:03<00:06, 32.83it/s] Loading 0: 43%|████▎ | 156/363 [00:03<00:05, 38.24it/s] Loading 0: 44%|████▍ | 161/363 [00:04<00:05, 39.26it/s] Loading 0: 46%|████▌ | 166/363 [00:04<00:04, 40.29it/s] Loading 0: 47%|████▋ | 172/363 [00:04<00:04, 39.54it/s] Loading 0: 49%|████▉ | 177/363 [00:04<00:04, 39.02it/s] Loading 0: 50%|█████ | 183/363 [00:04<00:04, 43.71it/s] Loading 0: 52%|█████▏ | 188/363 [00:04<00:04, 43.56it/s] Loading 0: 53%|█████▎ | 193/363 [00:04<00:03, 43.78it/s] Loading 0: 55%|█████▍ | 199/363 [00:04<00:03, 41.93it/s] Loading 0: 56%|█████▌ | 204/363 [00:05<00:03, 41.17it/s] Loading 0: 58%|█████▊ | 210/363 [00:05<00:03, 45.68it/s] Loading 0: 59%|█████▉ | 215/363 [00:05<00:03, 45.72it/s] Loading 0: 61%|██████ | 221/363 [00:05<00:02, 48.93it/s] Loading 0: 62%|██████▏ | 226/363 [00:05<00:04, 30.53it/s] Loading 0: 63%|██████▎ | 230/363 [00:05<00:04, 30.49it/s] Loading 0: 65%|██████▌ | 237/363 [00:05<00:03, 37.53it/s] Loading 0: 67%|██████▋ | 242/363 [00:06<00:03, 38.83it/s] Loading 0: 68%|██████▊ | 247/363 [00:06<00:02, 40.46it/s] Loading 0: 70%|██████▉ | 253/363 [00:06<00:02, 39.76it/s] Loading 0: 71%|███████ | 258/363 [00:06<00:02, 39.41it/s] Loading 0: 73%|███████▎ | 264/363 [00:06<00:02, 43.89it/s] Loading 0: 74%|███████▍ | 269/363 [00:06<00:02, 43.67it/s] Loading 0: 75%|███████▌ | 274/363 [00:06<00:02, 42.77it/s] Loading 0: 77%|███████▋ | 279/363 [00:06<00:01, 44.18it/s] Loading 0: 78%|███████▊ | 284/363 [00:07<00:02, 37.19it/s] Loading 0: 80%|████████ | 291/363 [00:07<00:01, 44.44it/s] Loading 0: 82%|████████▏ | 296/363 [00:07<00:01, 44.64it/s] Loading 0: 83%|████████▎ | 302/363 [00:07<00:01, 48.60it/s] Loading 0: 85%|████████▍ | 308/363 [00:14<00:20, 2.68it/s] Loading 0: 86%|████████▌ | 312/363 [00:14<00:15, 3.40it/s] Loading 0: 88%|████████▊ | 320/363 [00:14<00:07, 5.46it/s] Loading 0: 90%|████████▉ | 326/363 [00:14<00:05, 7.33it/s] Loading 0: 91%|█████████ | 331/363 [00:14<00:03, 9.35it/s] Loading 0: 93%|█████████▎| 337/363 [00:14<00:02, 12.65it/s] Loading 0: 94%|█████████▍| 342/363 [00:14<00:01, 15.72it/s] Loading 0: 96%|█████████▌| 347/363 [00:15<00:00, 19.31it/s] Loading 0: 97%|█████████▋| 353/363 [00:15<00:00, 23.16it/s] Loading 0: 99%|█████████▊| 358/363 [00:15<00:00, 26.41it/s] /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:957: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: warnings.warn(
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:785: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: warnings.warn(
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: warnings.warn(
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: Downloading shards: 0%| | 0/2 [00:00<?, ?it/s] Downloading shards: 50%|█████ | 1/2 [00:05<00:05, 5.34s/it] Downloading shards: 100%|██████████| 2/2 [00:08<00:00, 3.84s/it] Downloading shards: 100%|██████████| 2/2 [00:08<00:00, 4.07s/it]
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s] Loading checkpoint shards: 50%|█████ | 1/2 [00:00<00:00, 2.20it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 3.64it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 3.31it/s]
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: Saving duration: 1.388s
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: Processed model ChaiML/gpt2_xl_pairwise_89m_step_347634 in 13.158s
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: creating bucket guanaco-reward-models
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: Bucket 's3://guanaco-reward-models/' created
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/nothingiisreal-celeste-v1-9-1-v1_reward
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/nothingiisreal-celeste-v1-9-1-v1_reward/config.json
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/nothingiisreal-celeste-v1-9-1-v1_reward/special_tokens_map.json
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/nothingiisreal-celeste-v1-9-1-v1_reward/tokenizer_config.json
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/nothingiisreal-celeste-v1-9-1-v1_reward/merges.txt
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/nothingiisreal-celeste-v1-9-1-v1_reward/vocab.json
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/nothingiisreal-celeste-v1-9-1-v1_reward/tokenizer.json
nothingiisreal-celeste-v1-9-1-v1-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/nothingiisreal-celeste-v1-9-1-v1_reward/reward.tensors
Job nothingiisreal-celeste-v1-9-1-v1-mkmlizer completed after 135.77s with status: succeeded
Stopping job with name nothingiisreal-celeste-v1-9-1-v1-mkmlizer
Pipeline stage MKMLizer completed in 136.79s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.11s
Running pipeline stage ISVCDeployer
Creating inference service nothingiisreal-celeste-v1-9-1-v1
Waiting for inference service nothingiisreal-celeste-v1-9-1-v1 to be ready
Failed to get response for submission cycy233-l3-s-v0-c3_v1: ('http://cycy233-l3-s-v0-c3-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '{"error":"ValueError : [TypeError(\\"\'numpy.int64\' object is not iterable\\"), TypeError(\'vars() argument must have __dict__ attribute\')]"}')
Inference service nothingiisreal-celeste-v1-9-1-v1 ready after 171.06999969482422s
Pipeline stage ISVCDeployer completed in 173.19s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.5555472373962402s
Received healthy response to inference request in 1.745253562927246s
Received healthy response to inference request in 1.7301828861236572s
Received healthy response to inference request in 1.8094627857208252s
Received healthy response to inference request in 1.6887903213500977s
5 requests
0 failed requests
5th percentile: 1.6970688343048095
10th percentile: 1.7053473472595215
20th percentile: 1.7219043731689454
30th percentile: 1.733197021484375
40th percentile: 1.7392252922058105
50th percentile: 1.745253562927246
60th percentile: 1.7709372520446778
70th percentile: 1.7966209411621095
80th percentile: 1.9586796760559084
90th percentile: 2.257113456726074
95th percentile: 2.4063303470611572
99th percentile: 2.525703859329224
mean time: 1.9058473587036133
Pipeline stage StressChecker completed in 10.29s
nothingiisreal-celeste-v1-9-1_v1 status is now deployed due to DeploymentManager action
nothingiisreal-celeste-v1-9-1_v1 status is now inactive due to auto deactivation removed underperforming models
nothingiisreal-celeste-v1-9-1_v1 status is now torndown due to DeploymentManager action

Usage Metrics

Latency Metrics