submission_id: google-gemma-2-27b-it_v6
developer_uid: Jellywibble
alignment_samples: 0
best_of: 4
celo_rating: 1157.24
display_name: google-gemma-2-27b-it_v1
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 50, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64}
is_internal_developer: True
language_model: google/gemma-2-27b-it
max_input_tokens: 512
max_output_tokens: 64
model_architecture: Gemma2ForCausalLM
model_group: google/gemma-2-27b-it
model_name: google-gemma-2-27b-it_v1
model_num_parameters: 28731935232.0
model_repo: google/gemma-2-27b-it
model_size: 29B
num_battles: 112259
num_wins: 49555
propriety_score: 0.7413206964046565
propriety_total_count: 9707.0
ranking_group: single
reward_formatter: {'bot_template': '{bot_name}: {message}\n', 'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}\n'}
reward_repo: ChaiML/gpt2_xl_pairwise_89m_step_347634
status: torndown
submission_type: basic
timestamp: 2024-07-14T00:49:14+00:00
us_pacific_date: 2024-07-13
win_ratio: 0.44143453976963987
Resubmit model
Running pipeline stage MKMLizer
Starting job with name google-gemma-2-27b-it-v6-mkmlizer
Waiting for job on google-gemma-2-27b-it-v6-mkmlizer to finish
google-gemma-2-27b-it-v6-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
google-gemma-2-27b-it-v6-mkmlizer: ║ _____ __ __ ║
google-gemma-2-27b-it-v6-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
google-gemma-2-27b-it-v6-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
google-gemma-2-27b-it-v6-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
google-gemma-2-27b-it-v6-mkmlizer: ║ /___/ ║
google-gemma-2-27b-it-v6-mkmlizer: ║ ║
google-gemma-2-27b-it-v6-mkmlizer: ║ Version: 0.9.5.post2 ║
google-gemma-2-27b-it-v6-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
google-gemma-2-27b-it-v6-mkmlizer: ║ https://mk1.ai ║
google-gemma-2-27b-it-v6-mkmlizer: ║ ║
google-gemma-2-27b-it-v6-mkmlizer: ║ The license key for the current software has been verified as ║
google-gemma-2-27b-it-v6-mkmlizer: ║ belonging to: ║
google-gemma-2-27b-it-v6-mkmlizer: ║ ║
google-gemma-2-27b-it-v6-mkmlizer: ║ Chai Research Corp. ║
google-gemma-2-27b-it-v6-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
google-gemma-2-27b-it-v6-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
google-gemma-2-27b-it-v6-mkmlizer: ║ ║
google-gemma-2-27b-it-v6-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
google-gemma-2-27b-it-v6-mkmlizer: Downloaded to shared memory in 79.228s
google-gemma-2-27b-it-v6-mkmlizer: quantizing model to /dev/shm/model_cache
google-gemma-2-27b-it-v6-mkmlizer: Saving flywheel model at /dev/shm/model_cache
google-gemma-2-27b-it-v6-mkmlizer: quantized model in 84.779s
google-gemma-2-27b-it-v6-mkmlizer: Processed model google/gemma-2-27b-it in 164.007s
google-gemma-2-27b-it-v6-mkmlizer: creating bucket guanaco-mkml-models
google-gemma-2-27b-it-v6-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
google-gemma-2-27b-it-v6-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/google-gemma-2-27b-it-v6
google-gemma-2-27b-it-v6-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/google-gemma-2-27b-it-v6/config.json
google-gemma-2-27b-it-v6-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/google-gemma-2-27b-it-v6/tokenizer_config.json
google-gemma-2-27b-it-v6-mkmlizer: cp /dev/shm/model_cache/tokenizer.model s3://guanaco-mkml-models/google-gemma-2-27b-it-v6/tokenizer.model
google-gemma-2-27b-it-v6-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/google-gemma-2-27b-it-v6/special_tokens_map.json
google-gemma-2-27b-it-v6-mkmlizer: cp /dev/shm/model_cache/flywheel_model.2.safetensors s3://guanaco-mkml-models/google-gemma-2-27b-it-v6/flywheel_model.2.safetensors
google-gemma-2-27b-it-v6-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/google-gemma-2-27b-it-v6/flywheel_model.0.safetensors
google-gemma-2-27b-it-v6-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/google-gemma-2-27b-it-v6/flywheel_model.1.safetensors
google-gemma-2-27b-it-v6-mkmlizer: loading reward model from ChaiML/gpt2_xl_pairwise_89m_step_347634
google-gemma-2-27b-it-v6-mkmlizer: Loading 0: 0%| | 0/508 [00:00<?, ?it/s] Loading 0: 1%| | 4/508 [00:00<00:14, 35.99it/s] Loading 0: 3%|▎ | 14/508 [00:00<00:08, 60.46it/s] Loading 0: 4%|▍ | 20/508 [00:00<00:08, 57.10it/s] Loading 0: 6%|▌ | 28/508 [00:00<00:15, 30.54it/s] Loading 0: 6%|▋ | 33/508 [00:00<00:15, 30.65it/s] Loading 0: 7%|▋ | 37/508 [00:01<00:14, 32.17it/s] Loading 0: 9%|▉ | 47/508 [00:01<00:10, 44.27it/s] Loading 0: 10%|█ | 53/508 [00:01<00:09, 46.43it/s] Loading 0: 12%|█▏ | 59/508 [00:01<00:09, 47.38it/s] Loading 0: 13%|█▎ | 68/508 [00:01<00:08, 54.34it/s] Loading 0: 15%|█▍ | 74/508 [00:02<00:15, 27.82it/s] Loading 0: 16%|█▌ | 81/508 [00:02<00:12, 32.94it/s] Loading 0: 17%|█▋ | 87/508 [00:02<00:11, 37.16it/s] Loading 0: 19%|█▉ | 97/508 [00:02<00:08, 46.93it/s] Loading 0: 20%|██ | 103/508 [00:02<00:08, 48.27it/s] Loading 0: 21%|██▏ | 109/508 [00:02<00:08, 47.00it/s] Loading 0: 23%|██▎ | 118/508 [00:03<00:12, 31.66it/s] Loading 0: 24%|██▍ | 124/508 [00:03<00:10, 35.40it/s] Loading 0: 25%|██▌ | 129/508 [00:03<00:10, 37.27it/s] Loading 0: 27%|██▋ | 136/508 [00:03<00:08, 41.51it/s] Loading 0: 29%|██▊ | 146/508 [00:03<00:07, 50.45it/s] Loading 0: 30%|██▉ | 152/508 [00:03<00:06, 51.39it/s] Loading 0: 31%|███ | 158/508 [00:03<00:06, 51.14it/s] Loading 0: 34%|███▎ | 171/508 [00:04<00:09, 37.40it/s] Loading 0: 35%|███▍ | 176/508 [00:04<00:09, 36.03it/s] Loading 0: 36%|███▌ | 181/508 [00:04<00:09, 34.59it/s] Loading 0: 38%|███▊ | 191/508 [00:04<00:07, 44.21it/s] Loading 0: 38%|███▊ | 191/508 [00:25<00:07, 44.21it/s] Loading 0: 38%|███▊ | 192/508 [00:25<05:44, 1.09s/it] Loading 0: 40%|███▉ | 202/508 [00:25<03:12, 1.59it/s] Loading 0: 42%|████▏ | 211/508 [00:25<02:01, 2.44it/s] Loading 0: 43%|████▎ | 218/508 [00:25<01:31, 3.16it/s] Loading 0: 44%|████▍ | 224/508 [00:26<01:07, 4.18it/s] Loading 0: 46%|████▌ | 234/508 [00:26<00:42, 6.52it/s] Loading 0: 47%|████▋ | 240/508 [00:26<00:32, 8.31it/s] Loading 0: 48%|████▊ | 246/508 [00:26<00:24, 10.62it/s] Loading 0: 50%|█████ | 255/508 [00:26<00:16, 15.27it/s] Loading 0: 51%|█████▏ | 261/508 [00:26<00:16, 14.78it/s] Loading 0: 53%|█████▎ | 267/508 [00:27<00:13, 18.42it/s] Loading 0: 54%|█████▎ | 272/508 [00:27<00:10, 21.46it/s] Loading 0: 55%|█████▍ | 279/508 [00:27<00:08, 26.90it/s] Loading 0: 57%|█████▋ | 289/508 [00:27<00:05, 36.55it/s] Loading 0: 58%|█████▊ | 295/508 [00:27<00:05, 39.79it/s] Loading 0: 59%|█████▉ | 301/508 [00:27<00:04, 42.12it/s] Loading 0: 62%|██████▏ | 314/508 [00:28<00:05, 34.66it/s] Loading 0: 63%|██████▎ | 319/508 [00:28<00:05, 33.70it/s] Loading 0: 64%|██████▍ | 324/508 [00:28<00:05, 33.26it/s] Loading 0: 66%|██████▌ | 334/508 [00:28<00:04, 42.89it/s] Loading 0: 68%|██████▊ | 344/508 [00:28<00:03, 51.22it/s] Loading 0: 69%|██████▉ | 350/508 [00:28<00:03, 51.70it/s] Loading 0: 71%|███████ | 359/508 [00:29<00:04, 33.75it/s] Loading 0: 72%|███████▏ | 364/508 [00:29<00:04, 35.85it/s] Loading 0: 73%|███████▎ | 369/508 [00:29<00:04, 34.61it/s] Loading 0: 74%|███████▍ | 378/508 [00:29<00:03, 42.44it/s] Loading 0: 76%|███████▋ | 388/508 [00:29<00:02, 51.15it/s] Loading 0: 78%|███████▊ | 394/508 [00:29<00:02, 51.70it/s] Loading 0: 79%|███████▊ | 400/508 [00:29<00:02, 50.57it/s] Loading 0: 80%|███████▉ | 406/508 [00:30<00:03, 29.48it/s] Loading 0: 81%|████████ | 411/508 [00:30<00:02, 32.49it/s] Loading 0: 83%|████████▎ | 421/508 [00:30<00:02, 42.75it/s] Loading 0: 84%|████████▍ | 427/508 [00:30<00:01, 45.13it/s] Loading 0: 85%|████████▌ | 433/508 [00:30<00:01, 46.41it/s] Loading 0: 87%|████████▋ | 443/508 [00:31<00:01, 55.28it/s] Loading 0: 89%|████████▊ | 450/508 [00:31<00:01, 56.30it/s] Loading 0: 89%|████████▉ | 451/508 [00:51<00:01, 56.30it/s] Loading 0: 89%|████████▉ | 452/508 [00:51<01:00, 1.08s/it] Loading 0: 90%|████████▉ | 457/508 [00:52<00:41, 1.23it/s] Loading 0: 91%|█████████ | 462/508 [00:52<00:27, 1.68it/s] Loading 0: 92%|█████████▏| 467/508 [00:52<00:17, 2.30it/s] Loading 0: 94%|█████████▍| 477/508 [00:52<00:07, 4.06it/s] Loading 0: 96%|█████████▌| 487/508 [00:52<00:03, 6.45it/s] Loading 0: 97%|█████████▋| 493/508 [00:52<00:01, 8.27it/s] Loading 0: 99%|█████████▉| 502/508 [00:53<00:00, 10.35it/s] Loading 0: 100%|█████████▉| 507/508 [00:53<00:00, 12.44it/s] /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:950: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
google-gemma-2-27b-it-v6-mkmlizer: warnings.warn(
google-gemma-2-27b-it-v6-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:778: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
google-gemma-2-27b-it-v6-mkmlizer: warnings.warn(
google-gemma-2-27b-it-v6-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
google-gemma-2-27b-it-v6-mkmlizer: warnings.warn(
google-gemma-2-27b-it-v6-mkmlizer: Downloading shards: 0%| | 0/2 [00:00<?, ?it/s] Downloading shards: 50%|█████ | 1/2 [00:06<00:06, 6.93s/it] Downloading shards: 100%|██████████| 2/2 [00:09<00:00, 4.15s/it] Downloading shards: 100%|██████████| 2/2 [00:09<00:00, 4.56s/it]
google-gemma-2-27b-it-v6-mkmlizer: Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s] Loading checkpoint shards: 50%|█████ | 1/2 [00:00<00:00, 1.55it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 2.56it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 2.33it/s]
google-gemma-2-27b-it-v6-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
google-gemma-2-27b-it-v6-mkmlizer: Saving duration: 2.281s
google-gemma-2-27b-it-v6-mkmlizer: Processed model ChaiML/gpt2_xl_pairwise_89m_step_347634 in 14.236s
google-gemma-2-27b-it-v6-mkmlizer: creating bucket guanaco-reward-models
google-gemma-2-27b-it-v6-mkmlizer: Bucket 's3://guanaco-reward-models/' created
google-gemma-2-27b-it-v6-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/google-gemma-2-27b-it-v6_reward
google-gemma-2-27b-it-v6-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/google-gemma-2-27b-it-v6_reward/config.json
google-gemma-2-27b-it-v6-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/google-gemma-2-27b-it-v6_reward/special_tokens_map.json
google-gemma-2-27b-it-v6-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/google-gemma-2-27b-it-v6_reward/vocab.json
google-gemma-2-27b-it-v6-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/google-gemma-2-27b-it-v6_reward/tokenizer_config.json
google-gemma-2-27b-it-v6-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/google-gemma-2-27b-it-v6_reward/merges.txt
google-gemma-2-27b-it-v6-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/google-gemma-2-27b-it-v6_reward/tokenizer.json
google-gemma-2-27b-it-v6-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/google-gemma-2-27b-it-v6_reward/reward.tensors
Job google-gemma-2-27b-it-v6-mkmlizer completed after 216.92s with status: succeeded
Stopping job with name google-gemma-2-27b-it-v6-mkmlizer
Pipeline stage MKMLizer completed in 218.00s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.09s
Running pipeline stage ISVCDeployer
Creating inference service google-gemma-2-27b-it-v6
Waiting for inference service google-gemma-2-27b-it-v6 to be ready
Inference service google-gemma-2-27b-it-v6 ready after 60.24562668800354s
Pipeline stage ISVCDeployer completed in 67.38s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.357642889022827s
Received healthy response to inference request in 2.6192264556884766s
Received healthy response to inference request in 1.3009905815124512s
Received healthy response to inference request in 2.2596631050109863s
Received healthy response to inference request in 2.333787679672241s
5 requests
0 failed requests
5th percentile: 1.4927250862121582
10th percentile: 1.6844595909118651
20th percentile: 2.067928600311279
30th percentile: 2.274488019943237
40th percentile: 2.304137849807739
50th percentile: 2.333787679672241
60th percentile: 2.4479631900787355
70th percentile: 2.5621387004852294
80th percentile: 2.766909742355347
90th percentile: 3.062276315689087
95th percentile: 3.2099596023559567
99th percentile: 3.328106231689453
mean time: 2.3742621421813963
Pipeline stage StressChecker completed in 12.62s
google-gemma-2-27b-it_v6 status is now deployed due to DeploymentManager action
google-gemma-2-27b-it_v6 status is now inactive due to auto deactivation removed underperforming models
admin requested tearing down of google-gemma-2-27b-it_v6
Running pipeline stage ISVCDeleter
Checking if service google-gemma-2-27b-it-v6 is running
Skipping teardown as no inference service was found
Pipeline stage ISVCDeleter completed in 4.70s
Running pipeline stage MKMLModelDeleter
Cleaning model data from S3
Cleaning model data from model cache
Deleting key google-gemma-2-27b-it-v6/config.json from bucket guanaco-mkml-models
Deleting key google-gemma-2-27b-it-v6/flywheel_model.0.safetensors from bucket guanaco-mkml-models
Deleting key google-gemma-2-27b-it-v6/flywheel_model.1.safetensors from bucket guanaco-mkml-models
Deleting key google-gemma-2-27b-it-v6/flywheel_model.2.safetensors from bucket guanaco-mkml-models
Deleting key google-gemma-2-27b-it-v6/special_tokens_map.json from bucket guanaco-mkml-models
Deleting key google-gemma-2-27b-it-v6/tokenizer.json from bucket guanaco-mkml-models
Deleting key google-gemma-2-27b-it-v6/tokenizer.model from bucket guanaco-mkml-models
Deleting key google-gemma-2-27b-it-v6/tokenizer_config.json from bucket guanaco-mkml-models
Cleaning model data from model cache
Deleting key google-gemma-2-27b-it-v6_reward/config.json from bucket guanaco-reward-models
Deleting key google-gemma-2-27b-it-v6_reward/merges.txt from bucket guanaco-reward-models
Deleting key google-gemma-2-27b-it-v6_reward/reward.tensors from bucket guanaco-reward-models
Deleting key google-gemma-2-27b-it-v6_reward/special_tokens_map.json from bucket guanaco-reward-models
Deleting key google-gemma-2-27b-it-v6_reward/tokenizer.json from bucket guanaco-reward-models
Deleting key google-gemma-2-27b-it-v6_reward/tokenizer_config.json from bucket guanaco-reward-models
Deleting key google-gemma-2-27b-it-v6_reward/vocab.json from bucket guanaco-reward-models
Pipeline stage MKMLModelDeleter completed in 8.92s
google-gemma-2-27b-it_v6 status is now torndown due to DeploymentManager action

Usage Metrics

Latency Metrics