submission_id: google-gemma-2-27b_v2
developer_uid: chai_backend_admin
status: inactive
model_repo: google/gemma-2-27b
reward_repo: ChaiML/gpt2_xl_pairwise_89m_step_347634
generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 50, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
reward_formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
timestamp: 2024-07-10T18:05:39+00:00
model_name: google-gemma-2-27b_v2
model_group: google/gemma-2-27b
num_battles: 32524
num_wins: 12418
celo_rating: 1092.25
alignment_score: None
alignment_samples: 0
propriety_score: 0.7316025067144136
propriety_total_count: 5585.0
submission_type: basic
model_architecture: Gemma2ForCausalLM
model_num_parameters: 28731935232.0
best_of: 4
max_input_tokens: 512
max_output_tokens: 64
display_name: google-gemma-2-27b_v2
ineligible_reason: None
language_model: google/gemma-2-27b
model_size: 29B
reward_model: ChaiML/gpt2_xl_pairwise_89m_step_347634
us_pacific_date: 2024-07-10
win_ratio: 0.38181035542983643
preference_data_url: None
Resubmit model
Running pipeline stage MKMLizer
Starting job with name google-gemma-2-27b-v2-mkmlizer
Waiting for job on google-gemma-2-27b-v2-mkmlizer to finish
google-gemma-2-27b-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
google-gemma-2-27b-v2-mkmlizer: ║ _____ __ __ ║
google-gemma-2-27b-v2-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
google-gemma-2-27b-v2-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
google-gemma-2-27b-v2-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
google-gemma-2-27b-v2-mkmlizer: ║ /___/ ║
google-gemma-2-27b-v2-mkmlizer: ║ ║
google-gemma-2-27b-v2-mkmlizer: ║ Version: 0.9.5.post1 ║
google-gemma-2-27b-v2-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
google-gemma-2-27b-v2-mkmlizer: ║ https://mk1.ai ║
google-gemma-2-27b-v2-mkmlizer: ║ ║
google-gemma-2-27b-v2-mkmlizer: ║ The license key for the current software has been verified as ║
google-gemma-2-27b-v2-mkmlizer: ║ belonging to: ║
google-gemma-2-27b-v2-mkmlizer: ║ ║
google-gemma-2-27b-v2-mkmlizer: ║ Chai Research Corp. ║
google-gemma-2-27b-v2-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
google-gemma-2-27b-v2-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
google-gemma-2-27b-v2-mkmlizer: ║ ║
google-gemma-2-27b-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
google-gemma-2-27b-v2-mkmlizer: Downloaded to shared memory in 144.692s
google-gemma-2-27b-v2-mkmlizer: quantizing model to /dev/shm/model_cache
google-gemma-2-27b-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
google-gemma-2-27b-v2-mkmlizer: quantized model in 90.969s
google-gemma-2-27b-v2-mkmlizer: Processed model google/gemma-2-27b in 235.662s
google-gemma-2-27b-v2-mkmlizer: creating bucket guanaco-mkml-models
google-gemma-2-27b-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
google-gemma-2-27b-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/google-gemma-2-27b-v2
google-gemma-2-27b-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/google-gemma-2-27b-v2/special_tokens_map.json
google-gemma-2-27b-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/google-gemma-2-27b-v2/tokenizer_config.json
google-gemma-2-27b-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/google-gemma-2-27b-v2/config.json
google-gemma-2-27b-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.model s3://guanaco-mkml-models/google-gemma-2-27b-v2/tokenizer.model
google-gemma-2-27b-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/google-gemma-2-27b-v2/tokenizer.json
google-gemma-2-27b-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.2.safetensors s3://guanaco-mkml-models/google-gemma-2-27b-v2/flywheel_model.2.safetensors
google-gemma-2-27b-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/google-gemma-2-27b-v2/flywheel_model.0.safetensors
google-gemma-2-27b-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/google-gemma-2-27b-v2/flywheel_model.1.safetensors
google-gemma-2-27b-v2-mkmlizer: loading reward model from ChaiML/gpt2_xl_pairwise_89m_step_347634
google-gemma-2-27b-v2-mkmlizer: Loading 0: 0%| | 0/508 [00:00<?, ?it/s] Loading 0: 1%| | 6/508 [00:00<00:33, 15.21it/s] Loading 0: 2%|▏ | 8/508 [00:00<00:37, 13.29it/s] Loading 0: 2%|▏ | 10/508 [00:00<00:35, 13.92it/s] Loading 0: 3%|▎ | 14/508 [00:00<00:25, 19.22it/s] Loading 0: 3%|▎ | 17/508 [00:01<00:27, 17.67it/s] Loading 0: 6%|▌ | 28/508 [00:01<00:19, 24.66it/s] Loading 0: 6%|▌ | 31/508 [00:01<00:24, 19.20it/s] Loading 0: 7%|▋ | 37/508 [00:01<00:20, 22.55it/s] Loading 0: 8%|▊ | 43/508 [00:01<00:16, 28.35it/s] Loading 0: 10%|▉ | 50/508 [00:02<00:17, 26.12it/s] Loading 0: 11%|█ | 54/508 [00:02<00:20, 21.92it/s] Loading 0: 11%|█▏ | 58/508 [00:02<00:18, 24.39it/s] Loading 0: 12%|█▏ | 61/508 [00:02<00:20, 21.91it/s] Loading 0: 14%|█▍ | 72/508 [00:03<00:16, 26.99it/s] Loading 0: 15%|█▍ | 75/508 [00:03<00:20, 20.69it/s] Loading 0: 16%|█▌ | 80/508 [00:03<00:17, 24.75it/s] Loading 0: 17%|█▋ | 84/508 [00:03<00:17, 23.92it/s] Loading 0: 19%|█▊ | 94/508 [00:04<00:15, 26.23it/s] Loading 0: 19%|█▉ | 99/508 [00:04<00:14, 28.57it/s] Loading 0: 20%|██ | 103/508 [00:04<00:15, 26.82it/s] Loading 0: 21%|██ | 107/508 [00:04<00:15, 25.61it/s] Loading 0: 22%|██▏ | 113/508 [00:04<00:12, 31.43it/s] Loading 0: 23%|██▎ | 117/508 [00:05<00:18, 21.53it/s] Loading 0: 24%|██▎ | 120/508 [00:05<00:19, 19.78it/s] Loading 0: 25%|██▍ | 125/508 [00:05<00:17, 21.48it/s] Loading 0: 25%|██▌ | 128/508 [00:05<00:16, 22.43it/s] Loading 0: 27%|██▋ | 138/508 [00:05<00:14, 25.32it/s] Loading 0: 28%|██▊ | 141/508 [00:06<00:18, 19.69it/s] Loading 0: 29%|██▉ | 147/508 [00:06<00:16, 22.49it/s] Loading 0: 30%|██▉ | 152/508 [00:06<00:13, 26.68it/s] Loading 0: 31%|███▏ | 160/508 [00:06<00:13, 26.35it/s] Loading 0: 32%|███▏ | 164/508 [00:07<00:15, 22.22it/s] Loading 0: 33%|███▎ | 169/508 [00:07<00:14, 23.34it/s] Loading 0: 34%|███▍ | 174/508 [00:07<00:12, 27.51it/s] Loading 0: 36%|███▌ | 182/508 [00:07<00:12, 26.49it/s] Loading 0: 37%|███▋ | 186/508 [00:07<00:14, 22.22it/s] Loading 0: 38%|███▊ | 191/508 [00:08<00:13, 23.66it/s] Loading 0: 38%|███▊ | 191/508 [00:20<00:13, 23.66it/s] Loading 0: 38%|███▊ | 192/508 [00:29<08:25, 1.60s/it] Loading 0: 40%|████ | 204/508 [00:29<03:38, 1.39it/s] Loading 0: 41%|████ | 207/508 [00:29<03:04, 1.63it/s] Loading 0: 42%|████▏ | 213/508 [00:29<02:05, 2.35it/s] Loading 0: 43%|████▎ | 219/508 [00:29<01:25, 3.37it/s] Loading 0: 44%|████▍ | 226/508 [00:30<00:59, 4.78it/s] Loading 0: 45%|████▌ | 230/508 [00:30<00:50, 5.52it/s] Loading 0: 46%|████▌ | 234/508 [00:30<00:39, 6.90it/s] Loading 0: 47%|████▋ | 237/508 [00:30<00:35, 7.57it/s] Loading 0: 49%|████▉ | 248/508 [00:31<00:20, 12.45it/s] Loading 0: 49%|████▉ | 251/508 [00:31<00:22, 11.62it/s] Loading 0: 50%|█████ | 256/508 [00:31<00:17, 14.66it/s] Loading 0: 51%|█████ | 259/508 [00:32<00:17, 14.28it/s] Loading 0: 53%|█████▎ | 270/508 [00:32<00:12, 19.66it/s] Loading 0: 54%|█████▎ | 273/508 [00:32<00:14, 16.10it/s] Loading 0: 55%|█████▍ | 278/508 [00:32<00:11, 19.66it/s] Loading 0: 55%|█████▌ | 281/508 [00:33<00:12, 17.80it/s] Loading 0: 57%|█████▋ | 292/508 [00:33<00:09, 22.83it/s] Loading 0: 58%|█████▊ | 295/508 [00:33<00:11, 18.57it/s] Loading 0: 59%|█████▉ | 301/508 [00:33<00:09, 21.46it/s] Loading 0: 60%|█████▉ | 304/508 [00:34<00:09, 22.54it/s] Loading 0: 62%|██████▏ | 314/508 [00:34<00:07, 25.49it/s] Loading 0: 62%|██████▏ | 317/508 [00:34<00:09, 20.11it/s] Loading 0: 64%|██████▎ | 323/508 [00:34<00:08, 22.93it/s] Loading 0: 65%|██████▍ | 328/508 [00:34<00:06, 26.89it/s] Loading 0: 66%|██████▌ | 336/508 [00:35<00:06, 26.27it/s] Loading 0: 67%|██████▋ | 340/508 [00:35<00:07, 22.27it/s] Loading 0: 68%|██████▊ | 345/508 [00:35<00:06, 23.59it/s] Loading 0: 69%|██████▉ | 350/508 [00:35<00:05, 27.76it/s] Loading 0: 70%|███████ | 358/508 [00:36<00:05, 26.86it/s] Loading 0: 71%|███████▏ | 362/508 [00:36<00:06, 20.96it/s] Loading 0: 72%|███████▏ | 366/508 [00:36<00:06, 23.56it/s] Loading 0: 73%|███████▎ | 369/508 [00:36<00:06, 21.80it/s] Loading 0: 75%|███████▍ | 380/508 [00:37<00:04, 27.01it/s] Loading 0: 75%|███████▌ | 383/508 [00:37<00:05, 22.43it/s] Loading 0: 77%|███████▋ | 389/508 [00:37<00:04, 25.57it/s] Loading 0: 78%|███████▊ | 395/508 [00:37<00:03, 31.20it/s] Loading 0: 79%|███████▉ | 402/508 [00:37<00:03, 28.56it/s] Loading 0: 80%|███████▉ | 406/508 [00:38<00:04, 23.86it/s] Loading 0: 81%|████████ | 411/508 [00:38<00:03, 24.90it/s] Loading 0: 82%|████████▏ | 417/508 [00:38<00:02, 30.54it/s] Loading 0: 83%|████████▎ | 424/508 [00:38<00:03, 27.73it/s] Loading 0: 84%|████████▍ | 428/508 [00:38<00:03, 23.58it/s] Loading 0: 85%|████████▌ | 433/508 [00:39<00:03, 24.91it/s] Loading 0: 86%|████████▋ | 439/508 [00:39<00:02, 30.58it/s] Loading 0: 88%|████████▊ | 446/508 [00:39<00:02, 28.29it/s] Loading 0: 88%|████████▊ | 448/508 [00:50<00:02, 28.29it/s] Loading 0: 88%|████████▊ | 449/508 [01:02<01:20, 1.37s/it] Loading 0: 90%|████████▉ | 455/508 [01:02<00:48, 1.09it/s] Loading 0: 91%|█████████ | 461/508 [01:02<00:29, 1.60it/s] Loading 0: 92%|█████████▏| 468/508 [01:02<00:16, 2.38it/s] Loading 0: 93%|█████████▎| 472/508 [01:02<00:12, 2.93it/s] Loading 0: 94%|█████████▍| 477/508 [01:03<00:07, 3.92it/s] Loading 0: 95%|█████████▌| 483/508 [01:03<00:04, 5.65it/s] Loading 0: 96%|█████████▋| 490/508 [01:03<00:02, 7.71it/s] Loading 0: 97%|█████████▋| 494/508 [01:03<00:01, 8.68it/s] Loading 0: 98%|█████████▊| 499/508 [01:03<00:00, 10.77it/s] Loading 0: 99%|█████████▉| 505/508 [01:04<00:00, 14.69it/s] /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:950: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
google-gemma-2-27b-v2-mkmlizer: warnings.warn(
google-gemma-2-27b-v2-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:778: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
google-gemma-2-27b-v2-mkmlizer: warnings.warn(
google-gemma-2-27b-v2-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
google-gemma-2-27b-v2-mkmlizer: warnings.warn(
google-gemma-2-27b-v2-mkmlizer: Downloading shards: 0%| | 0/2 [00:00<?, ?it/s] Downloading shards: 50%|█████ | 1/2 [00:06<00:06, 6.11s/it] Downloading shards: 100%|██████████| 2/2 [00:08<00:00, 3.68s/it] Downloading shards: 100%|██████████| 2/2 [00:08<00:00, 4.04s/it]
google-gemma-2-27b-v2-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/google-gemma-2-27b-v2_reward/vocab.json
google-gemma-2-27b-v2-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/google-gemma-2-27b-v2_reward/merges.txt
google-gemma-2-27b-v2-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/google-gemma-2-27b-v2_reward/special_tokens_map.json
google-gemma-2-27b-v2-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/google-gemma-2-27b-v2_reward/reward.tensors
Job google-gemma-2-27b-v2-mkmlizer completed after 294.54s with status: succeeded
Stopping job with name google-gemma-2-27b-v2-mkmlizer
Pipeline stage MKMLizer completed in 295.73s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.32s
Running pipeline stage ISVCDeployer
Creating inference service google-gemma-2-27b-v2
Waiting for inference service google-gemma-2-27b-v2 to be ready
Tearing down inference service google-gemma-2-27b-v2
%s, retrying in %s seconds...
Creating inference service google-gemma-2-27b-v2
Waiting for inference service google-gemma-2-27b-v2 to be ready
Inference service google-gemma-2-27b-v2 ready after 203.46934485435486s
Pipeline stage ISVCDeployer completed in 228.69s
Running pipeline stage StressChecker
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 3.6711628437042236s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.8725018501281738s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.87762188911438s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.7657999992370605s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.9147312641143799s
5 requests
0 failed requests
5th percentile: 1.7871403694152832
10th percentile: 1.8084807395935059
20th percentile: 1.8511614799499512
30th percentile: 1.880947732925415
40th percentile: 1.8978394985198974
50th percentile: 1.9147312641143799
60th percentile: 2.29988751411438
70th percentile: 2.6850437641143796
80th percentile: 3.0363300800323487
90th percentile: 3.353746461868286
95th percentile: 3.512454652786255
99th percentile: 3.63942120552063
mean time: 2.4203635692596435
Pipeline stage StressChecker completed in 15.07s
google-gemma-2-27b_v2 status is now deployed due to DeploymentManager action
google-gemma-2-27b_v2 status is now inactive due to auto deactivation removed underperforming models

Usage Metrics

Latency Metrics