google-gemma-2-27b-it

submission_id: google-gemma-2-27b-it_v2
developer_uid: end_to_end_test
best_of: 4
display_name: google-gemma-2-27b-it_v2
family_friendly_score: 0.0
formatter: {'memory_template': 'character: {bot_name} {memory}\n', 'prompt_template': '{prompt}', 'bot_template': '{bot_name}: {message}', 'user_template': '{user_name}: {message}', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.0, 'top_p': 0.99, 'min_p': 0.1, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64}
ineligible_reason: model is only for e2e test
is_internal_developer: True
language_model: google/gemma-2-27b-it
max_input_tokens: 512
max_output_tokens: 64
model_architecture: Gemma2ForCausalLM
model_group: google/gemma-2-27b-it
model_name: google-gemma-2-27b-it_v2
model_num_parameters: 28731935232.0
model_repo: google/gemma-2-27b-it
model_size: 29B
num_battles: 222
num_wins: 92
ranking_group: single
reward_formatter: {'bot_template': '{bot_name}: {message}', 'memory_template': 'character: {bot_name} {memory}\n', 'prompt_template': '{prompt}', 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}'}
reward_repo: ChaiML/reward_models_100_170000000_cp_498032
status: torndown
submission_type: basic
timestamp: 2024-07-13T22:47:48+00:00
us_pacific_date: 2024-07-13
win_ratio: 0.4144144144144144
Resubmit model
Running pipeline stage MKMLizer
Starting job with name google-gemma-2-27b-it-v2-mkmlizer
Waiting for job on google-gemma-2-27b-it-v2-mkmlizer to finish
google-gemma-2-27b-it-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
google-gemma-2-27b-it-v2-mkmlizer: ║     _____            __           __                                ║
google-gemma-2-27b-it-v2-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
google-gemma-2-27b-it-v2-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
google-gemma-2-27b-it-v2-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
google-gemma-2-27b-it-v2-mkmlizer: ║       /___/                                                         ║
google-gemma-2-27b-it-v2-mkmlizer: ║                                                                     ║
google-gemma-2-27b-it-v2-mkmlizer: ║  Version: 0.9.5.post2                                               ║
google-gemma-2-27b-it-v2-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
google-gemma-2-27b-it-v2-mkmlizer: ║  https://mk1.ai                                                     ║
google-gemma-2-27b-it-v2-mkmlizer: ║                                                                     ║
google-gemma-2-27b-it-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
google-gemma-2-27b-it-v2-mkmlizer: ║  belonging to:                                                      ║
google-gemma-2-27b-it-v2-mkmlizer: ║                                                                     ║
google-gemma-2-27b-it-v2-mkmlizer: ║  Chai Research Corp.                                                ║
google-gemma-2-27b-it-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
google-gemma-2-27b-it-v2-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
google-gemma-2-27b-it-v2-mkmlizer: ║                                                                     ║
google-gemma-2-27b-it-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Stopping job with name google-gemma-2-27b-it-v2-mkmlizer
%s, retrying in %s seconds...
Stopping job with name google-gemma-2-27b-it-v2-mkmlizer
%s, retrying in %s seconds...
Stopping job with name google-gemma-2-27b-it-v2-mkmlizer
%s, retrying in %s seconds...
Starting job with name google-gemma-2-27b-it-v2-mkmlizer
Waiting for job on google-gemma-2-27b-it-v2-mkmlizer to finish
google-gemma-2-27b-it-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
google-gemma-2-27b-it-v2-mkmlizer: ║     _____            __           __                                ║
google-gemma-2-27b-it-v2-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
google-gemma-2-27b-it-v2-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
google-gemma-2-27b-it-v2-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
google-gemma-2-27b-it-v2-mkmlizer: ║       /___/                                                         ║
google-gemma-2-27b-it-v2-mkmlizer: ║                                                                     ║
google-gemma-2-27b-it-v2-mkmlizer: ║  Version: 0.9.5.post2                                               ║
google-gemma-2-27b-it-v2-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
google-gemma-2-27b-it-v2-mkmlizer: ║  https://mk1.ai                                                     ║
google-gemma-2-27b-it-v2-mkmlizer: ║                                                                     ║
google-gemma-2-27b-it-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
google-gemma-2-27b-it-v2-mkmlizer: ║  belonging to:                                                      ║
google-gemma-2-27b-it-v2-mkmlizer: ║                                                                     ║
google-gemma-2-27b-it-v2-mkmlizer: ║  Chai Research Corp.                                                ║
google-gemma-2-27b-it-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
google-gemma-2-27b-it-v2-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
google-gemma-2-27b-it-v2-mkmlizer: ║                                                                     ║
google-gemma-2-27b-it-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
google-gemma-2-27b-it-v2-mkmlizer: Downloaded to shared memory in 80.341s
google-gemma-2-27b-it-v2-mkmlizer: quantizing model to /dev/shm/model_cache
google-gemma-2-27b-it-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
google-gemma-2-27b-it-v2-mkmlizer: quantized model in 72.868s
google-gemma-2-27b-it-v2-mkmlizer: Processed model google/gemma-2-27b-it in 153.210s
google-gemma-2-27b-it-v2-mkmlizer: creating bucket guanaco-mkml-models
google-gemma-2-27b-it-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
google-gemma-2-27b-it-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/google-gemma-2-27b-it-v2
google-gemma-2-27b-it-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/google-gemma-2-27b-it-v2/config.json
google-gemma-2-27b-it-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/google-gemma-2-27b-it-v2/tokenizer_config.json
google-gemma-2-27b-it-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/google-gemma-2-27b-it-v2/special_tokens_map.json
google-gemma-2-27b-it-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/google-gemma-2-27b-it-v2/tokenizer.json
google-gemma-2-27b-it-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.model s3://guanaco-mkml-models/google-gemma-2-27b-it-v2/tokenizer.model
google-gemma-2-27b-it-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.2.safetensors s3://guanaco-mkml-models/google-gemma-2-27b-it-v2/flywheel_model.2.safetensors
google-gemma-2-27b-it-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/google-gemma-2-27b-it-v2/flywheel_model.0.safetensors
google-gemma-2-27b-it-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/google-gemma-2-27b-it-v2/flywheel_model.1.safetensors
google-gemma-2-27b-it-v2-mkmlizer: loading reward model from ChaiML/reward_models_100_170000000_cp_498032
google-gemma-2-27b-it-v2-mkmlizer: 
Loading 0:   0%|          | 0/508 [00:00<?, ?it/s]
Loading 0:   1%|          | 5/508 [00:00<00:13, 36.37it/s]
Loading 0:   3%|▎         | 15/508 [00:00<00:07, 65.33it/s]
Loading 0:   6%|▌         | 28/508 [00:00<00:11, 42.35it/s]
Loading 0:   7%|▋         | 34/508 [00:00<00:11, 42.75it/s]
Loading 0:   8%|▊         | 39/508 [00:01<00:14, 33.40it/s]
Loading 0:   9%|▉         | 48/508 [00:01<00:11, 41.49it/s]
Loading 0:  11%|█▏        | 58/508 [00:01<00:09, 48.86it/s]
Loading 0:  13%|█▎        | 64/508 [00:01<00:09, 47.86it/s]
Loading 0:  14%|█▍        | 73/508 [00:01<00:15, 28.31it/s]
Loading 0:  16%|█▌        | 80/508 [00:02<00:13, 31.12it/s]
Loading 0:  17%|█▋        | 86/508 [00:02<00:12, 33.45it/s]
Loading 0:  18%|█▊        | 91/508 [00:02<00:12, 33.63it/s]
Loading 0:  19%|█▉        | 97/508 [00:02<00:11, 35.72it/s]
Loading 0:  20%|██        | 102/508 [00:02<00:10, 36.93it/s]
Loading 0:  21%|██▏       | 109/508 [00:02<00:09, 39.97it/s]
Loading 0:  23%|██▎       | 118/508 [00:03<00:14, 26.10it/s]
Loading 0:  24%|██▍       | 124/508 [00:03<00:13, 29.40it/s]
Loading 0:  25%|██▌       | 128/508 [00:03<00:12, 30.08it/s]
Loading 0:  27%|██▋       | 135/508 [00:03<00:10, 36.52it/s]
Loading 0:  28%|██▊       | 140/508 [00:03<00:09, 37.39it/s]
Loading 0:  29%|██▉       | 147/508 [00:04<00:08, 40.35it/s]
Loading 0:  31%|███       | 157/508 [00:04<00:07, 47.84it/s]
Loading 0:  32%|███▏      | 163/508 [00:04<00:07, 47.22it/s]
Loading 0:  34%|███▎      | 171/508 [00:04<00:11, 28.72it/s]
Loading 0:  34%|███▍      | 175/508 [00:05<00:15, 21.55it/s]
Loading 0:  35%|███▌      | 180/508 [00:05<00:13, 23.83it/s]
Loading 0:  37%|███▋      | 190/508 [00:05<00:09, 32.87it/s]
Loading 0:  38%|███▊      | 191/508 [00:22<00:09, 32.87it/s]
Loading 0:  38%|███▊      | 192/508 [00:22<04:57,  1.06it/s]
Loading 0:  40%|███▉      | 201/508 [00:22<02:52,  1.78it/s]
Loading 0:  41%|████      | 207/508 [00:22<02:03,  2.44it/s]
Loading 0:  42%|████▏     | 213/508 [00:22<01:28,  3.34it/s]
Loading 0:  43%|████▎     | 219/508 [00:23<01:09,  4.19it/s]
Loading 0:  44%|████▍     | 224/508 [00:23<00:52,  5.44it/s]
Loading 0:  46%|████▌     | 234/508 [00:23<00:30,  8.93it/s]
Loading 0:  47%|████▋     | 239/508 [00:23<00:24, 10.90it/s]
Loading 0:  48%|████▊     | 246/508 [00:23<00:18, 14.51it/s]
Loading 0:  50%|█████     | 255/508 [00:24<00:12, 20.41it/s]
Loading 0:  51%|█████▏    | 261/508 [00:24<00:13, 17.77it/s]
Loading 0:  53%|█████▎    | 267/508 [00:24<00:11, 21.39it/s]
Loading 0:  54%|█████▎    | 272/508 [00:24<00:09, 24.35it/s]
Loading 0:  55%|█████▍    | 279/508 [00:24<00:07, 29.65it/s]
Loading 0:  57%|█████▋    | 289/508 [00:25<00:05, 38.80it/s]
Loading 0:  58%|█████▊    | 295/508 [00:25<00:05, 40.44it/s]
Loading 0:  59%|█████▉    | 301/508 [00:25<00:04, 41.84it/s]
Loading 0:  61%|██████    | 308/508 [00:25<00:04, 46.63it/s]
Loading 0:  62%|██████▏   | 314/508 [00:25<00:06, 28.29it/s]
Loading 0:  63%|██████▎   | 319/508 [00:26<00:06, 27.53it/s]
Loading 0:  64%|██████▎   | 323/508 [00:26<00:06, 28.71it/s]
Loading 0:  66%|██████▌   | 333/508 [00:26<00:04, 39.03it/s]
Loading 0:  67%|██████▋   | 338/508 [00:26<00:04, 39.57it/s]
Loading 0:  68%|██████▊   | 345/508 [00:26<00:03, 43.17it/s]
Loading 0:  70%|██████▉   | 354/508 [00:26<00:03, 49.90it/s]
Loading 0:  71%|███████   | 360/508 [00:27<00:05, 28.65it/s]
Loading 0:  72%|███████▏  | 366/508 [00:27<00:04, 32.58it/s]
Loading 0:  73%|███████▎  | 371/508 [00:27<00:03, 34.48it/s]
Loading 0:  74%|███████▍  | 378/508 [00:27<00:03, 39.01it/s]
Loading 0:  76%|███████▋  | 388/508 [00:27<00:02, 47.57it/s]
Loading 0:  78%|███████▊  | 394/508 [00:27<00:02, 47.95it/s]
Loading 0:  79%|███████▊  | 400/508 [00:27<00:02, 46.19it/s]
Loading 0:  80%|███████▉  | 405/508 [00:28<00:03, 27.04it/s]
Loading 0:  81%|████████  | 411/508 [00:28<00:03, 31.50it/s]
Loading 0:  83%|████████▎ | 421/508 [00:28<00:02, 41.41it/s]
Loading 0:  84%|████████▍ | 427/508 [00:28<00:01, 42.86it/s]
Loading 0:  85%|████████▌ | 433/508 [00:28<00:01, 43.00it/s]
Loading 0:  87%|████████▋ | 443/508 [00:28<00:01, 52.24it/s]
Loading 0:  88%|████████▊ | 449/508 [00:29<00:01, 51.57it/s]
Loading 0:  89%|████████▉ | 451/508 [00:45<00:01, 51.57it/s]
Loading 0:  89%|████████▉ | 452/508 [00:45<00:47,  1.17it/s]
Loading 0:  90%|████████▉ | 457/508 [00:46<00:33,  1.54it/s]
Loading 0:  91%|█████████ | 462/508 [00:46<00:22,  2.08it/s]
Loading 0:  92%|█████████▏| 466/508 [00:46<00:15,  2.67it/s]
Loading 0:  94%|█████████▎| 476/508 [00:46<00:06,  4.80it/s]
Loading 0:  95%|█████████▍| 481/508 [00:46<00:04,  6.11it/s]
Loading 0:  96%|█████████▌| 488/508 [00:46<00:02,  8.64it/s]
Loading 0:  98%|█████████▊| 497/508 [00:46<00:00, 12.88it/s]
Loading 0:  99%|█████████▉| 503/508 [00:47<00:00, 13.09it/s]
                                                            
/opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:950: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
google-gemma-2-27b-it-v2-mkmlizer:   warnings.warn(
google-gemma-2-27b-it-v2-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:778: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
google-gemma-2-27b-it-v2-mkmlizer:   warnings.warn(
google-gemma-2-27b-it-v2-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
google-gemma-2-27b-it-v2-mkmlizer:   warnings.warn(
google-gemma-2-27b-it-v2-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
google-gemma-2-27b-it-v2-mkmlizer: Saving duration: 3.485s
google-gemma-2-27b-it-v2-mkmlizer: Processed model ChaiML/reward_models_100_170000000_cp_498032 in 7.294s
google-gemma-2-27b-it-v2-mkmlizer: creating bucket guanaco-reward-models
google-gemma-2-27b-it-v2-mkmlizer: Bucket 's3://guanaco-reward-models/' created
google-gemma-2-27b-it-v2-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/google-gemma-2-27b-it-v2_reward
google-gemma-2-27b-it-v2-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/google-gemma-2-27b-it-v2_reward/special_tokens_map.json
google-gemma-2-27b-it-v2-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/google-gemma-2-27b-it-v2_reward/tokenizer_config.json
google-gemma-2-27b-it-v2-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/google-gemma-2-27b-it-v2_reward/config.json
google-gemma-2-27b-it-v2-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/google-gemma-2-27b-it-v2_reward/vocab.json
google-gemma-2-27b-it-v2-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/google-gemma-2-27b-it-v2_reward/merges.txt
google-gemma-2-27b-it-v2-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/google-gemma-2-27b-it-v2_reward/tokenizer.json
google-gemma-2-27b-it-v2-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/google-gemma-2-27b-it-v2_reward/reward.tensors
Job google-gemma-2-27b-it-v2-mkmlizer completed after 828.85s with status: succeeded
Stopping job with name google-gemma-2-27b-it-v2-mkmlizer
Pipeline stage MKMLizer completed in 1349.34s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.26s
Running pipeline stage ISVCDeployer
Creating inference service google-gemma-2-27b-it-v2
Waiting for inference service google-gemma-2-27b-it-v2 to be ready
Inference service google-gemma-2-27b-it-v2 ready after 182.4097559452057s
Pipeline stage ISVCDeployer completed in 189.08s
Running pipeline stage StressChecker
Received healthy response to inference request in 14.087478876113892s
Received healthy response to inference request in 1.7274742126464844s
Received healthy response to inference request in 1.9929749965667725s
Received healthy response to inference request in 1.6166088581085205s
Received healthy response to inference request in 3.49049711227417s
5 requests
0 failed requests
5th percentile: 1.6387819290161132
10th percentile: 1.660954999923706
20th percentile: 1.7053011417388917
30th percentile: 1.780574369430542
40th percentile: 1.8867746829986571
50th percentile: 1.9929749965667725
60th percentile: 2.5919838428497313
70th percentile: 3.19099268913269
80th percentile: 5.609893465042116
90th percentile: 9.848686170578004
95th percentile: 11.968082523345945
99th percentile: 13.663599605560302
mean time: 4.583006811141968
Pipeline stage StressChecker completed in 25.11s
google-gemma-2-27b-it_v2 status is now deployed due to DeploymentManager action
google-gemma-2-27b-it_v2 status is now inactive due to admin request
admin requested tearing down of google-gemma-2-27b-it_v2
Running pipeline stage ISVCDeleter
Pipeline stage %s skipped, reason=%s
Pipeline stage ISVCDeleter completed in 0.70s
Running pipeline stage MKMLModelDeleter
Pipeline stage %s skipped, reason=%s
Pipeline stage MKMLModelDeleter completed in 0.19s
google-gemma-2-27b-it_v2 status is now torndown due to DeploymentManager action