submission_id: google-gemma-2-27b-it_v9
developer_uid: chai_backend_admin
alignment_samples: 0
best_of: 4
celo_rating: 1178.39
display_name: google-gemma-2-27b-it_v9
formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': True}
generation_params: {'temperature': 1.2, 'top_p': 1.0, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<end_of_turn>', '<|eot_id|>'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64}
is_internal_developer: True
language_model: google/gemma-2-27b-it
max_input_tokens: 512
max_output_tokens: 64
model_architecture: Gemma2ForCausalLM
model_group: google/gemma-2-27b-it
model_name: google-gemma-2-27b-it_v9
model_num_parameters: 28731935232.0
model_repo: google/gemma-2-27b-it
model_size: 29B
num_battles: 92125
num_wins: 43320
propriety_score: 0.7397003745318352
propriety_total_count: 8010.0
ranking_group: single
reward_formatter: {'bot_template': '{bot_name}: {message}\n', 'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}\n'}
reward_repo: ChaiML/gpt2_xl_pairwise_89m_step_347634
status: torndown
submission_type: basic
timestamp: 2024-07-14T03:25:16+00:00
us_pacific_date: 2024-07-13
win_ratio: 0.4702306648575305
Resubmit model
Running pipeline stage MKMLizer
Starting job with name google-gemma-2-27b-it-v9-mkmlizer
Waiting for job on google-gemma-2-27b-it-v9-mkmlizer to finish
google-gemma-2-27b-it-v9-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
google-gemma-2-27b-it-v9-mkmlizer: ║ _____ __ __ ║
google-gemma-2-27b-it-v9-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
google-gemma-2-27b-it-v9-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
google-gemma-2-27b-it-v9-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
google-gemma-2-27b-it-v9-mkmlizer: ║ /___/ ║
google-gemma-2-27b-it-v9-mkmlizer: ║ ║
google-gemma-2-27b-it-v9-mkmlizer: ║ Version: 0.9.5.post3 ║
google-gemma-2-27b-it-v9-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
google-gemma-2-27b-it-v9-mkmlizer: ║ https://mk1.ai ║
google-gemma-2-27b-it-v9-mkmlizer: ║ ║
google-gemma-2-27b-it-v9-mkmlizer: ║ The license key for the current software has been verified as ║
google-gemma-2-27b-it-v9-mkmlizer: ║ belonging to: ║
google-gemma-2-27b-it-v9-mkmlizer: ║ ║
google-gemma-2-27b-it-v9-mkmlizer: ║ Chai Research Corp. ║
google-gemma-2-27b-it-v9-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
google-gemma-2-27b-it-v9-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
google-gemma-2-27b-it-v9-mkmlizer: ║ ║
google-gemma-2-27b-it-v9-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
google-gemma-2-27b-it-v9-mkmlizer: Downloaded to shared memory in 79.675s
google-gemma-2-27b-it-v9-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpd8p3ubiz, device:0
google-gemma-2-27b-it-v9-mkmlizer: Saving flywheel model at /dev/shm/model_cache
google-gemma-2-27b-it-v9-mkmlizer: quantized model in 88.439s
google-gemma-2-27b-it-v9-mkmlizer: Processed model google/gemma-2-27b-it in 168.114s
google-gemma-2-27b-it-v9-mkmlizer: creating bucket guanaco-mkml-models
google-gemma-2-27b-it-v9-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
google-gemma-2-27b-it-v9-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/google-gemma-2-27b-it-v9
google-gemma-2-27b-it-v9-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/google-gemma-2-27b-it-v9/config.json
google-gemma-2-27b-it-v9-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/google-gemma-2-27b-it-v9/special_tokens_map.json
google-gemma-2-27b-it-v9-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/google-gemma-2-27b-it-v9/tokenizer_config.json
google-gemma-2-27b-it-v9-mkmlizer: cp /dev/shm/model_cache/tokenizer.model s3://guanaco-mkml-models/google-gemma-2-27b-it-v9/tokenizer.model
google-gemma-2-27b-it-v9-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/google-gemma-2-27b-it-v9/tokenizer.json
google-gemma-2-27b-it-v9-mkmlizer: cp /dev/shm/model_cache/flywheel_model.2.safetensors s3://guanaco-mkml-models/google-gemma-2-27b-it-v9/flywheel_model.2.safetensors
google-gemma-2-27b-it-v9-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/google-gemma-2-27b-it-v9/flywheel_model.0.safetensors
google-gemma-2-27b-it-v9-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/google-gemma-2-27b-it-v9/flywheel_model.1.safetensors
google-gemma-2-27b-it-v9-mkmlizer: loading reward model from ChaiML/gpt2_xl_pairwise_89m_step_347634
google-gemma-2-27b-it-v9-mkmlizer: Loading 0: 0%| | 0/508 [00:00<?, ?it/s] Loading 0: 1%| | 4/508 [00:00<00:13, 36.11it/s] Loading 0: 3%|▎ | 14/508 [00:00<00:08, 60.66it/s] Loading 0: 4%|▍ | 20/508 [00:00<00:08, 57.13it/s] Loading 0: 6%|▌ | 28/508 [00:00<00:16, 29.07it/s] Loading 0: 6%|▋ | 33/508 [00:00<00:16, 28.46it/s] Loading 0: 7%|▋ | 37/508 [00:01<00:15, 29.97it/s] Loading 0: 9%|▉ | 47/508 [00:01<00:11, 41.24it/s] Loading 0: 10%|█ | 52/508 [00:01<00:10, 41.78it/s] Loading 0: 12%|█▏ | 59/508 [00:01<00:09, 45.19it/s] Loading 0: 13%|█▎ | 68/508 [00:01<00:08, 51.99it/s] Loading 0: 15%|█▍ | 74/508 [00:02<00:16, 26.89it/s] Loading 0: 16%|█▌ | 81/508 [00:02<00:13, 31.98it/s] Loading 0: 17%|█▋ | 87/508 [00:02<00:11, 36.26it/s] Loading 0: 19%|█▉ | 97/508 [00:02<00:08, 46.08it/s] Loading 0: 20%|██ | 103/508 [00:02<00:08, 47.61it/s] Loading 0: 21%|██▏ | 109/508 [00:02<00:08, 47.19it/s] Loading 0: 23%|██▎ | 118/508 [00:03<00:12, 31.56it/s] Loading 0: 24%|██▍ | 124/508 [00:03<00:10, 35.42it/s] Loading 0: 25%|██▌ | 129/508 [00:03<00:10, 37.04it/s] Loading 0: 27%|██▋ | 136/508 [00:03<00:09, 41.26it/s] Loading 0: 29%|██▊ | 146/508 [00:03<00:07, 50.28it/s] Loading 0: 30%|██▉ | 152/508 [00:03<00:07, 50.52it/s] Loading 0: 31%|███ | 158/508 [00:03<00:07, 49.65it/s] Loading 0: 34%|███▎ | 171/508 [00:04<00:09, 36.31it/s] Loading 0: 35%|███▍ | 176/508 [00:04<00:09, 34.69it/s] Loading 0: 36%|███▌ | 181/508 [00:04<00:09, 33.65it/s] Loading 0: 38%|███▊ | 191/508 [00:04<00:07, 43.06it/s] Loading 0: 38%|███▊ | 191/508 [00:26<00:07, 43.06it/s] Loading 0: 38%|███▊ | 192/508 [00:26<06:08, 1.17s/it] Loading 0: 40%|███▉ | 202/508 [00:26<03:25, 1.49it/s] Loading 0: 42%|████▏ | 211/508 [00:26<02:10, 2.28it/s] Loading 0: 43%|████▎ | 218/508 [00:27<01:37, 2.97it/s] Loading 0: 44%|████▍ | 224/508 [00:27<01:12, 3.92it/s] Loading 0: 46%|████▌ | 234/508 [00:27<00:44, 6.14it/s] Loading 0: 47%|████▋ | 240/508 [00:27<00:34, 7.83it/s] Loading 0: 48%|████▊ | 246/508 [00:27<00:26, 10.01it/s] Loading 0: 50%|█████ | 255/508 [00:28<00:17, 14.43it/s] Loading 0: 51%|█████▏ | 261/508 [00:28<00:17, 14.07it/s] Loading 0: 53%|█████▎ | 267/508 [00:28<00:13, 17.55it/s] Loading 0: 54%|█████▎ | 272/508 [00:28<00:11, 20.54it/s] Loading 0: 55%|█████▍ | 279/508 [00:28<00:08, 25.84it/s] Loading 0: 57%|█████▋ | 289/508 [00:29<00:06, 35.23it/s] Loading 0: 58%|█████▊ | 295/508 [00:29<00:05, 38.49it/s] Loading 0: 59%|█████▉ | 301/508 [00:29<00:05, 40.86it/s] Loading 0: 62%|██████▏ | 314/508 [00:29<00:05, 33.19it/s] Loading 0: 63%|██████▎ | 319/508 [00:29<00:05, 32.74it/s] Loading 0: 64%|██████▍ | 324/508 [00:30<00:05, 32.52it/s] Loading 0: 66%|██████▌ | 334/508 [00:30<00:04, 42.14it/s] Loading 0: 68%|██████▊ | 344/508 [00:30<00:03, 50.55it/s] Loading 0: 69%|██████▉ | 350/508 [00:30<00:03, 51.04it/s] Loading 0: 71%|███████ | 359/508 [00:30<00:04, 33.66it/s] Loading 0: 72%|███████▏ | 364/508 [00:30<00:04, 35.69it/s] Loading 0: 73%|███████▎ | 369/508 [00:31<00:04, 34.47it/s] Loading 0: 74%|███████▍ | 378/508 [00:31<00:03, 42.43it/s] Loading 0: 76%|███████▋ | 388/508 [00:31<00:02, 51.19it/s] Loading 0: 78%|███████▊ | 394/508 [00:31<00:02, 51.65it/s] Loading 0: 79%|███████▊ | 400/508 [00:31<00:02, 50.40it/s] Loading 0: 80%|███████▉ | 406/508 [00:32<00:03, 29.27it/s] Loading 0: 81%|████████ | 411/508 [00:32<00:03, 32.22it/s] Loading 0: 83%|████████▎ | 421/508 [00:32<00:02, 42.41it/s] Loading 0: 84%|████████▍ | 427/508 [00:32<00:01, 44.83it/s] Loading 0: 85%|████████▌ | 433/508 [00:32<00:01, 45.95it/s] Loading 0: 87%|████████▋ | 443/508 [00:32<00:01, 54.51it/s] Loading 0: 88%|████████▊ | 449/508 [00:32<00:01, 53.81it/s] Loading 0: 89%|████████▉ | 451/508 [00:54<00:01, 53.81it/s] Loading 0: 89%|████████▉ | 452/508 [00:54<01:01, 1.10s/it] Loading 0: 90%|████████▉ | 457/508 [00:54<00:42, 1.20it/s] Loading 0: 91%|█████████ | 462/508 [00:54<00:28, 1.63it/s] Loading 0: 92%|█████████▏| 466/508 [00:54<00:19, 2.12it/s] Loading 0: 94%|█████████▎| 476/508 [00:55<00:08, 3.84it/s] Loading 0: 95%|█████████▍| 482/508 [00:55<00:05, 5.20it/s] Loading 0: 96%|█████████▌| 488/508 [00:55<00:02, 6.99it/s] Loading 0: 98%|█████████▊| 497/508 [00:55<00:01, 10.65it/s] Loading 0: 99%|█████████▉| 503/508 [00:55<00:00, 11.25it/s] /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:950: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
google-gemma-2-27b-it-v9-mkmlizer: warnings.warn(
google-gemma-2-27b-it-v9-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:778: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
google-gemma-2-27b-it-v9-mkmlizer: warnings.warn(
google-gemma-2-27b-it-v9-mkmlizer: Downloading shards: 0%| | 0/2 [00:00<?, ?it/s] Downloading shards: 50%|█████ | 1/2 [00:06<00:06, 6.82s/it] Downloading shards: 100%|██████████| 2/2 [00:08<00:00, 4.05s/it] Downloading shards: 100%|██████████| 2/2 [00:08<00:00, 4.47s/it]
google-gemma-2-27b-it-v9-mkmlizer: Saving duration: 2.311s
google-gemma-2-27b-it-v9-mkmlizer: Processed model ChaiML/gpt2_xl_pairwise_89m_step_347634 in 14.064s
google-gemma-2-27b-it-v9-mkmlizer: creating bucket guanaco-reward-models
google-gemma-2-27b-it-v9-mkmlizer: Bucket 's3://guanaco-reward-models/' created
google-gemma-2-27b-it-v9-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/google-gemma-2-27b-it-v9_reward
google-gemma-2-27b-it-v9-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/google-gemma-2-27b-it-v9_reward/special_tokens_map.json
google-gemma-2-27b-it-v9-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/google-gemma-2-27b-it-v9_reward/config.json
google-gemma-2-27b-it-v9-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/google-gemma-2-27b-it-v9_reward/tokenizer_config.json
google-gemma-2-27b-it-v9-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/google-gemma-2-27b-it-v9_reward/merges.txt
google-gemma-2-27b-it-v9-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/google-gemma-2-27b-it-v9_reward/vocab.json
google-gemma-2-27b-it-v9-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/google-gemma-2-27b-it-v9_reward/tokenizer.json
google-gemma-2-27b-it-v9-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/google-gemma-2-27b-it-v9_reward/reward.tensors
Job google-gemma-2-27b-it-v9-mkmlizer completed after 244.97s with status: succeeded
Stopping job with name google-gemma-2-27b-it-v9-mkmlizer
Pipeline stage MKMLizer completed in 246.90s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.27s
Running pipeline stage ISVCDeployer
Creating inference service google-gemma-2-27b-it-v9
Waiting for inference service google-gemma-2-27b-it-v9 to be ready
Inference service google-gemma-2-27b-it-v9 ready after 133.27548265457153s
Pipeline stage ISVCDeployer completed in 134.91s
Running pipeline stage StressChecker
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 4.137565612792969s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.099836826324463s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.996016502380371s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 3.0999693870544434s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.927319049835205s
5 requests
0 failed requests
5th percentile: 2.0167805671691896
10th percentile: 2.0375446319580077
20th percentile: 2.0790727615356444
30th percentile: 2.2653332710266114
40th percentile: 2.5963261604309085
50th percentile: 2.927319049835205
60th percentile: 2.9963791847229
70th percentile: 3.065439319610596
80th percentile: 3.3074886322021486
90th percentile: 3.7225271224975587
95th percentile: 3.9300463676452635
99th percentile: 4.096061763763427
mean time: 2.8521414756774903
Pipeline stage StressChecker completed in 16.57s
google-gemma-2-27b-it_v9 status is now deployed due to DeploymentManager action
google-gemma-2-27b-it_v9 status is now inactive due to auto deactivation removed underperforming models
admin requested tearing down of google-gemma-2-27b-it_v9
Running pipeline stage ISVCDeleter
Checking if service google-gemma-2-27b-it-v9 is running
Tearing down inference service google-gemma-2-27b-it-v9
Service google-gemma-2-27b-it-v9 has been torndown
Pipeline stage ISVCDeleter completed in 5.17s
Running pipeline stage MKMLModelDeleter
Cleaning model data from S3
Cleaning model data from model cache
Deleting key google-gemma-2-27b-it-v9/config.json from bucket guanaco-mkml-models
Deleting key google-gemma-2-27b-it-v9/flywheel_model.0.safetensors from bucket guanaco-mkml-models
Deleting key google-gemma-2-27b-it-v9/flywheel_model.1.safetensors from bucket guanaco-mkml-models
Deleting key google-gemma-2-27b-it-v9/flywheel_model.2.safetensors from bucket guanaco-mkml-models
Deleting key google-gemma-2-27b-it-v9/special_tokens_map.json from bucket guanaco-mkml-models
Deleting key google-gemma-2-27b-it-v9/tokenizer.json from bucket guanaco-mkml-models
Deleting key google-gemma-2-27b-it-v9/tokenizer.model from bucket guanaco-mkml-models
Deleting key google-gemma-2-27b-it-v9/tokenizer_config.json from bucket guanaco-mkml-models
Cleaning model data from model cache
Deleting key google-gemma-2-27b-it-v9_reward/config.json from bucket guanaco-reward-models
Deleting key google-gemma-2-27b-it-v9_reward/merges.txt from bucket guanaco-reward-models
Deleting key google-gemma-2-27b-it-v9_reward/reward.tensors from bucket guanaco-reward-models
Deleting key google-gemma-2-27b-it-v9_reward/special_tokens_map.json from bucket guanaco-reward-models
Deleting key google-gemma-2-27b-it-v9_reward/tokenizer.json from bucket guanaco-reward-models
Deleting key google-gemma-2-27b-it-v9_reward/tokenizer_config.json from bucket guanaco-reward-models
Deleting key google-gemma-2-27b-it-v9_reward/vocab.json from bucket guanaco-reward-models
Pipeline stage MKMLModelDeleter completed in 9.46s
google-gemma-2-27b-it_v9 status is now torndown due to DeploymentManager action

Usage Metrics

Latency Metrics