submission_id: google-gemma-2-27b-it_v1
developer_uid: chai_backend_admin
status: inactive
model_repo: google/gemma-2-27b-it
reward_repo: ChaiML/gpt2_xl_pairwise_89m_step_347634
generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 50, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
reward_formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
timestamp: 2024-07-10T21:31:19+00:00
model_name: google-gemma-2-27b-it_v1
model_group: google/gemma-2-27b-it
num_battles: 38089
num_wins: 17943
celo_rating: 1161.15
alignment_score: None
alignment_samples: 0
propriety_score: 0.7412064062266128
propriety_total_count: 6681.0
submission_type: basic
model_architecture: Gemma2ForCausalLM
model_num_parameters: 28731935232.0
best_of: 4
max_input_tokens: 512
max_output_tokens: 64
display_name: google-gemma-2-27b-it_v1
ineligible_reason: None
language_model: google/gemma-2-27b-it
model_size: 29B
reward_model: ChaiML/gpt2_xl_pairwise_89m_step_347634
us_pacific_date: 2024-07-10
win_ratio: 0.4710808894956549
preference_data_url: None
Resubmit model
Running pipeline stage MKMLizer
Starting job with name google-gemma-2-27b-it-v1-mkmlizer
Waiting for job on google-gemma-2-27b-it-v1-mkmlizer to finish
HTTP Request: %s %s "%s %d %s"
HTTP Request: %s %s "%s %d %s"
HTTP Request: %s %s "%s %d %s"
HTTP Request: %s %s "%s %d %s"
google-gemma-2-27b-it-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
google-gemma-2-27b-it-v1-mkmlizer: ║ _____ __ __ ║
google-gemma-2-27b-it-v1-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
google-gemma-2-27b-it-v1-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
google-gemma-2-27b-it-v1-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
google-gemma-2-27b-it-v1-mkmlizer: ║ /___/ ║
google-gemma-2-27b-it-v1-mkmlizer: ║ ║
google-gemma-2-27b-it-v1-mkmlizer: ║ Version: 0.9.5.post1 ║
google-gemma-2-27b-it-v1-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
google-gemma-2-27b-it-v1-mkmlizer: ║ https://mk1.ai ║
google-gemma-2-27b-it-v1-mkmlizer: ║ ║
google-gemma-2-27b-it-v1-mkmlizer: ║ The license key for the current software has been verified as ║
google-gemma-2-27b-it-v1-mkmlizer: ║ belonging to: ║
google-gemma-2-27b-it-v1-mkmlizer: ║ ║
google-gemma-2-27b-it-v1-mkmlizer: ║ Chai Research Corp. ║
google-gemma-2-27b-it-v1-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
google-gemma-2-27b-it-v1-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
google-gemma-2-27b-it-v1-mkmlizer: ║ ║
google-gemma-2-27b-it-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
google-gemma-2-27b-it-v1-mkmlizer: Downloaded to shared memory in 100.014s
google-gemma-2-27b-it-v1-mkmlizer: quantizing model to /dev/shm/model_cache
google-gemma-2-27b-it-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
google-gemma-2-27b-it-v1-mkmlizer: quantized model in 82.509s
google-gemma-2-27b-it-v1-mkmlizer: Processed model google/gemma-2-27b-it in 182.524s
google-gemma-2-27b-it-v1-mkmlizer: creating bucket guanaco-mkml-models
google-gemma-2-27b-it-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
google-gemma-2-27b-it-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/google-gemma-2-27b-it-v1
google-gemma-2-27b-it-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/google-gemma-2-27b-it-v1/config.json
google-gemma-2-27b-it-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/google-gemma-2-27b-it-v1/tokenizer_config.json
google-gemma-2-27b-it-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/google-gemma-2-27b-it-v1/special_tokens_map.json
google-gemma-2-27b-it-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/google-gemma-2-27b-it-v1/tokenizer.json
google-gemma-2-27b-it-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.model s3://guanaco-mkml-models/google-gemma-2-27b-it-v1/tokenizer.model
google-gemma-2-27b-it-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.2.safetensors s3://guanaco-mkml-models/google-gemma-2-27b-it-v1/flywheel_model.2.safetensors
google-gemma-2-27b-it-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/google-gemma-2-27b-it-v1/flywheel_model.1.safetensors
google-gemma-2-27b-it-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/google-gemma-2-27b-it-v1/flywheel_model.0.safetensors
google-gemma-2-27b-it-v1-mkmlizer: loading reward model from ChaiML/gpt2_xl_pairwise_89m_step_347634
google-gemma-2-27b-it-v1-mkmlizer: Loading 0: 0%| | 0/508 [00:00<?, ?it/s] Loading 0: 1%| | 4/508 [00:00<00:13, 36.79it/s] Loading 0: 3%|▎ | 14/508 [00:00<00:08, 61.21it/s] Loading 0: 4%|▍ | 20/508 [00:00<00:08, 57.82it/s] Loading 0: 6%|▌ | 28/508 [00:00<00:15, 31.36it/s] Loading 0: 6%|▋ | 33/508 [00:00<00:15, 31.43it/s] Loading 0: 7%|▋ | 37/508 [00:01<00:14, 33.02it/s] Loading 0: 9%|▉ | 47/508 [00:01<00:10, 45.39it/s] Loading 0: 10%|█ | 53/508 [00:01<00:09, 47.56it/s] Loading 0: 12%|█▏ | 59/508 [00:01<00:09, 48.43it/s] Loading 0: 13%|█▎ | 68/508 [00:01<00:07, 55.01it/s] Loading 0: 15%|█▍ | 74/508 [00:01<00:15, 28.77it/s] Loading 0: 16%|█▌ | 81/508 [00:02<00:12, 33.84it/s] Loading 0: 17%|█▋ | 87/508 [00:02<00:11, 38.04it/s] Loading 0: 19%|█▉ | 97/508 [00:02<00:08, 47.93it/s] Loading 0: 20%|██ | 103/508 [00:02<00:08, 49.11it/s] Loading 0: 21%|██▏ | 109/508 [00:02<00:08, 49.31it/s] Loading 0: 23%|██▎ | 118/508 [00:03<00:12, 32.21it/s] Loading 0: 24%|██▍ | 124/508 [00:03<00:10, 35.71it/s] Loading 0: 25%|██▌ | 129/508 [00:03<00:09, 37.92it/s] Loading 0: 27%|██▋ | 136/508 [00:03<00:08, 42.53it/s] Loading 0: 29%|██▊ | 146/508 [00:03<00:06, 52.16it/s] Loading 0: 30%|██▉ | 152/508 [00:03<00:06, 52.67it/s] Loading 0: 31%|███ | 158/508 [00:03<00:06, 51.73it/s] Loading 0: 34%|███▎ | 171/508 [00:04<00:08, 38.39it/s] Loading 0: 35%|███▍ | 176/508 [00:04<00:09, 36.73it/s] Loading 0: 36%|███▌ | 181/508 [00:04<00:09, 35.60it/s] Loading 0: 38%|███▊ | 191/508 [00:04<00:06, 45.33it/s] Loading 0: 38%|███▊ | 191/508 [00:24<00:06, 45.33it/s] Loading 0: 38%|███▊ | 192/508 [00:24<05:41, 1.08s/it] Loading 0: 40%|███▉ | 202/508 [00:24<03:10, 1.60it/s] Loading 0: 42%|████▏ | 211/508 [00:25<02:00, 2.45it/s] Loading 0: 43%|████▎ | 218/508 [00:25<01:30, 3.20it/s] Loading 0: 44%|████▍ | 224/508 [00:25<01:07, 4.23it/s] Loading 0: 46%|████▌ | 234/508 [00:25<00:41, 6.61it/s] Loading 0: 47%|████▋ | 241/508 [00:25<00:30, 8.76it/s] Loading 0: 49%|████▉ | 248/508 [00:26<00:23, 11.28it/s] Loading 0: 50%|█████ | 256/508 [00:26<00:16, 15.27it/s] Loading 0: 52%|█████▏ | 262/508 [00:26<00:16, 14.92it/s] Loading 0: 53%|█████▎ | 268/508 [00:26<00:12, 18.54it/s] Loading 0: 55%|█████▍ | 278/508 [00:26<00:08, 26.47it/s] Loading 0: 56%|█████▌ | 284/508 [00:27<00:07, 30.33it/s] Loading 0: 57%|█████▋ | 290/508 [00:27<00:06, 33.90it/s] Loading 0: 59%|█████▉ | 300/508 [00:27<00:04, 43.57it/s] Loading 0: 60%|██████ | 307/508 [00:27<00:04, 47.15it/s] Loading 0: 62%|██████▏ | 314/508 [00:27<00:05, 32.47it/s] Loading 0: 63%|██████▎ | 319/508 [00:27<00:05, 31.88it/s] Loading 0: 64%|██████▍ | 324/508 [00:28<00:05, 32.04it/s] Loading 0: 66%|██████▌ | 334/508 [00:28<00:04, 42.54it/s] Loading 0: 68%|██████▊ | 344/508 [00:28<00:03, 51.46it/s] Loading 0: 69%|██████▉ | 351/508 [00:28<00:02, 53.44it/s] Loading 0: 71%|███████ | 359/508 [00:28<00:04, 33.82it/s] Loading 0: 72%|███████▏ | 364/508 [00:28<00:04, 35.99it/s] Loading 0: 73%|███████▎ | 369/508 [00:29<00:03, 34.79it/s] Loading 0: 74%|███████▍ | 378/508 [00:29<00:03, 43.29it/s] Loading 0: 76%|███████▋ | 388/508 [00:29<00:02, 52.38it/s] Loading 0: 78%|███████▊ | 395/508 [00:29<00:02, 54.38it/s] Loading 0: 79%|███████▉ | 402/508 [00:29<00:01, 54.28it/s] Loading 0: 80%|████████ | 408/508 [00:30<00:03, 31.69it/s] Loading 0: 81%|████████▏ | 413/508 [00:30<00:03, 30.92it/s] Loading 0: 83%|████████▎ | 422/508 [00:30<00:02, 38.92it/s] Loading 0: 85%|████████▌ | 432/508 [00:30<00:01, 48.36it/s] Loading 0: 86%|████████▌ | 438/508 [00:30<00:01, 50.05it/s] Loading 0: 87%|████████▋ | 444/508 [00:30<00:01, 50.23it/s] Loading 0: 89%|████████▉ | 451/508 [00:50<00:01, 50.23it/s] Loading 0: 89%|████████▉ | 452/508 [00:50<00:47, 1.18it/s] Loading 0: 90%|████████▉ | 457/508 [00:51<00:34, 1.48it/s] Loading 0: 91%|█████████ | 462/508 [00:51<00:23, 1.94it/s] Loading 0: 92%|█████████▏| 466/508 [00:51<00:17, 2.44it/s] Loading 0: 94%|█████████▎| 476/508 [00:51<00:07, 4.25it/s] Loading 0: 95%|█████████▍| 482/508 [00:51<00:04, 5.67it/s] Loading 0: 96%|█████████▌| 488/508 [00:52<00:02, 7.56it/s] Loading 0: 98%|█████████▊| 497/508 [00:52<00:00, 11.40it/s] Loading 0: 99%|█████████▉| 503/508 [00:52<00:00, 12.03it/s] /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:950: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
google-gemma-2-27b-it-v1-mkmlizer: warnings.warn(
google-gemma-2-27b-it-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:778: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
google-gemma-2-27b-it-v1-mkmlizer: warnings.warn(
google-gemma-2-27b-it-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
google-gemma-2-27b-it-v1-mkmlizer: warnings.warn(
google-gemma-2-27b-it-v1-mkmlizer: Downloading shards: 0%| | 0/2 [00:00<?, ?it/s] Downloading shards: 50%|█████ | 1/2 [00:09<00:09, 9.33s/it] Downloading shards: 100%|██████████| 2/2 [00:13<00:00, 6.13s/it] Downloading shards: 100%|██████████| 2/2 [00:13<00:00, 6.61s/it]
google-gemma-2-27b-it-v1-mkmlizer: Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s] Loading checkpoint shards: 50%|█████ | 1/2 [00:00<00:00, 1.56it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 2.58it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 2.35it/s]
google-gemma-2-27b-it-v1-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
google-gemma-2-27b-it-v1-mkmlizer: Saving duration: 2.175s
google-gemma-2-27b-it-v1-mkmlizer: Processed model ChaiML/gpt2_xl_pairwise_89m_step_347634 in 19.095s
google-gemma-2-27b-it-v1-mkmlizer: creating bucket guanaco-reward-models
google-gemma-2-27b-it-v1-mkmlizer: Bucket 's3://guanaco-reward-models/' created
google-gemma-2-27b-it-v1-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/google-gemma-2-27b-it-v1_reward
google-gemma-2-27b-it-v1-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/google-gemma-2-27b-it-v1_reward/config.json
google-gemma-2-27b-it-v1-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/google-gemma-2-27b-it-v1_reward/special_tokens_map.json
google-gemma-2-27b-it-v1-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/google-gemma-2-27b-it-v1_reward/tokenizer_config.json
google-gemma-2-27b-it-v1-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/google-gemma-2-27b-it-v1_reward/merges.txt
google-gemma-2-27b-it-v1-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/google-gemma-2-27b-it-v1_reward/tokenizer.json
google-gemma-2-27b-it-v1-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/google-gemma-2-27b-it-v1_reward/vocab.json
Job google-gemma-2-27b-it-v1-mkmlizer completed after 468.34s with status: succeeded
Stopping job with name google-gemma-2-27b-it-v1-mkmlizer
Pipeline stage MKMLizer completed in 469.39s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.29s
Running pipeline stage ISVCDeployer
Creating inference service google-gemma-2-27b-it-v1
Waiting for inference service google-gemma-2-27b-it-v1 to be ready
Inference service google-gemma-2-27b-it-v1 ready after 60.74240159988403s
Pipeline stage ISVCDeployer completed in 66.98s
Running pipeline stage StressChecker
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 3.663112163543701s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 3.0039141178131104s
%s, retrying in %s seconds...
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.5446019172668457s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.0280187129974365s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.1984548568725586s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 3.134810447692871s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.9138879776000977s
5 requests
0 failed requests
5th percentile: 1.9367141246795654
10th percentile: 1.9595402717590331
20th percentile: 2.0051925659179686
30th percentile: 2.0621059417724608
40th percentile: 2.1302803993225097
50th percentile: 2.1984548568725586
60th percentile: 2.3369136810302735
70th percentile: 2.475372505187988
80th percentile: 2.662643623352051
90th percentile: 2.898727035522461
95th percentile: 3.016768741607666
99th percentile: 3.11120210647583
mean time: 2.363954782485962
Pipeline stage StressChecker completed in 27.80s
google-gemma-2-27b-it_v1 status is now deployed due to DeploymentManager action
google-gemma-2-27b-it_v1 status is now inactive due to auto deactivation removed underperforming models

Usage Metrics

Latency Metrics