submission_id: chaiml-sao10k-l3-rp-v3-3_v37
developer_uid: chai_backend_admin
status: inactive
model_repo: ChaiML/sao10k-l3-rp-v3-3
reward_repo: ChaiML/gpt2_xl_pairwise_89m_step_347634
generation_params: {'temperature': 0.95, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|end_header_id|>,', '<|eot_id|>,', '\n\n{user_name}'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n{bot_name}'s Persona: {memory}\n\n", 'prompt_template': '{prompt}<|eot_id|>', 'bot_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}: {message}<|eot_id|>', 'user_template': '<|start_header_id|>user<|end_header_id|>\n\n{user_name}: {message}<|eot_id|>', 'response_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}:', 'truncate_by_message': False}
reward_formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
timestamp: 2024-07-12T18:20:36+00:00
model_name: chaiml-sao10k-l3-rp-v3-3_v37
model_group: ChaiML/sao10k-l3-rp-v3-3
num_battles: 1842812
num_wins: 1013526
celo_rating: 1234.03
alignment_score: None
alignment_samples: 0
propriety_score: 0.7250930159584006
propriety_total_count: 178464.0
submission_type: basic
model_architecture: LlamaForCausalLM
model_num_parameters: 8030261248.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
display_name: chaiml-sao10k-l3-rp-v3-3_v37
ineligible_reason: None
language_model: ChaiML/sao10k-l3-rp-v3-3
model_size: 8B
reward_model: ChaiML/gpt2_xl_pairwise_89m_step_347634
us_pacific_date: 2024-07-12
win_ratio: 0.5499888214315948
preference_data_url: None
Resubmit model
Running pipeline stage MKMLizer
Starting job with name chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer
Waiting for job on chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer to finish
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: ║ _____ __ __ ║
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: ║ /___/ ║
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: ║ ║
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: ║ Version: 0.9.5.post2 ║
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: ║ https://mk1.ai ║
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: ║ ║
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: ║ The license key for the current software has been verified as ║
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: ║ belonging to: ║
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: ║ ║
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: ║ Chai Research Corp. ║
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: ║ ║
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: Downloaded to shared memory in 25.117s
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpxfastjpj, device:0
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: lm_head.weight torch.Size([139542528])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.embed_tokens.weight torch.Size([139542528])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.0.input_layernorm.weight torch.Size([4096])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.0.mlp.down_proj.weight torch.Size([11927552])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.0.mlp.up_gate_proj.weight torch.Size([23855104])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.0.post_attention_layernorm.weight torch.Size([4096])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.0.self_attn.o_proj.weight torch.Size([3407872])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.0.self_attn.qkv_proj.weight torch.Size([5111808])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.1.input_layernorm.weight torch.Size([4096])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.1.mlp.down_proj.weight torch.Size([11927552])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.1.mlp.up_gate_proj.weight torch.Size([23855104])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.1.post_attention_layernorm.weight torch.Size([4096])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.1.self_attn.o_proj.weight torch.Size([3407872])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.1.self_attn.qkv_proj.weight torch.Size([5111808])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.10.input_layernorm.weight torch.Size([4096])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.10.mlp.down_proj.weight torch.Size([11927552])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.10.mlp.up_gate_proj.weight torch.Size([23855104])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.10.post_attention_layernorm.weight torch.Size([4096])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.10.self_attn.o_proj.weight torch.Size([3407872])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.10.self_attn.qkv_proj.weight torch.Size([5111808])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.11.input_layernorm.weight torch.Size([4096])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.11.mlp.down_proj.weight torch.Size([11927552])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.11.mlp.up_gate_proj.weight torch.Size([23855104])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.11.post_attention_layernorm.weight torch.Size([4096])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.11.self_attn.o_proj.weight torch.Size([3407872])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.11.self_attn.qkv_proj.weight torch.Size([5111808])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.12.input_layernorm.weight torch.Size([4096])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.12.mlp.down_proj.weight torch.Size([11927552])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.12.mlp.up_gate_proj.weight torch.Size([23855104])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.12.post_attention_layernorm.weight torch.Size([4096])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.12.self_attn.o_proj.weight torch.Size([3407872])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.12.self_attn.qkv_proj.weight torch.Size([5111808])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.13.input_layernorm.weight torch.Size([4096])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.13.mlp.down_proj.weight torch.Size([11927552])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.13.mlp.up_gate_proj.weight torch.Size([23855104])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.13.post_attention_layernorm.weight torch.Size([4096])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.13.self_attn.o_proj.weight torch.Size([3407872])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.13.self_attn.qkv_proj.weight torch.Size([5111808])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.14.input_layernorm.weight torch.Size([4096])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.14.mlp.down_proj.weight torch.Size([11927552])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.14.mlp.up_gate_proj.weight torch.Size([23855104])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.14.post_attention_layernorm.weight torch.Size([4096])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.14.self_attn.o_proj.weight torch.Size([3407872])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.14.self_attn.qkv_proj.weight torch.Size([5111808])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.15.input_layernorm.weight torch.Size([4096])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.15.mlp.down_proj.weight torch.Size([11927552])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.15.mlp.up_gate_proj.weight torch.Size([23855104])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.15.post_attention_layernorm.weight torch.Size([4096])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.15.self_attn.o_proj.weight torch.Size([3407872])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.15.self_attn.qkv_proj.weight torch.Size([5111808])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.16.input_layernorm.weight torch.Size([4096])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.16.mlp.down_proj.weight torch.Size([11927552])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.16.mlp.up_gate_proj.weight torch.Size([23855104])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.16.post_attention_layernorm.weight torch.Size([4096])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.16.self_attn.o_proj.weight torch.Size([3407872])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.16.self_attn.qkv_proj.weight torch.Size([5111808])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.17.input_layernorm.weight torch.Size([4096])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.17.mlp.down_proj.weight torch.Size([11927552])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.17.mlp.up_gate_proj.weight torch.Size([23855104])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.17.post_attention_layernorm.weight torch.Size([4096])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.17.self_attn.o_proj.weight torch.Size([3407872])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.17.self_attn.qkv_proj.weight torch.Size([5111808])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.18.input_layernorm.weight torch.Size([4096])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.18.mlp.down_proj.weight torch.Size([11927552])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.18.mlp.up_gate_proj.weight torch.Size([23855104])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.18.post_attention_layernorm.weight torch.Size([4096])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.18.self_attn.o_proj.weight torch.Size([3407872])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.18.self_attn.qkv_proj.weight torch.Size([5111808])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.19.input_layernorm.weight torch.Size([4096])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.19.mlp.down_proj.weight torch.Size([11927552])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.19.mlp.up_gate_proj.weight torch.Size([23855104])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.19.post_attention_layernorm.weight torch.Size([4096])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.19.self_attn.o_proj.weight torch.Size([3407872])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.19.self_attn.qkv_proj.weight torch.Size([5111808])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.2.input_layernorm.weight torch.Size([4096])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.2.mlp.down_proj.weight torch.Size([11927552])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.2.mlp.up_gate_proj.weight torch.Size([23855104])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.2.post_attention_layernorm.weight torch.Size([4096])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.2.self_attn.o_proj.weight torch.Size([3407872])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.2.self_attn.qkv_proj.weight torch.Size([5111808])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: model.layers.20.input_layernorm.weight torch.Size([4096])
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: | 179/291 [00:07<00:02, 55.60it/s] Loading 0: 62%|██████▏ | 181/291 [00:07<00:01, 55.60it/s] Loading 0: 63%|██████▎ | 182/291 [00:07<00:01, 55.60it/s] Loading 0: 63%|██████▎ | 183/291 [00:07<00:01, 55.60it/s] Loading 0: 63%|██████▎ | 184/291 [00:07<00:01, 67.79it/s] Loading 0: 64%|██████▎ | 185/291 [00:07<00:01, 67.79it/s] Loading 0: 64%|██████▍ | 186/291 [00:07<00:01, 67.79it/s] Loading 0: 65%|██████▍ | 188/291 [00:07<00:01, 67.79it/s] Loading 0: 65%|██████▌ | 190/291 [00:07<00:01, 67.79it/s] Loading 0: 66%|██████▌ | 191/291 [00:07<00:01, 67.79it/s] Loading 0: 66%|██████▌ | 192/291 [00:07<00:01, 67.79it/s] Loading 0: 67%|██████▋ | 194/291 [00:07<00:01, 67.79it/s] Loading 0: 67%|██████▋ | 195/291 [00:07<00:01, 70.65it/s] Loading 0: 67%|██████▋ | 195/291 [00:07<00:01, 70.65it/s] Loading 0: 68%|██████▊ | 197/291 [00:07<00:01, 70.65it/s] Loading 0: 68%|██████▊ | 199/291 [00:07<00:01, 70.65it/s] Loading 0: 69%|██████▊ | 200/291 [00:07<00:01, 70.65it/s] Loading 0: 69%|██████▉ | 201/291 [00:07<00:01, 70.65it/s] Loading 0: 70%|██████▉ | 203/291 [00:07<00:01, 70.65it/s] Loading 0: 70%|███████ | 204/291 [00:07<00:01, 70.65it/s] Loading 0: 71%|███████ | 206/291 [00:07<00:01, 70.65it/s] Loading 0: 71%|███████▏ | 208/291 [00:07<00:01, 70.65it/s] Loading 0: 72%|███████▏ | 209/291 [00:07<00:01, 70.65it/s] Loading 0: 72%|███████▏ | 210/291 [00:07<00:01, 70.65it/s] Loading 0: 73%|███████▎ | 211/291 [00:07<00:00, 85.11it/s] Loading 0: 73%|███████▎ | 212/291 [00:07<00:00, 85.11it/s] Loading 0: 73%|███████▎ | 213/291 [00:07<00:00, 85.11it/s] Loading 0: 74%|███████▍ | 215/291 [00:07<00:00, 85.11it/s] Loading 0: 75%|███████▍ | 217/291 [00:07<00:00, 85.11it/s] Loading 0: 75%|███████▍ | 218/291 [00:07<00:00, 85.11it/s] Loading 0: 75%|███████▌ | 219/291 [00:07<00:00, 85.11it/s] Loading 0: 76%|███████▌ | 221/291 [00:08<00:00, 85.11it/s] Loading 0: 76%|███████▋ | 222/291 [00:08<00:00, 88.94it/s] Loading 0: 76%|███████▋ | 222/291 [00:08<00:00, 88.94it/s] Loading 0: 77%|███████▋ | 224/291 [00:08<00:00, 88.94it/s] Loading 0: 78%|███████▊ | 226/291 [00:08<00:00, 88.94it/s] Loading 0: 78%|███████▊ | 227/291 [00:08<00:00, 88.94it/s] Loading 0: 78%|███████▊ | 228/291 [00:08<00:00, 88.94it/s] Loading 0: 79%|███████▉ | 230/291 [00:08<00:00, 88.94it/s] Loading 0: 79%|███████▉ | 231/291 [00:08<00:00, 88.94it/s] Loading 0: 80%|████████ | 233/291 [00:08<00:00, 88.94it/s] Loading 0: 81%|████████ | 235/291 [00:08<00:00, 88.94it/s] Loading 0: 81%|████████ | 236/291 [00:08<00:00, 88.94it/s] Loading 0: 81%|████████▏ | 237/291 [00:08<00:00, 88.94it/s] Loading 0: 82%|████████▏ | 238/291 [00:08<00:00, 104.18it/s] Loading 0: 82%|████████▏ | 239/291 [00:08<00:00, 104.18it/s] Loading 0: 82%|████████▏ | 240/291 [00:08<00:00, 104.18it/s] Loading 0: 83%|████████▎ | 242/291 [00:08<00:00, 104.18it/s] Loading 0: 84%|████████▍ | 244/291 [00:08<00:00, 104.18it/s] Loading 0: 84%|████████▍ | 245/291 [00:08<00:00, 104.18it/s] Loading 0: 85%|████████▍ | 246/291 [00:08<00:00, 104.18it/s] Loading 0: 85%|████████▌ | 248/291 [00:08<00:00, 104.18it/s] Loading 0: 86%|████████▌ | 249/291 [00:08<00:00, 104.18it/s] Loading 0: 86%|████████▋ | 251/291 [00:08<00:00, 107.66it/s] Loading 0: 86%|████████▋ | 251/291 [00:08<00:00, 107.66it/s] Loading 0: 87%|████████▋ | 253/291 [00:08<00:00, 107.66it/s] Loading 0: 87%|████████▋ | 254/291 [00:08<00:00, 107.66it/s] Loading 0: 88%|████████▊ | 255/291 [00:08<00:00, 107.66it/s] Loading 0: 88%|████████▊ | 257/291 [00:08<00:00, 107.66it/s] Loading 0: 89%|████████▊ | 258/291 [00:08<00:00, 107.66it/s] Loading 0: 89%|████████▉ | 260/291 [00:08<00:00, 107.66it/s] Loading 0: 90%|█████████ | 262/291 [00:08<00:00, 107.66it/s] Loading 0: 90%|█████████ | 263/291 [00:08<00:00, 107.66it/s] Loading 0: 91%|█████████ | 264/291 [00:08<00:00, 107.66it/s] Loading 0: 91%|█████████ | 265/291 [00:08<00:00, 113.17it/s] Loading 0: 91%|█████████▏| 266/291 [00:08<00:00, 113.17it/s] Loading 0: 92%|█████████▏| 267/291 [00:08<00:00, 113.17it/s] Loading 0: 92%|█████████▏| 269/291 [00:08<00:00, 113.17it/s] Loading 0: 93%|█████████▎| 271/291 [00:08<00:00, 113.17it/s] Loading 0: 93%|█████████▎| 272/291 [00:08<00:00, 113.17it/s] Loading 0: 94%|█████████▍| 273/291 [00:08<00:00, 113.17it/s] Loading 0: 95%|█████████▍| 275/291 [00:08<00:00, 113.17it/s] Loading 0: 95%|█████████▍| 276/291 [00:08<00:00, 113.17it/s] Loading 0: 96%|█████████▌| 278/291 [00:08<00:00, 60.00it/s] Loading 0: 96%|█████████▌| 278/291 [00:08<00:00, 60.00it/s] Loading 0: 96%|█████████▌| 280/291 [00:08<00:00, 60.00it/s] Loading 0: 97%|█████████▋| 281/291 [00:08<00:00, 60.00it/s] Loading 0: 97%|█████████▋| 282/291 [00:08<00:00, 60.00it/s] Loading 0: 98%|█████████▊| 284/291 [00:08<00:00, 60.00it/s] Loading 0: 98%|█████████▊| 285/291 [00:08<00:00, 60.00it/s] Loading 0: 99%|█████████▊| 287/291 [00:08<00:00, 60.00it/s] Loading 0: 99%|█████████▉| 289/291 [00:08<00:00, 60.00it/s] Loading 0: 100%|█████████▉| 290/291 [00:08<00:00, 60.00it/s] Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: quantized model in 28.721s
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: Processed model ChaiML/sao10k-l3-rp-v3-3 in 53.838s
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: creating bucket guanaco-mkml-models
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-sao10k-l3-rp-v3-3-v37
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-sao10k-l3-rp-v3-3-v37/config.json
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-sao10k-l3-rp-v3-3-v37/special_tokens_map.json
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-sao10k-l3-rp-v3-3-v37/tokenizer_config.json
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-sao10k-l3-rp-v3-3-v37/tokenizer.json
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-sao10k-l3-rp-v3-3-v37/flywheel_model.0.safetensors
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: loading reward model from ChaiML/gpt2_xl_pairwise_89m_step_347634
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:950: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: warnings.warn(
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:778: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: warnings.warn(
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: Downloading shards: 0%| | 0/2 [00:00<?, ?it/s] Downloading shards: 50%|█████ | 1/2 [00:06<00:06, 6.96s/it] Downloading shards: 100%|██████████| 2/2 [00:09<00:00, 4.28s/it] Downloading shards: 100%|██████████| 2/2 [00:09<00:00, 4.68s/it]
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: Loading checkpoint shards: 0%| | 0/2 [00:00<?, ?it/s] Loading checkpoint shards: 50%|█████ | 1/2 [00:00<00:00, 1.65it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 2.67it/s] Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00, 2.44it/s]
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: Saving duration: 2.149s
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: Processed model ChaiML/gpt2_xl_pairwise_89m_step_347634 in 14.131s
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: creating bucket guanaco-reward-models
chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/chaiml-sao10k-l3-rp-v3-3-v37_reward/reward.tensors
Job chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer completed after 185.06s with status: succeeded
Stopping job with name chaiml-sao10k-l3-rp-v3-3-v37-mkmlizer
Pipeline stage MKMLizer completed in 187.59s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.41s
Running pipeline stage ISVCDeployer
Creating inference service chaiml-sao10k-l3-rp-v3-3-v37
Waiting for inference service chaiml-sao10k-l3-rp-v3-3-v37 to be ready
Inference service chaiml-sao10k-l3-rp-v3-3-v37 ready after 143.71484375s
Pipeline stage ISVCDeployer completed in 146.39s
Running pipeline stage StressChecker
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.885317087173462s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.8839225769042969s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.7596306800842285s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 2.0662636756896973s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 1.9164326190948486s
5 requests
0 failed requests
5th percentile: 1.784489059448242
10th percentile: 1.809347438812256
20th percentile: 1.8590641975402833
30th percentile: 1.8904245853424073
40th percentile: 1.903428602218628
50th percentile: 1.9164326190948486
60th percentile: 1.976365041732788
70th percentile: 2.0362974643707275
80th percentile: 2.2300743579864504
90th percentile: 2.557695722579956
95th percentile: 2.721506404876709
99th percentile: 2.8525549507141115
mean time: 2.1023133277893065
Pipeline stage StressChecker completed in 14.54s
chaiml-sao10k-l3-rp-v3-3_v37 status is now deployed due to DeploymentManager action
chaiml-sao10k-l3-rp-v3-3_v37 status is now inactive due to auto deactivation removed underperforming models
chaiml-sao10k-l3-rp-v3-3_v37 status is now deployed due to admin request

Usage Metrics

Latency Metrics