mistralai-mixtral-8x7b-_3473

submission_id: mistralai-mixtral-8x7b-_3473_v66

developer_uid: robert_irvine

status: inactive

model_repo: mistralai/Mixtral-8x7B-Instruct-v0.1

reward_repo: ChaiML/gpt2_xl_pairwise_89m_step_347634

generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 50, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n', '</s>', '<|user|>', '###'], 'max_input_tokens': 512, 'best_of': 4, 'reward_max_token_input': 512, 'max_output_tokens': 64}

formatter: {'memory_template': '<s>[INST] This is an entertaining conversation. You are {bot_name} who has the persona: {memory}.\nEngage in a chat with {user_name} while staying in character. Try to flirt with {user_name}. Engage in *roleplay* actions. Describe the scene dramatically. \n', 'prompt_template': '{prompt}\n', 'bot_template': '{bot_name}: {message}</s>', 'user_template': '[INST] {user_name}: {message} [/INST]', 'response_template': '{bot_name}:', 'truncate_by_message': False}

reward_formatter: {'memory_template': '""', 'prompt_template': '""', 'bot_template': 'Bot: {message}\n', 'user_template': 'User: {message}\n', 'response_template': 'Bot:', 'truncate_by_message': False}

timestamp: 2024-07-10T18:03:17+00:00

model_name: mistralai-mixtral-8x7b-_3473_v66

model_group: mistralai/Mixtral-8x7B-I

num_battles: 33406

num_wins: 16949

celo_rating: 1177.34

alignment_score: None

alignment_samples: 0

propriety_score: 0.7387023034992087

propriety_total_count: 5687.0

submission_type: basic

model_architecture: MixtralForCausalLM

model_num_parameters: 46702792704.0

best_of: 4

max_input_tokens: 512

max_output_tokens: 64

display_name: mistralai-mixtral-8x7b-_3473_v66

ineligible_reason: None

language_model: mistralai/Mixtral-8x7B-Instruct-v0.1

model_size: 47B

reward_model: ChaiML/gpt2_xl_pairwise_89m_step_347634

us_pacific_date: 2024-07-10

win_ratio: 0.5073639465964198

preference_data_url: None

Resubmit model

Reference Models

Running pipeline stage MKMLizer
Starting job with name mistralai-mixtral-8x7b-3473-v66-mkmlizer
Waiting for job on mistralai-mixtral-8x7b-3473-v66-mkmlizer to finish
Failed to get response for submission blend_pefis_2024-07-04: ('http://mistralai-mixtral-8x7b-3473-v33-predictor-default.tenant-chaiml-guanaco.knative.ord1.coreweave.cloud/v1/models/GPT-J-6B-lit-v2:predict', '{"error":"TypeError : SamplingParameters.__init__() got an unexpected keyword argument \'min_p\'"}')
mistralai-mixtral-8x7b-3473-v66-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mistralai-mixtral-8x7b-3473-v66-mkmlizer: ║     _____            __           __                                ║
mistralai-mixtral-8x7b-3473-v66-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
mistralai-mixtral-8x7b-3473-v66-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
mistralai-mixtral-8x7b-3473-v66-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
mistralai-mixtral-8x7b-3473-v66-mkmlizer: ║       /___/                                                         ║
mistralai-mixtral-8x7b-3473-v66-mkmlizer: ║                                                                     ║
mistralai-mixtral-8x7b-3473-v66-mkmlizer: ║  Version: 0.8.14                                                    ║
mistralai-mixtral-8x7b-3473-v66-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
mistralai-mixtral-8x7b-3473-v66-mkmlizer: ║  https://mk1.ai                                                     ║
mistralai-mixtral-8x7b-3473-v66-mkmlizer: ║                                                                     ║
mistralai-mixtral-8x7b-3473-v66-mkmlizer: ║  The license key for the current software has been verified as      ║
mistralai-mixtral-8x7b-3473-v66-mkmlizer: ║  belonging to:                                                      ║
mistralai-mixtral-8x7b-3473-v66-mkmlizer: ║                                                                     ║
mistralai-mixtral-8x7b-3473-v66-mkmlizer: ║  Chai Research Corp.                                                ║
mistralai-mixtral-8x7b-3473-v66-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
mistralai-mixtral-8x7b-3473-v66-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
mistralai-mixtral-8x7b-3473-v66-mkmlizer: ║                                                                     ║
mistralai-mixtral-8x7b-3473-v66-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Connection pool is full, discarding connection: %s
Connection pool is full, discarding connection: %s
Connection pool is full, discarding connection: %s
Connection pool is full, discarding connection: %s
Connection pool is full, discarding connection: %s
Connection pool is full, discarding connection: %s
Connection pool is full, discarding connection: %s
Connection pool is full, discarding connection: %s
Connection pool is full, discarding connection: %s
Connection pool is full, discarding connection: %s
mistralai-mixtral-8x7b-3473-v66-mkmlizer: Downloaded to shared memory in 234.706s
mistralai-mixtral-8x7b-3473-v66-mkmlizer: quantizing model to /dev/shm/model_cache
mistralai-mixtral-8x7b-3473-v66-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Connection pool is full, discarding connection: %s
Connection pool is full, discarding connection: %s
Connection pool is full, discarding connection: %s
mistralai-mixtral-8x7b-3473-v66-mkmlizer: 
Loading 0:   0%|          | 0/995 [00:00<?, ?it/s]
Loading 0:   0%|          | 2/995 [00:00<01:41,  9.82it/s]
Loading 0:   0%|          | 3/995 [00:00<02:16,  7.26it/s]
Loading 0:   0%|          | 4/995 [00:00<02:59,  5.51it/s]
Loading 0:   1%|          | 5/995 [00:00<03:11,  5.18it/s]
Loading 0:   1%|          | 6/995 [00:01<03:40,  4.48it/s]
Loading 0:   1%|          | 7/995 [00:01<03:47,  4.34it/s]
Loading 0:   1%|          | 8/995 [00:01<03:21,  4.89it/s]
Loading 0:   1%|          | 9/995 [00:01<03:27,  4.74it/s]
Loading 0:   1%|          | 10/995 [00:02<03:40,  4.46it/s]
Loading 0:   1%|          | 11/995 [00:02<04:00,  4.09it/s]
Loading 0:   1%|          | 12/995 [00:02<03:55,  4.18it/s]
Loading 0:   1%|▏         | 13/995 [00:02<04:00,  4.08it/s]
Loading 0:   1%|▏         | 14/995 [00:03<03:49,  4.27it/s]
Loading 0:   2%|▏         | 15/995 [00:03<04:00,  4.07it/s]
Loading 0:   2%|▏         | 16/995 [00:03<03:36,  4.52it/s]
Loading 0:   2%|▏         | 17/995 [00:03<03:23,  4.80it/s]
Loading 0:   2%|▏         | 18/995 [00:03<03:13,  5.05it/s]
Loading 0:   2%|▏         | 19/995 [00:03<03:03,  5.32it/s]
Loading 0:   2%|▏         | 20/995 [00:04<02:59,  5.43it/s]
Loading 0:   2%|▏         | 21/995 [00:04<02:43,  5.94it/s]
Loading 0:   2%|▏         | 22/995 [00:04<02:26,  6.63it/s]
Loading 0:   2%|▏         | 24/995 [00:04<02:02,  7.91it/s]
Loading 0:   3%|▎         | 25/995 [00:04<02:07,  7.61it/s]
Loading 0:   3%|▎         | 31/995 [00:04<00:51, 18.62it/s]
Loading 0:   3%|▎         | 34/995 [00:05<01:06, 14.35it/s]
Loading 0:   4%|▎         | 36/995 [00:05<01:17, 12.30it/s]
Loading 0:   4%|▍         | 38/995 [00:05<01:23, 11.43it/s]
Loading 0:   4%|▍         | 40/995 [00:05<01:44,  9.17it/s]
Loading 0:   4%|▍         | 42/995 [00:06<02:06,  7.54it/s]
Loading 0:   4%|▍         | 43/995 [00:06<02:16,  6.95it/s]
Loading 0:   4%|▍         | 44/995 [00:06<02:31,  6.28it/s]
Loading 0:   5%|▍         | 45/995 [00:07<03:06,  5.10it/s]
Loading 0:   5%|▍         | 46/995 [00:07<03:27,  4.58it/s]
Loading 0:   5%|▍         | 49/995 [00:07<02:01,  7.77it/s]
Loading 0:   5%|▌         | 52/995 [00:07<02:18,  6.82it/s]
Loading 0:   5%|▌         | 53/995 [00:08<02:33,  6.14it/s]
Loading 0:   5%|▌         | 54/995 [00:08<02:36,  6.01it/s]
Loading 0:   6%|▌         | 55/995 [00:08<02:42,  5.78it/s]
Loading 0:   6%|▌         | 56/995 [00:08<02:42,  5.77it/s]
Loading 0:   6%|▌         | 57/995 [00:08<02:40,  5.84it/s]
Loading 0:   6%|▌         | 58/995 [00:09<02:27,  6.35it/s]
Loading 0:   6%|▌         | 59/995 [00:09<02:28,  6.30it/s]
Loading 0:   6%|▌         | 60/995 [00:09<02:25,  6.43it/s]
Loading 0:   6%|▌         | 61/995 [00:09<02:21,  6.62it/s]
Loading 0:   6%|▋         | 64/995 [00:09<01:39,  9.40it/s]
Loading 0:   7%|▋         | 65/995 [00:09<01:53,  8.16it/s]
Loading 0:   7%|▋         | 66/995 [00:10<02:20,  6.61it/s]
Loading 0:   7%|▋         | 67/995 [00:10<02:31,  6.14it/s]
Loading 0:   7%|▋         | 68/995 [00:10<02:42,  5.72it/s]
Loading 0:   7%|▋         | 69/995 [00:10<02:52,  5.37it/s]
Loading 0:   7%|▋         | 70/995 [00:10<02:51,  5.38it/s]
Loading 0:   7%|▋         | 71/995 [00:11<02:42,  5.67it/s]
Loading 0:   7%|▋         | 72/995 [00:11<02:32,  6.04it/s]
Loading 0:   7%|▋         | 73/995 [00:11<02:16,  6.76it/s]
Loading 0:   8%|▊         | 75/995 [00:11<02:02,  7.48it/s]
Loading 0:   8%|▊         | 76/995 [00:11<02:16,  6.72it/s]
Loading 0:   8%|▊         | 77/995 [00:11<02:26,  6.25it/s]
Loading 0:   8%|▊         | 79/995 [00:12<02:11,  6.99it/s]
Loading 0:   8%|▊         | 80/995 [00:12<02:09,  7.04it/s]
Loading 0:   8%|▊         | 81/995 [00:12<02:07,  7.15it/s]
Loading 0:   8%|▊         | 82/995 [00:12<02:06,  7.24it/s]
Loading 0:   8%|▊         | 83/995 [00:12<02:18,  6.57it/s]
Loading 0:   8%|▊         | 84/995 [00:12<02:24,  6.30it/s]
Loading 0:   9%|▊         | 85/995 [00:13<02:22,  6.37it/s]
Loading 0:   9%|▊         | 86/995 [00:13<02:23,  6.33it/s]
Loading 0:   9%|▊         | 87/995 [00:13<02:25,  6.25it/s]
Loading 0:   9%|▉         | 93/995 [00:13<00:54, 16.55it/s]
Loading 0:  10%|▉         | 95/995 [00:13<01:05, 13.77it/s]
Loading 0:  10%|▉         | 97/995 [00:14<01:14, 11.98it/s]
Loading 0:  10%|▉         | 99/995 [00:14<01:15, 11.82it/s]
Loading 0:  10%|█         | 101/995 [00:14<01:12, 12.30it/s]
Loading 0:  11%|█         | 107/995 [00:14<01:13, 12.11it/s]
Loading 0:  11%|█         | 109/995 [00:15<01:24, 10.44it/s]
Loading 0:  11%|█         | 111/995 [00:15<01:40,  8.81it/s]
Loading 0:  11%|█▏        | 112/995 [00:15<01:47,  8.21it/s]
Loading 0:  11%|█▏        | 113/995 [00:15<01:53,  7.79it/s]
Loading 0:  11%|█▏        | 114/995 [00:16<02:00,  7.28it/s]
Loading 0:  12%|█▏        | 115/995 [00:16<02:10,  6.73it/s]
Loading 0:  12%|█▏        | 116/995 [00:16<03:14,  4.52it/s]
Loading 0:  12%|█▏        | 117/995 [00:17<03:46,  3.88it/s]
Loading 0:  12%|█▏        | 118/995 [00:17<03:58,  3.67it/s]
Loading 0:  12%|█▏        | 119/995 [00:17<03:52,  3.76it/s]
Loading 0:  12%|█▏        | 120/995 [00:17<03:29,  4.17it/s]
Loading 0:  12%|█▏        | 121/995 [00:17<03:17,  4.43it/s]
Loading 0:  12%|█▏        | 122/995 [00:18<03:09,  4.61it/s]
Loading 0:  12%|█▏        | 123/995 [00:18<02:57,  4.92it/s]
Loading 0:  13%|█▎        | 126/995 [00:18<01:51,  7.78it/s]
Loading 0:  13%|█▎        | 127/995 [00:18<02:01,  7.13it/s]
Loading 0:  13%|█▎        | 128/995 [00:18<02:16,  6.36it/s]
Loading 0:  13%|█▎        | 129/995 [00:19<02:39,  5.44it/s]
Loading 0:  13%|█▎        | 130/995 [00:19<02:52,  5.01it/s]
Loading 0:  13%|█▎        | 131/995 [00:19<02:59,  4.82it/s]
Loading 0:  13%|█▎        | 132/995 [00:19<02:48,  5.12it/s]
Loading 0:  13%|█▎        | 133/995 [00:19<02:41,  5.34it/s]
Loading 0:  13%|█▎        | 134/995 [00:20<02:50,  5.06it/s]
Loading 0:  14%|█▎        | 135/995 [00:20<02:48,  5.11it/s]
Loading 0:  14%|█▎        | 136/995 [00:20<02:46,  5.16it/s]
Loading 0:  14%|█▍        | 137/995 [00:20<02:41,  5.31it/s]
Loading 0:  14%|█▍        | 138/995 [00:20<02:26,  5.86it/s]
Loading 0:  14%|█▍        | 139/995 [00:21<02:15,  6.32it/s]
Loading 0:  14%|█▍        | 140/995 [00:21<02:13,  6.43it/s]
Loading 0:  14%|█▍        | 141/995 [00:21<02:14,  6.34it/s]
Loading 0:  14%|█▍        | 142/995 [00:21<02:12,  6.42it/s]
Loading 0:  14%|█▍        | 143/995 [00:21<02:24,  5.90it/s]
Loading 0:  14%|█▍        | 144/995 [00:21<02:28,  5.74it/s]
Loading 0:  15%|█▍        | 145/995 [00:22<02:24,  5.88it/s]
Loading 0:  15%|█▍        | 146/995 [00:22<02:30,  5.63it/s]
Loading 0:  15%|█▍        | 147/995 [00:22<02:30,  5.63it/s]
Loading 0:  15%|█▍        | 148/995 [00:22<02:24,  5.84it/s]
Loading 0:  15%|█▍        | 149/995 [00:22<02:37,  5.37it/s]
Loading 0:  16%|█▌        | 155/995 [00:22<00:58, 14.31it/s]
Loading 0:  16%|█▋        | 162/995 [00:23<00:55, 14.92it/s]
Loading 0:  16%|█▋        | 164/995 [00:23<00:54, 15.37it/s]
Loading 0:  17%|█▋        | 167/995 [00:23<00:54, 15.29it/s]
Loading 0:  17%|█▋        | 170/995 [00:23<00:51, 16.15it/s]
Loading 0:  17%|█▋        | 172/995 [00:24<01:02, 13.22it/s]
Loading 0:  17%|█▋        | 174/995 [00:24<01:38,  8.31it/s]
Loading 0:  18%|█▊        | 176/995 [00:25<02:24,  5.68it/s]
Loading 0:  18%|█▊        | 177/995 [00:25<02:40,  5.08it/s]
Loading 0:  18%|█▊        | 178/995 [00:25<02:50,  4.80it/s]
Loading 0:  18%|█▊        | 179/995 [00:26<02:39,  5.12it/s]
Loading 0:  18%|█▊        | 180/995 [00:26<02:24,  5.65it/s]
Loading 0:  18%|█▊        | 182/995 [00:26<01:49,  7.40it/s]
Loading 0:  18%|█▊        | 183/995 [00:26<01:52,  7.25it/s]
Loading 0:  18%|█▊        | 184/995 [00:26<01:57,  6.91it/s]
Loading 0:  19%|█▊        | 185/995 [00:26<01:54,  7.06it/s]
Loading 0:  19%|█▉        | 188/995 [00:26<01:17, 10.42it/s]
Loading 0:  19%|█▉        | 190/995 [00:27<01:31,  8.83it/s]
Loading 0:  19%|█▉        | 191/995 [00:27<01:34,  8.51it/s]
Loading 0:  19%|█▉        | 192/995 [00:27<01:40,  7.96it/s]
Loading 0:  19%|█▉        | 193/995 [00:27<01:48,  7.36it/s]
Loading 0:  19%|█▉        | 194/995 [00:27<01:57,  6.81it/s]
Loading 0:  20%|█▉        | 195/995 [00:28<01:59,  6.70it/s]
Loading 0:  20%|█▉        | 196/995 [00:28<01:55,  6.94it/s]
Loading 0:  20%|█▉        | 197/995 [00:28<01:47,  7.40it/s]
Loading 0:  20%|█▉        | 198/995 [00:28<01:49,  7.25it/s]
Loading 0:  20%|██        | 199/995 [00:28<01:49,  7.29it/s]
Loading 0:  20%|██        | 200/995 [00:28<01:46,  7.45it/s]
Loading 0:  20%|██        | 201/995 [00:28<01:38,  8.03it/s]
Loading 0:  20%|██        | 203/995 [00:28<01:25,  9.30it/s]
Loading 0:  21%|██        | 204/995 [00:29<01:28,  8.94it/s]
Loading 0:  21%|██        | 210/995 [00:29<01:08, 11.51it/s]
Loading 0:  21%|██▏       | 212/995 [00:29<01:29,  8.78it/s]
Loading 0:  21%|██▏       | 213/995 [00:30<01:39,  7.89it/s]
Loading 0:  22%|██▏       | 214/995 [00:30<01:47,  7.26it/s]
Loading 0:  22%|██▏       | 215/995 [00:30<01:53,  6.85it/s]
Loading 0:  22%|██▏       | 216/995 [00:30<01:57,  6.63it/s]
Loading 0:  22%|██▏       | 219/995 [00:30<01:22,  9.41it/s]
Loading 0:  22%|██▏       | 220/995 [00:30<01:30,  8.52it/s]
Loading 0:  22%|██▏       | 221/995 [00:31<01:59,  6.50it/s]
Loading 0:  22%|██▏       | 222/995 [00:31<02:30,  5.14it/s]
Loading 0:  22%|██▏       | 223/995 [00:31<02:45,  4.66it/s]
Loading 0:  23%|██▎       | 224/995 [00:32<02:43,  4.71it/s]
Loading 0:  23%|██▎       | 225/995 [00:32<02:41,  4.78it/s]
Loading 0:  23%|██▎       | 226/995 [00:32<02:42,  4.74it/s]
Loading 0:  23%|██▎       | 227/995 [00:32<02:55,  4.36it/s]
Loading 0:  23%|██▎       | 228/995 [00:33<03:12,  3.99it/s]
Loading 0:  23%|██▎       | 229/995 [00:33<03:19,  3.83it/s]
Loading 0:  23%|██▎       | 230/995 [00:33<03:24,  3.74it/s]
Loading 0:  23%|██▎       | 231/995 [00:33<02:53,  4.41it/s]
Loading 0:  23%|██▎       | 233/995 [00:34<02:16,  5.60it/s]
Loading 0:  24%|██▎       | 234/995 [00:34<02:13,  5.69it/s]
Loading 0:  24%|██▎       | 235/995 [00:34<02:04,  6.10it/s]
Loading 0:  24%|██▎       | 236/995 [00:34<02:03,  6.15it/s]
Loading 0:  24%|██▍       | 237/995 [00:34<02:20,  5.39it/s]
Loading 0:  24%|██▍       | 238/995 [00:34<02:40,  4.72it/s]
Loading 0:  24%|██▍       | 239/995 [00:35<02:40,  4.71it/s]
Loading 0:  24%|██▍       | 240/995 [00:35<02:54,  4.33it/s]
Loading 0:  24%|██▍       | 241/995 [00:35<03:05,  4.07it/s]
Loading 0:  24%|██▍       | 242/995 [00:36<03:08,  3.99it/s]
Loading 0:  25%|██▍       | 247/995 [00:36<01:12, 10.28it/s]
Loading 0:  25%|██▌       | 249/995 [00:36<01:06, 11.27it/s]
Loading 0:  25%|██▌       | 251/995 [00:36<01:35,  7.78it/s]
Loading 0:  25%|██▌       | 253/995 [00:37<01:53,  6.55it/s]
Loading 0:  26%|██▌       | 254/995 [00:37<01:55,  6.44it/s]
Loading 0:  26%|██▌       | 255/995 [00:37<01:56,  6.37it/s]
Loading 0:  26%|██▌       | 256/995 [00:37<01:57,  6.30it/s]
Loading 0:  26%|██▌       | 257/995 [00:37<01:56,  6.33it/s]
Loading 0:  26%|██▌       | 259/995 [00:38<01:44,  7.08it/s]
Loading 0:  26%|██▋       | 263/995 [00:38<00:59, 12.23it/s]
Loading 0:  27%|██▋       | 265/995 [00:38<01:22,  8.80it/s]
Loading 0:  27%|██▋       | 267/995 [00:38<01:29,  8.17it/s]
Loading 0:  27%|██▋       | 271/995 [00:38<00:59, 12.20it/s]
Loading 0:  27%|██▋       | 273/995 [00:39<01:05, 10.94it/s]
Loading 0:  28%|██▊       | 275/995 [00:39<01:10, 10.17it/s]
Loading 0:  28%|██▊       | 277/995 [00:39<01:17,  9.22it/s]
Loading 0:  28%|██▊       | 279/995 [00:39<01:10, 10.09it/s]
Loading 0:  28%|██▊       | 281/995 [00:40<01:07, 10.62it/s]
Loading 0:  28%|██▊       | 281/995 [00:58<01:07, 10.62it/s]
Loading 0:  28%|██▊       | 282/995 [00:58<37:45,  3.18s/it]
Loading 0:  28%|██▊       | 283/995 [00:58<31:05,  2.62s/it]
Loading 0:  29%|██▊       | 284/995 [00:59<25:02,  2.11s/it]
Loading 0:  29%|██▊       | 285/995 [00:59<19:54,  1.68s/it]
Loading 0:  29%|██▊       | 286/995 [00:59<15:47,  1.34s/it]
Loading 0:  29%|██▉       | 289/995 [01:00<08:25,  1.40it/s]
Loading 0:  29%|██▉       | 290/995 [01:00<07:39,  1.53it/s]
Loading 0:  29%|██▉       | 291/995 [01:00<07:02,  1.66it/s]
Loading 0:  29%|██▉       | 292/995 [01:01<06:08,  1.91it/s]
Loading 0:  29%|██▉       | 293/995 [01:01<05:33,  2.10it/s]
Loading 0:  30%|██▉       | 294/995 [01:01<05:25,  2.16it/s]
Loading 0:  30%|██▉       | 295/995 [01:02<04:46,  2.45it/s]
Loading 0:  30%|██▉       | 296/995 [01:02<04:19,  2.69it/s]
Loading 0:  30%|██▉       | 297/995 [01:02<04:03,  2.87it/s]
Loading 0:  30%|██▉       | 298/995 [01:03<04:02,  2.88it/s]
Loading 0:  30%|███       | 299/995 [01:03<03:57,  2.93it/s]
Loading 0:  30%|███       | 300/995 [01:03<04:07,  2.81it/s]
Loading 0:  30%|███       | 301/995 [01:04<04:19,  2.67it/s]
Loading 0:  30%|███       | 302/995 [01:04<04:20,  2.66it/s]
Loading 0:  30%|███       | 303/995 [01:04<04:17,  2.69it/s]
Loading 0:  31%|███       | 304/995 [01:05<04:10,  2.76it/s]
Loading 0:  31%|███       | 305/995 [01:05<03:45,  3.06it/s]
Loading 0:  31%|███       | 306/995 [01:05<03:45,  3.06it/s]
Loading 0:  31%|███       | 307/995 [01:06<03:47,  3.03it/s]
Loading 0:  31%|███       | 308/995 [01:06<03:52,  2.96it/s]
Loading 0:  31%|███       | 309/995 [01:06<03:52,  2.95it/s]
Loading 0:  31%|███       | 310/995 [01:07<03:40,  3.10it/s]
Loading 0:  31%|███▏      | 311/995 [01:07<03:39,  3.12it/s]
Loading 0:  31%|███▏      | 312/995 [01:07<03:32,  3.21it/s]
Loading 0:  32%|███▏      | 318/995 [01:07<01:10,  9.67it/s]
Loading 0:  32%|███▏      | 320/995 [01:08<01:45,  6.40it/s]
Loading 0:  32%|███▏      | 322/995 [01:09<02:30,  4.46it/s]
Loading 0:  32%|███▏      | 323/995 [01:09<02:54,  3.85it/s]
Loading 0:  33%|███▎      | 324/995 [01:10<02:58,  3.75it/s]
Loading 0:  33%|███▎      | 325/995 [01:10<03:03,  3.65it/s]
Loading 0:  33%|███▎      | 326/995 [01:10<03:11,  3.50it/s]
Loading 0:  33%|███▎      | 327/995 [01:11<03:43,  2.99it/s]
Loading 0:  33%|███▎      | 328/995 [01:11<04:03,  2.74it/s]
Loading 0:  33%|███▎      | 329/995 [01:12<04:04,  2.72it/s]
Loading 0:  33%|███▎      | 330/995 [01:12<03:50,  2.88it/s]
Loading 0:  33%|███▎      | 332/995 [01:12<02:28,  4.47it/s]
Loading 0:  34%|███▎      | 334/995 [01:12<01:52,  5.90it/s]
Loading 0:  34%|███▍      | 336/995 [01:12<01:26,  7.64it/s]
Loading 0:  34%|███▍      | 338/995 [01:12<01:12,  9.11it/s]
Loading 0:  34%|███▍      | 343/995 [01:13<00:41, 15.63it/s]
Loading 0:  35%|███▍      | 347/995 [01:13<00:33, 19.10it/s]
Loading 0:  35%|███▌      | 351/995 [01:13<00:28, 22.53it/s]
Loading 0:  36%|███▌      | 354/995 [01:13<00:29, 21.89it/s]
Loading 0:  36%|███▌      | 357/995 [01:13<00:33, 18.81it/s]
Loading 0:  36%|███▌      | 360/995 [01:13<00:42, 14.93it/s]
Loading 0:  36%|███▋      | 362/995 [01:14<00:45, 14.01it/s]
Loading 0:  37%|███▋      | 368/995 [01:14<00:43, 14.26it/s]
Loading 0:  37%|███▋      | 370/995 [01:14<00:46, 13.47it/s]
Loading 0:  38%|███▊      | 374/995 [01:14<00:40, 15.41it/s]
Loading 0:  38%|███▊      | 376/995 [01:15<00:43, 14.19it/s]
Loading 0:  38%|███▊      | 378/995 [01:15<00:45, 13.42it/s]
Loading 0:  38%|███▊      | 380/995 [01:15<00:45, 13.66it/s]
Loading 0:  38%|███▊      | 382/995 [01:15<00:44, 13.72it/s]
Loading 0:  39%|███▊      | 384/995 [01:15<00:44, 13.60it/s]
Loading 0:  39%|███▉      | 386/995 [01:15<00:45, 13.32it/s]
Loading 0:  39%|███▉      | 388/995 [01:16<00:46, 13.14it/s]
Loading 0:  39%|███▉      | 390/995 [0
mistralai-mixtral-8x7b-3473-v66-mkmlizer: quantized model in 154.359s
mistralai-mixtral-8x7b-3473-v66-mkmlizer: Processed model mistralai/Mixtral-8x7B-Instruct-v0.1 in 389.065s
mistralai-mixtral-8x7b-3473-v66-mkmlizer: creating bucket guanaco-mkml-models
mistralai-mixtral-8x7b-3473-v66-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
mistralai-mixtral-8x7b-3473-v66-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v66
mistralai-mixtral-8x7b-3473-v66-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v66/config.json
mistralai-mixtral-8x7b-3473-v66-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v66/special_tokens_map.json
mistralai-mixtral-8x7b-3473-v66-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v66/tokenizer_config.json
mistralai-mixtral-8x7b-3473-v66-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v66/tokenizer.json
mistralai-mixtral-8x7b-3473-v66-mkmlizer: cp /dev/shm/model_cache/tokenizer.model s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v66/tokenizer.model
mistralai-mixtral-8x7b-3473-v66-mkmlizer: cp /dev/shm/model_cache/flywheel_model.3.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v66/flywheel_model.3.safetensors
mistralai-mixtral-8x7b-3473-v66-mkmlizer: cp /dev/shm/model_cache/flywheel_model.2.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v66/flywheel_model.2.safetensors
mistralai-mixtral-8x7b-3473-v66-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v66/flywheel_model.0.safetensors
mistralai-mixtral-8x7b-3473-v66-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v66/flywheel_model.1.safetensors
mistralai-mixtral-8x7b-3473-v66-mkmlizer: loading reward model from ChaiML/gpt2_xl_pairwise_89m_step_347634
mistralai-mixtral-8x7b-3473-v66-mkmlizer: 1:16<00:45, 13.34it/s]
Loading 0:  39%|███▉      | 392/995 [01:16<00:44, 13.49it/s]
Loading 0:  40%|███▉      | 394/995 [01:16<00:44, 13.48it/s]
Loading 0:  40%|███▉      | 396/995 [01:16<00:43, 13.67it/s]
Loading 0:  40%|████      | 402/995 [01:16<00:25, 23.12it/s]
Loading 0:  41%|████      | 406/995 [01:16<00:24, 23.88it/s]
Loading 0:  41%|████      | 409/995 [01:17<00:29, 20.12it/s]
Loading 0:  41%|████▏     | 412/995 [01:17<00:31, 18.33it/s]
Loading 0:  42%|████▏     | 414/995 [01:17<00:47, 12.35it/s]
Loading 0:  42%|████▏     | 416/995 [01:18<01:12,  8.04it/s]
Loading 0:  42%|████▏     | 418/995 [01:18<01:12,  7.92it/s]
Loading 0:  42%|████▏     | 421/995 [01:18<01:01,  9.37it/s]
Loading 0:  43%|████▎     | 423/995 [01:19<01:25,  6.66it/s]
Loading 0:  43%|████▎     | 424/995 [01:19<01:37,  5.83it/s]
Loading 0:  43%|████▎     | 425/995 [01:19<01:48,  5.25it/s]
Loading 0:  43%|████▎     | 426/995 [01:20<01:54,  4.98it/s]
Loading 0:  43%|████▎     | 427/995 [01:20<02:10,  4.34it/s]
Loading 0:  43%|████▎     | 428/995 [01:20<02:17,  4.11it/s]
Loading 0:  43%|████▎     | 429/995 [01:20<02:24,  3.91it/s]
Loading 0:  43%|████▎     | 430/995 [01:21<02:22,  3.96it/s]
Loading 0:  43%|████▎     | 431/995 [01:21<02:11,  4.30it/s]
Loading 0:  43%|████▎     | 432/995 [01:21<02:26,  3.84it/s]
Loading 0:  44%|████▎     | 433/995 [01:22<03:00,  3.11it/s]
Loading 0:  44%|████▍     | 436/995 [01:22<02:01,  4.60it/s]
Loading 0:  44%|████▍     | 437/995 [01:22<02:20,  3.97it/s]
Loading 0:  44%|████▍     | 438/995 [01:23<02:47,  3.33it/s]
Loading 0:  44%|████▍     | 439/995 [01:23<03:07,  2.96it/s]
Loading 0:  44%|████▍     | 440/995 [01:24<03:20,  2.76it/s]
Loading 0:  44%|████▍     | 441/995 [01:24<03:10,  2.91it/s]
Loading 0:  44%|████▍     | 442/995 [01:24<02:59,  3.09it/s]
Loading 0:  45%|████▍     | 444/995 [01:25<01:57,  4.67it/s]
Loading 0:  45%|████▍     | 446/995 [01:25<01:25,  6.45it/s]
Loading 0:  45%|████▌     | 450/995 [01:25<00:48, 11.22it/s]
Loading 0:  46%|████▌     | 456/995 [01:25<00:27, 19.49it/s]
Loading 0:  47%|████▋     | 469/995 [01:25<00:12, 40.57it/s]
Loading 0:  48%|████▊     | 478/995 [01:25<00:15, 33.05it/s]
Loading 0:  49%|████▊     | 484/995 [01:25<00:13, 37.40it/s]
Loading 0:  49%|████▉     | 490/995 [01:26<00:12, 41.60it/s]
Loading 0:  50%|█████     | 498/995 [01:26<00:10, 49.70it/s]
Loading 0:  51%|█████     | 505/995 [01:26<00:09, 52.51it/s]
Loading 0:  51%|█████▏    | 512/995 [01:26<00:08, 54.32it/s]
Loading 0:  52%|█████▏    | 519/995 [01:26<00:08, 55.66it/s]
Loading 0:  53%|█████▎    | 526/995 [01:26<00:12, 38.34it/s]
Loading 0:  54%|█████▎    | 534/995 [01:26<00:10, 46.08it/s]
Loading 0:  54%|█████▍    | 540/995 [01:27<00:09, 47.09it/s]
Loading 0:  55%|█████▍    | 546/995 [01:27<00:09, 49.42it/s]
Loading 0:  56%|█████▌    | 557/995 [01:27<00:06, 63.78it/s]
Loading 0:  56%|█████▋    | 560/995 [01:38<00:06, 63.78it/s]
Loading 0:  56%|█████▋    | 561/995 [01:48<06:55,  1.05it/s]
Loading 0:  57%|█████▋    | 563/995 [01:48<06:11,  1.16it/s]
Loading 0:  57%|█████▋    | 569/995 [01:49<04:15,  1.67it/s]
Loading 0:  58%|█████▊    | 574/995 [01:49<03:04,  2.28it/s]
Loading 0:  58%|█████▊    | 581/995 [01:49<02:05,  3.31it/s]
Loading 0:  59%|█████▉    | 587/995 [01:49<01:27,  4.64it/s]
Loading 0:  60%|█████▉    | 594/995 [01:49<00:59,  6.74it/s]
Loading 0:  60%|██████    | 599/995 [01:49<00:45,  8.65it/s]
Loading 0:  61%|██████    | 605/995 [01:50<00:33, 11.60it/s]
Loading 0:  61%|██████▏   | 611/995 [01:50<00:25, 15.15it/s]
Loading 0:  63%|██████▎   | 623/995 [01:50<00:14, 25.06it/s]
Loading 0:  63%|██████▎   | 630/995 [01:50<00:12, 29.69it/s]
Loading 0:  64%|██████▍   | 637/995 [01:50<00:13, 26.59it/s]
Loading 0:  65%|██████▍   | 644/995 [01:50<00:11, 31.80it/s]
Loading 0:  65%|██████▌   | 651/995 [01:50<00:09, 37.83it/s]
Loading 0:  66%|██████▌   | 658/995 [01:51<00:07, 43.57it/s]
Loading 0:  67%|██████▋   | 665/995 [01:51<00:07, 44.61it/s]
Loading 0:  68%|██████▊   | 672/995 [01:51<00:06, 48.40it/s]
Loading 0:  69%|██████▉   | 685/995 [01:51<00:04, 64.83it/s]
Loading 0:  70%|██████▉   | 693/995 [01:51<00:07, 43.11it/s]
Loading 0:  70%|███████   | 700/995 [01:51<00:06, 46.81it/s]
Loading 0:  71%|███████   | 707/995 [01:52<00:05, 49.80it/s]
Loading 0:  72%|███████▏  | 715/995 [01:52<00:05, 55.26it/s]
Loading 0:  73%|███████▎  | 722/995 [01:52<00:04, 55.96it/s]
Loading 0:  73%|███████▎  | 729/995 [01:52<00:04, 57.24it/s]
Loading 0:  74%|███████▍  | 739/995 [01:52<00:06, 40.63it/s]
Loading 0:  75%|███████▌  | 747/995 [01:52<00:05, 47.08it/s]
Loading 0:  76%|███████▌  | 753/995 [01:52<00:04, 49.41it/s]
Loading 0:  76%|███████▋  | 759/995 [01:53<00:04, 51.16it/s]
Loading 0:  77%|███████▋  | 765/995 [01:53<00:04, 51.85it/s]
Loading 0:  78%|███████▊  | 778/995 [01:53<00:03, 69.61it/s]
Loading 0:  79%|███████▉  | 786/995 [01:53<00:03, 67.53it/s]
Loading 0:  80%|███████▉  | 794/995 [01:53<00:04, 43.91it/s]
Loading 0:  80%|████████  | 800/995 [01:53<00:04, 46.04it/s]
Loading 0:  81%|████████  | 808/995 [01:53<00:03, 52.90it/s]
Loading 0:  82%|████████▏ | 815/995 [01:54<00:03, 53.62it/s]
Loading 0:  83%|████████▎ | 822/995 [01:54<00:03, 53.95it/s]
Loading 0:  83%|████████▎ | 829/995 [01:54<00:02, 55.71it/s]
Loading 0:  85%|████████▍ | 841/995 [02:08<00:02, 55.71it/s]
Loading 0:  85%|████████▍ | 842/995 [02:16<01:54,  1.33it/s]
Loading 0:  85%|████████▌ | 849/995 [02:17<01:25,  1.71it/s]
Loading 0:  86%|████████▌ | 854/995 [02:17<01:06,  2.12it/s]
Loading 0:  86%|████████▋ | 859/995 [02:17<00:50,  2.71it/s]
Loading 0:  87%|████████▋ | 864/995 [02:17<00:37,  3.51it/s]
Loading 0:  88%|████████▊ | 871/995 [02:17<00:24,  5.10it/s]
Loading 0:  88%|████████▊ | 877/995 [02:17<00:17,  6.85it/s]
Loading 0:  89%|████████▊ | 883/995 [02:17<00:12,  9.11it/s]
Loading 0:  89%|████████▉ | 889/995 [02:17<00:08, 12.03it/s]
Loading 0:  90%|█████████ | 897/995 [02:18<00:06, 14.37it/s]
Loading 0:  91%|█████████ | 904/995 [02:18<00:04, 18.93it/s]
Loading 0:  91%|█████████▏| 909/995 [02:18<00:03, 22.23it/s]
Loading 0:  92%|█████████▏| 914/995 [02:18<00:03, 25.86it/s]
Loading 0:  92%|█████████▏| 919/995 [02:18<00:02, 29.68it/s]
Loading 0:  93%|█████████▎| 924/995 [02:18<00:02, 33.18it/s]
Loading 0:  94%|█████████▍| 936/995 [02:18<00:01, 49.96it/s]
Loading 0:  95%|█████████▍| 943/995 [02:19<00:01, 50.07it/s]
Loading 0:  96%|█████████▌| 952/995 [02:20<00:03, 12.63it/s]
Loading 0:  96%|█████████▌| 957/995 [02:20<00:02, 14.98it/s]
Loading 0:  97%|█████████▋| 964/995 [02:20<00:01, 19.63it/s]
Loading 0:  97%|█████████▋| 970/995 [02:21<00:01, 23.51it/s]
Loading 0:  98%|█████████▊| 976/995 [02:21<00:00, 27.33it/s]
Loading 0:  99%|█████████▊| 982/995 [02:21<00:00, 31.39it/s]
Loading 0:  99%|█████████▉| 988/995 [02:21<00:00, 36.34it/s]
                                                            
/opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:919: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mistralai-mixtral-8x7b-3473-v66-mkmlizer:   warnings.warn(
mistralai-mixtral-8x7b-3473-v66-mkmlizer: /opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
mistralai-mixtral-8x7b-3473-v66-mkmlizer:   warnings.warn(
mistralai-mixtral-8x7b-3473-v66-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:769: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mistralai-mixtral-8x7b-3473-v66-mkmlizer:   warnings.warn(
mistralai-mixtral-8x7b-3473-v66-mkmlizer: 
Downloading shards:   0%|          | 0/2 [00:00<?, ?it/s]
Downloading shards:  50%|█████     | 1/2 [00:06<00:06,  6.03s/it]
Downloading shards: 100%|██████████| 2/2 [00:08<00:00,  3.79s/it]
Downloading shards: 100%|██████████| 2/2 [00:08<00:00,  4.13s/it]
mistralai-mixtral-8x7b-3473-v66-mkmlizer: 
Loading checkpoint shards:   0%|          | 0/2 [00:00<?, ?it/s]
Loading checkpoint shards:  50%|█████     | 1/2 [00:00<00:00,  1.86it/s]
Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00,  3.07it/s]
Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00,  2.79it/s]
mistralai-mixtral-8x7b-3473-v66-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
mistralai-mixtral-8x7b-3473-v66-mkmlizer: Saving duration: 1.877s
mistralai-mixtral-8x7b-3473-v66-mkmlizer: Processed model ChaiML/gpt2_xl_pairwise_89m_step_347634 in 12.264s
mistralai-mixtral-8x7b-3473-v66-mkmlizer: creating bucket guanaco-reward-models
mistralai-mixtral-8x7b-3473-v66-mkmlizer: Bucket 's3://guanaco-reward-models/' created
mistralai-mixtral-8x7b-3473-v66-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v66_reward
mistralai-mixtral-8x7b-3473-v66-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v66_reward/config.json
mistralai-mixtral-8x7b-3473-v66-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v66_reward/special_tokens_map.json
mistralai-mixtral-8x7b-3473-v66-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v66_reward/tokenizer_config.json
mistralai-mixtral-8x7b-3473-v66-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v66_reward/merges.txt
mistralai-mixtral-8x7b-3473-v66-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v66_reward/vocab.json
mistralai-mixtral-8x7b-3473-v66-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v66_reward/tokenizer.json
mistralai-mixtral-8x7b-3473-v66-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v66_reward/reward.tensors
Job mistralai-mixtral-8x7b-3473-v66-mkmlizer completed after 446.36s with status: succeeded
Stopping job with name mistralai-mixtral-8x7b-3473-v66-mkmlizer
Pipeline stage MKMLizer completed in 447.39s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.17s
Running pipeline stage ISVCDeployer
Creating inference service mistralai-mixtral-8x7b-3473-v66
Waiting for inference service mistralai-mixtral-8x7b-3473-v66 to be ready
Inference service mistralai-mixtral-8x7b-3473-v66 ready after 80.45538520812988s
Pipeline stage ISVCDeployer completed in 90.65s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.5945746898651123s
Received healthy response to inference request in 1.6540045738220215s
Received healthy response to inference request in 1.8298492431640625s
Received healthy response to inference request in 1.6539368629455566s
Received healthy response to inference request in 1.1942565441131592s
5 requests
0 failed requests
5th percentile: 1.2861926078796386
10th percentile: 1.378128671646118
20th percentile: 1.5620007991790772
30th percentile: 1.6539504051208496
40th percentile: 1.6539774894714356
50th percentile: 1.6540045738220215
60th percentile: 1.724342441558838
70th percentile: 1.7946803092956543
80th percentile: 1.9827943325042725
90th percentile: 2.2886845111846923
95th percentile: 2.441629600524902
99th percentile: 2.56398567199707
mean time: 1.7853243827819825
Pipeline stage StressChecker completed in 9.85s
mistralai-mixtral-8x7b-_3473_v66 status is now deployed due to DeploymentManager action
mistralai-mixtral-8x7b-_3473_v66 status is now inactive due to auto deactivation removed underperforming models

Start Time:

End Time:

Step (in hours or minutes, e.g. 1h, 10m):

Usage Metrics

Latency Metrics

Model Developer	Developer Rank	Model Rank	Model Rank By Developer	Elo Rating	Elo Rating By Developer	Auto Alignment Score	Auto Alignment Samples	User Alignment Score	User Alignment Question Responses	Submission ID	Parameters	Huggingface Repository	Variant	Number of Battles	Number of Battles By Developer	Win Ratio	Win Ratio By Developer	Type	Reward Repo	Best of	Max Input Tokens	Max Output Tokens	Model Architecture	Model Group	Date	Elo Norm	Alignment Norm	Alt Ranking	Alt Ranking By Developer	Elo Rank	Alignment Rank
chai_backend_admin	1	1	1	1273.33	1273.33		0	0.76	887.0	chaiml-elo-alignment-run-3_v25	8B	ChaiML/elo_alignment_run_3	e2e-test-chaiml-v25	10147	22459786	0.56	0.60	basic	ChaiML/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	ChaiML/elo_alignment_run	2024-08-05	1.62	1.81	1	1	19	69
zonemercy	2	2	2	1275.12	1289.29		0	0.75	4615.0	zonemercy-burly-blue-cp2500_v5	8B	zonemercy/Burly_Blue_cp2500	zonemercy-burly-blue-cp2500_v5	7875	4840314	0.61	0.64	basic	ChaiML/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	zonemercy/Burly_Blue_cp2	2024-07-19	1.65	1.71	2	2	12	79
PawanOsman	3	3	3	1277.56	1277.56		0	0.75	977.0	pawankrd-cosmosrp_v77	8B	PawanKrd/CosmosRP	pawankrd-cosmosrp_v77	10992	4069431	0.58	0.75	basic	Jellywibble/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	PawanKrd/CosmosRP	2024-07-20	1.71	1.43	3	3	6	144
zonemercy	2	4	2	1267.28	1289.29		0	0.75	1010.0	zonemercy-lexical-nemov9_2587_v1	13B	zonemercy/Lexical-Nemov9-1k5e6	zonemercy-lexical-nemov9_2587_v1	11494	4840314	0.55	0.64	basic	ChaiML/gpt2_xl_pairwise_89m_step_347634	8.0	1024.0	64.0	MistralForCausalLM	zonemercy/Lexical-Nemov9	2024-08-06	1.49	1.53	4	2	59	121
albert_chai	4	5	5	1272.80	1272.80		0	0.74	1338.0	chaiml-elo-alignment-run-3_v11	8B	ChaiML/elo_alignment_run_3	chaiml-elo-alignment-run-3_v3	11278	18928	0.61	0.61	basic	ChaiML/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	ChaiML/elo_alignment_run	2024-07-19	1.60	1.29	5	5	23	178
PawanOsman	3	6	3	1274.85	1277.56		0	0.74	1001.0	pawankrd-cosmosrp_v76	8B	PawanKrd/CosmosRP	pawankrd-cosmosrp_v76	10908	4069431	0.58	0.75	basic	Jellywibble/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	PawanKrd/CosmosRP	2024-07-20	1.65	1.15	6	3	13	215
PawanOsman	3	7	3	1273.13	1277.56		0	0.74	1068.0	pawankrd-cosmosrp_v79	8B	PawanKrd/CosmosRP	pawankrd-cosmosrp_v79	10793	4069431	0.57	0.75	basic	Jellywibble/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	PawanKrd/CosmosRP	2024-07-20	1.61	1.09	7	3	20	234
zonemercy	2	8	2	1274.76	1289.29		0	0.74	1017.0	chaiml-elo-alignment-run-3_v30	8B	ChaiML/elo_alignment_run_3	chaiml-elo-alignment-run-3_v30	11476	4840314	0.56	0.64	basic	ChaiML/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	ChaiML/elo_alignment_run	2024-08-06	1.65	0.95	11	2	14	299
PawanOsman	3	9	3	1274.06	1277.56		0	0.74	1197.0	pawankrd-cosmosrp_v78	8B	PawanKrd/CosmosRP	pawankrd-cosmosrp_v78	12920	4069431	0.58	0.75	basic	Jellywibble/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	PawanKrd/CosmosRP	2024-07-20	1.63	0.98	10	3	16	286
PawanOsman	3	10	3	1273.12	1277.56		0	0.73	1196.0	pawankrd-cosmosrp_v73	8B	PawanKrd/CosmosRP	pawankrd-cosmosrp_v73	13819	4069431	0.61	0.75	basic	Jellywibble/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	PawanKrd/CosmosRP	2024-07-18	1.61	0.93	12	3	21	308
PawanOsman	3	11	3	1268.48	1277.56		0	0.74	1548.0	pawankrd-cosmosrp_v72	8B	PawanKrd/CosmosRP	pawankrd-cosmosrp_v72	12678	4069431	0.60	0.75	basic	Jellywibble/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	PawanKrd/CosmosRP	2024-07-18	1.51	1.07	8	3	51	246
zonemercy	2	12	2	1273.77	1289.29		0	0.73	924.0	zonemercy-cogent-nemo-v1_4327_v1	13B	zonemercy/Cogent-Nemo-v1-1k1e5-ep1	zonemercy-cogent-nemo-v1_4327_v1	10226	4840314	0.56	0.64	basic	ChaiML/gpt2_xl_pairwise_89m_step_347634	8.0	1024.0	64.0	MistralForCausalLM	zonemercy/Cogent-Nemo-v1	2024-08-07	1.63	0.84	16	2	18	343
chai_backend_admin	1	13	1	1263.14	1273.33		0	0.74	866.0	chaiml-elo-alignment-run-3_v26	8B	ChaiML/elo_alignment_run_3	e2e-test-chaiml-v26	9925	22459786	0.55	0.60	basic	ChaiML/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	ChaiML/elo_alignment_run	2024-08-05	1.40	1.19	9	1	102	197
zonemercy	2	14	2	1273.95	1289.29		0	0.73	10695.0	chaiml-elo-alignment-run-3_v8	8B	ChaiML/elo_alignment_run_3	chaiml-elo-alignment-run-3_v8	210506	4840314	0.58	0.64	basic	ChaiML/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	ChaiML/elo_alignment_run	2024-07-15	1.63	0.73	28	2	17	412
PawanOsman	3	15	3	1272.23	1277.56		0	0.73	1746.0	pawankrd-cosmosrp_v71	8B	PawanKrd/CosmosRP	pawankrd-cosmosrp_v71	24610	4069431	0.62	0.75	basic	Jellywibble/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	PawanKrd/CosmosRP	2024-07-17	1.59	0.79	22	3	25	376
PawanOsman	3	16	3	1269.64	1277.56		0	0.73	5002.0	pawankrd-cosmosrp_v69	8B	PawanKrd/CosmosRP	pawankrd-cosmosrp_v69	132481	4069431	0.59	0.75	basic	Jellywibble/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	PawanKrd/CosmosRP	2024-07-16	1.54	0.84	18	3	41	342
zonemercy	2	17	2	1270.04	1289.29		0	0.73	9211.0	chaiml-elo-alignment-run-3_v5	8B	ChaiML/elo_alignment_run_3	chaiml-elo-alignment-run-3_v3	94158	4840314	0.55	0.64	basic	ChaiML/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	ChaiML/elo_alignment_run	2024-07-12	1.55	0.81	21	2	37	363
zonemercy	2	18	2	1268.38	1289.29		0	0.73	5476.0	chaiml-elo-alignment-run-3_v4	8B	ChaiML/elo_alignment_run_3	chaiml-elo-alignment-run-3_v3	50315	4840314	0.57	0.64	basic	ChaiML/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	ChaiML/elo_alignment_run	2024-07-12	1.51	0.82	23	2	53	353
Jellywibble	5	19	19	1267.07	1276.15		0	0.73	5329.0	jellywibble-lora-120k-pr_2801_v2	8B	Jellywibble/lora_120k_pref_data_ep3_stacked_elo_alignment	nitral-ai-hathor-l3-8b-v-01_v1	30298	7581067	0.60	0.61	basic	ChaiML/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	Jellywibble/lora_120k_pr	2024-07-09	1.48	0.86	19	19	62	333
Jellywibble	5	20	19	1265.69	1276.15		0	0.73	1032.0	chaiml-elo-alignment-run-3_v18	8B	ChaiML/elo_alignment_run_3	chaiml-elo-alignment-run-3_v3	11675	7581067	0.58	0.61	basic	ChaiML/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	ChaiML/elo_alignment_run	2024-07-30	1.45	0.87	24	19	82	328
zonemercy	2	21	2	1265.75	1289.29		0	0.73	5639.0	chaiml-elo-alignment-run-3_v3	8B	ChaiML/elo_alignment_run_3	chaiml-elo-alignment-run-3_v3	11314	4840314	0.62	0.64	basic	ChaiML/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	ChaiML/elo_alignment_run	2024-07-12	1.45	0.87	26	2	80	330
zonemercy	2	22	2	1264.17	1289.29		0	0.73	10173.0	zonemercy-burly-blue-cp2500_v1	8B	zonemercy/Burly_Blue_cp2500	zonemercy-burly-blue-cp2500_v1	124947	4840314	0.59	0.64	basic	ChaiML/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	zonemercy/Burly_Blue_cp2	2024-07-14	1.42	0.87	27	2	96	329
PawanOsman	3	23	3	1267.21	1277.56		0	0.73	6358.0	pawankrd-cosmosrp_v65	8B	PawanKrd/CosmosRP	pawankrd-cosmosrp_v65	64112	4069431	0.58	0.75	basic	Jellywibble/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	PawanKrd/CosmosRP	2024-07-13	1.48	0.72	37	3	60	424
PawanOsman	3	24	3	1272.07	1277.56		0	0.72	50018.0	pawankrd-cosmosrp_v66	8B	PawanKrd/CosmosRP	pawankrd-cosmosrp_v66	353310	4069431	0.56	0.75	basic	Jellywibble/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	PawanKrd/CosmosRP	2024-07-13	1.59	0.51	66	3	28	574
Bbbrun0	6	25	25	1267.13	1267.13		0	0.73	1019.0	jellywibble-wibblewobble_v1	8B	Jellywibble/WibbleWobble	auto	11246	3305336	0.57	0.59	basic	Jellywibble/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	Jellywibble/WibbleWobble	2024-08-04	1.48	0.70	41	13	61	436
chai_backend_admin	1	26	1	1266.50	1273.33		0	0.73	862.0	chaiml-elo-alignment-run-3_v27	8B	ChaiML/elo_alignment_run_3	chaiml-elo-alignment-run-3_v3	9878	22459786	0.57	0.60	basic	ChaiML/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	ChaiML/elo_alignment_run	2024-08-05	1.47	0.72	39	1	74	420
zonemercy	2	27	2	1269.60	1289.29		0	0.73	5801.0	chaiml-elo-alignment-run-3_v6	8B	ChaiML/elo_alignment_run_3	chaiml-elo-alignment-run-3_v3	52400	4840314	0.57	0.64	basic	ChaiML/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	ChaiML/elo_alignment_run	2024-07-12	1.54	0.59	52	2	42	505
zonemercy	2	28	2	1264.09	1289.29		0	0.73	5613.0	jellywibble-lora-120k-pr_2801_v6	8B	Jellywibble/lora_120k_pref_data_ep3_stacked_elo_alignment	jellywibble-lora-120k-pr_2801_v6	11193	4840314	0.62	0.64	basic	ChaiML/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	Jellywibble/lora_120k_pr	2024-07-12	1.42	0.80	33	2	98	368
PawanOsman	3	29	3	1266.54	1277.56		0	0.73	10925.0	pawankrd-cosmosrp_v64	8B	PawanKrd/CosmosRP	pawankrd-cosmosrp_v64	112573	4069431	0.56	0.75	basic	Jellywibble/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	PawanKrd/CosmosRP	2024-07-12	1.47	0.68	45	3	73	443
Bbbrun0	6	30	25	1245.67	1267.13		0	0.75	861.0	bbchicago-brt-8b-l3-v1-9_2176_v1	8B	BBChicago/Brt-8B-L3_v1.9_with_dpo_lora_s5900	bbchicago-brt-v1_9-dpo-s5900	19612	3305336	0.55	0.59	basic	Jellywibble/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	BBChicago/Brt-8B-L3_v1.9	2024-07-24	1.02	1.51	13	13	211	126
shiroe40	7	31	31	1275.61	1277.83		0	0.72	1010.0	cycy233-l3-ba-e-v-c2_v1	8B	cycy233/L3-ba-e-v-c2	auto	11705	11611290	0.57	0.61	basic	Jellywibble/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	cycy233/L3-ba-e-v-c2	2024-07-29	1.66	0.32	117	15	10	747
zonemercy	2	32	2	1264.76	1289.29		0	0.73	12325.0	chaiml-elo-alignment-run_5486_v1	8B	ChaiML/elo_alignment_run_3_ep1	dpo-v2-0710-rm-score-completed	28117	4840314	0.60	0.64	basic	ChaiML/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	ChaiML/elo_alignment_run	2024-07-12	1.43	0.74	38	2	89	401
Jellywibble	5	33	19	1276.15	1276.15		0	0.72	954.0	chaiml-elo-alignment-run-3_v16	8B	ChaiML/elo_alignment_run_3	chaiml-elo-alignment-run-3_v3	10540	7581067	0.60	0.61	basic	ChaiML/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	ChaiML/elo_alignment_run	2024-07-30	1.68	0.29	130	19	9	770
zonemercy	2	34	2	1277.63	1289.29		0	0.72	12582.0	chaiml-elo-run-3-cp7000_v1	8B	ChaiML/elo_run_3_cp7000	nitral-ai-hathor-l3-8b-v-01_v1	24560	4840314	0.62	0.64	basic	ChaiML/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	ChaiML/elo_run_3_cp7000	2024-07-12	1.71	0.21	158	2	5	842
zonemercy	2	35	2	1246.89	1289.29		0	0.75	5478.0	chaiml-elo-alignment-run-3_v7	8B	ChaiML/elo_alignment_run_3	chaiml-elo-alignment-run-3_v3	57438	4840314	0.58	0.64	basic	ChaiML/gpt2_xl_pairwise_89m_step_347634	4.0	512.0	64.0	LlamaForCausalLM	ChaiML/elo_alignment_run	2024-07-13	1.05	1.44	14	2	202	140
shiroe40	7	36	31	1270.79	1277.83		0	0.72	1741.0	cycy233-l3-e-v2-c1_v2	8B	cycy233/L3-e-v2-c1	auto	6789	11611290	0.58	0.61	basic	Jellywibble/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	cycy233/L3-e-v2-c1	2024-07-19	1.56	0.48	77	15	33	599
zonemercy	2	37	2	1235.09	1289.29		0	0.76	919.0	zonemercy-lexical-nemov6_2011_v6	13B	zonemercy/Lexical-Nemov6-1k1e5	0805v1-1	9881	4840314	0.51	0.64	basic	ChaiML/gpt2_xl_pairwise_89m_step_347634	4.0	1024.0	64.0	MistralForCausalLM	zonemercy/Lexical-Nemov6	2024-08-05	0.79	1.89	25	2	351	59
Jellywibble	5	38	19	1266.70	1276.15	-0.87	79114	0.73	428961.0	chaiml-elo-alignment-run-3_v17	8B	ChaiML/elo_alignment_run_3	chaiml-elo-alignment-run-3_v3	5047098	7581067	0.57	0.61	basic	ChaiML/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	ChaiML/elo_alignment_run	2024-07-30	1.47	0.63	51	19	68	476
zonemercy	2	39	2	1283.66	1289.29		0	0.71	5498.0	chaiml-elo-run-3-cp7000_v4	8B	ChaiML/elo_run_3_cp7000	nitral-ai-hathor-l3-8b-v-01_v1	60274	4840314	0.64	0.64	basic	ChaiML/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	ChaiML/elo_run_3_cp7000	2024-07-13	1.84	-0.05	267	2	3	1088
shiroe40	7	40	31	1247.29	1277.83		0	0.75	1005.0	cycy233-l3-bc-v0-c1_v1	8B	cycy233/L3-bc-v0-c1	auto	11910	11611290	0.56	0.61	basic	Jellywibble/gpt2_xl_pairwise_89m_step_347634	16.0	512.0	64.0	LlamaForCausalLM	cycy233/L3-bc-v0-c1	2024-07-20	1.06	1.39	15	15	198	153