submission_id: mlabonne-ultramerge-7b_v1
developer_uid: Meliodia
status: torndown
model_repo: mlabonne/UltraMerge-7B
reward_repo: ChaiML/reward_gpt2_medium_preference_24m_e2
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
reward_formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
timestamp: 2024-04-16T05:02:04+00:00
model_name: mlabonne-ultramerge-7b_v1
model_eval_status: success
model_group: mlabonne/UltraMerge-7B
num_battles: 6112
num_wins: 3017
celo_rating: 1139.88
propriety_score: 0.0
propriety_total_count: 0.0
submission_type: basic
model_architecture: MistralForCausalLM
model_num_parameters: 7241732096.0
best_of: 16
max_input_tokens: 512
max_output_tokens: 64
display_name: mlabonne-ultramerge-7b_v1
ineligible_reason: propriety_total_count < 800
language_model: mlabonne/UltraMerge-7B
model_size: 7B
reward_model: ChaiML/reward_gpt2_medium_preference_24m_e2
us_pacific_date: 2024-04-15
win_ratio: 0.49361910994764396
preference_data_url: None
Resubmit model
Running pipeline stage MKMLizer
Starting job with name mlabonne-ultramerge-7b-v1-mkmlizer
Waiting for job on mlabonne-ultramerge-7b-v1-mkmlizer to finish
mlabonne-ultramerge-7b-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mlabonne-ultramerge-7b-v1-mkmlizer: ║     _____            __           __                                ║
mlabonne-ultramerge-7b-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
mlabonne-ultramerge-7b-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
mlabonne-ultramerge-7b-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
mlabonne-ultramerge-7b-v1-mkmlizer: ║       /___/                                                         ║
mlabonne-ultramerge-7b-v1-mkmlizer: ║                                                                     ║
mlabonne-ultramerge-7b-v1-mkmlizer: ║  Version: 0.6.11                                                    ║
mlabonne-ultramerge-7b-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
mlabonne-ultramerge-7b-v1-mkmlizer: ║                                                                     ║
mlabonne-ultramerge-7b-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
mlabonne-ultramerge-7b-v1-mkmlizer: ║  belonging to:                                                      ║
mlabonne-ultramerge-7b-v1-mkmlizer: ║                                                                     ║
mlabonne-ultramerge-7b-v1-mkmlizer: ║  Chai Research Corp.                                                ║
mlabonne-ultramerge-7b-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
mlabonne-ultramerge-7b-v1-mkmlizer: ║  Expiration: 2024-07-15 23:59:59                                    ║
mlabonne-ultramerge-7b-v1-mkmlizer: ║                                                                     ║
mlabonne-ultramerge-7b-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
mlabonne-ultramerge-7b-v1-mkmlizer: 
.gitattributes:   0%|          | 0.00/1.52k [00:00<?, ?B/s]
.gitattributes: 100%|██████████| 1.52k/1.52k [00:00<00:00, 3.83MB/s]
mlabonne-ultramerge-7b-v1-mkmlizer: 
model-00001-of-00003.safetensors:   0%|          | 0.00/4.94G [00:00<?, ?B/s]
model-00001-of-00003.safetensors:   0%|          | 10.5M/4.94G [00:00<05:18, 15.5MB/s]
model-00001-of-00003.safetensors:   1%|          | 31.5M/4.94G [00:01<04:37, 17.7MB/s]
model-00001-of-00003.safetensors:   1%|          | 41.9M/4.94G [00:02<05:01, 16.3MB/s]
model-00001-of-00003.safetensors:   1%|          | 52.4M/4.94G [00:02<03:39, 22.2MB/s]
model-00001-of-00003.safetensors:   1%|▏         | 73.4M/4.94G [00:02<02:07, 38.3MB/s]
model-00001-of-00003.safetensors:   2%|▏         | 94.4M/4.94G [00:02<01:32, 52.2MB/s]
model-00001-of-00003.safetensors:   3%|▎         | 136M/4.94G [00:03<00:49, 96.2MB/s] 
model-00001-of-00003.safetensors:   6%|▌         | 283M/4.94G [00:03<00:15, 296MB/s] 
model-00001-of-00003.safetensors:   7%|▋         | 346M/4.94G [00:03<00:17, 270MB/s]
model-00001-of-00003.safetensors:   9%|▉         | 461M/4.94G [00:03<00:11, 407MB/s]
model-00001-of-00003.safetensors:  19%|█▉        | 954M/4.94G [00:03<00:03, 1.24GB/s]
model-00001-of-00003.safetensors:  24%|██▍       | 1.20G/4.94G [00:03<00:02, 1.33GB/s]
model-00001-of-00003.safetensors:  28%|██▊       | 1.38G/4.94G [00:04<00:04, 840MB/s] 
model-00001-of-00003.safetensors:  31%|███       | 1.53G/4.94G [00:04<00:06, 567MB/s]
model-00001-of-00003.safetensors:  34%|███▎      | 1.67G/4.94G [00:04<00:04, 658MB/s]
model-00001-of-00003.safetensors:  36%|███▌      | 1.78G/4.94G [00:05<00:04, 659MB/s]
model-00001-of-00003.safetensors:  38%|███▊      | 1.89G/4.94G [00:05<00:04, 694MB/s]
model-00001-of-00003.safetensors:  44%|████▎     | 2.16G/4.94G [00:05<00:02, 1.05GB/s]
model-00001-of-00003.safetensors:  52%|█████▏    | 2.57G/4.94G [00:05<00:01, 1.63GB/s]
model-00001-of-00003.safetensors:  56%|█████▋    | 2.79G/4.94G [00:06<00:02, 827MB/s] 
model-00001-of-00003.safetensors:  60%|█████▉    | 2.96G/4.94G [00:06<00:02, 769MB/s]
model-00001-of-00003.safetensors:  63%|██████▎   | 3.14G/4.94G [00:06<00:02, 896MB/s]
model-00001-of-00003.safetensors:  68%|██████▊   | 3.36G/4.94G [00:06<00:01, 1.09GB/s]
model-00001-of-00003.safetensors:  71%|███████▏  | 3.52G/4.94G [00:06<00:01, 1.12GB/s]
model-00001-of-00003.safetensors:  77%|███████▋  | 3.83G/4.94G [00:06<00:00, 1.42GB/s]
model-00001-of-00003.safetensors:  81%|████████  | 4.01G/4.94G [00:07<00:01, 828MB/s] 
model-00001-of-00003.safetensors:  84%|████████▍ | 4.14G/4.94G [00:07<00:01, 748MB/s]
model-00001-of-00003.safetensors:  86%|████████▌ | 4.26G/4.94G [00:07<00:00, 691MB/s]
model-00001-of-00003.safetensors:  89%|████████▉ | 4.41G/4.94G [00:07<00:00, 804MB/s]
model-00001-of-00003.safetensors:  92%|█████████▏| 4.54G/4.94G [00:07<00:00, 887MB/s]
model-00001-of-00003.safetensors: 100%|█████████▉| 4.94G/4.94G [00:08<00:00, 610MB/s]
mlabonne-ultramerge-7b-v1-mkmlizer: 
model-00002-of-00003.safetensors:   0%|          | 0.00/5.00G [00:00<?, ?B/s]
model-00002-of-00003.safetensors:   0%|          | 10.5M/5.00G [00:02<16:35, 5.01MB/s]
model-00002-of-00003.safetensors:   0%|          | 21.0M/5.00G [00:02<09:32, 8.70MB/s]
model-00002-of-00003.safetensors:   1%|▏         | 62.9M/5.00G [00:02<02:20, 35.2MB/s]
model-00002-of-00003.safetensors:   2%|▏         | 94.4M/5.00G [00:02<01:27, 55.8MB/s]
model-00002-of-00003.safetensors:   3%|▎         | 168M/5.00G [00:03<00:39, 123MB/s]  
model-00002-of-00003.safetensors:   8%|▊         | 388M/5.00G [00:03<00:14, 312MB/s]
model-00002-of-00003.safetensors:  10%|▉         | 482M/5.00G [00:03<00:11, 392MB/s]
model-00002-of-00003.safetensors:  20%|██        | 1.02G/5.00G [00:03<00:03, 1.15GB/s]
model-00002-of-00003.safetensors:  25%|██▍       | 1.24G/5.00G [00:04<00:06, 562MB/s] 
model-00002-of-00003.safetensors:  28%|██▊       | 1.39G/5.00G [00:04<00:05, 634MB/s]
model-00002-of-00003.safetensors:  31%|███       | 1.54G/5.00G [00:04<00:05, 595MB/s]
model-00002-of-00003.safetensors:  33%|███▎      | 1.66G/5.00G [00:05<00:05, 633MB/s]
model-00002-of-00003.safetensors:  35%|███▌      | 1.76G/5.00G [00:05<00:04, 662MB/s]
model-00002-of-00003.safetensors:  37%|███▋      | 1.87G/5.00G [00:05<00:04, 707MB/s]
model-00002-of-00003.safetensors:  39%|███▉      | 1.97G/5.00G [00:05<00:04, 756MB/s]
model-00002-of-00003.safetensors:  42%|████▏     | 2.08G/5.00G [00:05<00:04, 726MB/s]
model-00002-of-00003.safetensors:  43%|████▎     | 2.17G/5.00G [00:05<00:04, 685MB/s]
model-00002-of-00003.safetensors:  46%|████▌     | 2.30G/5.00G [00:05<00:03, 784MB/s]
model-00002-of-00003.safetensors:  48%|████▊     | 2.39G/5.00G [00:05<00:03, 753MB/s]
model-00002-of-00003.safetensors:  53%|█████▎    | 2.67G/5.00G [00:06<00:01, 1.23GB/s]
model-00002-of-00003.safetensors:  60%|██████    | 3.01G/5.00G [00:06<00:01, 1.74GB/s]
model-00002-of-00003.safetensors:  65%|██████▍   | 3.24G/5.00G [00:06<00:00, 1.78GB/s]
model-00002-of-00003.safetensors:  69%|██████▉   | 3.44G/5.00G [00:07<00:02, 682MB/s] 
model-00002-of-00003.safetensors:  72%|███████▏  | 3.59G/5.00G [00:07<00:02, 682MB/s]
model-00002-of-00003.safetensors:  74%|███████▍  | 3.71G/5.00G [00:07<00:01, 695MB/s]
model-00002-of-00003.safetensors:  77%|███████▋  | 3.83G/5.00G [00:07<00:01, 750MB/s]
model-00002-of-00003.safetensors:  80%|████████  | 4.02G/5.00G [00:07<00:01, 938MB/s]
model-00002-of-00003.safetensors:  85%|████████▍ | 4.23G/5.00G [00:07<00:00, 1.16GB/s]
model-00002-of-00003.safetensors:  88%|████████▊ | 4.38G/5.00G [00:07<00:00, 1.10GB/s]
model-00002-of-00003.safetensors:  90%|█████████ | 4.52G/5.00G [00:08<00:00, 943MB/s] 
model-00002-of-00003.safetensors:  94%|█████████▎| 4.68G/5.00G [00:08<00:00, 1.06GB/s]
model-00002-of-00003.safetensors:  97%|█████████▋| 4.84G/5.00G [00:08<00:00, 1.19GB/s]
model-00002-of-00003.safetensors: 100%|█████████▉| 5.00G/5.00G [00:08<00:00, 665MB/s] 
model-00002-of-00003.safetensors: 100%|█████████▉| 5.00G/5.00G [00:08<00:00, 565MB/s]
mlabonne-ultramerge-7b-v1-mkmlizer: 
model-00003-of-00003.safetensors:   0%|          | 0.00/4.54G [00:00<?, ?B/s]
model-00003-of-00003.safetensors:   0%|          | 10.5M/4.54G [00:01<09:28, 7.96MB/s]
model-00003-of-00003.safetensors:   1%|          | 31.5M/4.54G [00:01<02:50, 26.5MB/s]
model-00003-of-00003.safetensors:   1%|          | 52.4M/4.54G [00:01<01:33, 47.9MB/s]
model-00003-of-00003.safetensors:   2%|▏         | 73.4M/4.54G [00:01<01:13, 61.0MB/s]
model-00003-of-00003.safetensors:   2%|▏         | 105M/4.54G [00:01<00:49, 89.9MB/s] 
model-00003-of-00003.safetensors:   3%|▎         | 126M/4.54G [00:02<00:45, 97.3MB/s]
model-00003-of-00003.safetensors:   3%|▎         | 147M/4.54G [00:02<00:41, 106MB/s] 
model-00003-of-00003.safetensors:   4%|▍         | 189M/4.54G [00:02<00:26, 162MB/s]
model-00003-of-00003.safetensors:   5%|▍         | 220M/4.54G [00:02<00:23, 184MB/s]
model-00003-of-00003.safetensors:   6%|▌         | 273M/4.54G [00:02<00:16, 257MB/s]
model-00003-of-00003.safetensors:   8%|▊         | 357M/4.54G [00:02<00:11, 373MB/s]
model-00003-of-00003.safetensors:  12%|█▏        | 535M/4.54G [00:02<00:05, 710MB/s]
model-00003-of-00003.safetensors:  14%|█▎        | 619M/4.54G [00:02<00:05, 714MB/s]
model-00003-of-00003.safetensors:  17%|█▋        | 765M/4.54G [00:03<00:04, 905MB/s]
model-00003-of-00003.safetensors:  27%|██▋       | 1.24G/4.54G [00:03<00:02, 1.34GB/s]
model-00003-of-00003.safetensors:  30%|███       | 1.36G/4.54G [00:03<00:04, 719MB/s] 
model-00003-of-00003.safetensors:  32%|███▏      | 1.46G/4.54G [00:04<00:04, 659MB/s]
model-00003-of-00003.safetensors:  36%|███▌      | 1.64G/4.54G [00:04<00:03, 824MB/s]
model-00003-of-00003.safetensors:  39%|███▊      | 1.75G/4.54G [00:04<00:03, 725MB/s]
model-00003-of-00003.safetensors:  42%|████▏     | 1.92G/4.54G [00:04<00:02, 884MB/s]
model-00003-of-00003.safetensors:  46%|████▌     | 2.08G/4.54G [00:04<00:02, 1.00GB/s]
model-00003-of-00003.safetensors:  49%|████▊     | 2.21G/4.54G [00:04<00:02, 1.07GB/s]
model-00003-of-00003.safetensors:  52%|█████▏    | 2.37G/4.54G [00:04<00:01, 1.18GB/s]
model-00003-of-00003.safetensors:  55%|█████▌    | 2.51G/4.54G [00:04<00:01, 1.15GB/s]
model-00003-of-00003.safetensors:  61%|██████    | 2.78G/4.54G [00:04<00:01, 1.53GB/s]
model-00003-of-00003.safetensors:  65%|██████▍   | 2.95G/4.54G [00:05<00:01, 1.47GB/s]
model-00003-of-00003.safetensors:  68%|██████▊   | 3.10G/4.54G [00:05<00:01, 986MB/s] 
model-00003-of-00003.safetensors:  71%|███████   | 3.23G/4.54G [00:05<00:01, 929MB/s]
model-00003-of-00003.safetensors:  74%|███████▎  | 3.34G/4.54G [00:05<00:01, 755MB/s]
model-00003-of-00003.safetensors:  76%|███████▌  | 3.44G/4.54G [00:05<00:01, 774MB/s]
model-00003-of-00003.safetensors:  78%|███████▊  | 3.55G/4.54G [00:06<00:01, 850MB/s]
model-00003-of-00003.safetensors:  82%|████████▏ | 3.71G/4.54G [00:06<00:00, 973MB/s]
model-00003-of-00003.safetensors:  84%|████████▍ | 3.83G/4.54G [00:06<00:00, 989MB/s]
model-00003-of-00003.safetensors:  91%|█████████ | 4.11G/4.54G [00:06<00:00, 1.44GB/s]
model-00003-of-00003.safetensors:  94%|█████████▍| 4.28G/4.54G [00:06<00:00, 1.27GB/s]
model-00003-of-00003.safetensors:  98%|█████████▊| 4.46G/4.54G [00:06<00:00, 1.29GB/s]
model-00003-of-00003.safetensors: 100%|█████████▉| 4.54G/4.54G [00:06<00:00, 673MB/s] 
mlabonne-ultramerge-7b-v1-mkmlizer: 
model.safetensors.index.json:   0%|          | 0.00/23.9k [00:00<?, ?B/s]
model.safetensors.index.json: 100%|██████████| 23.9k/23.9k [00:00<00:00, 123MB/s]
mlabonne-ultramerge-7b-v1-mkmlizer: 
special_tokens_map.json:   0%|          | 0.00/552 [00:00<?, ?B/s]
special_tokens_map.json: 100%|██████████| 552/552 [00:00<00:00, 9.04MB/s]
mlabonne-ultramerge-7b-v1-mkmlizer: 
tokenizer.json:   0%|          | 0.00/1.80M [00:00<?, ?B/s]
tokenizer.json: 100%|██████████| 1.80M/1.80M [00:00<00:00, 29.4MB/s]
mlabonne-ultramerge-7b-v1-mkmlizer: 
tokenizer.model:   0%|          | 0.00/493k [00:00<?, ?B/s]
tokenizer.model: 100%|██████████| 493k/493k [00:00<00:00, 52.5MB/s]
mlabonne-ultramerge-7b-v1-mkmlizer: 
tokenizer_config.json:   0%|          | 0.00/971 [00:00<?, ?B/s]
tokenizer_config.json: 100%|██████████| 971/971 [00:00<00:00, 7.99MB/s]
mlabonne-ultramerge-7b-v1-mkmlizer: Downloaded to shared memory in 26.044s
mlabonne-ultramerge-7b-v1-mkmlizer: quantizing model to /dev/shm/model_cache
mlabonne-ultramerge-7b-v1-mkmlizer: Saving mkml model at /dev/shm/model_cache
mlabonne-ultramerge-7b-v1-mkmlizer: Reading /tmp/tmpx9rjgee_/model.safetensors.index.json
mlabonne-ultramerge-7b-v1-mkmlizer: 
Profiling:   0%|          | 0/291 [00:00<?, ?it/s]
Profiling:   0%|          | 1/291 [00:01<07:22,  1.53s/it]
Profiling:   4%|▍         | 12/291 [00:01<00:28,  9.96it/s]
Profiling:   7%|▋         | 21/291 [00:01<00:14, 18.54it/s]
Profiling:  12%|█▏        | 36/291 [00:01<00:07, 35.17it/s]
Profiling:  16%|█▌        | 46/291 [00:01<00:05, 44.78it/s]
Profiling:  20%|█▉        | 57/291 [00:02<00:04, 56.51it/s]
Profiling:  25%|██▍       | 72/291 [00:02<00:02, 73.36it/s]
Profiling:  29%|██▊       | 83/291 [00:02<00:02, 77.90it/s]
Profiling:  33%|███▎      | 97/291 [00:02<00:02, 92.18it/s]
Profiling:  37%|███▋      | 109/291 [00:02<00:03, 54.85it/s]
Profiling:  41%|████      | 120/291 [00:02<00:02, 63.30it/s]
Profiling:  45%|████▍     | 130/291 [00:03<00:02, 69.52it/s]
Profiling:  48%|████▊     | 140/291 [00:03<00:02, 74.30it/s]
Profiling:  53%|█████▎    | 154/291 [00:03<00:01, 89.14it/s]
Profiling:  57%|█████▋    | 165/291 [00:03<00:01, 91.64it/s]
Profiling:  60%|██████    | 176/291 [00:03<00:01, 91.41it/s]
Profiling:  65%|██████▍   | 189/291 [00:03<00:01, 101.21it/s]
Profiling:  70%|███████   | 204/291 [00:05<00:04, 20.79it/s] 
Profiling:  73%|███████▎  | 212/291 [00:05<00:03, 24.45it/s]
Profiling:  76%|███████▋  | 222/291 [00:05<00:02, 30.76it/s]
Profiling:  82%|████████▏ | 238/291 [00:05<00:01, 43.79it/s]
Profiling:  85%|████████▌ | 248/291 [00:05<00:00, 50.92it/s]
Profiling:  89%|████████▉ | 259/291 [00:05<00:00, 60.12it/s]
Profiling:  94%|█████████▍| 274/291 [00:06<00:00, 75.10it/s]
Profiling:  98%|█████████▊| 286/291 [00:06<00:00, 80.58it/s]
Profiling: 100%|██████████| 291/291 [00:06<00:00, 45.84it/s]
mlabonne-ultramerge-7b-v1-mkmlizer: quantized model in 17.834s
mlabonne-ultramerge-7b-v1-mkmlizer: Processed model mlabonne/UltraMerge-7B in 45.055s
mlabonne-ultramerge-7b-v1-mkmlizer: creating bucket guanaco-mkml-models
mlabonne-ultramerge-7b-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
mlabonne-ultramerge-7b-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/mlabonne-ultramerge-7b-v1
mlabonne-ultramerge-7b-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/mlabonne-ultramerge-7b-v1/special_tokens_map.json
mlabonne-ultramerge-7b-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/mlabonne-ultramerge-7b-v1/tokenizer_config.json
mlabonne-ultramerge-7b-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/mlabonne-ultramerge-7b-v1/config.json
mlabonne-ultramerge-7b-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/mlabonne-ultramerge-7b-v1/tokenizer.json
mlabonne-ultramerge-7b-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.model s3://guanaco-mkml-models/mlabonne-ultramerge-7b-v1/tokenizer.model
mlabonne-ultramerge-7b-v1-mkmlizer: cp /dev/shm/model_cache/mkml_model.tensors s3://guanaco-mkml-models/mlabonne-ultramerge-7b-v1/mkml_model.tensors
mlabonne-ultramerge-7b-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:1067: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mlabonne-ultramerge-7b-v1-mkmlizer:   warnings.warn(
mlabonne-ultramerge-7b-v1-mkmlizer: loading reward model from ChaiML/reward_gpt2_medium_preference_24m_e2
mlabonne-ultramerge-7b-v1-mkmlizer: 
config.json:   0%|          | 0.00/1.05k [00:00<?, ?B/s]
config.json: 100%|██████████| 1.05k/1.05k [00:00<00:00, 8.13MB/s]
mlabonne-ultramerge-7b-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:690: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mlabonne-ultramerge-7b-v1-mkmlizer:   warnings.warn(
mlabonne-ultramerge-7b-v1-mkmlizer: 
tokenizer_config.json:   0%|          | 0.00/234 [00:00<?, ?B/s]
tokenizer_config.json: 100%|██████████| 234/234 [00:00<00:00, 2.21MB/s]
mlabonne-ultramerge-7b-v1-mkmlizer: 
vocab.json:   0%|          | 0.00/1.04M [00:00<?, ?B/s]
vocab.json: 100%|██████████| 1.04M/1.04M [00:00<00:00, 25.4MB/s]
mlabonne-ultramerge-7b-v1-mkmlizer: 
tokenizer.json:   0%|          | 0.00/2.11M [00:00<?, ?B/s]
tokenizer.json: 100%|██████████| 2.11M/2.11M [00:00<00:00, 35.1MB/s]
mlabonne-ultramerge-7b-v1-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:472: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mlabonne-ultramerge-7b-v1-mkmlizer:   warnings.warn(
mlabonne-ultramerge-7b-v1-mkmlizer: 
pytorch_model.bin:   0%|          | 0.00/1.44G [00:00<?, ?B/s]
pytorch_model.bin:   1%|          | 10.5M/1.44G [00:00<00:43, 32.8MB/s]
pytorch_model.bin:   1%|▏         | 21.0M/1.44G [00:00<00:26, 53.4MB/s]
pytorch_model.bin:   3%|▎         | 41.9M/1.44G [00:00<00:18, 77.2MB/s]
pytorch_model.bin:   5%|▌         | 73.4M/1.44G [00:00<00:10, 129MB/s] 
pytorch_model.bin:   8%|▊         | 115M/1.44G [00:01<00:09, 139MB/s] 
pytorch_model.bin:  11%|█         | 157M/1.44G [00:01<00:06, 191MB/s]
pytorch_model.bin:  16%|█▌        | 231M/1.44G [00:01<00:03, 307MB/s]
pytorch_model.bin:  23%|██▎       | 325M/1.44G [00:01<00:02, 452MB/s]
pytorch_model.bin:  27%|██▋       | 388M/1.44G [00:01<00:02, 453MB/s]
pytorch_model.bin:  41%|████▏     | 598M/1.44G [00:01<00:01, 725MB/s]
pytorch_model.bin:  46%|████▋     | 671M/1.44G [00:03<00:04, 181MB/s]
pytorch_model.bin:  53%|█████▎    | 765M/1.44G [00:03<00:02, 228MB/s]
pytorch_model.bin:  57%|█████▋    | 828M/1.44G [00:03<00:02, 216MB/s]
pytorch_model.bin:  99%|█████████▊| 1.42G/1.44G [00:03<00:00, 776MB/s]
pytorch_model.bin: 100%|█████████▉| 1.44G/1.44G [00:03<00:00, 384MB/s]
mlabonne-ultramerge-7b-v1-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
mlabonne-ultramerge-7b-v1-mkmlizer: Saving duration: 0.261s
mlabonne-ultramerge-7b-v1-mkmlizer: Processed model ChaiML/reward_gpt2_medium_preference_24m_e2 in 7.589s
mlabonne-ultramerge-7b-v1-mkmlizer: creating bucket guanaco-reward-models
mlabonne-ultramerge-7b-v1-mkmlizer: Bucket 's3://guanaco-reward-models/' created
mlabonne-ultramerge-7b-v1-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/mlabonne-ultramerge-7b-v1_reward
mlabonne-ultramerge-7b-v1-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/mlabonne-ultramerge-7b-v1_reward/config.json
mlabonne-ultramerge-7b-v1-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/mlabonne-ultramerge-7b-v1_reward/tokenizer_config.json
mlabonne-ultramerge-7b-v1-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/mlabonne-ultramerge-7b-v1_reward/merges.txt
mlabonne-ultramerge-7b-v1-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/mlabonne-ultramerge-7b-v1_reward/special_tokens_map.json
mlabonne-ultramerge-7b-v1-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/mlabonne-ultramerge-7b-v1_reward/vocab.json
mlabonne-ultramerge-7b-v1-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/mlabonne-ultramerge-7b-v1_reward/tokenizer.json
mlabonne-ultramerge-7b-v1-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/mlabonne-ultramerge-7b-v1_reward/reward.tensors
Job mlabonne-ultramerge-7b-v1-mkmlizer completed after 84.09s with status: succeeded
Stopping job with name mlabonne-ultramerge-7b-v1-mkmlizer
Pipeline stage MKMLizer completed in 86.95s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.09s
Running pipeline stage ISVCDeployer
Creating inference service mlabonne-ultramerge-7b-v1
Waiting for inference service mlabonne-ultramerge-7b-v1 to be ready
Inference service mlabonne-ultramerge-7b-v1 ready after 40.21697473526001s
Pipeline stage ISVCDeployer completed in 47.30s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.7271618843078613s
Received healthy response to inference request in 1.1826939582824707s
Received healthy response to inference request in 1.186736822128296s
Received healthy response to inference request in 1.2090601921081543s
Received healthy response to inference request in 1.1978521347045898s
5 requests
0 failed requests
5th percentile: 1.1835025310516358
10th percentile: 1.1843111038208007
20th percentile: 1.1859282493591308
30th percentile: 1.1889598846435547
40th percentile: 1.1934060096740722
50th percentile: 1.1978521347045898
60th percentile: 1.2023353576660156
70th percentile: 1.2068185806274414
80th percentile: 1.3126805305480957
90th percentile: 1.5199212074279786
95th percentile: 1.6235415458679199
99th percentile: 1.706437816619873
mean time: 1.3007009983062745
Pipeline stage StressChecker completed in 7.27s
Running pipeline stage DaemonicModelEvalScorer
Pipeline stage DaemonicModelEvalScorer completed in 0.03s
Running pipeline stage DaemonicSafetyScorer
Pipeline stage DaemonicSafetyScorer completed in 0.04s
Running M-Eval for topic stay_in_character
mlabonne-ultramerge-7b_v1 status is now deployed due to DeploymentManager action
M-Eval Dataset for topic stay_in_character is loaded
mlabonne-ultramerge-7b_v1 status is now inactive due to auto deactivation removed underperforming models
admin requested tearing down of mlabonne-ultramerge-7b_v1
Toredown service mistralai-mixtral-8x7b-3473-v15
Running pipeline stage ISVCDeleter
Pipeline stage ISVCDeleter completed in 3.57s
Running pipeline stage MKMLModelDeleter
Checking if service mlabonne-ultramerge-7b-v1 is running
Cleaning model data from S3
Cleaning model data from model cache
Deleting key mistralai-mixtral-8x7b-3473-v15/config.json from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v15/flywheel_model.0.safetensors from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v15/flywheel_model.1.safetensors from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v15/flywheel_model.2.safetensors from bucket guanaco-mkml-models
Tearing down inference service mlabonne-ultramerge-7b-v1
Toredown service mlabonne-ultramerge-7b-v1
Pipeline stage ISVCDeleter completed in 3.92s
Running pipeline stage MKMLModelDeleter
admin requested tearing down of nitral-ai-eris-primev4-4_7044_v2
Cleaning model data from S3
Running pipeline stage ISVCDeleter
Cleaning model data from model cache
Deleting key mistralai-mixtral-8x7b-3473-v15/flywheel_model.3.safetensors from bucket guanaco-mkml-models
Checking if service nitral-ai-eris-primev4-4-7044-v2 is running
Deleting key mlabonne-ultramerge-7b-v1/config.json from bucket guanaco-mkml-models
Deleting key mlabonne-ultramerge-7b-v1/mkml_model.tensors from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v15/special_tokens_map.json from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v15/tokenizer.json from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v15/tokenizer.model from bucket guanaco-mkml-models
Deleting key mlabonne-ultramerge-7b-v1/special_tokens_map.json from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v15/tokenizer_config.json from bucket guanaco-mkml-models
Deleting key mlabonne-ultramerge-7b-v1/tokenizer.json from bucket guanaco-mkml-models
Cleaning model data from model cache
Deleting key mlabonne-ultramerge-7b-v1/tokenizer.model from bucket guanaco-mkml-models
Deleting key mlabonne-ultramerge-7b-v1/tokenizer_config.json from bucket guanaco-mkml-models
Deleting key mistralai-mixtral-8x7b-3473-v15_reward/config.json from bucket guanaco-reward-models
Cleaning model data from model cache
Deleting key mistralai-mixtral-8x7b-3473-v15_reward/merges.txt from bucket guanaco-reward-models
Deleting key mlabonne-ultramerge-7b-v1_reward/config.json from bucket guanaco-reward-models
Deleting key mistralai-mixtral-8x7b-3473-v15_reward/reward.tensors from bucket guanaco-reward-models
Deleting key mlabonne-ultramerge-7b-v1_reward/merges.txt from bucket guanaco-reward-models
Deleting key mistralai-mixtral-8x7b-3473-v15_reward/special_tokens_map.json from bucket guanaco-reward-models
Deleting key mlabonne-ultramerge-7b-v1_reward/reward.tensors from bucket guanaco-reward-models
Deleting key mistralai-mixtral-8x7b-3473-v15_reward/tokenizer.json from bucket guanaco-reward-models
Tearing down inference service nitral-ai-eris-primev4-4-7044-v2
Deleting key mlabonne-ultramerge-7b-v1_reward/special_tokens_map.json from bucket guanaco-reward-models
Deleting key mistralai-mixtral-8x7b-3473-v15_reward/tokenizer_config.json from bucket guanaco-reward-models
Toredown service nitral-ai-eris-primev4-4-7044-v2
Deleting key mlabonne-ultramerge-7b-v1_reward/tokenizer.json from bucket guanaco-reward-models
Deleting key mistralai-mixtral-8x7b-3473-v15_reward/vocab.json from bucket guanaco-reward-models
Deleting key mlabonne-ultramerge-7b-v1_reward/tokenizer_config.json from bucket guanaco-reward-models
Pipeline stage ISVCDeleter completed in 4.22s
Pipeline stage MKMLModelDeleter completed in 8.55s
Deleting key mlabonne-ultramerge-7b-v1_reward/vocab.json from bucket guanaco-reward-models
Running pipeline stage MKMLModelDeleter
mistralai-mixtral-8x7b-_3473_v15 status is now torndown due to DeploymentManager action
Cleaning model data from S3
Cleaning model data from model cache
mlabonne-ultramerge-7b_v1 status is now torndown due to DeploymentManager action

Usage Metrics

Latency Metrics