developer_uid: zonemercy
submission_id: zonemercy-gemini-edit-v1_1351_v2
model_name: 0821v1-3
model_group: zonemercy/Gemini-Edit-v1
status: torndown
timestamp: 2024-08-21T21:04:20+00:00
num_battles: 10950
num_wins: 5413
celo_rating: 1238.76
family_friendly_score: 0.0
submission_type: basic
model_repo: zonemercy/Gemini-Edit-v1-1e5ep1
model_architecture: MistralForCausalLM
model_num_parameters: 12772070400.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
display_name: 0821v1-3
is_internal_developer: True
language_model: zonemercy/Gemini-Edit-v1-1e5ep1
model_size: 13B
ranking_group: single
us_pacific_date: 2024-08-21
win_ratio: 0.494337899543379
generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '###', 'Bot:', 'User:', 'You:', '<|im_end|>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Running pipeline stage MKMLizer
Starting job with name zonemercy-gemini-edit-v1-1351-v2-mkmlizer
Waiting for job on zonemercy-gemini-edit-v1-1351-v2-mkmlizer to finish
zonemercy-gemini-edit-v1-1351-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
zonemercy-gemini-edit-v1-1351-v2-mkmlizer: ║     _____            __           __                                ║
zonemercy-gemini-edit-v1-1351-v2-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
zonemercy-gemini-edit-v1-1351-v2-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
zonemercy-gemini-edit-v1-1351-v2-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
zonemercy-gemini-edit-v1-1351-v2-mkmlizer: ║       /___/                                                         ║
zonemercy-gemini-edit-v1-1351-v2-mkmlizer: ║                                                                     ║
zonemercy-gemini-edit-v1-1351-v2-mkmlizer: ║  Version: 0.10.1                                                    ║
zonemercy-gemini-edit-v1-1351-v2-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
zonemercy-gemini-edit-v1-1351-v2-mkmlizer: ║  https://mk1.ai                                                     ║
zonemercy-gemini-edit-v1-1351-v2-mkmlizer: ║                                                                     ║
zonemercy-gemini-edit-v1-1351-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
zonemercy-gemini-edit-v1-1351-v2-mkmlizer: ║  belonging to:                                                      ║
zonemercy-gemini-edit-v1-1351-v2-mkmlizer: ║                                                                     ║
zonemercy-gemini-edit-v1-1351-v2-mkmlizer: ║  Chai Research Corp.                                                ║
zonemercy-gemini-edit-v1-1351-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
zonemercy-gemini-edit-v1-1351-v2-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
zonemercy-gemini-edit-v1-1351-v2-mkmlizer: ║                                                                     ║
zonemercy-gemini-edit-v1-1351-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
zonemercy-gemini-edit-v1-1351-v2-mkmlizer: Downloaded to shared memory in 59.687s
zonemercy-gemini-edit-v1-1351-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpreaxrix7, device:0
zonemercy-gemini-edit-v1-1351-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
zonemercy-gemini-edit-v1-1351-v2-mkmlizer: quantized model in 41.733s
zonemercy-gemini-edit-v1-1351-v2-mkmlizer: Processed model zonemercy/Gemini-Edit-v1-1e5ep1 in 101.420s
zonemercy-gemini-edit-v1-1351-v2-mkmlizer: creating bucket guanaco-mkml-models
zonemercy-gemini-edit-v1-1351-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
zonemercy-gemini-edit-v1-1351-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/zonemercy-gemini-edit-v1-1351-v2
zonemercy-gemini-edit-v1-1351-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/zonemercy-gemini-edit-v1-1351-v2/config.json
zonemercy-gemini-edit-v1-1351-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/zonemercy-gemini-edit-v1-1351-v2/special_tokens_map.json
zonemercy-gemini-edit-v1-1351-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/zonemercy-gemini-edit-v1-1351-v2/tokenizer_config.json
zonemercy-gemini-edit-v1-1351-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/zonemercy-gemini-edit-v1-1351-v2/tokenizer.json
zonemercy-gemini-edit-v1-1351-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/zonemercy-gemini-edit-v1-1351-v2/flywheel_model.0.safetensors
zonemercy-gemini-edit-v1-1351-v2-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:16, 21.72it/s]
Loading 0:   3%|▎         | 10/363 [00:00<00:12, 27.83it/s]
Loading 0:   4%|▍         | 14/363 [00:00<00:14, 24.53it/s]
Loading 0:   6%|▌         | 20/363 [00:00<00:10, 33.31it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:14, 23.10it/s]
Loading 0:   7%|▋         | 27/363 [00:01<00:15, 21.91it/s]
Loading 0:   9%|▊         | 31/363 [00:01<00:13, 25.14it/s]
Loading 0:   9%|▉         | 34/363 [00:01<00:13, 24.49it/s]
Loading 0:  11%|█         | 39/363 [00:01<00:11, 27.63it/s]
Loading 0:  12%|█▏        | 42/363 [00:01<00:12, 25.18it/s]
Loading 0:  13%|█▎        | 48/363 [00:01<00:10, 30.00it/s]
Loading 0:  14%|█▍        | 52/363 [00:01<00:10, 28.75it/s]
Loading 0:  15%|█▌        | 56/363 [00:02<00:10, 29.04it/s]
Loading 0:  17%|█▋        | 61/363 [00:02<00:12, 24.57it/s]
Loading 0:  18%|█▊        | 64/363 [00:02<00:13, 21.50it/s]
Loading 0:  20%|█▉        | 71/363 [00:02<00:10, 28.12it/s]
Loading 0:  21%|██        | 75/363 [00:02<00:10, 27.12it/s]
Loading 0:  21%|██▏       | 78/363 [00:03<00:11, 24.68it/s]
Loading 0:  23%|██▎       | 84/363 [00:03<00:09, 28.94it/s]
Loading 0:  24%|██▍       | 87/363 [00:03<00:10, 25.91it/s]
Loading 0:  26%|██▌       | 93/363 [00:03<00:08, 30.39it/s]
Loading 0:  27%|██▋       | 97/363 [00:03<00:09, 28.91it/s]
Loading 0:  28%|██▊       | 101/363 [00:03<00:11, 22.58it/s]
Loading 0:  29%|██▊       | 104/363 [00:04<00:12, 20.35it/s]
Loading 0:  31%|███       | 111/363 [00:04<00:09, 26.90it/s]
Loading 0:  31%|███▏      | 114/363 [00:04<00:09, 25.22it/s]
Loading 0:  33%|███▎      | 120/363 [00:04<00:08, 29.58it/s]
Loading 0:  34%|███▍      | 124/363 [00:04<00:08, 28.16it/s]
Loading 0:  36%|███▌      | 129/363 [00:04<00:07, 30.36it/s]
Loading 0:  37%|███▋      | 133/363 [00:05<00:07, 28.96it/s]
Loading 0:  38%|███▊      | 137/363 [00:05<00:07, 29.31it/s]
Loading 0:  39%|███▉      | 142/363 [00:05<00:09, 23.99it/s]
Loading 0:  40%|███▉      | 145/363 [00:05<00:09, 22.82it/s]
Loading 0:  41%|████      | 149/363 [00:05<00:09, 22.13it/s]
Loading 0:  43%|████▎     | 156/363 [00:05<00:07, 28.46it/s]
Loading 0:  44%|████▍     | 160/363 [00:06<00:07, 27.83it/s]
Loading 0:  45%|████▌     | 165/363 [00:06<00:06, 30.05it/s]
Loading 0:  47%|████▋     | 169/363 [00:06<00:06, 28.28it/s]
Loading 0:  48%|████▊     | 174/363 [00:06<00:06, 30.42it/s]
Loading 0:  49%|████▉     | 178/363 [00:06<00:06, 28.71it/s]
Loading 0:  50%|█████     | 182/363 [00:06<00:07, 23.17it/s]
Loading 0:  51%|█████     | 185/363 [00:07<00:08, 20.79it/s]
Loading 0:  53%|█████▎    | 192/363 [00:07<00:06, 27.45it/s]
Loading 0:  54%|█████▎    | 195/363 [00:07<00:06, 25.67it/s]
Loading 0:  55%|█████▌    | 201/363 [00:07<00:05, 30.16it/s]
Loading 0:  56%|█████▋    | 205/363 [00:07<00:05, 28.20it/s]
Loading 0:  57%|█████▋    | 208/363 [00:07<00:05, 28.52it/s]
Loading 0:  58%|█████▊    | 212/363 [00:08<00:05, 25.46it/s]
Loading 0:  60%|██████    | 218/363 [00:08<00:04, 30.17it/s]
Loading 0:  61%|██████▏   | 223/363 [00:08<00:05, 25.73it/s]
Loading 0:  62%|██████▏   | 226/363 [00:08<00:05, 24.16it/s]
Loading 0:  63%|██████▎   | 230/363 [00:08<00:05, 23.13it/s]
Loading 0:  65%|██████▌   | 237/363 [00:08<00:04, 29.48it/s]
Loading 0:  66%|██████▋   | 241/363 [00:09<00:04, 28.18it/s]
Loading 0:  68%|██████▊   | 246/363 [00:09<00:03, 30.01it/s]
Loading 0:  69%|██████▉   | 250/363 [00:09<00:03, 28.91it/s]
Loading 0:  70%|███████   | 255/363 [00:09<00:03, 31.29it/s]
Loading 0:  71%|███████▏  | 259/363 [00:09<00:03, 29.66it/s]
Loading 0:  72%|███████▏  | 263/363 [00:09<00:04, 23.96it/s]
Loading 0:  73%|███████▎  | 266/363 [00:10<00:04, 21.28it/s]
Loading 0:  75%|███████▌  | 273/363 [00:10<00:03, 27.80it/s]
Loading 0:  76%|███████▌  | 276/363 [00:10<00:03, 25.46it/s]
Loading 0:  77%|███████▋  | 280/363 [00:10<00:02, 28.21it/s]
Loading 0:  78%|███████▊  | 284/363 [00:10<00:03, 25.27it/s]
Loading 0:  80%|████████  | 291/363 [00:10<00:02, 31.80it/s]
Loading 0:  81%|████████▏ | 295/363 [00:11<00:02, 29.69it/s]
Loading 0:  82%|████████▏ | 299/363 [00:11<00:02, 29.68it/s]
Loading 0:  84%|████████▎ | 304/363 [00:11<00:02, 25.09it/s]
Loading 0:  85%|████████▍ | 307/363 [00:11<00:02, 23.73it/s]
Loading 0:  86%|████████▌ | 311/363 [00:11<00:02, 22.51it/s]
Loading 0:  88%|████████▊ | 318/363 [00:11<00:01, 29.12it/s]
Loading 0:  89%|████████▊ | 322/363 [00:12<00:01, 28.19it/s]
Loading 0:  90%|█████████ | 327/363 [00:12<00:01, 30.71it/s]
Loading 0:  91%|█████████ | 331/363 [00:12<00:01, 29.21it/s]
Loading 0:  93%|█████████▎| 336/363 [00:12<00:00, 31.78it/s]
Loading 0:  94%|█████████▎| 340/363 [00:12<00:00, 30.10it/s]
Loading 0:  95%|█████████▍| 344/363 [00:19<00:09,  1.97it/s]
Loading 0:  96%|█████████▌| 348/363 [00:19<00:05,  2.65it/s]
Loading 0:  97%|█████████▋| 353/363 [00:19<00:02,  3.84it/s]
Loading 0:  98%|█████████▊| 357/363 [00:20<00:01,  4.98it/s]
                                                            

Job zonemercy-gemini-edit-v1-1351-v2-mkmlizer completed after 162.71s with status: succeeded
Stopping job with name zonemercy-gemini-edit-v1-1351-v2-mkmlizer
Pipeline stage MKMLizer completed in 163.61s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.09s
Running pipeline stage ISVCDeployer
Creating inference service zonemercy-gemini-edit-v1-1351-v2
Waiting for inference service zonemercy-gemini-edit-v1-1351-v2 to be ready
Failed to get response for submission blend_filor_2024-08-16: ('http://zonemercy-lexical-nemo-1518-v18-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:46334->127.0.0.1:8080: read: connection reset by peer\n')
Failed to get response for submission zonemercy-graft-cogent-v_7573_v7: ('http://zonemercy-graft-cogent-v-7573-v7-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'EOF\n')
Failed to get response for submission zonemercy-graft-cogent-v_7573_v7: ('http://zonemercy-graft-cogent-v-7573-v7-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'request timeout')
Inference service zonemercy-gemini-edit-v1-1351-v2 ready after 252.69587397575378s
Pipeline stage ISVCDeployer completed in 253.35s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.526567220687866s
Received healthy response to inference request in 1.6421422958374023s
Received healthy response to inference request in 2.3880679607391357s
Received healthy response to inference request in 1.195979356765747s
Received healthy response to inference request in 1.9678130149841309s
5 requests
0 failed requests
5th percentile: 1.285211944580078
10th percentile: 1.3744445323944092
20th percentile: 1.5529097080230714
30th percentile: 1.7072764396667481
40th percentile: 1.8375447273254395
50th percentile: 1.9678130149841309
60th percentile: 2.135914993286133
70th percentile: 2.3040169715881347
80th percentile: 2.4157678127288817
90th percentile: 2.471167516708374
95th percentile: 2.49886736869812
99th percentile: 2.521027250289917
mean time: 1.9441139698028564
Pipeline stage StressChecker completed in 10.46s
zonemercy-gemini-edit-v1_1351_v2 status is now deployed due to DeploymentManager action
zonemercy-gemini-edit-v1_1351_v2 status is now inactive due to auto deactivation removed underperforming models
zonemercy-gemini-edit-v1_1351_v2 status is now torndown due to DeploymentManager action