zonemercy-lexens-v1-ipo-_2405

developer_uid: zonemercy

submission_id: zonemercy-lexens-v1-ipo-_2405_v1

model_name: zonemercy-lexens-v1-ipo-_2405_v1

model_group: zonemercy/LexEns-v1-ipo-

status: inactive

timestamp: 2024-12-17T15:40:06+00:00

num_battles: 15314

num_wins: 6149

celo_rating: 1193.61

family_friendly_score: 0.607

family_friendly_standard_error: 0.006907257053273753

submission_type: basic

model_repo: zonemercy/LexEns-v1-ipo-12b01e5r256

model_architecture: MistralForCausalLM

model_num_parameters: 12772070400.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

latencies: [{'batch_size': 1, 'throughput': 0.6154567997288536, 'latency_mean': 1.6247420144081115, 'latency_p50': 1.6148866415023804, 'latency_p90': 1.7884423017501831}, {'batch_size': 3, 'throughput': 1.1355446001312428, 'latency_mean': 2.6340157425403596, 'latency_p50': 2.6593074798583984, 'latency_p90': 2.8842220544815063}, {'batch_size': 5, 'throughput': 1.3801884358026237, 'latency_mean': 3.600907691717148, 'latency_p50': 3.612542510032654, 'latency_p90': 4.050090765953064}, {'batch_size': 6, 'throughput': 1.4597101671888406, 'latency_mean': 4.091187624931336, 'latency_p50': 4.111448764801025, 'latency_p90': 4.575250577926636}, {'batch_size': 8, 'throughput': 1.524304869507064, 'latency_mean': 5.217192913293839, 'latency_p50': 5.2044161558151245, 'latency_p90': 6.005741429328919}, {'batch_size': 10, 'throughput': 1.558004517309865, 'latency_mean': 6.3681541240215305, 'latency_p50': 6.353451251983643, 'latency_p90': 7.0789025783538815}]

gpu_counts: {'NVIDIA RTX A5000': 1}

display_name: zonemercy-lexens-v1-ipo-_2405_v1

is_internal_developer: True

language_model: zonemercy/LexEns-v1-ipo-12b01e5r256

model_size: 13B

ranking_group: single

throughput_3p7s: 1.41

us_pacific_date: 2024-12-17

win_ratio: 0.40152801358234297

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name zonemercy-lexens-v1-ipo-2405-v1-mkmlizer
Waiting for job on zonemercy-lexens-v1-ipo-2405-v1-mkmlizer to finish
zonemercy-lexens-v1-ipo-2405-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
zonemercy-lexens-v1-ipo-2405-v1-mkmlizer: ║     _____            __           __                                ║
zonemercy-lexens-v1-ipo-2405-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
zonemercy-lexens-v1-ipo-2405-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
zonemercy-lexens-v1-ipo-2405-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
zonemercy-lexens-v1-ipo-2405-v1-mkmlizer: ║       /___/                                                         ║
zonemercy-lexens-v1-ipo-2405-v1-mkmlizer: ║                                                                     ║
zonemercy-lexens-v1-ipo-2405-v1-mkmlizer: ║  Version: 0.11.12                                                   ║
zonemercy-lexens-v1-ipo-2405-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
zonemercy-lexens-v1-ipo-2405-v1-mkmlizer: ║  https://mk1.ai                                                     ║
zonemercy-lexens-v1-ipo-2405-v1-mkmlizer: ║                                                                     ║
zonemercy-lexens-v1-ipo-2405-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
zonemercy-lexens-v1-ipo-2405-v1-mkmlizer: ║  belonging to:                                                      ║
zonemercy-lexens-v1-ipo-2405-v1-mkmlizer: ║                                                                     ║
zonemercy-lexens-v1-ipo-2405-v1-mkmlizer: ║  Chai Research Corp.                                                ║
zonemercy-lexens-v1-ipo-2405-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
zonemercy-lexens-v1-ipo-2405-v1-mkmlizer: ║  Expiration: 2025-01-15 23:59:59                                    ║
zonemercy-lexens-v1-ipo-2405-v1-mkmlizer: ║                                                                     ║
zonemercy-lexens-v1-ipo-2405-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
zonemercy-lexens-v1-ipo-2405-v1-mkmlizer: Downloaded to shared memory in 49.169s
zonemercy-lexens-v1-ipo-2405-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp1aabq4b5, device:0
zonemercy-lexens-v1-ipo-2405-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
zonemercy-lexens-v1-ipo-2405-v1-mkmlizer: quantized model in 36.565s
zonemercy-lexens-v1-ipo-2405-v1-mkmlizer: Processed model zonemercy/LexEns-v1-ipo-12b01e5r256 in 85.733s
zonemercy-lexens-v1-ipo-2405-v1-mkmlizer: creating bucket guanaco-mkml-models
zonemercy-lexens-v1-ipo-2405-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
zonemercy-lexens-v1-ipo-2405-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/zonemercy-lexens-v1-ipo-2405-v1
zonemercy-lexens-v1-ipo-2405-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/zonemercy-lexens-v1-ipo-2405-v1/config.json
zonemercy-lexens-v1-ipo-2405-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/zonemercy-lexens-v1-ipo-2405-v1/special_tokens_map.json
zonemercy-lexens-v1-ipo-2405-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/zonemercy-lexens-v1-ipo-2405-v1/tokenizer_config.json
zonemercy-lexens-v1-ipo-2405-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/zonemercy-lexens-v1-ipo-2405-v1/tokenizer.json
zonemercy-lexens-v1-ipo-2405-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/zonemercy-lexens-v1-ipo-2405-v1/flywheel_model.0.safetensors
zonemercy-lexens-v1-ipo-2405-v1-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:12, 27.71it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:07, 47.58it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:08, 42.53it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:08, 40.40it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:07, 46.36it/s]
Loading 0:  10%|▉         | 36/363 [00:00<00:06, 46.76it/s]
Loading 0:  11%|█▏        | 41/363 [00:01<00:08, 38.28it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:06, 45.89it/s]
Loading 0:  15%|█▍        | 54/363 [00:01<00:06, 46.60it/s]
Loading 0:  17%|█▋        | 60/363 [00:01<00:06, 44.73it/s]
Loading 0:  18%|█▊        | 65/363 [00:01<00:10, 29.31it/s]
Loading 0:  20%|█▉        | 71/363 [00:01<00:08, 33.83it/s]
Loading 0:  21%|██        | 76/363 [00:01<00:08, 35.57it/s]
Loading 0:  22%|██▏       | 81/363 [00:02<00:07, 37.51it/s]
Loading 0:  24%|██▎       | 86/363 [00:02<00:07, 39.10it/s]
Loading 0:  25%|██▌       | 91/363 [00:02<00:08, 33.73it/s]
Loading 0:  27%|██▋       | 98/363 [00:02<00:06, 40.86it/s]
Loading 0:  28%|██▊       | 103/363 [00:02<00:06, 40.93it/s]
Loading 0:  30%|██▉       | 108/363 [00:02<00:05, 42.77it/s]
Loading 0:  31%|███       | 113/363 [00:02<00:06, 36.41it/s]
Loading 0:  33%|███▎      | 118/363 [00:03<00:06, 36.37it/s]
Loading 0:  34%|███▍      | 125/363 [00:03<00:05, 43.60it/s]
Loading 0:  36%|███▌      | 130/363 [00:03<00:05, 42.61it/s]
Loading 0:  37%|███▋      | 135/363 [00:03<00:05, 43.42it/s]
Loading 0:  39%|███▊      | 140/363 [00:03<00:04, 44.66it/s]
Loading 0:  40%|███▉      | 145/363 [00:03<00:07, 27.27it/s]
Loading 0:  41%|████      | 149/363 [00:03<00:07, 27.59it/s]
Loading 0:  43%|████▎     | 156/363 [00:04<00:05, 34.98it/s]
Loading 0:  44%|████▍     | 161/363 [00:04<00:05, 35.79it/s]
Loading 0:  46%|████▌     | 166/363 [00:04<00:05, 37.27it/s]
Loading 0:  47%|████▋     | 171/363 [00:04<00:04, 39.46it/s]
Loading 0:  48%|████▊     | 176/363 [00:04<00:05, 34.19it/s]
Loading 0:  51%|█████     | 184/363 [00:04<00:04, 42.38it/s]
Loading 0:  52%|█████▏    | 189/363 [00:04<00:03, 44.14it/s]
Loading 0:  53%|█████▎    | 194/363 [00:05<00:04, 37.10it/s]
Loading 0:  55%|█████▌    | 201/363 [00:05<00:03, 42.63it/s]
Loading 0:  57%|█████▋    | 206/363 [00:05<00:03, 43.18it/s]
Loading 0:  58%|█████▊    | 211/363 [00:05<00:03, 41.97it/s]
Loading 0:  60%|█████▉    | 216/363 [00:05<00:03, 43.65it/s]
Loading 0:  61%|██████    | 222/363 [00:05<00:03, 40.46it/s]
Loading 0:  63%|██████▎   | 227/363 [00:05<00:04, 29.54it/s]
Loading 0:  64%|██████▎   | 231/363 [00:06<00:04, 29.38it/s]
Loading 0:  65%|██████▌   | 237/363 [00:06<00:03, 34.96it/s]
Loading 0:  66%|██████▋   | 241/363 [00:06<00:03, 35.10it/s]
Loading 0:  68%|██████▊   | 247/363 [00:06<00:02, 39.52it/s]
Loading 0:  70%|██████▉   | 253/363 [00:06<00:02, 39.71it/s]
Loading 0:  71%|███████   | 258/363 [00:06<00:02, 39.31it/s]
Loading 0:  73%|███████▎  | 265/363 [00:06<00:02, 44.74it/s]
Loading 0:  75%|███████▍  | 271/363 [00:06<00:02, 43.33it/s]
Loading 0:  76%|███████▌  | 276/363 [00:07<00:02, 42.08it/s]
Loading 0:  78%|███████▊  | 283/363 [00:07<00:01, 46.71it/s]
Loading 0:  79%|███████▉  | 288/363 [00:07<00:01, 47.33it/s]
Loading 0:  81%|████████  | 293/363 [00:07<00:01, 38.80it/s]
Loading 0:  83%|████████▎ | 301/363 [00:07<00:01, 47.72it/s]
Loading 0:  85%|████████▍ | 307/363 [00:14<00:19,  2.83it/s]
Loading 0:  86%|████████▌ | 312/363 [00:14<00:13,  3.73it/s]
Loading 0:  88%|████████▊ | 320/363 [00:14<00:07,  5.75it/s]
Loading 0:  90%|████████▉ | 326/363 [00:14<00:04,  7.61it/s]
Loading 0:  91%|█████████ | 331/363 [00:15<00:03,  9.52it/s]
Loading 0:  93%|█████████▎| 338/363 [00:15<00:01, 13.22it/s]
Loading 0:  94%|█████████▍| 343/363 [00:15<00:01, 16.25it/s]
Loading 0:  96%|█████████▌| 348/363 [00:15<00:00, 17.94it/s]
Loading 0:  98%|█████████▊| 356/363 [00:15<00:00, 25.01it/s]
Loading 0: 100%|█████████▉| 362/363 [00:15<00:00, 28.36it/s]
                                                            
Job zonemercy-lexens-v1-ipo-2405-v1-mkmlizer completed after 115.63s with status: succeeded
Stopping job with name zonemercy-lexens-v1-ipo-2405-v1-mkmlizer
Pipeline stage MKMLizer completed in 116.16s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.18s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service zonemercy-lexens-v1-ipo-2405-v1
Waiting for inference service zonemercy-lexens-v1-ipo-2405-v1 to be ready
Inference service zonemercy-lexens-v1-ipo-2405-v1 ready after 231.74286675453186s
Pipeline stage MKMLDeployer completed in 232.30s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.99918532371521s
Received healthy response to inference request in 1.1533596515655518s
Received healthy response to inference request in 1.2287302017211914s
read tcp 127.0.0.1:51932->127.0.0.1:8080: read: connection reset by peer
Received unhealthy response to inference request!
Received healthy response to inference request in 1.3844311237335205s
5 requests
1 failed requests
5th percentile: 0.514114236831665
10th percentile: 0.6739255905151367
20th percentile: 0.9935482978820801
30th percentile: 1.1684337615966798
40th percentile: 1.1985819816589356
50th percentile: 1.2287302017211914
60th percentile: 1.291010570526123
70th percentile: 1.3532909393310546
80th percentile: 1.5073819637298584
90th percentile: 1.7532836437225343
95th percentile: 1.876234483718872
99th percentile: 1.9745951557159425
mean time: 1.2240018367767334
%s, retrying in %s seconds...
Received healthy response to inference request in 1.220789909362793s
Received healthy response to inference request in 1.7954328060150146s
Received healthy response to inference request in 1.45975661277771s
Received healthy response to inference request in 1.223266839981079s
Received healthy response to inference request in 1.3189620971679688s
5 requests
0 failed requests
5th percentile: 1.2212852954864502
10th percentile: 1.2217806816101073
20th percentile: 1.222771453857422
30th percentile: 1.242405891418457
40th percentile: 1.280683994293213
50th percentile: 1.3189620971679688
60th percentile: 1.3752799034118652
70th percentile: 1.4315977096557617
80th percentile: 1.526891851425171
90th percentile: 1.6611623287200927
95th percentile: 1.7282975673675536
99th percentile: 1.7820057582855224
mean time: 1.403641653060913
Pipeline stage StressChecker completed in 16.26s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.20s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 2.06s
Shutdown handler de-registered
zonemercy-lexens-v1-ipo-_2405_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3078.16s
Shutdown handler de-registered
zonemercy-lexens-v1-ipo-_2405_v1 status is now inactive due to auto deactivation removed underperforming models