submission_id: hastagaras-mn-base-12b-e_9769_v1
developer_uid: Hastagaras
best_of: 8
celo_rating: 1245.73
display_name: test
family_friendly_score: 0.573
family_friendly_standard_error: 0.006995298421082549
formatter: {'memory_template': "[PREFIX]system[MIDDLE]You're {bot_name}\n{memory}\n", 'prompt_template': '{prompt}[SUFFIX]', 'bot_template': '[PREFIX]model[MIDDLE]{bot_name}: {message}[SUFFIX]', 'user_template': '[PREFIX]user[MIDDLE]{user_name}: {message}[SUFFIX]', 'response_template': '[PREFIX]model[MIDDLE]{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 0.85, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '[SUFFIX]'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
gpu_counts: {'NVIDIA RTX A5000': 1}
is_internal_developer: False
language_model: Hastagaras/MN-Base-12B-Experiment-LMH-ET-10.20.30
latencies: [{'batch_size': 1, 'throughput': 0.6154463686634699, 'latency_mean': 1.6247795975208283, 'latency_p50': 1.6327276229858398, 'latency_p90': 1.7716447591781617}, {'batch_size': 3, 'throughput': 1.1431873699347257, 'latency_mean': 2.6193230986595153, 'latency_p50': 2.630236864089966, 'latency_p90': 2.9591696500778197}, {'batch_size': 5, 'throughput': 1.380427306104375, 'latency_mean': 3.6098186337947844, 'latency_p50': 3.6283698081970215, 'latency_p90': 4.043443608283996}, {'batch_size': 6, 'throughput': 1.4447813544342605, 'latency_mean': 4.122050507068634, 'latency_p50': 4.137305021286011, 'latency_p90': 4.653046822547912}, {'batch_size': 8, 'throughput': 1.517871704519896, 'latency_mean': 5.23478373169899, 'latency_p50': 5.224444031715393, 'latency_p90': 5.897012495994567}, {'batch_size': 10, 'throughput': 1.5637263756743658, 'latency_mean': 6.34233766913414, 'latency_p50': 6.365033268928528, 'latency_p90': 7.225018048286438}]
max_input_tokens: 1024
max_output_tokens: 64
model_architecture: MistralForCausalLM
model_group: Hastagaras/MN-Base-12B-E
model_name: test
model_num_parameters: 12772070400.0
model_repo: Hastagaras/MN-Base-12B-Experiment-LMH-ET-10.20.30
model_size: 13B
num_battles: 15148
num_wins: 7167
ranking_group: single
status: inactive
submission_type: basic
throughput_3p7s: 1.4
timestamp: 2024-11-29T07:35:28+00:00
us_pacific_date: 2024-11-28
win_ratio: 0.4731317665698442
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name hastagaras-mn-base-12b-e-9769-v1-mkmlizer
Waiting for job on hastagaras-mn-base-12b-e-9769-v1-mkmlizer to finish
hastagaras-mn-base-12b-e-9769-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
hastagaras-mn-base-12b-e-9769-v1-mkmlizer: ║ _____ __ __ ║
hastagaras-mn-base-12b-e-9769-v1-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
hastagaras-mn-base-12b-e-9769-v1-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
hastagaras-mn-base-12b-e-9769-v1-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
hastagaras-mn-base-12b-e-9769-v1-mkmlizer: ║ /___/ ║
hastagaras-mn-base-12b-e-9769-v1-mkmlizer: ║ ║
hastagaras-mn-base-12b-e-9769-v1-mkmlizer: ║ Version: 0.11.12 ║
hastagaras-mn-base-12b-e-9769-v1-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
hastagaras-mn-base-12b-e-9769-v1-mkmlizer: ║ https://mk1.ai ║
hastagaras-mn-base-12b-e-9769-v1-mkmlizer: ║ ║
hastagaras-mn-base-12b-e-9769-v1-mkmlizer: ║ The license key for the current software has been verified as ║
hastagaras-mn-base-12b-e-9769-v1-mkmlizer: ║ belonging to: ║
hastagaras-mn-base-12b-e-9769-v1-mkmlizer: ║ ║
hastagaras-mn-base-12b-e-9769-v1-mkmlizer: ║ Chai Research Corp. ║
hastagaras-mn-base-12b-e-9769-v1-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
hastagaras-mn-base-12b-e-9769-v1-mkmlizer: ║ Expiration: 2025-01-15 23:59:59 ║
hastagaras-mn-base-12b-e-9769-v1-mkmlizer: ║ ║
hastagaras-mn-base-12b-e-9769-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
hastagaras-mn-base-12b-e-9769-v1-mkmlizer: Downloaded to shared memory in 49.062s
hastagaras-mn-base-12b-e-9769-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpkf7icv42, device:0
hastagaras-mn-base-12b-e-9769-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
hastagaras-mn-base-12b-e-9769-v1-mkmlizer: quantized model in 36.700s
hastagaras-mn-base-12b-e-9769-v1-mkmlizer: Processed model Hastagaras/MN-Base-12B-Experiment-LMH-ET-10.20.30 in 85.763s
hastagaras-mn-base-12b-e-9769-v1-mkmlizer: creating bucket guanaco-mkml-models
hastagaras-mn-base-12b-e-9769-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
hastagaras-mn-base-12b-e-9769-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/hastagaras-mn-base-12b-e-9769-v1
hastagaras-mn-base-12b-e-9769-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/hastagaras-mn-base-12b-e-9769-v1/config.json
hastagaras-mn-base-12b-e-9769-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/hastagaras-mn-base-12b-e-9769-v1/special_tokens_map.json
hastagaras-mn-base-12b-e-9769-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/hastagaras-mn-base-12b-e-9769-v1/tokenizer_config.json
hastagaras-mn-base-12b-e-9769-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/hastagaras-mn-base-12b-e-9769-v1/tokenizer.json
hastagaras-mn-base-12b-e-9769-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/hastagaras-mn-base-12b-e-9769-v1/flywheel_model.0.safetensors
hastagaras-mn-base-12b-e-9769-v1-mkmlizer: Loading 0: 0%| | 0/363 [00:00<?, ?it/s] Loading 0: 1%|▏ | 5/363 [00:00<00:11, 30.33it/s] Loading 0: 4%|▎ | 13/363 [00:00<00:06, 50.82it/s] Loading 0: 5%|▌ | 19/363 [00:00<00:07, 47.91it/s] Loading 0: 7%|▋ | 25/363 [00:00<00:06, 48.61it/s] Loading 0: 9%|▊ | 31/363 [00:00<00:06, 49.89it/s] Loading 0: 10%|█ | 37/363 [00:00<00:06, 47.00it/s] Loading 0: 12%|█▏ | 42/363 [00:00<00:07, 45.78it/s] Loading 0: 13%|█▎ | 49/363 [00:01<00:06, 50.96it/s] Loading 0: 15%|█▌ | 55/363 [00:01<00:06, 46.65it/s] Loading 0: 17%|█▋ | 60/363 [00:01<00:06, 46.72it/s] Loading 0: 18%|█▊ | 65/363 [00:01<00:10, 29.44it/s] Loading 0: 20%|█▉ | 72/363 [00:01<00:08, 35.61it/s] Loading 0: 21%|██ | 77/363 [00:01<00:07, 38.10it/s] Loading 0: 23%|██▎ | 82/363 [00:02<00:08, 33.09it/s] Loading 0: 25%|██▍ | 89/363 [00:02<00:06, 40.60it/s] Loading 0: 26%|██▌ | 94/363 [00:02<00:06, 41.56it/s] Loading 0: 27%|██▋ | 99/363 [00:02<00:06, 42.11it/s] Loading 0: 29%|██▉ | 105/363 [00:02<00:06, 41.69it/s] Loading 0: 31%|███ | 112/363 [00:02<00:05, 46.20it/s] Loading 0: 32%|███▏ | 117/363 [00:02<00:05, 44.49it/s] Loading 0: 34%|███▎ | 122/363 [00:02<00:05, 44.01it/s] Loading 0: 35%|███▍ | 127/363 [00:03<00:06, 35.94it/s] Loading 0: 37%|███▋ | 135/363 [00:03<00:05, 43.75it/s] Loading 0: 39%|███▉ | 141/363 [00:03<00:05, 42.25it/s] Loading 0: 40%|████ | 146/363 [00:03<00:07, 30.37it/s] Loading 0: 41%|████▏ | 150/363 [00:03<00:07, 28.99it/s] Loading 0: 43%|████▎ | 156/363 [00:03<00:06, 33.33it/s] Loading 0: 44%|████▍ | 160/363 [00:04<00:06, 33.48it/s] Loading 0: 45%|████▌ | 165/363 [00:04<00:05, 36.86it/s] Loading 0: 47%|████▋ | 169/363 [00:04<00:05, 36.50it/s] Loading 0: 48%|████▊ | 174/363 [00:04<00:04, 38.74it/s] Loading 0: 49%|████▉ | 179/363 [00:04<00:04, 39.17it/s] Loading 0: 51%|█████ | 184/363 [00:04<00:04, 40.54it/s] Loading 0: 52%|█████▏ | 189/363 [00:04<00:04, 42.80it/s] Loading 0: 53%|█████▎ | 194/363 [00:04<00:04, 35.47it/s] Loading 0: 55%|█████▌ | 201/363 [00:05<00:03, 42.90it/s] Loading 0: 57%|█████▋ | 206/363 [00:05<00:03, 42.16it/s] Loading 0: 58%|█████▊ | 211/363 [00:05<00:03, 40.77it/s] Loading 0: 60%|█████▉ | 216/363 [00:05<00:03, 42.07it/s] Loading 0: 61%|██████ | 221/363 [00:05<00:03, 42.34it/s] Loading 0: 62%|██████▏ | 226/363 [00:05<00:05, 25.93it/s] Loading 0: 63%|██████▎ | 230/363 [00:06<00:05, 26.23it/s] Loading 0: 65%|██████▌ | 237/363 [00:06<00:03, 33.72it/s] Loading 0: 67%|██████▋ | 242/363 [00:06<00:03, 35.01it/s] Loading 0: 68%|██████▊ | 247/363 [00:06<00:03, 36.25it/s] Loading 0: 69%|██████▉ | 252/363 [00:06<00:02, 38.55it/s] Loading 0: 71%|███████ | 257/363 [00:06<00:03, 33.05it/s] Loading 0: 73%|███████▎ | 264/363 [00:06<00:02, 39.14it/s] Loading 0: 74%|███████▍ | 269/363 [00:06<00:02, 39.64it/s] Loading 0: 75%|███████▌ | 274/363 [00:07<00:02, 39.87it/s] Loading 0: 77%|███████▋ | 279/363 [00:07<00:02, 41.54it/s] Loading 0: 78%|███████▊ | 284/363 [00:07<00:02, 34.07it/s] Loading 0: 80%|████████ | 291/363 [00:07<00:01, 40.62it/s] Loading 0: 82%|████████▏ | 296/363 [00:07<00:01, 40.00it/s] Loading 0: 83%|████████▎ | 301/363 [00:07<00:01, 41.47it/s] Loading 0: 84%|████████▍ | 306/363 [00:14<00:23, 2.45it/s] Loading 0: 85%|████████▌ | 310/363 [00:14<00:16, 3.18it/s] Loading 0: 87%|████████▋ | 314/363 [00:14<00:11, 4.18it/s] Loading 0: 88%|████████▊ | 320/363 [00:14<00:06, 6.24it/s] Loading 0: 90%|████████▉ | 326/363 [00:15<00:04, 8.74it/s] Loading 0: 91%|█████████ | 331/363 [00:15<00:02, 11.16it/s] Loading 0: 93%|█████████▎| 338/363 [00:15<00:01, 15.95it/s] Loading 0: 95%|█████████▍| 344/363 [00:15<00:00, 19.61it/s] Loading 0: 96%|█████████▌| 349/363 [00:15<00:00, 22.58it/s] Loading 0: 98%|█████████▊| 356/363 [00:15<00:00, 28.75it/s] Loading 0: 100%|█████████▉| 362/363 [00:15<00:00, 31.53it/s]
Job hastagaras-mn-base-12b-e-9769-v1-mkmlizer completed after 114.46s with status: succeeded
Stopping job with name hastagaras-mn-base-12b-e-9769-v1-mkmlizer
Pipeline stage MKMLizer completed in 115.01s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service hastagaras-mn-base-12b-e-9769-v1
Waiting for inference service hastagaras-mn-base-12b-e-9769-v1 to be ready
Connection pool is full, discarding connection: %s. Connection pool size: %s
Inference service hastagaras-mn-base-12b-e-9769-v1 ready after 140.4789080619812s
Pipeline stage MKMLDeployer completed in 141.02s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.1923136711120605s
Received healthy response to inference request in 1.868908405303955s
Received healthy response to inference request in 1.9027905464172363s
Received healthy response to inference request in 1.6078033447265625s
Received healthy response to inference request in 2.001307964324951s
5 requests
0 failed requests
5th percentile: 1.660024356842041
10th percentile: 1.7122453689575194
20th percentile: 1.8166873931884766
30th percentile: 1.8756848335266114
40th percentile: 1.8892376899719239
50th percentile: 1.9027905464172363
60th percentile: 1.9421975135803222
70th percentile: 1.9816044807434081
80th percentile: 2.039509105682373
90th percentile: 2.1159113883972167
95th percentile: 2.154112529754639
99th percentile: 2.1846734428405763
mean time: 1.9146247863769532
Pipeline stage StressChecker completed in 10.96s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
Connection pool is full, discarding connection: %s. Connection pool size: %s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.47s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 2.34s
Shutdown handler de-registered
hastagaras-mn-base-12b-e_9769_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage MKMLProfilerDeleter completed in 0.11s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.09s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service hastagaras-mn-base-12b-e-9769-v1-profiler
Waiting for inference service hastagaras-mn-base-12b-e-9769-v1-profiler to be ready
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3215.33s
Shutdown handler de-registered
hastagaras-mn-base-12b-e_9769_v1 status is now inactive due to auto deactivation removed underperforming models