nischaydnk-mistralnemo-_40585

developer_uid: NischayDnk

submission_id: nischaydnk-mistralnemo-_40585_v1

model_name: nischaydnk-mistralnemo-_40585_v1

model_group: NischayDnk/Mistralnemo-d

status: torndown

timestamp: 2025-01-05T03:09:44+00:00

num_battles: 8920

num_wins: 4324

celo_rating: 1249.94

family_friendly_score: 0.5968

family_friendly_standard_error: 0.006937287077813632

submission_type: basic

model_repo: NischayDnk/Mistralnemo-dpo-v15-rp-pantsftv1_24k

model_architecture: MistralForCausalLM

model_num_parameters: 12772070400.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

latencies: [{'batch_size': 1, 'throughput': 0.6210361228833704, 'latency_mean': 1.610145581960678, 'latency_p50': 1.6058611869812012, 'latency_p90': 1.777272915840149}, {'batch_size': 3, 'throughput': 1.1327123834976394, 'latency_mean': 2.6367904210090636, 'latency_p50': 2.636436343193054, 'latency_p90': 2.9171599626541136}, {'batch_size': 5, 'throughput': 1.3803805507722031, 'latency_mean': 3.6079068005084993, 'latency_p50': 3.5565359592437744, 'latency_p90': 4.086317658424377}, {'batch_size': 6, 'throughput': 1.4464955013913827, 'latency_mean': 4.1372387969493865, 'latency_p50': 4.1410980224609375, 'latency_p90': 4.68681914806366}, {'batch_size': 8, 'throughput': 1.5063104860898413, 'latency_mean': 5.279909337759018, 'latency_p50': 5.294153928756714, 'latency_p90': 5.985531234741211}, {'batch_size': 10, 'throughput': 1.5567727637365936, 'latency_mean': 6.372018325328827, 'latency_p50': 6.414892077445984, 'latency_p90': 7.1405092716217045}]

gpu_counts: {'NVIDIA RTX A5000': 1}

display_name: nischaydnk-mistralnemo-_40585_v1

is_internal_developer: False

language_model: NischayDnk/Mistralnemo-dpo-v15-rp-pantsftv1_24k

model_size: 13B

ranking_group: single

throughput_3p7s: 1.4

us_pacific_date: 2025-01-04

win_ratio: 0.48475336322869955

generation_params: {'temperature': 0.94, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name nischaydnk-mistralnemo-40585-v1-mkmlizer
Waiting for job on nischaydnk-mistralnemo-40585-v1-mkmlizer to finish
nischaydnk-mistralnemo-40585-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
nischaydnk-mistralnemo-40585-v1-mkmlizer: ║     _____            __           __                                ║
nischaydnk-mistralnemo-40585-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
nischaydnk-mistralnemo-40585-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
nischaydnk-mistralnemo-40585-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
nischaydnk-mistralnemo-40585-v1-mkmlizer: ║       /___/                                                         ║
nischaydnk-mistralnemo-40585-v1-mkmlizer: ║                                                                     ║
nischaydnk-mistralnemo-40585-v1-mkmlizer: ║  Version: 0.11.12                                                   ║
nischaydnk-mistralnemo-40585-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
nischaydnk-mistralnemo-40585-v1-mkmlizer: ║  https://mk1.ai                                                     ║
nischaydnk-mistralnemo-40585-v1-mkmlizer: ║                                                                     ║
nischaydnk-mistralnemo-40585-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
nischaydnk-mistralnemo-40585-v1-mkmlizer: ║  belonging to:                                                      ║
nischaydnk-mistralnemo-40585-v1-mkmlizer: ║                                                                     ║
nischaydnk-mistralnemo-40585-v1-mkmlizer: ║  Chai Research Corp.                                                ║
nischaydnk-mistralnemo-40585-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
nischaydnk-mistralnemo-40585-v1-mkmlizer: ║  Expiration: 2025-04-15 23:59:59                                    ║
nischaydnk-mistralnemo-40585-v1-mkmlizer: ║                                                                     ║
nischaydnk-mistralnemo-40585-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
nischaydnk-mistralnemo-40585-v1-mkmlizer: Downloaded to shared memory in 49.928s
nischaydnk-mistralnemo-40585-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpguu08ty_, device:0
nischaydnk-mistralnemo-40585-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
nischaydnk-mistralnemo-40585-v1-mkmlizer: quantized model in 35.496s
nischaydnk-mistralnemo-40585-v1-mkmlizer: Processed model NischayDnk/Mistralnemo-dpo-v15-rp-pantsftv1_24k in 85.425s
nischaydnk-mistralnemo-40585-v1-mkmlizer: creating bucket guanaco-mkml-models
nischaydnk-mistralnemo-40585-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
nischaydnk-mistralnemo-40585-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/nischaydnk-mistralnemo-40585-v1
nischaydnk-mistralnemo-40585-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/nischaydnk-mistralnemo-40585-v1/config.json
nischaydnk-mistralnemo-40585-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/nischaydnk-mistralnemo-40585-v1/special_tokens_map.json
nischaydnk-mistralnemo-40585-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/nischaydnk-mistralnemo-40585-v1/tokenizer_config.json
nischaydnk-mistralnemo-40585-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/nischaydnk-mistralnemo-40585-v1/tokenizer.json
nischaydnk-mistralnemo-40585-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/nischaydnk-mistralnemo-40585-v1/flywheel_model.0.safetensors
nischaydnk-mistralnemo-40585-v1-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:13, 26.74it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:07, 47.93it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:07, 44.87it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:07, 44.07it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:06, 50.47it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:06, 47.27it/s]
Loading 0:  12%|█▏        | 42/363 [00:00<00:07, 45.78it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:06, 50.41it/s]
Loading 0:  15%|█▌        | 55/363 [00:01<00:06, 45.28it/s]
Loading 0:  17%|█▋        | 61/363 [00:01<00:08, 36.25it/s]
Loading 0:  18%|█▊        | 66/363 [00:01<00:08, 36.82it/s]
Loading 0:  20%|█▉        | 72/363 [00:01<00:07, 41.16it/s]
Loading 0:  21%|██▏       | 78/363 [00:01<00:07, 40.42it/s]
Loading 0:  23%|██▎       | 83/363 [00:01<00:07, 39.76it/s]
Loading 0:  25%|██▍       | 89/363 [00:02<00:06, 43.92it/s]
Loading 0:  26%|██▌       | 94/363 [00:02<00:06, 43.88it/s]
Loading 0:  27%|██▋       | 99/363 [00:02<00:06, 43.92it/s]
Loading 0:  29%|██▊       | 104/363 [00:02<00:05, 44.34it/s]
Loading 0:  30%|███       | 110/363 [00:02<00:06, 41.83it/s]
Loading 0:  32%|███▏      | 115/363 [00:02<00:05, 43.14it/s]
Loading 0:  33%|███▎      | 120/363 [00:02<00:05, 41.19it/s]
Loading 0:  35%|███▍      | 126/363 [00:02<00:05, 44.39it/s]
Loading 0:  36%|███▋      | 132/363 [00:03<00:05, 41.58it/s]
Loading 0:  38%|███▊      | 137/363 [00:03<00:05, 41.21it/s]
Loading 0:  39%|███▉      | 142/363 [00:03<00:06, 33.14it/s]
Loading 0:  40%|████      | 147/363 [00:03<00:06, 34.88it/s]
Loading 0:  42%|████▏     | 151/363 [00:03<00:05, 36.00it/s]
Loading 0:  43%|████▎     | 157/363 [00:03<00:05, 40.62it/s]
Loading 0:  45%|████▍     | 163/363 [00:03<00:04, 40.48it/s]
Loading 0:  46%|████▋     | 168/363 [00:04<00:04, 41.28it/s]
Loading 0:  48%|████▊     | 175/363 [00:04<00:04, 46.88it/s]
Loading 0:  50%|████▉     | 181/363 [00:04<00:04, 44.55it/s]
Loading 0:  51%|█████     | 186/363 [00:04<00:04, 43.26it/s]
Loading 0:  53%|█████▎    | 193/363 [00:04<00:03, 48.11it/s]
Loading 0:  55%|█████▍    | 199/363 [00:04<00:03, 45.69it/s]
Loading 0:  56%|█████▌    | 204/363 [00:04<00:03, 44.29it/s]
Loading 0:  58%|█████▊    | 211/363 [00:04<00:03, 49.01it/s]
Loading 0:  60%|█████▉    | 217/363 [00:05<00:03, 46.17it/s]
Loading 0:  61%|██████    | 222/363 [00:05<00:03, 46.61it/s]
Loading 0:  63%|██████▎   | 227/363 [00:05<00:04, 33.81it/s]
Loading 0:  64%|██████▎   | 231/363 [00:05<00:04, 32.94it/s]
Loading 0:  65%|██████▌   | 237/363 [00:05<00:03, 38.61it/s]
Loading 0:  67%|██████▋   | 242/363 [00:05<00:03, 40.22it/s]
Loading 0:  68%|██████▊   | 247/363 [00:05<00:02, 41.59it/s]
Loading 0:  70%|██████▉   | 253/363 [00:06<00:02, 40.60it/s]
Loading 0:  71%|███████   | 258/363 [00:06<00:02, 40.66it/s]
Loading 0:  73%|███████▎  | 265/363 [00:06<00:02, 45.34it/s]
Loading 0:  74%|███████▍  | 270/363 [00:06<00:02, 46.10it/s]
Loading 0:  76%|███████▌  | 275/363 [00:06<00:02, 38.75it/s]
Loading 0:  78%|███████▊  | 283/363 [00:06<00:01, 46.81it/s]
Loading 0:  80%|███████▉  | 289/363 [00:06<00:01, 44.53it/s]
Loading 0:  81%|████████  | 294/363 [00:06<00:01, 43.21it/s]
Loading 0:  83%|████████▎ | 300/363 [00:07<00:01, 47.16it/s]
Loading 0:  84%|████████▍ | 305/363 [00:13<00:21,  2.68it/s]
Loading 0:  85%|████████▌ | 309/363 [00:13<00:15,  3.42it/s]
Loading 0:  86%|████████▌ | 313/363 [00:13<00:11,  4.42it/s]
Loading 0:  88%|████████▊ | 320/363 [00:14<00:06,  6.90it/s]
Loading 0:  90%|████████▉ | 326/363 [00:14<00:03,  9.43it/s]
Loading 0:  91%|█████████ | 331/363 [00:14<00:02, 12.04it/s]
Loading 0:  93%|█████████▎| 338/363 [00:14<00:01, 16.94it/s]
Loading 0:  95%|█████████▍| 344/363 [00:14<00:00, 20.58it/s]
Loading 0:  96%|█████████▌| 349/363 [00:14<00:00, 23.84it/s]
Loading 0:  98%|█████████▊| 356/363 [00:14<00:00, 30.04it/s]
Loading 0:  99%|█████████▉| 361/363 [00:14<00:00, 33.17it/s]
                                                            
Job nischaydnk-mistralnemo-40585-v1-mkmlizer completed after 238.31s with status: succeeded
Stopping job with name nischaydnk-mistralnemo-40585-v1-mkmlizer
Pipeline stage MKMLizer completed in 238.77s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.18s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service nischaydnk-mistralnemo-40585-v1
Waiting for inference service nischaydnk-mistralnemo-40585-v1 to be ready
Failed to get response for submission mistralai-mistral-sma_53415_v100: HTTPConnectionPool(host='mistralai-mistral-sma-53415-v100-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission mistralai-mistral-sma_53415_v100: HTTPConnectionPool(host='mistralai-mistral-sma-53415-v100-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Failed to get response for submission mistralai-mistral-sma_53415_v100: HTTPConnectionPool(host='mistralai-mistral-sma-53415-v100-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission cgato-nemo-12b-humanize_77076_v2: HTTPConnectionPool(host='cgato-nemo-12b-humanize-77076-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission cgato-nemo-12b-humanize_77076_v2: HTTPConnectionPool(host='cgato-nemo-12b-humanize-77076-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission mistralai-mistral-sma_53415_v100: HTTPConnectionPool(host='mistralai-mistral-sma-53415-v100-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission cgato-nemo-12b-humanize_77076_v2: HTTPConnectionPool(host='cgato-nemo-12b-humanize-77076-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission mistralai-mistral-sma_53415_v100: HTTPConnectionPool(host='mistralai-mistral-sma-53415-v100-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Inference service nischaydnk-mistralnemo-40585-v1 ready after 342.1387162208557s
Pipeline stage MKMLDeployer completed in 342.64s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.294921636581421s
Received healthy response to inference request in 1.6474907398223877s
Failed to get response for submission mistralai-mistral-sma_53415_v100: HTTPConnectionPool(host='mistralai-mistral-sma-53415-v100-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Received healthy response to inference request in 2.061800718307495s
Received healthy response to inference request in 1.6132991313934326s
Failed to get response for submission cgato-nemo-12b-humanize_77076_v2: HTTPConnectionPool(host='cgato-nemo-12b-humanize-77076-v2-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Received healthy response to inference request in 1.8462083339691162s
5 requests
0 failed requests
5th percentile: 1.6201374530792236
10th percentile: 1.6269757747650146
20th percentile: 1.6406524181365967
30th percentile: 1.6872342586517335
40th percentile: 1.7667212963104248
50th percentile: 1.8462083339691162
60th percentile: 1.9324452877044678
70th percentile: 2.0186822414398193
80th percentile: 2.1084249019622803
90th percentile: 2.2016732692718506
95th percentile: 2.2482974529266357
99th percentile: 2.285596799850464
mean time: 1.8927441120147706
Pipeline stage StressChecker completed in 10.71s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.67s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.60s
Shutdown handler de-registered
nischaydnk-mistralnemo-_40585_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage MKMLProfilerDeleter completed in 0.11s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.10s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service nischaydnk-mistralnemo-40585-v1-profiler
Waiting for inference service nischaydnk-mistralnemo-40585-v1-profiler to be ready
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3055.24s
Shutdown handler de-registered
nischaydnk-mistralnemo-_40585_v1 status is now inactive due to auto deactivation removed underperforming models
nischaydnk-mistralnemo-_40585_v1 status is now torndown due to DeploymentManager action
nischaydnk-mistralnemo-_40585_v1 status is now torndown due to DeploymentManager action
nischaydnk-mistralnemo-_40585_v1 status is now torndown due to DeploymentManager action