developer_uid: NischayDnk
submission_id: nischaydnk-mistralnemo-_72991_v2
model_name: nischaydnk-mistralnemo-_72991_v2
model_group: NischayDnk/Mistralnemo-d
status: torndown
timestamp: 2025-01-04T21:01:07+00:00
num_battles: 20553
num_wins: 10163
celo_rating: 1257.66
family_friendly_score: 0.5898
family_friendly_standard_error: 0.006956090281185258
submission_type: basic
model_repo: NischayDnk/Mistralnemo-dpo-v7-rp-pantsftv1
model_architecture: MistralForCausalLM
model_num_parameters: 12772070400.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
latencies: [{'batch_size': 1, 'throughput': 0.6164404300136956, 'latency_mean': 1.6221530890464784, 'latency_p50': 1.6164511442184448, 'latency_p90': 1.8004503011703492}, {'batch_size': 3, 'throughput': 1.1389259027687504, 'latency_mean': 2.6278377771377563, 'latency_p50': 2.6293615102767944, 'latency_p90': 2.8630738496780395}, {'batch_size': 5, 'throughput': 1.376857272387615, 'latency_mean': 3.619813321828842, 'latency_p50': 3.619023084640503, 'latency_p90': 3.992852807044983}, {'batch_size': 6, 'throughput': 1.457504717052467, 'latency_mean': 4.0954618072509765, 'latency_p50': 4.112325072288513, 'latency_p90': 4.6257587432861325}, {'batch_size': 8, 'throughput': 1.5235816351837472, 'latency_mean': 5.213965719938278, 'latency_p50': 5.223191738128662, 'latency_p90': 5.776784038543701}, {'batch_size': 10, 'throughput': 1.5589295964661085, 'latency_mean': 6.35057173371315, 'latency_p50': 6.347656965255737, 'latency_p90': 7.137407398223877}]
gpu_counts: {'NVIDIA RTX A5000': 1}
display_name: nischaydnk-mistralnemo-_72991_v2
is_internal_developer: False
language_model: NischayDnk/Mistralnemo-dpo-v7-rp-pantsftv1
model_size: 13B
ranking_group: single
throughput_3p7s: 1.4
us_pacific_date: 2025-01-04
win_ratio: 0.4944776918211453
generation_params: {'temperature': 0.9, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name nischaydnk-mistralnemo-72991-v2-mkmlizer
Waiting for job on nischaydnk-mistralnemo-72991-v2-mkmlizer to finish
nischaydnk-mistralnemo-72991-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
nischaydnk-mistralnemo-72991-v2-mkmlizer: ║     _____            __           __                                ║
nischaydnk-mistralnemo-72991-v2-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
nischaydnk-mistralnemo-72991-v2-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
nischaydnk-mistralnemo-72991-v2-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
nischaydnk-mistralnemo-72991-v2-mkmlizer: ║       /___/                                                         ║
nischaydnk-mistralnemo-72991-v2-mkmlizer: ║                                                                     ║
nischaydnk-mistralnemo-72991-v2-mkmlizer: ║  Version: 0.11.12                                                   ║
nischaydnk-mistralnemo-72991-v2-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
nischaydnk-mistralnemo-72991-v2-mkmlizer: ║  https://mk1.ai                                                     ║
nischaydnk-mistralnemo-72991-v2-mkmlizer: ║                                                                     ║
nischaydnk-mistralnemo-72991-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
nischaydnk-mistralnemo-72991-v2-mkmlizer: ║  belonging to:                                                      ║
nischaydnk-mistralnemo-72991-v2-mkmlizer: ║                                                                     ║
nischaydnk-mistralnemo-72991-v2-mkmlizer: ║  Chai Research Corp.                                                ║
nischaydnk-mistralnemo-72991-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
nischaydnk-mistralnemo-72991-v2-mkmlizer: ║  Expiration: 2025-04-15 23:59:59                                    ║
nischaydnk-mistralnemo-72991-v2-mkmlizer: ║                                                                     ║
nischaydnk-mistralnemo-72991-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
nischaydnk-mistralnemo-72991-v2-mkmlizer: Downloaded to shared memory in 31.842s
nischaydnk-mistralnemo-72991-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpfmeuqtya, device:0
nischaydnk-mistralnemo-72991-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
nischaydnk-mistralnemo-72991-v2-mkmlizer: quantized model in 36.669s
nischaydnk-mistralnemo-72991-v2-mkmlizer: Processed model NischayDnk/Mistralnemo-dpo-v7-rp-pantsftv1 in 68.512s
nischaydnk-mistralnemo-72991-v2-mkmlizer: creating bucket guanaco-mkml-models
nischaydnk-mistralnemo-72991-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
nischaydnk-mistralnemo-72991-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/nischaydnk-mistralnemo-72991-v2
nischaydnk-mistralnemo-72991-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/nischaydnk-mistralnemo-72991-v2/config.json
nischaydnk-mistralnemo-72991-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/nischaydnk-mistralnemo-72991-v2/special_tokens_map.json
nischaydnk-mistralnemo-72991-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/nischaydnk-mistralnemo-72991-v2/tokenizer_config.json
nischaydnk-mistralnemo-72991-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/nischaydnk-mistralnemo-72991-v2/tokenizer.json
nischaydnk-mistralnemo-72991-v2-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:15, 23.76it/s]
Loading 0:   3%|▎         | 12/363 [00:00<00:08, 40.55it/s]
Loading 0:   5%|▍         | 17/363 [00:00<00:08, 42.10it/s]
Loading 0:   6%|▌         | 22/363 [00:00<00:07, 43.58it/s]
Loading 0:   8%|▊         | 28/363 [00:00<00:08, 41.19it/s]
Loading 0:   9%|▉         | 33/363 [00:00<00:08, 40.96it/s]
Loading 0:  11%|█         | 40/363 [00:00<00:06, 47.26it/s]
Loading 0:  13%|█▎        | 46/363 [00:01<00:07, 44.94it/s]
Loading 0:  14%|█▍        | 51/363 [00:01<00:07, 43.12it/s]
Loading 0:  16%|█▌        | 58/363 [00:01<00:06, 49.39it/s]
Loading 0:  18%|█▊        | 64/363 [00:01<00:09, 30.74it/s]
Loading 0:  20%|█▉        | 72/363 [00:01<00:07, 38.44it/s]
Loading 0:  21%|██▏       | 78/363 [00:01<00:07, 38.62it/s]
Loading 0:  23%|██▎       | 83/363 [00:02<00:07, 39.09it/s]
Loading 0:  25%|██▍       | 89/363 [00:02<00:06, 43.18it/s]
Loading 0:  26%|██▌       | 94/363 [00:02<00:06, 39.52it/s]
Loading 0:  27%|██▋       | 99/363 [00:02<00:06, 39.99it/s]
Loading 0:  29%|██▉       | 105/363 [00:02<00:06, 39.76it/s]
Loading 0:  31%|███       | 112/363 [00:02<00:05, 45.02it/s]
Loading 0:  32%|███▏      | 117/363 [00:02<00:05, 43.93it/s]
Loading 0:  34%|███▍      | 123/363 [00:03<00:05, 42.20it/s]
Loading 0:  35%|███▌      | 128/363 [00:03<00:05, 39.29it/s]
Loading 0:  37%|███▋      | 133/363 [00:03<00:05, 41.77it/s]
Loading 0:  38%|███▊      | 138/363 [00:03<00:05, 40.35it/s]
Loading 0:  39%|███▉      | 143/363 [00:03<00:06, 31.82it/s]
Loading 0:  40%|████      | 147/363 [00:03<00:06, 32.53it/s]
Loading 0:  42%|████▏     | 151/363 [00:03<00:06, 33.50it/s]
Loading 0:  43%|████▎     | 157/363 [00:03<00:05, 37.57it/s]
Loading 0:  44%|████▍     | 161/363 [00:04<00:05, 36.99it/s]
Loading 0:  45%|████▌     | 165/363 [00:04<00:05, 37.32it/s]
Loading 0:  47%|████▋     | 169/363 [00:04<00:05, 36.18it/s]
Loading 0:  48%|████▊     | 175/363 [00:04<00:04, 40.41it/s]
Loading 0:  50%|████▉     | 181/363 [00:04<00:04, 40.22it/s]
Loading 0:  51%|█████     | 186/363 [00:04<00:04, 40.47it/s]
Loading 0:  53%|█████▎    | 193/363 [00:04<00:03, 46.77it/s]
Loading 0:  55%|█████▍    | 198/363 [00:04<00:03, 47.39it/s]
Loading 0:  56%|█████▌    | 203/363 [00:05<00:04, 35.27it/s]
Loading 0:  58%|█████▊    | 210/363 [00:05<00:03, 41.44it/s]
Loading 0:  59%|█████▉    | 215/363 [00:05<00:03, 41.39it/s]
Loading 0:  61%|██████    | 220/363 [00:05<00:03, 43.33it/s]
Loading 0:  62%|██████▏   | 225/363 [00:05<00:04, 28.93it/s]
Loading 0:  63%|██████▎   | 230/363 [00:05<00:04, 30.01it/s]
Loading 0:  65%|██████▌   | 237/363 [00:06<00:03, 36.73it/s]
Loading 0:  67%|██████▋   | 242/363 [00:06<00:03, 37.66it/s]
Loading 0:  68%|██████▊   | 247/363 [00:06<00:02, 39.70it/s]
Loading 0:  69%|██████▉   | 252/363 [00:06<00:02, 42.01it/s]
Loading 0:  71%|███████   | 257/363 [00:06<00:02, 37.05it/s]
Loading 0:  73%|███████▎  | 265/363 [00:06<00:02, 45.91it/s]
Loading 0:  75%|███████▍  | 271/363 [00:06<00:02, 43.91it/s]
Loading 0:  76%|███████▌  | 276/363 [00:06<00:02, 42.89it/s]
Loading 0:  78%|███████▊  | 282/363 [00:07<00:01, 46.77it/s]
Loading 0:  79%|███████▉  | 287/363 [00:07<00:01, 46.05it/s]
Loading 0:  80%|████████  | 292/363 [00:07<00:01, 45.54it/s]
Loading 0:  82%|████████▏ | 297/363 [00:07<00:01, 46.13it/s]
Loading 0:  83%|████████▎ | 303/363 [00:07<00:01, 43.40it/s]
Loading 0:  85%|████████▍ | 308/363 [00:14<00:22,  2.48it/s]
Loading 0:  86%|████████▌ | 312/363 [00:14<00:16,  3.17it/s]
Loading 0:  88%|████████▊ | 319/363 [00:14<00:08,  4.94it/s]
Loading 0:  89%|████████▉ | 323/363 [00:14<00:06,  6.19it/s]
Loading 0:  91%|█████████ | 329/363 [00:14<00:03,  8.83it/s]
Loading 0:  92%|█████████▏| 335/363 [00:15<00:02, 11.87it/s]
Loading 0:  94%|█████████▎| 340/363 [00:15<00:01, 14.78it/s]
Loading 0:  96%|█████████▌| 347/363 [00:15<00:00, 20.22it/s]
Loading 0:  97%|█████████▋| 352/363 [00:15<00:00, 23.38it/s]
Loading 0:  98%|█████████▊| 357/363 [00:15<00:00, 23.03it/s]
                                                            

Job nischaydnk-mistralnemo-72991-v2-mkmlizer completed after 94.51s with status: succeeded
Stopping job with name nischaydnk-mistralnemo-72991-v2-mkmlizer
Pipeline stage MKMLizer completed in 94.98s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.18s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service nischaydnk-mistralnemo-72991-v2
Waiting for inference service nischaydnk-mistralnemo-72991-v2 to be ready
Inference service nischaydnk-mistralnemo-72991-v2 ready after 331.52740836143494s
Pipeline stage MKMLDeployer completed in 332.32s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 7.4522764682769775s
Received healthy response to inference request in 6.846953392028809s
Received healthy response to inference request in 1.6911063194274902s
Received healthy response to inference request in 1.6837000846862793s
Received healthy response to inference request in 1.6863038539886475s
5 requests
0 failed requests
5th percentile: 1.684220838546753
10th percentile: 1.6847415924072267
20th percentile: 1.6857831001281738
30th percentile: 1.687264347076416
40th percentile: 1.689185333251953
50th percentile: 1.6911063194274902
60th percentile: 3.753445148468017
70th percentile: 5.815783977508544
80th percentile: 6.968018007278443
90th percentile: 7.21014723777771
95th percentile: 7.331211853027344
99th percentile: 7.428063545227051
mean time: 3.8720680236816407
%s, retrying in %s seconds...
Received healthy response to inference request in 2.0308837890625s
Received healthy response to inference request in 1.7652628421783447s
Received healthy response to inference request in 1.7226536273956299s
Received healthy response to inference request in 1.7274541854858398s
Received healthy response to inference request in 7.556981563568115s
5 requests
0 failed requests
5th percentile: 1.7236137390136719
10th percentile: 1.7245738506317139
20th percentile: 1.7264940738677979
30th percentile: 1.7350159168243409
40th percentile: 1.7501393795013427
50th percentile: 1.7652628421783447
60th percentile: 1.8715112209320068
70th percentile: 1.977759599685669
80th percentile: 3.136103343963624
90th percentile: 5.34654245376587
95th percentile: 6.451762008666991
99th percentile: 7.335937652587891
mean time: 2.960647201538086
Pipeline stage StressChecker completed in 36.83s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.67s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 6.70s
Shutdown handler de-registered
nischaydnk-mistralnemo-_72991_v2 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 8630.95s
Shutdown handler de-registered
nischaydnk-mistralnemo-_72991_v2 status is now inactive due to auto deactivation removed underperforming models
nischaydnk-mistralnemo-_72991_v2 status is now torndown due to DeploymentManager action
nischaydnk-mistralnemo-_72991_v2 status is now torndown due to DeploymentManager action
nischaydnk-mistralnemo-_72991_v2 status is now torndown due to DeploymentManager action