developer_uid: NischayDnk
submission_id: nischaydnk-mistralnemo-_50963_v1
model_name: nischaydnk-mistralnemo-_50963_v1
model_group: NischayDnk/Mistralnemo-d
status: torndown
timestamp: 2025-01-05T17:02:41+00:00
num_battles: 16730
num_wins: 8417
celo_rating: 1263.89
family_friendly_score: 0.583
family_friendly_standard_error: 0.006972962067873307
submission_type: basic
model_repo: NischayDnk/Mistralnemo-dpo-v7-rp-lumsftv1
model_architecture: MistralForCausalLM
model_num_parameters: 12772070400.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
latencies: [{'batch_size': 1, 'throughput': 0.6188930458080542, 'latency_mean': 1.6156997549533845, 'latency_p50': 1.61280357837677, 'latency_p90': 1.7926740884780883}, {'batch_size': 3, 'throughput': 1.1354207156438985, 'latency_mean': 2.629100874662399, 'latency_p50': 2.6197487115859985, 'latency_p90': 2.8867588758468625}, {'batch_size': 5, 'throughput': 1.3826121415124972, 'latency_mean': 3.6017215394973756, 'latency_p50': 3.5722122192382812, 'latency_p90': 4.030300712585449}, {'batch_size': 6, 'throughput': 1.4441285379219597, 'latency_mean': 4.13107567191124, 'latency_p50': 4.092423796653748, 'latency_p90': 4.732860350608826}, {'batch_size': 8, 'throughput': 1.5068239123080835, 'latency_mean': 5.259898475408554, 'latency_p50': 5.279614567756653, 'latency_p90': 6.001260304450988}, {'batch_size': 10, 'throughput': 1.5429228804394879, 'latency_mean': 6.434209638834, 'latency_p50': 6.464462995529175, 'latency_p90': 7.206098198890686}]
gpu_counts: {'NVIDIA RTX A5000': 1}
display_name: nischaydnk-mistralnemo-_50963_v1
is_internal_developer: False
language_model: NischayDnk/Mistralnemo-dpo-v7-rp-lumsftv1
model_size: 13B
ranking_group: single
throughput_3p7s: 1.4
us_pacific_date: 2025-01-05
win_ratio: 0.5031081888822475
generation_params: {'temperature': 0.94, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\nYou:', '\n\n', '</s>', '[/INST]', '<|im_end|>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name nischaydnk-mistralnemo-50963-v1-mkmlizer
Waiting for job on nischaydnk-mistralnemo-50963-v1-mkmlizer to finish
nischaydnk-mistralnemo-50963-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
nischaydnk-mistralnemo-50963-v1-mkmlizer: ║     _____            __           __                                ║
nischaydnk-mistralnemo-50963-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
nischaydnk-mistralnemo-50963-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
nischaydnk-mistralnemo-50963-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
nischaydnk-mistralnemo-50963-v1-mkmlizer: ║       /___/                                                         ║
nischaydnk-mistralnemo-50963-v1-mkmlizer: ║                                                                     ║
nischaydnk-mistralnemo-50963-v1-mkmlizer: ║  Version: 0.11.12                                                   ║
nischaydnk-mistralnemo-50963-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
nischaydnk-mistralnemo-50963-v1-mkmlizer: ║  https://mk1.ai                                                     ║
nischaydnk-mistralnemo-50963-v1-mkmlizer: ║                                                                     ║
nischaydnk-mistralnemo-50963-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
nischaydnk-mistralnemo-50963-v1-mkmlizer: ║  belonging to:                                                      ║
nischaydnk-mistralnemo-50963-v1-mkmlizer: ║                                                                     ║
nischaydnk-mistralnemo-50963-v1-mkmlizer: ║  Chai Research Corp.                                                ║
nischaydnk-mistralnemo-50963-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
nischaydnk-mistralnemo-50963-v1-mkmlizer: ║  Expiration: 2025-04-15 23:59:59                                    ║
nischaydnk-mistralnemo-50963-v1-mkmlizer: ║                                                                     ║
nischaydnk-mistralnemo-50963-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
nischaydnk-mistralnemo-50963-v1-mkmlizer: Downloaded to shared memory in 49.337s
nischaydnk-mistralnemo-50963-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpfspeznpx, device:0
nischaydnk-mistralnemo-50963-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
nischaydnk-mistralnemo-50963-v1-mkmlizer: quantized model in 35.865s
nischaydnk-mistralnemo-50963-v1-mkmlizer: Processed model NischayDnk/Mistralnemo-dpo-v7-rp-lumsftv1 in 85.202s
nischaydnk-mistralnemo-50963-v1-mkmlizer: creating bucket guanaco-mkml-models
nischaydnk-mistralnemo-50963-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/nischaydnk-mistralnemo-50963-v1/special_tokens_map.json
nischaydnk-mistralnemo-50963-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/nischaydnk-mistralnemo-50963-v1/config.json
nischaydnk-mistralnemo-50963-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/nischaydnk-mistralnemo-50963-v1/tokenizer_config.json
nischaydnk-mistralnemo-50963-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/nischaydnk-mistralnemo-50963-v1/tokenizer.json
nischaydnk-mistralnemo-50963-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/nischaydnk-mistralnemo-50963-v1/flywheel_model.0.safetensors
nischaydnk-mistralnemo-50963-v1-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:12, 27.95it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:07, 49.22it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:07, 46.21it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:07, 44.35it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:06, 51.67it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:06, 47.55it/s]
Loading 0:  12%|█▏        | 42/363 [00:00<00:07, 45.10it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:06, 50.55it/s]
Loading 0:  15%|█▌        | 55/363 [00:01<00:06, 46.32it/s]
Loading 0:  17%|█▋        | 60/363 [00:01<00:06, 47.12it/s]
Loading 0:  18%|█▊        | 65/363 [00:01<00:09, 31.59it/s]
Loading 0:  20%|█▉        | 72/363 [00:01<00:07, 37.72it/s]
Loading 0:  21%|██▏       | 78/363 [00:01<00:07, 38.00it/s]
Loading 0:  23%|██▎       | 83/363 [00:01<00:07, 38.43it/s]
Loading 0:  25%|██▍       | 90/363 [00:02<00:06, 43.71it/s]
Loading 0:  26%|██▌       | 95/363 [00:02<00:05, 45.03it/s]
Loading 0:  28%|██▊       | 100/363 [00:02<00:07, 37.51it/s]
Loading 0:  29%|██▉       | 106/363 [00:02<00:06, 41.07it/s]
Loading 0:  31%|███       | 112/363 [00:02<00:05, 45.10it/s]
Loading 0:  32%|███▏      | 117/363 [00:02<00:05, 42.38it/s]
Loading 0:  34%|███▎      | 122/363 [00:02<00:05, 42.87it/s]
Loading 0:  35%|███▍      | 127/363 [00:03<00:06, 37.18it/s]
Loading 0:  37%|███▋      | 135/363 [00:03<00:05, 45.43it/s]
Loading 0:  39%|███▉      | 141/363 [00:03<00:05, 43.40it/s]
Loading 0:  40%|████      | 146/363 [00:03<00:06, 32.52it/s]
Loading 0:  41%|████▏     | 150/363 [00:03<00:06, 32.14it/s]
Loading 0:  43%|████▎     | 156/363 [00:03<00:05, 37.49it/s]
Loading 0:  44%|████▍     | 161/363 [00:03<00:05, 38.22it/s]
Loading 0:  46%|████▌     | 166/363 [00:04<00:04, 40.52it/s]
Loading 0:  47%|████▋     | 171/363 [00:04<00:04, 42.65it/s]
Loading 0:  48%|████▊     | 176/363 [00:04<00:05, 36.11it/s]
Loading 0:  50%|█████     | 183/363 [00:04<00:04, 43.34it/s]
Loading 0:  52%|█████▏    | 188/363 [00:04<00:04, 42.51it/s]
Loading 0:  53%|█████▎    | 193/363 [00:04<00:03, 43.60it/s]
Loading 0:  55%|█████▍    | 198/363 [00:04<00:03, 44.61it/s]
Loading 0:  56%|█████▌    | 203/363 [00:04<00:04, 35.25it/s]
Loading 0:  58%|█████▊    | 210/363 [00:05<00:03, 42.51it/s]
Loading 0:  59%|█████▉    | 215/363 [00:05<00:03, 42.72it/s]
Loading 0:  61%|██████    | 220/363 [00:05<00:03, 44.54it/s]
Loading 0:  62%|██████▏   | 225/363 [00:05<00:04, 28.34it/s]
Loading 0:  63%|██████▎   | 230/363 [00:05<00:04, 29.55it/s]
Loading 0:  65%|██████▌   | 237/363 [00:05<00:03, 37.39it/s]
Loading 0:  67%|██████▋   | 242/363 [00:06<00:03, 38.58it/s]
Loading 0:  68%|██████▊   | 247/363 [00:06<00:02, 40.31it/s]
Loading 0:  69%|██████▉   | 252/363 [00:06<00:02, 41.93it/s]
Loading 0:  71%|███████   | 257/363 [00:06<00:02, 35.60it/s]
Loading 0:  73%|███████▎  | 265/363 [00:06<00:02, 43.83it/s]
Loading 0:  75%|███████▍  | 271/363 [00:06<00:02, 42.04it/s]
Loading 0:  76%|███████▌  | 276/363 [00:06<00:02, 41.30it/s]
Loading 0:  78%|███████▊  | 283/363 [00:06<00:01, 46.25it/s]
Loading 0:  79%|███████▉  | 288/363 [00:07<00:01, 46.78it/s]
Loading 0:  81%|████████  | 293/363 [00:07<00:01, 37.96it/s]
Loading 0:  82%|████████▏ | 299/363 [00:07<00:01, 42.30it/s]
Loading 0:  84%|████████▎ | 304/363 [00:14<00:22,  2.57it/s]
Loading 0:  85%|████████▍ | 308/363 [00:14<00:16,  3.31it/s]
Loading 0:  86%|████████▌ | 312/363 [00:14<00:11,  4.29it/s]
Loading 0:  88%|████████▊ | 320/363 [00:14<00:06,  7.11it/s]
Loading 0:  90%|████████▉ | 326/363 [00:14<00:03,  9.52it/s]
Loading 0:  91%|█████████ | 331/363 [00:14<00:02, 11.92it/s]
Loading 0:  93%|█████████▎| 337/363 [00:14<00:01, 15.93it/s]
Loading 0:  94%|█████████▍| 342/363 [00:14<00:01, 18.96it/s]
Loading 0:  96%|█████████▌| 347/363 [00:15<00:00, 22.27it/s]
Loading 0:  97%|█████████▋| 352/363 [00:15<00:00, 25.89it/s]
Loading 0:  98%|█████████▊| 357/363 [00:15<00:00, 25.97it/s]
                                                            

Job nischaydnk-mistralnemo-50963-v1-mkmlizer completed after 104.15s with status: succeeded
Stopping job with name nischaydnk-mistralnemo-50963-v1-mkmlizer
Pipeline stage MKMLizer completed in 104.65s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service nischaydnk-mistralnemo-50963-v1
Waiting for inference service nischaydnk-mistralnemo-50963-v1 to be ready
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Failed to get response for submission nischaydnk-mistralnemo-_72991_v4: HTTPConnectionPool(host='nischaydnk-mistralnemo-72991-v4-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission nischaydnk-mistralnemo-_72991_v4: HTTPConnectionPool(host='nischaydnk-mistralnemo-72991-v4-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission nischaydnk-mistralnemo-_72991_v4: HTTPConnectionPool(host='nischaydnk-mistralnemo-72991-v4-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission nischaydnk-mistralnemo-_72991_v4: HTTPConnectionPool(host='nischaydnk-mistralnemo-72991-v4-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Inference service nischaydnk-mistralnemo-50963-v1 ready after 341.12781524658203s
Pipeline stage MKMLDeployer completed in 341.64s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.3974900245666504s
Received healthy response to inference request in 1.8373570442199707s
Failed to get response for submission nischaydnk-mistralnemo-_72991_v4: HTTPConnectionPool(host='nischaydnk-mistralnemo-72991-v4-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Received healthy response to inference request in 2.0089313983917236s
Received healthy response to inference request in 1.692603349685669s
Received healthy response to inference request in 1.8320422172546387s
5 requests
0 failed requests
5th percentile: 1.7204911231994628
10th percentile: 1.7483788967132567
20th percentile: 1.8041544437408448
30th percentile: 1.8331051826477052
40th percentile: 1.835231113433838
50th percentile: 1.8373570442199707
60th percentile: 1.9059867858886719
70th percentile: 1.974616527557373
80th percentile: 2.086643123626709
90th percentile: 2.24206657409668
95th percentile: 2.319778299331665
99th percentile: 2.3819476795196532
mean time: 1.9536848068237305
Pipeline stage StressChecker completed in 11.19s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.72s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 1.07s
Shutdown handler de-registered
nischaydnk-mistralnemo-_50963_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 8805.44s
Shutdown handler de-registered
nischaydnk-mistralnemo-_50963_v1 status is now inactive due to auto deactivation removed underperforming models
nischaydnk-mistralnemo-_50963_v1 status is now torndown due to DeploymentManager action
nischaydnk-mistralnemo-_50963_v1 status is now torndown due to DeploymentManager action
nischaydnk-mistralnemo-_50963_v1 status is now torndown due to DeploymentManager action