nischaydnk-mistralnemo-_74255

developer_uid: NischayDnk

submission_id: nischaydnk-mistralnemo-_74255_v3

model_name: nischaydnk-mistralnemo-_74255_v3

model_group: NischayDnk/Mistralnemo-d

status: torndown

timestamp: 2025-02-21T04:48:13+00:00

num_battles: 7484

num_wins: 3434

celo_rating: 1242.88

family_friendly_score: 0.5682

family_friendly_standard_error: 0.0070049805138915275

submission_type: basic

model_repo: NischayDnk/Mistralnemo-dpo-v7-rp-lumsftv1_re

model_architecture: MistralForCausalLM

model_num_parameters: 12772070400.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

latencies: [{'batch_size': 1, 'throughput': 0.596208682002505, 'latency_mean': 1.677197983264923, 'latency_p50': 1.6831244230270386, 'latency_p90': 1.8337578535079957}, {'batch_size': 3, 'throughput': 1.0950844672937017, 'latency_mean': 2.7318533849716187, 'latency_p50': 2.7276114225387573, 'latency_p90': 3.073390173912048}, {'batch_size': 5, 'throughput': 1.326316141141842, 'latency_mean': 3.7566099524497987, 'latency_p50': 3.7588131427764893, 'latency_p90': 4.15250780582428}, {'batch_size': 6, 'throughput': 1.3948594869301454, 'latency_mean': 4.2839857912063595, 'latency_p50': 4.290739893913269, 'latency_p90': 4.802562022209168}, {'batch_size': 8, 'throughput': 1.4564836452005752, 'latency_mean': 5.461609213352204, 'latency_p50': 5.530112624168396, 'latency_p90': 6.148206949234009}, {'batch_size': 10, 'throughput': 1.4903398978539142, 'latency_mean': 6.666306393146515, 'latency_p50': 6.661488652229309, 'latency_p90': 7.503261923789978}]

gpu_counts: {'NVIDIA RTX A5000': 1}

display_name: nischaydnk-mistralnemo-_74255_v3

is_internal_developer: True

language_model: NischayDnk/Mistralnemo-dpo-v7-rp-lumsftv1_re

model_size: 13B

ranking_group: single

throughput_3p7s: 1.32

us_pacific_date: 2025-02-20

win_ratio: 0.45884553714591125

generation_params: {'temperature': 0.92, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\nYou:', '\n\n', '</s>', '[/INST]', '<|im_end|>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name nischaydnk-mistralnemo-74255-v3-mkmlizer
Waiting for job on nischaydnk-mistralnemo-74255-v3-mkmlizer to finish
nischaydnk-mistralnemo-74255-v3-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
nischaydnk-mistralnemo-74255-v3-mkmlizer: ║     _____            __           __                                ║
nischaydnk-mistralnemo-74255-v3-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
nischaydnk-mistralnemo-74255-v3-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
nischaydnk-mistralnemo-74255-v3-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
nischaydnk-mistralnemo-74255-v3-mkmlizer: ║       /___/                                                         ║
nischaydnk-mistralnemo-74255-v3-mkmlizer: ║                                                                     ║
nischaydnk-mistralnemo-74255-v3-mkmlizer: ║  Version: 0.12.8                                                    ║
nischaydnk-mistralnemo-74255-v3-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
nischaydnk-mistralnemo-74255-v3-mkmlizer: ║  https://mk1.ai                                                     ║
nischaydnk-mistralnemo-74255-v3-mkmlizer: ║                                                                     ║
nischaydnk-mistralnemo-74255-v3-mkmlizer: ║  The license key for the current software has been verified as      ║
nischaydnk-mistralnemo-74255-v3-mkmlizer: ║  belonging to:                                                      ║
nischaydnk-mistralnemo-74255-v3-mkmlizer: ║                                                                     ║
nischaydnk-mistralnemo-74255-v3-mkmlizer: ║  Chai Research Corp.                                                ║
nischaydnk-mistralnemo-74255-v3-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
nischaydnk-mistralnemo-74255-v3-mkmlizer: ║  Expiration: 2025-04-15 23:59:59                                    ║
nischaydnk-mistralnemo-74255-v3-mkmlizer: ║                                                                     ║
nischaydnk-mistralnemo-74255-v3-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
nischaydnk-mistralnemo-74255-v3-mkmlizer: Downloaded to shared memory in 48.558s
nischaydnk-mistralnemo-74255-v3-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp3kkxjqeg, device:0
nischaydnk-mistralnemo-74255-v3-mkmlizer: Saving flywheel model at /dev/shm/model_cache
nischaydnk-mistralnemo-74255-v3-mkmlizer: quantized model in 36.164s
nischaydnk-mistralnemo-74255-v3-mkmlizer: Processed model NischayDnk/Mistralnemo-dpo-v7-rp-lumsftv1_re in 84.723s
nischaydnk-mistralnemo-74255-v3-mkmlizer: creating bucket guanaco-mkml-models
nischaydnk-mistralnemo-74255-v3-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
nischaydnk-mistralnemo-74255-v3-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/nischaydnk-mistralnemo-74255-v3
nischaydnk-mistralnemo-74255-v3-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/nischaydnk-mistralnemo-74255-v3/config.json
nischaydnk-mistralnemo-74255-v3-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/nischaydnk-mistralnemo-74255-v3/special_tokens_map.json
nischaydnk-mistralnemo-74255-v3-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/nischaydnk-mistralnemo-74255-v3/tokenizer_config.json
nischaydnk-mistralnemo-74255-v3-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/nischaydnk-mistralnemo-74255-v3/tokenizer.json
nischaydnk-mistralnemo-74255-v3-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/nischaydnk-mistralnemo-74255-v3/flywheel_model.0.safetensors
Job nischaydnk-mistralnemo-74255-v3-mkmlizer completed after 114.43s with status: succeeded
Stopping job with name nischaydnk-mistralnemo-74255-v3-mkmlizer
Pipeline stage MKMLizer completed in 114.89s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service nischaydnk-mistralnemo-74255-v3
Waiting for inference service nischaydnk-mistralnemo-74255-v3 to be ready
Inference service nischaydnk-mistralnemo-74255-v3 ready after 210.98111701011658s
Pipeline stage MKMLDeployer completed in 211.49s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.3513715267181396s
Received healthy response to inference request in 1.618307113647461s
Received healthy response to inference request in 1.7622966766357422s
Received healthy response to inference request in 1.7266438007354736s
Received healthy response to inference request in 1.4910519123077393s
5 requests
0 failed requests
5th percentile: 1.5165029525756837
10th percentile: 1.5419539928436279
20th percentile: 1.5928560733795165
30th percentile: 1.6399744510650636
40th percentile: 1.6833091259002686
50th percentile: 1.7266438007354736
60th percentile: 1.740904951095581
70th percentile: 1.7551661014556885
80th percentile: 1.8801116466522219
90th percentile: 2.1157415866851808
95th percentile: 2.23355655670166
99th percentile: 2.327808532714844
mean time: 1.7899342060089112
Pipeline stage StressChecker completed in 10.15s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.59s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.68s
Shutdown handler de-registered
nischaydnk-mistralnemo-_74255_v3 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage MKMLProfilerDeleter completed in 0.08s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.08s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service nischaydnk-mistralnemo-74255-v3-profiler
Waiting for inference service nischaydnk-mistralnemo-74255-v3-profiler to be ready
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2846.80s
Shutdown handler de-registered
nischaydnk-mistralnemo-_74255_v3 status is now inactive due to auto deactivation removed underperforming models
nischaydnk-mistralnemo-_74255_v3 status is now torndown due to DeploymentManager action

ChatRequest

Bot Name

Generation Params

Prompt Formatter

ChatMessage 1

Sender

Message

User Message