developer_uid: cycy233
submission_id: cycy233-model4-6_v1
model_name: 486
model_group: cycy233/model4-6
status: torndown
timestamp: 2025-03-03T07:34:38+00:00
num_battles: 6261
num_wins: 2988
celo_rating: 1246.2
family_friendly_score: 0.514
family_friendly_standard_error: 0.00706829540978587
submission_type: basic
model_repo: cycy233/model4-6
model_architecture: LlamaForCausalLM
model_num_parameters: 8030261248.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
reward_model: default
latencies: [{'batch_size': 1, 'throughput': 0.8365733858346075, 'latency_mean': 1.1952886307239532, 'latency_p50': 1.2073794603347778, 'latency_p90': 1.3156967163085938}, {'batch_size': 4, 'throughput': 1.7551531803529001, 'latency_mean': 2.2740281188488005, 'latency_p50': 2.289913535118103, 'latency_p90': 2.5242912769317627}, {'batch_size': 5, 'throughput': 1.8612191473450508, 'latency_mean': 2.6648361110687255, 'latency_p50': 2.669623017311096, 'latency_p90': 2.956023359298706}, {'batch_size': 8, 'throughput': 2.0694319078584384, 'latency_mean': 3.8332259571552276, 'latency_p50': 3.8202929496765137, 'latency_p90': 4.3505104064941404}, {'batch_size': 10, 'throughput': 2.1200788560614745, 'latency_mean': 4.680753387212754, 'latency_p50': 4.706368446350098, 'latency_p90': 5.237169790267944}, {'batch_size': 12, 'throughput': 2.1560426668526533, 'latency_mean': 5.513712884187698, 'latency_p50': 5.511221528053284, 'latency_p90': 6.25094256401062}, {'batch_size': 15, 'throughput': 2.149341664112117, 'latency_mean': 6.880506391525269, 'latency_p50': 6.904094099998474, 'latency_p90': 7.65619740486145}]
gpu_counts: {'NVIDIA RTX A5000': 1}
display_name: 486
is_internal_developer: False
language_model: cycy233/model4-6
model_size: 8B
ranking_group: single
throughput_3p7s: 2.07
us_pacific_date: 2025-03-02
win_ratio: 0.4772400574988021
generation_params: {'temperature': 0.95, 'top_p': 1.0, 'min_p': 0.08, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['<|eot_id|>', '\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n{bot_name}'s Persona: {memory}\n\n", 'prompt_template': '{prompt}<|eot_id|>', 'bot_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}: {message}<|eot_id|>', 'user_template': '<|start_header_id|>user<|end_header_id|>\n\n{user_name}: {message}<|eot_id|>', 'response_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name cycy233-model4-6-v1-mkmlizer
Waiting for job on cycy233-model4-6-v1-mkmlizer to finish
cycy233-model4-6-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
cycy233-model4-6-v1-mkmlizer: ║     _____            __           __                                ║
cycy233-model4-6-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
cycy233-model4-6-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
cycy233-model4-6-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
cycy233-model4-6-v1-mkmlizer: ║       /___/                                                         ║
cycy233-model4-6-v1-mkmlizer: ║                                                                     ║
cycy233-model4-6-v1-mkmlizer: ║  Version: 0.12.8                                                    ║
cycy233-model4-6-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
cycy233-model4-6-v1-mkmlizer: ║  https://mk1.ai                                                     ║
cycy233-model4-6-v1-mkmlizer: ║                                                                     ║
cycy233-model4-6-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
cycy233-model4-6-v1-mkmlizer: ║  belonging to:                                                      ║
cycy233-model4-6-v1-mkmlizer: ║                                                                     ║
cycy233-model4-6-v1-mkmlizer: ║  Chai Research Corp.                                                ║
cycy233-model4-6-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
cycy233-model4-6-v1-mkmlizer: ║  Expiration: 2025-04-15 23:59:59                                    ║
cycy233-model4-6-v1-mkmlizer: ║                                                                     ║
cycy233-model4-6-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
cycy233-model4-6-v1-mkmlizer: Downloaded to shared memory in 30.297s
cycy233-model4-6-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpbpnyr5lj, device:0
cycy233-model4-6-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Job cycy233-model4-6-v1-mkmlizer completed after 93.0s with status: succeeded
Stopping job with name cycy233-model4-6-v1-mkmlizer
Pipeline stage MKMLizer completed in 93.45s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service cycy233-model4-6-v1
Waiting for inference service cycy233-model4-6-v1 to be ready
Failed to get response for submission nischaydnk-exp14-exp9dp_94555_v3: HTTPConnectionPool(host='nischaydnk-exp14-exp9dp-94555-v3-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Inference service cycy233-model4-6-v1 ready after 140.47565841674805s
Pipeline stage MKMLDeployer completed in 140.98s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.9750785827636719s
Received healthy response to inference request in 1.446241855621338s
Failed to get response for submission nischaydnk-exp14-exp9dp_94555_v3: HTTPConnectionPool(host='nischaydnk-exp14-exp9dp-94555-v3-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Received healthy response to inference request in 1.333073377609253s
Received healthy response to inference request in 1.1378834247589111s
Received healthy response to inference request in 1.3686492443084717s
5 requests
0 failed requests
5th percentile: 1.1769214153289795
10th percentile: 1.215959405899048
20th percentile: 1.2940353870391845
30th percentile: 1.3401885509490967
40th percentile: 1.354418897628784
50th percentile: 1.3686492443084717
60th percentile: 1.3996862888336181
70th percentile: 1.4307233333587646
80th percentile: 1.5520092010498048
90th percentile: 1.7635438919067383
95th percentile: 1.869311237335205
99th percentile: 1.9539251136779785
mean time: 1.452185297012329
Pipeline stage StressChecker completed in 8.66s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.78s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.65s
Shutdown handler de-registered
cycy233-model4-6_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2345.95s
Shutdown handler de-registered
cycy233-model4-6_v1 status is now inactive due to auto deactivation removed underperforming models
cycy233-model4-6_v1 status is now torndown due to DeploymentManager action