cycy233-modelv-fusion-s_35728

developer_uid: cycy233

submission_id: cycy233-modelv-fusion-s_35728_v1

model_name: cycy233-modelv-fusion-s_35728_v1

model_group: cycy233/modelV_fusion_st

status: torndown

timestamp: 2025-05-12T04:16:15+00:00

num_battles: 354

num_wins: 177

family_friendly_score: 0.0

family_friendly_standard_error: 0.0

submission_type: basic

model_repo: cycy233/modelV_fusion_step13000

model_architecture: MistralForCausalLM

model_num_parameters: 12772070400.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

reward_model: default

latencies: [{'batch_size': 1, 'throughput': 0.6038898609061621, 'latency_mean': 1.6558381116390228, 'latency_p50': 1.6434372663497925, 'latency_p90': 1.8337727069854737}, {'batch_size': 3, 'throughput': 1.0943951212531455, 'latency_mean': 2.732685500383377, 'latency_p50': 2.753031611442566, 'latency_p90': 2.9724667072296143}, {'batch_size': 5, 'throughput': 1.3181583635261331, 'latency_mean': 3.7711998236179354, 'latency_p50': 3.7868704795837402, 'latency_p90': 4.169117331504822}, {'batch_size': 6, 'throughput': 1.3859270956386096, 'latency_mean': 4.323243176937103, 'latency_p50': 4.309789538383484, 'latency_p90': 4.8233977317810055}, {'batch_size': 8, 'throughput': 1.475203980705224, 'latency_mean': 5.384501311779022, 'latency_p50': 5.395363926887512, 'latency_p90': 6.104105114936829}, {'batch_size': 10, 'throughput': 1.5020442821806677, 'latency_mean': 6.598118846416473, 'latency_p50': 6.601970553398132, 'latency_p90': 7.468645524978638}]

gpu_counts: {'NVIDIA RTX A5000': 1}

display_name: cycy233-modelv-fusion-s_35728_v1

ineligible_reason: num_battles<5000

is_internal_developer: False

language_model: cycy233/modelV_fusion_step13000

model_size: 13B

ranking_group: single

throughput_3p7s: 1.31

us_pacific_date: 2025-05-11

win_ratio: 0.5

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name cycy233-modelv-fusion-s-35728-v1-mkmlizer
Waiting for job on cycy233-modelv-fusion-s-35728-v1-mkmlizer to finish
cycy233-modelv-fusion-s-35728-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
cycy233-modelv-fusion-s-35728-v1-mkmlizer: ║     _____            __           __                                ║
cycy233-modelv-fusion-s-35728-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
cycy233-modelv-fusion-s-35728-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
cycy233-modelv-fusion-s-35728-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
cycy233-modelv-fusion-s-35728-v1-mkmlizer: ║       /___/                                                         ║
cycy233-modelv-fusion-s-35728-v1-mkmlizer: ║                                                                     ║
cycy233-modelv-fusion-s-35728-v1-mkmlizer: ║  Version: 0.12.8                                                    ║
cycy233-modelv-fusion-s-35728-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
cycy233-modelv-fusion-s-35728-v1-mkmlizer: ║  https://mk1.ai                                                     ║
cycy233-modelv-fusion-s-35728-v1-mkmlizer: ║                                                                     ║
cycy233-modelv-fusion-s-35728-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
cycy233-modelv-fusion-s-35728-v1-mkmlizer: ║  belonging to:                                                      ║
cycy233-modelv-fusion-s-35728-v1-mkmlizer: ║                                                                     ║
cycy233-modelv-fusion-s-35728-v1-mkmlizer: ║  Chai Research Corp.                                                ║
cycy233-modelv-fusion-s-35728-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
cycy233-modelv-fusion-s-35728-v1-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
cycy233-modelv-fusion-s-35728-v1-mkmlizer: ║                                                                     ║
cycy233-modelv-fusion-s-35728-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
cycy233-modelv-fusion-s-35728-v1-mkmlizer: Downloaded to shared memory in 42.020s
cycy233-modelv-fusion-s-35728-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpaopujiib, device:0
cycy233-modelv-fusion-s-35728-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
cycy233-modelv-fusion-s-35728-v1-mkmlizer: quantized model in 37.021s
cycy233-modelv-fusion-s-35728-v1-mkmlizer: Processed model cycy233/modelV_fusion_step13000 in 79.041s
cycy233-modelv-fusion-s-35728-v1-mkmlizer: creating bucket guanaco-mkml-models
cycy233-modelv-fusion-s-35728-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
cycy233-modelv-fusion-s-35728-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/cycy233-modelv-fusion-s-35728-v1
cycy233-modelv-fusion-s-35728-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/cycy233-modelv-fusion-s-35728-v1/config.json
cycy233-modelv-fusion-s-35728-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/cycy233-modelv-fusion-s-35728-v1/special_tokens_map.json
cycy233-modelv-fusion-s-35728-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/cycy233-modelv-fusion-s-35728-v1/tokenizer_config.json
cycy233-modelv-fusion-s-35728-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/cycy233-modelv-fusion-s-35728-v1/tokenizer.json
cycy233-modelv-fusion-s-35728-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/cycy233-modelv-fusion-s-35728-v1/flywheel_model.0.safetensors
cycy233-modelv-fusion-s-35728-v1-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:12, 28.35it/s]
Loading 0:   3%|▎         | 12/363 [00:00<00:07, 44.12it/s]
Loading 0:   5%|▍         | 17/363 [00:00<00:08, 42.01it/s]
Loading 0:   6%|▌         | 22/363 [00:00<00:08, 40.95it/s]
Loading 0:   7%|▋         | 27/363 [00:00<00:07, 42.17it/s]
Loading 0:   9%|▉         | 32/363 [00:00<00:09, 33.15it/s]
Loading 0:  11%|█         | 39/363 [00:00<00:08, 40.41it/s]
Loading 0:  12%|█▏        | 44/363 [00:01<00:07, 39.96it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:07, 39.44it/s]
Loading 0:  15%|█▍        | 54/363 [00:01<00:07, 40.41it/s]
Loading 0:  16%|█▋        | 59/363 [00:01<00:07, 41.00it/s]
Loading 0:  18%|█▊        | 64/363 [00:01<00:12, 23.18it/s]
Loading 0:  20%|█▉        | 71/363 [00:02<00:09, 30.58it/s]
Loading 0:  21%|██        | 76/363 [00:02<00:08, 32.53it/s]
Loading 0:  22%|██▏       | 81/363 [00:02<00:08, 34.99it/s]
Loading 0:  24%|██▎       | 86/363 [00:02<00:07, 38.23it/s]
Loading 0:  25%|██▌       | 91/363 [00:02<00:08, 33.92it/s]
Loading 0:  27%|██▋       | 98/363 [00:02<00:06, 41.32it/s]
Loading 0:  28%|██▊       | 103/363 [00:02<00:06, 41.50it/s]
Loading 0:  30%|███       | 109/363 [00:02<00:05, 45.69it/s]
Loading 0:  31%|███▏      | 114/363 [00:03<00:06, 39.33it/s]
Loading 0:  33%|███▎      | 119/363 [00:03<00:06, 38.82it/s]
Loading 0:  34%|███▍      | 125/363 [00:03<00:05, 43.21it/s]
Loading 0:  36%|███▌      | 130/363 [00:03<00:05, 42.52it/s]
Loading 0:  37%|███▋      | 135/363 [00:03<00:05, 41.45it/s]
Loading 0:  39%|███▊      | 140/363 [00:03<00:05, 43.10it/s]
Loading 0:  40%|███▉      | 145/363 [00:04<00:08, 24.98it/s]
Loading 0:  41%|████      | 149/363 [00:04<00:08, 25.67it/s]
Loading 0:  43%|████▎     | 156/363 [00:04<00:06, 32.96it/s]
Loading 0:  44%|████▍     | 161/363 [00:04<00:05, 34.53it/s]
Loading 0:  46%|████▌     | 166/363 [00:04<00:05, 35.27it/s]
Loading 0:  47%|████▋     | 171/363 [00:04<00:04, 38.46it/s]
Loading 0:  48%|████▊     | 176/363 [00:04<00:06, 30.94it/s]
Loading 0:  50%|█████     | 183/363 [00:05<00:04, 37.48it/s]
Loading 0:  52%|█████▏    | 188/363 [00:05<00:04, 38.81it/s]
Loading 0:  53%|█████▎    | 193/363 [00:05<00:04, 39.76it/s]
Loading 0:  55%|█████▍    | 198/363 [00:05<00:03, 42.14it/s]
Loading 0:  56%|█████▌    | 203/363 [00:05<00:04, 35.59it/s]
Loading 0:  58%|█████▊    | 210/363 [00:05<00:03, 42.52it/s]
Loading 0:  59%|█████▉    | 215/363 [00:05<00:03, 42.71it/s]
Loading 0:  61%|██████    | 220/363 [00:05<00:03, 43.59it/s]
Loading 0:  62%|██████▏   | 225/363 [00:06<00:04, 27.76it/s]
Loading 0:  63%|██████▎   | 230/363 [00:06<00:04, 29.93it/s]
Loading 0:  65%|██████▌   | 237/363 [00:06<00:03, 36.94it/s]
Loading 0:  67%|██████▋   | 242/363 [00:06<00:03, 38.33it/s]
Loading 0:  68%|██████▊   | 247/363 [00:06<00:02, 39.23it/s]
Loading 0:  69%|██████▉   | 252/363 [00:06<00:02, 41.64it/s]
Loading 0:  71%|███████   | 257/363 [00:07<00:02, 35.59it/s]
Loading 0:  73%|███████▎  | 264/363 [00:07<00:02, 42.04it/s]
Loading 0:  74%|███████▍  | 269/363 [00:07<00:02, 41.50it/s]
Loading 0:  75%|███████▌  | 274/363 [00:07<00:02, 41.36it/s]
Loading 0:  77%|███████▋  | 279/363 [00:07<00:01, 43.24it/s]
Loading 0:  78%|███████▊  | 284/363 [00:07<00:02, 36.24it/s]
Loading 0:  80%|████████  | 291/363 [00:07<00:01, 43.23it/s]
Loading 0:  82%|████████▏ | 296/363 [00:07<00:01, 43.68it/s]
Loading 0:  83%|████████▎ | 302/363 [00:08<00:01, 47.37it/s]
Loading 0:  85%|████████▍ | 307/363 [00:14<00:21,  2.55it/s]
Loading 0:  86%|████████▌ | 312/363 [00:15<00:14,  3.45it/s]
Loading 0:  88%|████████▊ | 319/363 [00:15<00:08,  5.24it/s]
Loading 0:  89%|████████▉ | 324/363 [00:15<00:05,  6.85it/s]
Loading 0:  91%|█████████ | 329/363 [00:15<00:03,  8.94it/s]
Loading 0:  92%|█████████▏| 334/363 [00:15<00:02, 11.60it/s]
Loading 0:  93%|█████████▎| 339/363 [00:15<00:01, 13.71it/s]
Loading 0:  96%|█████████▌| 347/363 [00:15<00:00, 20.06it/s]
Loading 0:  97%|█████████▋| 352/363 [00:15<00:00, 23.65it/s]
Loading 0:  98%|█████████▊| 357/363 [00:16<00:00, 23.86it/s]
                                                            
Job cycy233-modelv-fusion-s-35728-v1-mkmlizer completed after 103.94s with status: succeeded
Stopping job with name cycy233-modelv-fusion-s-35728-v1-mkmlizer
Pipeline stage MKMLizer completed in 104.64s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.34s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service cycy233-modelv-fusion-s-35728-v1
Waiting for inference service cycy233-modelv-fusion-s-35728-v1 to be ready
Inference service cycy233-modelv-fusion-s-35728-v1 ready after 120.7723708152771s
Pipeline stage MKMLDeployer completed in 121.28s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.5254132747650146s
Received healthy response to inference request in 1.3173112869262695s
Received healthy response to inference request in 1.7742505073547363s
Received healthy response to inference request in 1.371028184890747s
5 requests
1 failed requests
5th percentile: 1.328054666519165
10th percentile: 1.3387980461120605
20th percentile: 1.3602848052978516
30th percentile: 1.451672649383545
40th percentile: 1.6129615783691407
50th percentile: 1.7742505073547363
60th percentile: 2.0747156143188477
70th percentile: 2.375180721282959
80th percentile: 6.047257566452029
90th percentile: 13.090946149826053
95th percentile: 16.61279044151306
99th percentile: 19.43026587486267
mean time: 5.424527597427368
%s, retrying in %s seconds...
Received healthy response to inference request in 1.867152452468872s
Received healthy response to inference request in 1.799600601196289s
Received healthy response to inference request in 1.6575660705566406s
Received healthy response to inference request in 1.6063804626464844s
Received healthy response to inference request in 1.2768745422363281s
5 requests
0 failed requests
5th percentile: 1.3427757263183593
10th percentile: 1.4086769104003907
20th percentile: 1.5404792785644532
30th percentile: 1.6166175842285155
40th percentile: 1.637091827392578
50th percentile: 1.6575660705566406
60th percentile: 1.7143798828125
70th percentile: 1.7711936950683593
80th percentile: 1.8131109714508056
90th percentile: 1.8401317119598388
95th percentile: 1.8536420822143556
99th percentile: 1.8644503784179687
mean time: 1.641514825820923
Pipeline stage StressChecker completed in 38.07s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.93s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.69s
Shutdown handler de-registered
cycy233-modelv-fusion-s_35728_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
clean up pipeline due to error=DeploymentChecksError('None: None')
Shutdown handler de-registered
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
clean up pipeline due to error=DeploymentChecksError('None: None')
Shutdown handler de-registered
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
clean up pipeline due to error=DeploymentChecksError('None: None')
Shutdown handler de-registered
cycy233-modelv-fusion-s_35728_v1 status is now torndown due to DeploymentManager action