latitudegames-wayfarer-12b

developer_uid: zmeeks

submission_id: latitudegames-wayfarer-12b_v12

model_name: way-1109_k10

model_group: LatitudeGames/Wayfarer-1

status: torndown

timestamp: 2025-06-29T21:45:15+00:00

num_battles: 7530

num_wins: 3930

celo_rating: 1263.75

family_friendly_score: 0.5696

family_friendly_standard_error: 0.007002225931801972

submission_type: basic

model_repo: LatitudeGames/Wayfarer-12B

model_architecture: MistralForCausalLM

model_num_parameters: 12772090880.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

reward_model: default

latencies: [{'batch_size': 1, 'throughput': 0.6006529166734013, 'latency_mean': 1.664673626422882, 'latency_p50': 1.653498888015747, 'latency_p90': 1.8599868297576905}, {'batch_size': 3, 'throughput': 1.0764637388634661, 'latency_mean': 2.780066044330597, 'latency_p50': 2.8050457239151, 'latency_p90': 3.042129063606262}, {'batch_size': 5, 'throughput': 1.2883066369717846, 'latency_mean': 3.8659256899356844, 'latency_p50': 3.8820823431015015, 'latency_p90': 4.353687453269958}, {'batch_size': 6, 'throughput': 1.3410039744729523, 'latency_mean': 4.449179096221924, 'latency_p50': 4.433582186698914, 'latency_p90': 4.995176315307617}, {'batch_size': 8, 'throughput': 1.4114487620670144, 'latency_mean': 5.629961488246917, 'latency_p50': 5.635342121124268, 'latency_p90': 6.36455352306366}, {'batch_size': 10, 'throughput': 1.4477410289336927, 'latency_mean': 6.858260110616684, 'latency_p50': 6.826170921325684, 'latency_p90': 7.774037003517151}]

gpu_counts: {'NVIDIA RTX A5000': 1}

display_name: way-1109_k10

is_internal_developer: False

language_model: LatitudeGames/Wayfarer-12B

model_size: 13B

ranking_group: single

throughput_3p7s: 1.27

us_pacific_date: 2025-06-29

win_ratio: 0.5219123505976095

generation_params: {'temperature': 1.1, 'top_p': 0.9, 'min_p': 0.0, 'top_k': 10, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name latitudegames-wayfarer-12b-v12-mkmlizer
Waiting for job on latitudegames-wayfarer-12b-v12-mkmlizer to finish
latitudegames-wayfarer-12b-v12-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
latitudegames-wayfarer-12b-v12-mkmlizer: ║                                                                     ║
latitudegames-wayfarer-12b-v12-mkmlizer: ║   ██████   ██████  █████   ████  ████                               ║
latitudegames-wayfarer-12b-v12-mkmlizer: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
latitudegames-wayfarer-12b-v12-mkmlizer: ║   ░███░█████░███   ░███  ███     ░███                               ║
latitudegames-wayfarer-12b-v12-mkmlizer: ║   ░███░░███ ░███   ░███████      ░███                               ║
latitudegames-wayfarer-12b-v12-mkmlizer: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
latitudegames-wayfarer-12b-v12-mkmlizer: ║   ░███      ░███   ░███ ░░███    ░███                               ║
latitudegames-wayfarer-12b-v12-mkmlizer: ║   █████     █████  █████ ░░████  █████                              ║
latitudegames-wayfarer-12b-v12-mkmlizer: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
latitudegames-wayfarer-12b-v12-mkmlizer: ║                                                                     ║
latitudegames-wayfarer-12b-v12-mkmlizer: ║  Version: 0.29.3                                                    ║
latitudegames-wayfarer-12b-v12-mkmlizer: ║  Features: FLYWHEEL, CUDA                                           ║
latitudegames-wayfarer-12b-v12-mkmlizer: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
latitudegames-wayfarer-12b-v12-mkmlizer: ║  https://mk1.ai                                                     ║
latitudegames-wayfarer-12b-v12-mkmlizer: ║                                                                     ║
latitudegames-wayfarer-12b-v12-mkmlizer: ║  The license key for the current software has been verified as      ║
latitudegames-wayfarer-12b-v12-mkmlizer: ║  belonging to:                                                      ║
latitudegames-wayfarer-12b-v12-mkmlizer: ║                                                                     ║
latitudegames-wayfarer-12b-v12-mkmlizer: ║  Chai Research Corp.                                                ║
latitudegames-wayfarer-12b-v12-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
latitudegames-wayfarer-12b-v12-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
latitudegames-wayfarer-12b-v12-mkmlizer: ║                                                                     ║
latitudegames-wayfarer-12b-v12-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
latitudegames-wayfarer-12b-v12-mkmlizer: Downloaded to shared memory in 31.825s
latitudegames-wayfarer-12b-v12-mkmlizer: Checking if LatitudeGames/Wayfarer-12B already exists in ChaiML
latitudegames-wayfarer-12b-v12-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp7fise5_7, device:0
latitudegames-wayfarer-12b-v12-mkmlizer: Saving flywheel model at /dev/shm/model_cache
latitudegames-wayfarer-12b-v12-mkmlizer: quantized model in 31.005s
latitudegames-wayfarer-12b-v12-mkmlizer: Processed model LatitudeGames/Wayfarer-12B in 62.911s
latitudegames-wayfarer-12b-v12-mkmlizer: creating bucket guanaco-mkml-models
latitudegames-wayfarer-12b-v12-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
latitudegames-wayfarer-12b-v12-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/latitudegames-wayfarer-12b-v12
latitudegames-wayfarer-12b-v12-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/latitudegames-wayfarer-12b-v12/config.json
latitudegames-wayfarer-12b-v12-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/latitudegames-wayfarer-12b-v12/special_tokens_map.json
latitudegames-wayfarer-12b-v12-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/latitudegames-wayfarer-12b-v12/tokenizer_config.json
latitudegames-wayfarer-12b-v12-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/latitudegames-wayfarer-12b-v12/tokenizer.json
latitudegames-wayfarer-12b-v12-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/latitudegames-wayfarer-12b-v12/flywheel_model.0.safetensors
latitudegames-wayfarer-12b-v12-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:11, 32.19it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:06, 51.51it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:07, 44.86it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:07, 43.41it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:06, 49.02it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:07, 45.38it/s]
Loading 0:  12%|█▏        | 42/363 [00:00<00:07, 44.24it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:06, 49.21it/s]
Loading 0:  15%|█▌        | 55/363 [00:01<00:06, 45.75it/s]
Loading 0:  17%|█▋        | 61/363 [00:01<00:08, 34.67it/s]
Loading 0:  18%|█▊        | 65/363 [00:01<00:08, 33.33it/s]
Loading 0:  20%|█▉        | 71/363 [00:01<00:07, 38.82it/s]
Loading 0:  21%|██        | 76/363 [00:01<00:07, 39.69it/s]
Loading 0:  22%|██▏       | 81/363 [00:01<00:06, 41.19it/s]
Loading 0:  24%|██▍       | 87/363 [00:02<00:06, 39.95it/s]
Loading 0:  25%|██▌       | 92/363 [00:02<00:06, 38.91it/s]
Loading 0:  27%|██▋       | 98/363 [00:02<00:06, 43.62it/s]
Loading 0:  28%|██▊       | 103/363 [00:02<00:06, 42.70it/s]
Loading 0:  30%|██▉       | 108/363 [00:02<00:05, 44.54it/s]
Loading 0:  31%|███       | 113/363 [00:02<00:06, 37.62it/s]
Loading 0:  33%|███▎      | 118/363 [00:02<00:06, 38.19it/s]
Loading 0:  34%|███▍      | 125/363 [00:02<00:05, 45.17it/s]
Loading 0:  36%|███▌      | 130/363 [00:03<00:05, 44.83it/s]
Loading 0:  37%|███▋      | 135/363 [00:03<00:05, 43.53it/s]
Loading 0:  39%|███▉      | 141/363 [00:03<00:05, 41.30it/s]
Loading 0:  40%|████      | 146/363 [00:03<00:07, 29.49it/s]
Loading 0:  41%|████▏     | 150/363 [00:03<00:07, 28.98it/s]
Loading 0:  43%|████▎     | 156/363 [00:03<00:06, 34.06it/s]
Loading 0:  44%|████▍     | 160/363 [00:04<00:05, 34.44it/s]
Loading 0:  45%|████▌     | 165/363 [00:04<00:05, 37.20it/s]
Loading 0:  47%|████▋     | 169/363 [00:04<00:05, 36.44it/s]
Loading 0:  48%|████▊     | 174/363 [00:04<00:04, 39.76it/s]
Loading 0:  49%|████▉     | 179/363 [00:04<00:04, 40.50it/s]
Loading 0:  51%|█████     | 184/363 [00:04<00:04, 40.99it/s]
Loading 0:  52%|█████▏    | 189/363 [00:04<00:04, 41.60it/s]
Loading 0:  53%|█████▎    | 194/363 [00:04<00:04, 33.92it/s]
Loading 0:  55%|█████▌    | 201/363 [00:05<00:03, 41.59it/s]
Loading 0:  57%|█████▋    | 206/363 [00:05<00:03, 41.79it/s]
Loading 0:  58%|█████▊    | 211/363 [00:05<00:03, 41.78it/s]
Loading 0:  60%|█████▉    | 216/363 [00:05<00:03, 42.32it/s]
Loading 0:  61%|██████    | 221/363 [00:05<00:03, 43.78it/s]
Loading 0:  62%|██████▏   | 226/363 [00:05<00:05, 24.43it/s]
Loading 0:  63%|██████▎   | 230/363 [00:06<00:05, 25.63it/s]
Loading 0:  65%|██████▌   | 237/363 [00:06<00:03, 33.00it/s]
Loading 0:  67%|██████▋   | 242/363 [00:06<00:03, 34.02it/s]
Loading 0:  68%|██████▊   | 247/363 [00:06<00:03, 36.15it/s]
Loading 0:  69%|██████▉   | 252/363 [00:06<00:02, 39.29it/s]
Loading 0:  71%|███████   | 257/363 [00:06<00:03, 33.04it/s]
Loading 0:  73%|███████▎  | 264/363 [00:06<00:02, 40.52it/s]
Loading 0:  74%|███████▍  | 269/363 [00:06<00:02, 41.06it/s]
Loading 0:  75%|███████▌  | 274/363 [00:07<00:02, 41.63it/s]
Loading 0:  77%|███████▋  | 279/363 [00:07<00:01, 42.99it/s]
Loading 0:  78%|███████▊  | 284/363 [00:07<00:02, 35.46it/s]
Loading 0:  80%|████████  | 291/363 [00:07<00:01, 42.97it/s]
Loading 0:  82%|████████▏ | 296/363 [00:07<00:01, 42.67it/s]
Loading 0:  83%|████████▎ | 301/363 [00:07<00:01, 43.76it/s]
Loading 0:  84%|████████▍ | 306/363 [00:08<00:02, 22.79it/s]
Loading 0:  85%|████████▌ | 310/363 [00:08<00:02, 23.17it/s]
Loading 0:  87%|████████▋ | 314/363 [00:08<00:01, 25.62it/s]
Loading 0:  88%|████████▊ | 320/363 [00:08<00:01, 30.85it/s]
Loading 0:  89%|████████▉ | 324/363 [00:08<00:01, 32.34it/s]
Loading 0:  91%|█████████ | 329/363 [00:08<00:00, 35.11it/s]
Loading 0:  92%|█████████▏| 335/363 [00:08<00:00, 35.98it/s]
Loading 0:  93%|█████████▎| 339/363 [00:09<00:00, 34.98it/s]
Loading 0:  95%|█████████▌| 346/363 [00:09<00:00, 42.21it/s]
Loading 0:  97%|█████████▋| 351/363 [00:09<00:00, 42.60it/s]
Loading 0:  98%|█████████▊| 356/363 [00:09<00:00, 43.25it/s]
Loading 0: 100%|█████████▉| 362/363 [00:09<00:00, 41.08it/s]
                                                            
Job latitudegames-wayfarer-12b-v12-mkmlizer completed after 85.22s with status: succeeded
Stopping job with name latitudegames-wayfarer-12b-v12-mkmlizer
Pipeline stage MKMLizer completed in 85.90s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service latitudegames-wayfarer-12b-v12
Waiting for inference service latitudegames-wayfarer-12b-v12 to be ready
Inference service latitudegames-wayfarer-12b-v12 ready after 190.82618498802185s
Pipeline stage MKMLDeployer completed in 191.58s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.1639821529388428s
Received healthy response to inference request in 1.7731404304504395s
Received healthy response to inference request in 1.8171687126159668s
Received healthy response to inference request in 1.6915576457977295s
5 requests
1 failed requests
5th percentile: 1.7078742027282714
10th percentile: 1.7241907596588135
20th percentile: 1.7568238735198975
30th percentile: 1.7819460868835448
40th percentile: 1.7995573997497558
50th percentile: 1.8171687126159668
60th percentile: 1.9558940887451173
70th percentile: 2.0946194648742673
80th percentile: 5.758467102050784
90th percentile: 12.947437000274661
95th percentile: 16.541921949386595
99th percentile: 19.41750990867615
mean time: 5.516451168060303
%s, retrying in %s seconds...
Received healthy response to inference request in 1.4826488494873047s
Received healthy response to inference request in 1.8023812770843506s
Received healthy response to inference request in 1.5536530017852783s
Received healthy response to inference request in 2.000042676925659s
Received healthy response to inference request in 1.7342729568481445s
5 requests
0 failed requests
5th percentile: 1.4968496799468993
10th percentile: 1.5110505104064942
20th percentile: 1.5394521713256837
30th percentile: 1.5897769927978516
40th percentile: 1.662024974822998
50th percentile: 1.7342729568481445
60th percentile: 1.761516284942627
70th percentile: 1.7887596130371093
80th percentile: 1.8419135570526124
90th percentile: 1.9209781169891358
95th percentile: 1.9605103969573974
99th percentile: 1.9921362209320068
mean time: 1.7145997524261474
Pipeline stage StressChecker completed in 38.60s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.64s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 1.86s
Shutdown handler de-registered
latitudegames-wayfarer-12b_v12 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage MKMLProfilerDeleter completed in 0.12s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.11s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service latitudegames-wayfarer-12b-v12-profiler
Waiting for inference service latitudegames-wayfarer-12b-v12-profiler to be ready
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 4320.49s
Shutdown handler de-registered
latitudegames-wayfarer-12b_v12 status is now inactive due to auto deactivation removed underperforming models
latitudegames-wayfarer-12b_v12 status is now torndown due to DeploymentManager action