randomforest1024-v25

developer_uid: RandomForest1024

submission_id: randomforest1024-v25_v12

model_name: randomforest1024-v25_v12

model_group: RandomForest1024/V25

status: torndown

timestamp: 2025-02-10T19:32:41+00:00

num_battles: 11449

num_wins: 5565

celo_rating: 1251.0

family_friendly_score: 0.5964

family_friendly_standard_error: 0.006938400968522935

submission_type: basic

model_repo: RandomForest1024/V25

model_architecture: MistralForCausalLM

model_num_parameters: 12772070400.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

reward_model: default

display_name: randomforest1024-v25_v12

is_internal_developer: False

language_model: RandomForest1024/V25

model_size: 13B

ranking_group: single

us_pacific_date: 2025-02-10

win_ratio: 0.4860686522840423

generation_params: {'temperature': 0.7, 'top_p': 0.85, 'min_p': 0.0, 'top_k': 50, 'presence_penalty': 0.5, 'frequency_penalty': 0.3, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name randomforest1024-v25-v12-mkmlizer
Waiting for job on randomforest1024-v25-v12-mkmlizer to finish
randomforest1024-v25-v12-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
randomforest1024-v25-v12-mkmlizer: ║     _____            __           __                                ║
randomforest1024-v25-v12-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
randomforest1024-v25-v12-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
randomforest1024-v25-v12-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
randomforest1024-v25-v12-mkmlizer: ║       /___/                                                         ║
randomforest1024-v25-v12-mkmlizer: ║                                                                     ║
randomforest1024-v25-v12-mkmlizer: ║  Version: 0.11.12                                                   ║
randomforest1024-v25-v12-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
randomforest1024-v25-v12-mkmlizer: ║  https://mk1.ai                                                     ║
randomforest1024-v25-v12-mkmlizer: ║                                                                     ║
randomforest1024-v25-v12-mkmlizer: ║  The license key for the current software has been verified as      ║
randomforest1024-v25-v12-mkmlizer: ║  belonging to:                                                      ║
randomforest1024-v25-v12-mkmlizer: ║                                                                     ║
randomforest1024-v25-v12-mkmlizer: ║  Chai Research Corp.                                                ║
randomforest1024-v25-v12-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
randomforest1024-v25-v12-mkmlizer: ║  Expiration: 2025-04-15 23:59:59                                    ║
randomforest1024-v25-v12-mkmlizer: ║                                                                     ║
randomforest1024-v25-v12-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
randomforest1024-v25-v12-mkmlizer: Downloaded to shared memory in 35.849s
randomforest1024-v25-v12-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpy1r18d4p, device:0
randomforest1024-v25-v12-mkmlizer: Saving flywheel model at /dev/shm/model_cache
randomforest1024-v25-v12-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/randomforest1024-v25-v12/flywheel_model.0.safetensors
randomforest1024-v25-v12-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:10, 35.20it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:06, 54.58it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:07, 47.76it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:07, 46.49it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:06, 52.57it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:06, 48.16it/s]
Loading 0:  12%|█▏        | 42/363 [00:00<00:06, 46.47it/s]
Loading 0:  13%|█▎        | 49/363 [00:00<00:06, 51.03it/s]
Loading 0:  15%|█▌        | 55/363 [00:01<00:06, 47.59it/s]
Loading 0:  17%|█▋        | 61/363 [00:01<00:09, 33.49it/s]
Loading 0:  18%|█▊        | 65/363 [00:01<00:09, 32.96it/s]
Loading 0:  20%|█▉        | 72/363 [00:01<00:07, 39.50it/s]
Loading 0:  21%|██        | 77/363 [00:01<00:06, 41.82it/s]
Loading 0:  23%|██▎       | 82/363 [00:01<00:07, 37.15it/s]
Loading 0:  25%|██▍       | 89/363 [00:02<00:06, 44.49it/s]
Loading 0:  26%|██▌       | 94/363 [00:02<00:05, 45.02it/s]
Loading 0:  27%|██▋       | 99/363 [00:02<00:05, 45.99it/s]
Loading 0:  29%|██▉       | 105/363 [00:02<00:05, 43.75it/s]
Loading 0:  31%|███       | 112/363 [00:02<00:05, 46.95it/s]
Loading 0:  32%|███▏      | 117/363 [00:02<00:05, 45.17it/s]
Loading 0:  34%|███▍      | 123/363 [00:02<00:05, 42.92it/s]
Loading 0:  35%|███▌      | 128/363 [00:02<00:05, 41.93it/s]
Loading 0:  37%|███▋      | 134/363 [00:03<00:05, 45.05it/s]
Loading 0:  38%|███▊      | 139/363 [00:03<00:04, 45.24it/s]
Loading 0:  40%|███▉      | 144/363 [00:03<00:08, 26.72it/s]
Loading 0:  41%|████      | 149/363 [00:03<00:07, 29.22it/s]
Loading 0:  43%|████▎     | 156/363 [00:03<00:05, 36.56it/s]
Loading 0:  44%|████▍     | 161/363 [00:03<00:05, 36.87it/s]
Loading 0:  46%|████▌     | 166/363 [00:04<00:05, 37.96it/s]
Loading 0:  47%|████▋     | 172/363 [00:04<00:04, 38.42it/s]
Loading 0:  49%|████▉     | 177/363 [00:04<00:04, 38.87it/s]
Loading 0:  50%|█████     | 183/363 [00:04<00:04, 43.79it/s]
Loading 0:  52%|█████▏    | 188/363 [00:04<00:04, 43.47it/s]
Loading 0:  53%|█████▎    | 193/363 [00:04<00:03, 43.92it/s]
Loading 0:  55%|█████▍    | 199/363 [00:04<00:03, 42.08it/s]
Loading 0:  56%|█████▌    | 204/363 [00:04<00:03, 42.00it/s]
Loading 0:  58%|█████▊    | 210/363 [00:05<00:03, 45.62it/s]
Loading 0:  59%|█████▉    | 215/363 [00:05<00:03, 44.99it/s]
Loading 0:  61%|██████    | 221/363 [00:05<00:02, 48.71it/s]
Loading 0:  62%|██████▏   | 226/363 [00:05<00:05, 25.05it/s]
Loading 0:  63%|██████▎   | 230/363 [00:05<00:05, 26.22it/s]
Loading 0:  65%|██████▌   | 237/363 [00:05<00:03, 33.25it/s]
Loading 0:  67%|██████▋   | 242/363 [00:06<00:03, 35.46it/s]
Loading 0:  68%|██████▊   | 247/363 [00:06<00:03, 36.86it/s]
Loading 0:  70%|██████▉   | 253/363 [00:06<00:02, 36.74it/s]
Loading 0:  71%|███████   | 258/363 [00:06<00:02, 37.63it/s]
Loading 0:  73%|███████▎  | 264/363 [00:06<00:02, 41.86it/s]
Loading 0:  74%|███████▍  | 269/363 [00:06<00:02, 41.62it/s]
Loading 0:  75%|███████▌  | 274/363 [00:06<00:02, 42.51it/s]
Loading 0:  77%|███████▋  | 279/363 [00:06<00:01, 43.75it/s]
Loading 0:  78%|███████▊  | 284/363 [00:07<00:02, 36.25it/s]
Loading 0:  80%|████████  | 291/363 [00:07<00:01, 42.30it/s]
Loading 0:  82%|████████▏ | 296/363 [00:07<00:01, 42.36it/s]
Loading 0:  83%|████████▎ | 301/363 [00:07<00:01, 43.92it/s]
Loading 0:  84%|████████▍ | 306/363 [00:14<00:23,  2.47it/s]
Loading 0:  86%|████████▌ | 311/363 [00:14<00:15,  3.40it/s]
Loading 0:  87%|████████▋ | 317/363 [00:14<00:09,  4.90it/s]
Loading 0:  88%|████████▊ | 321/363 [00:14<00:06,  6.17it/s]
Loading 0:  91%|█████████ | 329/363 [00:14<00:03,  9.83it/s]
Loading 0:  92%|█████████▏| 335/363 [00:14<00:02, 12.78it/s]
Loading 0:  94%|█████████▎| 340/363 [00:14<00:01, 15.75it/s]
Loading 0:  96%|█████████▌| 347/363 [00:15<00:00, 21.35it/s]
Loading 0:  97%|█████████▋| 353/363 [00:15<00:00, 24.94it/s]
Loading 0:  99%|█████████▊| 358/363 [00:15<00:00, 28.07it/s]
                                                            
Job randomforest1024-v25-v12-mkmlizer completed after 109.25s with status: succeeded
Stopping job with name randomforest1024-v25-v12-mkmlizer
Pipeline stage MKMLizer completed in 109.68s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service randomforest1024-v25-v12
Waiting for inference service randomforest1024-v25-v12 to be ready
Inference service randomforest1024-v25-v12 ready after 200.67518830299377s
Pipeline stage MKMLDeployer completed in 201.14s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.266202211380005s
Received healthy response to inference request in 1.654585599899292s
Received healthy response to inference request in 1.6524255275726318s
Received healthy response to inference request in 1.6267855167388916s
Received healthy response to inference request in 1.6687901020050049s
5 requests
0 failed requests
5th percentile: 1.6319135189056397
10th percentile: 1.6370415210723877
20th percentile: 1.6472975254058837
30th percentile: 1.6528575420379639
40th percentile: 1.653721570968628
50th percentile: 1.654585599899292
60th percentile: 1.6602674007415772
70th percentile: 1.6659492015838624
80th percentile: 1.788272523880005
90th percentile: 2.027237367630005
95th percentile: 2.146719789505005
99th percentile: 2.242305727005005
mean time: 1.7737577915191651
Pipeline stage StressChecker completed in 9.98s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.57s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.59s
Shutdown handler de-registered
randomforest1024-v25_v12 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2519.03s
Shutdown handler de-registered
randomforest1024-v25_v12 status is now inactive due to auto deactivation removed underperforming models
randomforest1024-v25_v12 status is now torndown due to DeploymentManager action
randomforest1024-v25_v12 status is now torndown due to DeploymentManager action
randomforest1024-v25_v12 status is now torndown due to DeploymentManager action