chaiml-llama31-204m-exp_85038

developer_uid: NischayDnk
submission_id: chaiml-llama31-204m-exp_85038_v1
model_name: chaiml-llama31-204m-exp_85038_v1
model_group: ChaiML/llama31-204m-exp3
status: torndown
timestamp: 2025-07-01T20:20:17+00:00
num_battles: 8853
num_wins: 4460
celo_rating: 1292.72
family_friendly_score: 0.5316000000000001
family_friendly_standard_error: 0.007056931911248684
submission_type: basic
model_repo: ChaiML/llama31-204m-exp3-data-dynamicv1-900k-full-1e-s05-noff-1-1
model_architecture: LlamaForSequenceClassification
model_num_parameters: 8030261248.0
best_of: 1
max_input_tokens: 512
max_output_tokens: 1
reward_model: default
display_name: chaiml-llama31-204m-exp_85038_v1
ineligible_reason: max_output_tokens!=64
is_internal_developer: False
language_model: ChaiML/llama31-204m-exp3-data-dynamicv1-900k-full-1e-s05-noff-1-1
model_size: 8B
ranking_group: single
us_pacific_date: 2025-07-01
win_ratio: 0.503784028013103
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 1, 'max_output_tokens': 1}
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-llama31-204m-exp-85038-v1-mkmlizer
Waiting for job on chaiml-llama31-204m-exp-85038-v1-mkmlizer to finish
chaiml-llama31-204m-exp-85038-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-llama31-204m-exp-85038-v1-mkmlizer: ║                                                                     ║
chaiml-llama31-204m-exp-85038-v1-mkmlizer: ║   ██████   ██████  █████   ████  ████                               ║
chaiml-llama31-204m-exp-85038-v1-mkmlizer: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
chaiml-llama31-204m-exp-85038-v1-mkmlizer: ║   ░███░█████░███   ░███  ███     ░███                               ║
chaiml-llama31-204m-exp-85038-v1-mkmlizer: ║   ░███░░███ ░███   ░███████      ░███                               ║
chaiml-llama31-204m-exp-85038-v1-mkmlizer: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
chaiml-llama31-204m-exp-85038-v1-mkmlizer: ║   ░███      ░███   ░███ ░░███    ░███                               ║
chaiml-llama31-204m-exp-85038-v1-mkmlizer: ║   █████     █████  █████ ░░████  █████                              ║
chaiml-llama31-204m-exp-85038-v1-mkmlizer: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
chaiml-llama31-204m-exp-85038-v1-mkmlizer: ║                                                                     ║
chaiml-llama31-204m-exp-85038-v1-mkmlizer: ║  Version: 0.29.3                                                    ║
chaiml-llama31-204m-exp-85038-v1-mkmlizer: ║  Features: FLYWHEEL, CUDA                                           ║
chaiml-llama31-204m-exp-85038-v1-mkmlizer: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
chaiml-llama31-204m-exp-85038-v1-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-llama31-204m-exp-85038-v1-mkmlizer: ║                                                                     ║
chaiml-llama31-204m-exp-85038-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-llama31-204m-exp-85038-v1-mkmlizer: ║  belonging to:                                                      ║
chaiml-llama31-204m-exp-85038-v1-mkmlizer: ║                                                                     ║
chaiml-llama31-204m-exp-85038-v1-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-llama31-204m-exp-85038-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-llama31-204m-exp-85038-v1-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
chaiml-llama31-204m-exp-85038-v1-mkmlizer: ║                                                                     ║
chaiml-llama31-204m-exp-85038-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-llama31-204m-exp-85038-v1-mkmlizer: Downloaded to shared memory in 29.513s
chaiml-llama31-204m-exp-85038-v1-mkmlizer: Checking if ChaiML/llama31-204m-exp3-data-dynamicv1-900k-full-1e-s05-noff-1-1 already exists in ChaiML
chaiml-llama31-204m-exp-85038-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:t0, folder:/tmp/tmpu_7emtge, device:0
chaiml-llama31-204m-exp-85038-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-llama31-204m-exp-85038-v1-mkmlizer: quantized model in 20.041s
chaiml-llama31-204m-exp-85038-v1-mkmlizer: Processed model ChaiML/llama31-204m-exp3-data-dynamicv1-900k-full-1e-s05-noff-1-1 in 49.555s
chaiml-llama31-204m-exp-85038-v1-mkmlizer: creating bucket guanaco-mkml-models
chaiml-llama31-204m-exp-85038-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-llama31-204m-exp-85038-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-llama31-204m-exp-85038-v1
chaiml-llama31-204m-exp-85038-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-llama31-204m-exp-85038-v1/config.json
chaiml-llama31-204m-exp-85038-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-llama31-204m-exp-85038-v1/tokenizer_config.json
chaiml-llama31-204m-exp-85038-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-llama31-204m-exp-85038-v1/special_tokens_map.json
chaiml-llama31-204m-exp-85038-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-llama31-204m-exp-85038-v1/tokenizer.json
chaiml-llama31-204m-exp-85038-v1-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 5/291 [00:00<00:08, 33.11it/s]
Loading 0:   4%|▍         | 13/291 [00:00<00:05, 52.49it/s]
Loading 0:   7%|▋         | 19/291 [00:00<00:05, 46.02it/s]
Loading 0:   8%|▊         | 24/291 [00:00<00:05, 44.86it/s]
Loading 0:  11%|█         | 31/291 [00:00<00:05, 50.48it/s]
Loading 0:  13%|█▎        | 37/291 [00:00<00:05, 44.42it/s]
Loading 0:  14%|█▍        | 42/291 [00:00<00:05, 43.17it/s]
Loading 0:  17%|█▋        | 49/291 [00:01<00:05, 48.07it/s]
Loading 0:  19%|█▉        | 55/291 [00:01<00:05, 44.51it/s]
Loading 0:  21%|██        | 60/291 [00:01<00:05, 44.47it/s]
Loading 0:  23%|██▎       | 67/291 [00:01<00:04, 49.82it/s]
Loading 0:  25%|██▌       | 73/291 [00:01<00:04, 45.99it/s]
Loading 0:  27%|██▋       | 78/291 [00:01<00:04, 44.15it/s]
Loading 0:  29%|██▊       | 83/291 [00:02<00:06, 30.06it/s]
Loading 0:  30%|██▉       | 87/291 [00:02<00:06, 30.19it/s]
Loading 0:  32%|███▏      | 94/291 [00:02<00:05, 37.20it/s]
Loading 0:  34%|███▍      | 100/291 [00:02<00:05, 37.34it/s]
Loading 0:  36%|███▌      | 105/291 [00:02<00:05, 36.06it/s]
Loading 0:  38%|███▊      | 111/291 [00:02<00:04, 40.98it/s]
Loading 0:  40%|███▉      | 116/291 [00:02<00:04, 41.85it/s]
Loading 0:  42%|████▏     | 121/291 [00:02<00:03, 43.14it/s]
Loading 0:  44%|████▎     | 127/291 [00:03<00:03, 41.39it/s]
Loading 0:  45%|████▌     | 132/291 [00:03<00:03, 41.66it/s]
Loading 0:  48%|████▊     | 139/291 [00:03<00:03, 46.79it/s]
Loading 0:  49%|████▉     | 144/291 [00:03<00:03, 46.60it/s]
Loading 0:  51%|█████     | 149/291 [00:03<00:03, 36.93it/s]
Loading 0:  54%|█████▎    | 156/291 [00:03<00:03, 42.80it/s]
Loading 0:  55%|█████▌    | 161/291 [00:03<00:03, 42.84it/s]
Loading 0:  57%|█████▋    | 166/291 [00:03<00:02, 43.73it/s]
Loading 0:  59%|█████▉    | 171/291 [00:04<00:02, 45.28it/s]
Loading 0:  61%|██████    | 177/291 [00:04<00:02, 41.86it/s]
Loading 0:  63%|██████▎   | 182/291 [00:04<00:02, 40.92it/s]
Loading 0:  64%|██████▍   | 187/291 [00:04<00:03, 28.91it/s]
Loading 0:  66%|██████▌   | 191/291 [00:04<00:03, 30.39it/s]
Loading 0:  67%|██████▋   | 195/291 [00:04<00:03, 30.68it/s]
Loading 0:  69%|██████▉   | 201/291 [00:04<00:02, 37.14it/s]
Loading 0:  71%|███████   | 206/291 [00:05<00:02, 37.39it/s]
Loading 0:  73%|███████▎  | 211/291 [00:05<00:02, 39.43it/s]
Loading 0:  74%|███████▍  | 216/291 [00:05<00:01, 41.96it/s]
Loading 0:  76%|███████▌  | 221/291 [00:05<00:01, 36.19it/s]
Loading 0:  79%|███████▊  | 229/291 [00:05<00:01, 43.80it/s]
Loading 0:  81%|████████  | 235/291 [00:05<00:01, 41.93it/s]
Loading 0:  82%|████████▏ | 240/291 [00:05<00:01, 40.60it/s]
Loading 0:  85%|████████▍ | 247/291 [00:06<00:00, 46.04it/s]
Loading 0:  87%|████████▋ | 253/291 [00:06<00:00, 43.78it/s]
Loading 0:  89%|████████▊ | 258/291 [00:06<00:00, 43.02it/s]
Loading 0:  91%|█████████ | 264/291 [00:06<00:00, 46.24it/s]
Loading 0:  92%|█████████▏| 269/291 [00:06<00:00, 45.83it/s]
Loading 0:  94%|█████████▍| 274/291 [00:06<00:00, 45.58it/s]
Loading 0:  96%|█████████▌| 280/291 [00:06<00:00, 43.43it/s]
Loading 0:  98%|█████████▊| 285/291 [00:06<00:00, 43.79it/s]
Loading 0: 100%|█████████▉| 290/291 [00:07<00:00, 31.16it/s]
                                                            
Job chaiml-llama31-204m-exp-85038-v1-mkmlizer completed after 74.91s with status: succeeded
Stopping job with name chaiml-llama31-204m-exp-85038-v1-mkmlizer
Pipeline stage MKMLizer completed in 75.58s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.19s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-llama31-204m-exp-85038-v1
Waiting for inference service chaiml-llama31-204m-exp-85038-v1 to be ready
Inference service chaiml-llama31-204m-exp-85038-v1 ready after 220.86661314964294s
Pipeline stage MKMLDeployer completed in 221.30s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 4.849214792251587s
Received healthy response to inference request in 4.735633611679077s
Received healthy response to inference request in 2.09399676322937s
Received healthy response to inference request in 3.420259475708008s
5 requests
1 failed requests
5th percentile: 2.359249305725098
10th percentile: 2.624501848220825
20th percentile: 3.15500693321228
30th percentile: 3.683334302902222
40th percentile: 4.209483957290649
50th percentile: 4.735633611679077
60th percentile: 4.781066083908081
70th percentile: 4.826498556137085
80th percentile: 7.908925819396975
90th percentile: 14.028347873687746
95th percentile: 17.088058900833126
99th percentile: 19.53582772254944
mean time: 7.049374914169311
%s, retrying in %s seconds...
Received healthy response to inference request in 2.6460654735565186s
Received healthy response to inference request in 3.063596248626709s
Received healthy response to inference request in 4.634184837341309s
Received healthy response to inference request in 3.483914613723755s
Received healthy response to inference request in 2.765256404876709s
5 requests
0 failed requests
5th percentile: 2.6699036598205566
10th percentile: 2.6937418460845945
20th percentile: 2.741418218612671
30th percentile: 2.824924373626709
40th percentile: 2.944260311126709
50th percentile: 3.063596248626709
60th percentile: 3.231723594665527
70th percentile: 3.399850940704346
80th percentile: 3.713968658447266
90th percentile: 4.174076747894287
95th percentile: 4.4041307926177975
99th percentile: 4.588174028396606
mean time: 3.318603515625
Pipeline stage StressChecker completed in 54.60s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.72s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.74s
Shutdown handler de-registered
chaiml-llama31-204m-exp_85038_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
clean up pipeline due to error=DeploymentChecksError('None: None')
Shutdown handler de-registered
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 4056.39s
Shutdown handler de-registered
chaiml-llama31-204m-exp_85038_v1 status is now inactive due to auto deactivation removed underperforming models
chaiml-llama31-204m-exp_85038_v1 status is now torndown due to DeploymentManager action