dtnewman-daniel-202502_12303

developer_uid: chai_backend_admin
submission_id: dtnewman-daniel-202502_12303_v14
model_name: dtnewman-daniel-202502_12303_v14
model_group: dtnewman/daniel-20250211
status: torndown
timestamp: 2025-02-14T06:59:11+00:00
num_battles: 5323
num_wins: 2923
celo_rating: 1297.67
family_friendly_score: 0.5598000000000001
family_friendly_standard_error: 0.007020312813543283
submission_type: basic
model_repo: dtnewman/daniel-20250211-c-lt4000-4epochs-embeddings
model_architecture: MistralForCausalLM
model_num_parameters: 12772070400.0
best_of: 4
max_input_tokens: 1300
max_output_tokens: 100
reward_model: default
display_name: dtnewman-daniel-202502_12303_v14
ineligible_reason: max_output_tokens!=64
is_internal_developer: True
language_model: dtnewman/daniel-20250211-c-lt4000-4epochs-embeddings
model_size: 13B
ranking_group: single
us_pacific_date: 2025-02-13
win_ratio: 0.5491264324628968
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>'], 'max_input_tokens': 1300, 'best_of': 4, 'max_output_tokens': 100}
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': 'You: {message}\n', 'response_template': '####\n{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name dtnewman-daniel-202502-12303-v14-mkmlizer
Waiting for job on dtnewman-daniel-202502-12303-v14-mkmlizer to finish
dtnewman-daniel-202502-12303-v14-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
dtnewman-daniel-202502-12303-v14-mkmlizer: ║     _____            __           __                                ║
dtnewman-daniel-202502-12303-v14-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
dtnewman-daniel-202502-12303-v14-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
dtnewman-daniel-202502-12303-v14-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
dtnewman-daniel-202502-12303-v14-mkmlizer: ║       /___/                                                         ║
dtnewman-daniel-202502-12303-v14-mkmlizer: ║                                                                     ║
dtnewman-daniel-202502-12303-v14-mkmlizer: ║  Version: 0.12.8                                                    ║
dtnewman-daniel-202502-12303-v14-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
dtnewman-daniel-202502-12303-v14-mkmlizer: ║  https://mk1.ai                                                     ║
dtnewman-daniel-202502-12303-v14-mkmlizer: ║                                                                     ║
dtnewman-daniel-202502-12303-v14-mkmlizer: ║  The license key for the current software has been verified as      ║
dtnewman-daniel-202502-12303-v14-mkmlizer: ║  belonging to:                                                      ║
dtnewman-daniel-202502-12303-v14-mkmlizer: ║                                                                     ║
dtnewman-daniel-202502-12303-v14-mkmlizer: ║  Chai Research Corp.                                                ║
dtnewman-daniel-202502-12303-v14-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
dtnewman-daniel-202502-12303-v14-mkmlizer: ║  Expiration: 2025-04-15 23:59:59                                    ║
dtnewman-daniel-202502-12303-v14-mkmlizer: ║                                                                     ║
dtnewman-daniel-202502-12303-v14-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
dtnewman-daniel-202502-12303-v14-mkmlizer: Downloaded to shared memory in 31.756s
dtnewman-daniel-202502-12303-v14-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmprbkqujw0, device:0
dtnewman-daniel-202502-12303-v14-mkmlizer: Saving flywheel model at /dev/shm/model_cache
dtnewman-daniel-202502-12303-v14-mkmlizer: quantized model in 36.220s
dtnewman-daniel-202502-12303-v14-mkmlizer: Processed model dtnewman/daniel-20250211-c-lt4000-4epochs-embeddings in 67.977s
dtnewman-daniel-202502-12303-v14-mkmlizer: creating bucket guanaco-mkml-models
dtnewman-daniel-202502-12303-v14-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
dtnewman-daniel-202502-12303-v14-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/dtnewman-daniel-202502-12303-v14
dtnewman-daniel-202502-12303-v14-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/dtnewman-daniel-202502-12303-v14/config.json
dtnewman-daniel-202502-12303-v14-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/dtnewman-daniel-202502-12303-v14/special_tokens_map.json
dtnewman-daniel-202502-12303-v14-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/dtnewman-daniel-202502-12303-v14/tokenizer_config.json
dtnewman-daniel-202502-12303-v14-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/dtnewman-daniel-202502-12303-v14/tokenizer.json
dtnewman-daniel-202502-12303-v14-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/dtnewman-daniel-202502-12303-v14/flywheel_model.0.safetensors
dtnewman-daniel-202502-12303-v14-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:12, 29.59it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:07, 49.48it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:07, 43.60it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:07, 43.27it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:06, 49.49it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:07, 45.57it/s]
Loading 0:  12%|█▏        | 42/363 [00:00<00:07, 44.35it/s]
Loading 0:  13%|█▎        | 48/363 [00:01<00:06, 48.27it/s]
Loading 0:  15%|█▍        | 53/363 [00:01<00:06, 45.71it/s]
Loading 0:  16%|█▋        | 59/363 [00:01<00:06, 49.38it/s]
Loading 0:  18%|█▊        | 65/363 [00:01<00:10, 27.43it/s]
Loading 0:  20%|█▉        | 71/363 [00:01<00:08, 32.77it/s]
Loading 0:  21%|██        | 76/363 [00:01<00:08, 35.27it/s]
Loading 0:  22%|██▏       | 81/363 [00:02<00:07, 37.12it/s]
Loading 0:  24%|██▎       | 86/363 [00:02<00:07, 39.45it/s]
Loading 0:  25%|██▌       | 91/363 [00:02<00:07, 34.35it/s]
Loading 0:  27%|██▋       | 98/363 [00:02<00:06, 41.39it/s]
Loading 0:  28%|██▊       | 103/363 [00:02<00:06, 41.78it/s]
Loading 0:  30%|██▉       | 108/363 [00:02<00:05, 43.57it/s]
Loading 0:  31%|███       | 113/363 [00:02<00:06, 37.27it/s]
Loading 0:  33%|███▎      | 118/363 [00:02<00:06, 37.43it/s]
Loading 0:  35%|███▍      | 126/363 [00:03<00:05, 45.49it/s]
Loading 0:  36%|███▌      | 131/363 [00:03<00:05, 46.25it/s]
Loading 0:  37%|███▋      | 136/363 [00:03<00:05, 38.75it/s]
Loading 0:  39%|███▉      | 142/363 [00:03<00:07, 30.53it/s]
Loading 0:  40%|████      | 146/363 [00:03<00:06, 31.78it/s]
Loading 0:  41%|████▏     | 150/363 [00:03<00:06, 31.18it/s]
Loading 0:  43%|████▎     | 156/363 [00:04<00:05, 37.35it/s]
Loading 0:  44%|████▍     | 161/363 [00:04<00:05, 39.06it/s]
Loading 0:  46%|████▌     | 166/363 [00:04<00:04, 40.89it/s]
Loading 0:  47%|████▋     | 171/363 [00:04<00:04, 43.22it/s]
Loading 0:  48%|████▊     | 176/363 [00:04<00:05, 36.89it/s]
Loading 0:  50%|█████     | 183/363 [00:04<00:04, 44.20it/s]
Loading 0:  52%|█████▏    | 188/363 [00:04<00:03, 43.76it/s]
Loading 0:  53%|█████▎    | 193/363 [00:04<00:03, 43.69it/s]
Loading 0:  55%|█████▍    | 199/363 [00:05<00:03, 42.28it/s]
Loading 0:  56%|█████▌    | 204/363 [00:05<00:03, 42.17it/s]
Loading 0:  58%|█████▊    | 211/363 [00:05<00:03, 47.24it/s]
Loading 0:  60%|█████▉    | 217/363 [00:05<00:03, 44.31it/s]
Loading 0:  61%|██████▏   | 223/363 [00:05<00:04, 30.71it/s]
Loading 0:  63%|██████▎   | 227/363 [00:05<00:04, 31.52it/s]
Loading 0:  64%|██████▎   | 231/363 [00:05<00:04, 30.66it/s]
Loading 0:  65%|██████▌   | 237/363 [00:06<00:03, 36.27it/s]
Loading 0:  67%|██████▋   | 242/363 [00:06<00:03, 37.94it/s]
Loading 0:  68%|██████▊   | 247/363 [00:06<00:02, 39.14it/s]
Loading 0:  70%|██████▉   | 253/363 [00:06<00:02, 38.65it/s]
Loading 0:  71%|███████   | 258/363 [00:06<00:02, 38.36it/s]
Loading 0:  73%|███████▎  | 265/363 [00:06<00:02, 43.92it/s]
Loading 0:  74%|███████▍  | 270/363 [00:06<00:02, 45.23it/s]
Loading 0:  76%|███████▌  | 275/363 [00:07<00:02, 36.32it/s]
Loading 0:  78%|███████▊  | 283/363 [00:07<00:01, 44.29it/s]
Loading 0:  80%|███████▉  | 289/363 [00:07<00:01, 42.15it/s]
Loading 0:  81%|████████  | 294/363 [00:07<00:01, 40.67it/s]
Loading 0:  82%|████████▏ | 299/363 [00:07<00:01, 42.73it/s]
Loading 0:  84%|████████▎ | 304/363 [00:14<00:23,  2.55it/s]
Loading 0:  85%|████████▍ | 308/363 [00:14<00:16,  3.27it/s]
Loading 0:  86%|████████▌ | 312/363 [00:14<00:11,  4.26it/s]
Loading 0:  88%|████████▊ | 319/363 [00:14<00:06,  6.71it/s]
Loading 0:  89%|████████▉ | 324/363 [00:14<00:04,  8.86it/s]
Loading 0:  91%|█████████ | 329/363 [00:14<00:02, 11.58it/s]
Loading 0:  92%|█████████▏| 335/363 [00:15<00:01, 15.07it/s]
Loading 0:  94%|█████████▎| 340/363 [00:15<00:01, 18.13it/s]
Loading 0:  95%|█████████▌| 346/363 [00:15<00:00, 23.28it/s]
Loading 0:  97%|█████████▋| 351/363 [00:15<00:00, 26.77it/s]
Loading 0:  98%|█████████▊| 356/363 [00:15<00:00, 30.40it/s]
Loading 0:  99%|█████████▉| 361/363 [00:15<00:00, 33.67it/s]
                                                            
Job dtnewman-daniel-202502-12303-v14-mkmlizer completed after 93.61s with status: succeeded
Stopping job with name dtnewman-daniel-202502-12303-v14-mkmlizer
Pipeline stage MKMLizer completed in 94.06s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.14s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service dtnewman-daniel-202502-12303-v14
Waiting for inference service dtnewman-daniel-202502-12303-v14 to be ready
Inference service dtnewman-daniel-202502-12303-v14 ready after 190.78098154067993s
Pipeline stage MKMLDeployer completed in 191.60s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.7363295555114746s
Received healthy response to inference request in 2.5210726261138916s
Received healthy response to inference request in 2.4052555561065674s
Received healthy response to inference request in 2.1734166145324707s
Received healthy response to inference request in 2.260319948196411s
5 requests
0 failed requests
5th percentile: 2.190797281265259
10th percentile: 2.208177947998047
20th percentile: 2.242939281463623
30th percentile: 2.2893070697784426
40th percentile: 2.347281312942505
50th percentile: 2.4052555561065674
60th percentile: 2.4515823841094972
70th percentile: 2.4979092121124267
80th percentile: 2.564124011993408
90th percentile: 2.6502267837524416
95th percentile: 2.693278169631958
99th percentile: 2.7277192783355715
mean time: 2.419278860092163
Pipeline stage StressChecker completed in 13.32s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.62s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.68s
Shutdown handler de-registered
dtnewman-daniel-202502_12303_v14 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage MKMLProfilerDeleter completed in 0.08s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.08s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service dtnewman-daniel-202502-12303-v14-profiler
Waiting for inference service dtnewman-daniel-202502-12303-v14-profiler to be ready
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2800.65s
Shutdown handler de-registered
dtnewman-daniel-202502_12303_v14 status is now inactive due to auto deactivation removed underperforming models
dtnewman-daniel-202502_12303_v14 status is now torndown due to DeploymentManager action
dtnewman-daniel-202502_12303_v14 status is now torndown due to DeploymentManager action
dtnewman-daniel-202502_12303_v14 status is now torndown due to DeploymentManager action