chaiml-ezstorytellingedi_3985

developer_uid: ebony59

submission_id: chaiml-ezstorytellingedi_3985_v1

model_name: chaiml-ezstorytellingedi_3985_v1

model_group: ChaiML/EZStorytellingEdi

status: inactive

timestamp: 2024-12-05T05:58:53+00:00

num_battles: 11456

num_wins: 5376

celo_rating: 1241.32

family_friendly_score: 0.5602

family_friendly_standard_error: 0.007019629050028214

submission_type: basic

model_repo: ChaiML/EZStorytellingEditsSFT_Qi6_71convos_reward_4epoch

model_architecture: MistralForCausalLM

model_num_parameters: 22247282688.0

best_of: 4

max_input_tokens: 1024

max_output_tokens: 64

latencies: [{'batch_size': 1, 'throughput': 0.38403870712393345, 'latency_mean': 2.603825581073761, 'latency_p50': 2.611825704574585, 'latency_p90': 2.8734701395034787}, {'batch_size': 3, 'throughput': 0.8131051220208777, 'latency_mean': 3.67486886382103, 'latency_p50': 3.729366183280945, 'latency_p90': 4.0140389680862425}, {'batch_size': 5, 'throughput': 1.0794355900911248, 'latency_mean': 4.607983961105346, 'latency_p50': 4.603312611579895, 'latency_p90': 5.164565372467041}, {'batch_size': 6, 'throughput': 1.1668497260174469, 'latency_mean': 5.101895154714584, 'latency_p50': 5.132061719894409, 'latency_p90': 5.78669159412384}, {'batch_size': 10, 'throughput': 1.3775821281441747, 'latency_mean': 7.170490814447403, 'latency_p50': 7.171633958816528, 'latency_p90': 8.185191655158997}]

gpu_counts: {'NVIDIA RTX A6000': 1}

display_name: chaiml-ezstorytellingedi_3985_v1

is_internal_developer: True

language_model: ChaiML/EZStorytellingEditsSFT_Qi6_71convos_reward_4epoch

model_size: 22B

ranking_group: single

throughput_3p7s: 0.83

us_pacific_date: 2024-12-04

win_ratio: 0.4692737430167598

generation_params: {'temperature': 0.9, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 4, 'max_output_tokens': 64}

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-ezstorytellingedi-3985-v1-mkmlizer
Waiting for job on chaiml-ezstorytellingedi-3985-v1-mkmlizer to finish
chaiml-ezstorytellingedi-3985-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-ezstorytellingedi-3985-v1-mkmlizer: ║     _____            __           __                                ║
chaiml-ezstorytellingedi-3985-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
chaiml-ezstorytellingedi-3985-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
chaiml-ezstorytellingedi-3985-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
chaiml-ezstorytellingedi-3985-v1-mkmlizer: ║       /___/                                                         ║
chaiml-ezstorytellingedi-3985-v1-mkmlizer: ║                                                                     ║
chaiml-ezstorytellingedi-3985-v1-mkmlizer: ║  Version: 0.11.12                                                   ║
chaiml-ezstorytellingedi-3985-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
chaiml-ezstorytellingedi-3985-v1-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-ezstorytellingedi-3985-v1-mkmlizer: ║                                                                     ║
chaiml-ezstorytellingedi-3985-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-ezstorytellingedi-3985-v1-mkmlizer: ║  belonging to:                                                      ║
chaiml-ezstorytellingedi-3985-v1-mkmlizer: ║                                                                     ║
chaiml-ezstorytellingedi-3985-v1-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-ezstorytellingedi-3985-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-ezstorytellingedi-3985-v1-mkmlizer: ║  Expiration: 2025-01-15 23:59:59                                    ║
chaiml-ezstorytellingedi-3985-v1-mkmlizer: ║                                                                     ║
chaiml-ezstorytellingedi-3985-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-ezstorytellingedi-3985-v1-mkmlizer: Downloaded to shared memory in 69.733s
chaiml-ezstorytellingedi-3985-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmplas_28dq, device:0
chaiml-ezstorytellingedi-3985-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-ezstorytellingedi-3985-v1-mkmlizer: quantized model in 45.191s
chaiml-ezstorytellingedi-3985-v1-mkmlizer: Processed model ChaiML/EZStorytellingEditsSFT_Qi6_71convos_reward_4epoch in 114.925s
chaiml-ezstorytellingedi-3985-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-ezstorytellingedi-3985-v1/special_tokens_map.json
chaiml-ezstorytellingedi-3985-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-ezstorytellingedi-3985-v1/tokenizer_config.json
chaiml-ezstorytellingedi-3985-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-ezstorytellingedi-3985-v1/tokenizer.json
chaiml-ezstorytellingedi-3985-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/chaiml-ezstorytellingedi-3985-v1/flywheel_model.1.safetensors
chaiml-ezstorytellingedi-3985-v1-mkmlizer: 
Loading 0:   0%|          | 0/507 [00:00<?, ?it/s]
Loading 0:   1%|          | 5/507 [00:00<00:20, 24.51it/s]
Loading 0:   2%|▏         | 12/507 [00:00<00:12, 40.59it/s]
Loading 0:   3%|▎         | 17/507 [00:00<00:12, 39.31it/s]
Loading 0:   4%|▍         | 22/507 [00:00<00:12, 38.89it/s]
Loading 0:   5%|▌         | 27/507 [00:00<00:12, 39.50it/s]
Loading 0:   6%|▋         | 32/507 [00:00<00:14, 32.11it/s]
Loading 0:   8%|▊         | 39/507 [00:01<00:11, 39.35it/s]
Loading 0:   9%|▊         | 44/507 [00:01<00:12, 38.00it/s]
Loading 0:  10%|▉         | 49/507 [00:01<00:13, 33.16it/s]
Loading 0:  10%|█         | 53/507 [00:01<00:17, 25.90it/s]
Loading 0:  11%|█         | 56/507 [00:01<00:18, 24.16it/s]
Loading 0:  12%|█▏        | 63/507 [00:01<00:13, 31.87it/s]
Loading 0:  13%|█▎        | 67/507 [00:02<00:13, 32.54it/s]
Loading 0:  14%|█▍        | 73/507 [00:02<00:12, 34.91it/s]
Loading 0:  16%|█▌        | 80/507 [00:02<00:12, 35.55it/s]
Loading 0:  17%|█▋        | 87/507 [00:02<00:10, 41.10it/s]
Loading 0:  18%|█▊        | 92/507 [00:02<00:10, 40.52it/s]
Loading 0:  19%|█▉        | 97/507 [00:02<00:10, 39.85it/s]
Loading 0:  20%|██        | 102/507 [00:02<00:09, 41.21it/s]
Loading 0:  21%|██        | 107/507 [00:03<00:11, 35.47it/s]
Loading 0:  22%|██▏       | 113/507 [00:03<00:13, 30.17it/s]
Loading 0:  23%|██▎       | 117/507 [00:03<00:13, 28.90it/s]
Loading 0:  24%|██▍       | 122/507 [00:03<00:12, 29.72it/s]
Loading 0:  25%|██▌       | 129/507 [00:03<00:10, 36.22it/s]
Loading 0:  26%|██▌       | 133/507 [00:03<00:10, 35.86it/s]
Loading 0:  27%|██▋       | 138/507 [00:03<00:09, 38.95it/s]
Loading 0:  28%|██▊       | 143/507 [00:04<00:09, 39.01it/s]
Loading 0:  29%|██▉       | 148/507 [00:04<00:09, 39.76it/s]
Loading 0:  30%|███       | 153/507 [00:04<00:08, 41.04it/s]
Loading 0:  31%|███       | 158/507 [00:04<00:09, 35.94it/s]
Loading 0:  32%|███▏      | 164/507 [00:04<00:08, 41.53it/s]
Loading 0:  33%|███▎      | 169/507 [00:04<00:10, 31.87it/s]
Loading 0:  34%|███▍      | 173/507 [00:04<00:10, 31.96it/s]
Loading 0:  35%|███▍      | 177/507 [00:05<00:10, 30.88it/s]
Loading 0:  36%|███▌      | 183/507 [00:05<00:09, 35.72it/s]
Loading 0:  37%|███▋      | 187/507 [00:05<00:08, 35.65it/s]
Loading 0:  38%|███▊      | 192/507 [00:05<00:08, 39.13it/s]
Loading 0:  39%|███▉      | 197/507 [00:05<00:07, 39.06it/s]
Loading 0:  40%|███▉      | 202/507 [00:05<00:07, 40.25it/s]
Loading 0:  41%|████      | 207/507 [00:05<00:07, 41.02it/s]
Loading 0:  42%|████▏     | 212/507 [00:05<00:08, 35.58it/s]
Loading 0:  43%|████▎     | 218/507 [00:06<00:07, 41.19it/s]
Loading 0:  44%|████▍     | 223/507 [00:06<00:07, 36.14it/s]
Loading 0:  45%|████▍     | 227/507 [00:06<00:09, 29.93it/s]
Loading 0:  46%|████▌     | 231/507 [00:06<00:09, 28.78it/s]
Loading 0:  47%|████▋     | 237/507 [00:06<00:08, 33.16it/s]
Loading 0:  48%|████▊     | 241/507 [00:06<00:08, 32.02it/s]
Loading 0:  49%|████▊     | 246/507 [00:07<00:07, 33.34it/s]
Loading 0:  49%|████▉     | 250/507 [00:07<00:07, 32.45it/s]
Loading 0:  50%|█████     | 255/507 [00:07<00:07, 34.68it/s]
Loading 0:  51%|█████     | 259/507 [00:07<00:07, 33.43it/s]
Loading 0:  52%|█████▏    | 264/507 [00:07<00:06, 36.01it/s]
Loading 0:  53%|█████▎    | 268/507 [00:07<00:06, 35.10it/s]
Loading 0:  54%|█████▍    | 273/507 [00:07<00:06, 37.43it/s]
Loading 0:  55%|█████▍    | 277/507 [00:07<00:06, 35.87it/s]
Loading 0:  56%|█████▌    | 283/507 [00:08<00:05, 38.02it/s]
Loading 0:  57%|█████▋    | 287/507 [00:08<00:09, 23.28it/s]
Loading 0:  58%|█████▊    | 293/507 [00:08<00:08, 26.18it/s]
Loading 0:  59%|█████▉    | 299/507 [00:23<00:07, 26.18it/s]
Loading 0:  59%|█████▉    | 300/507 [00:23<02:50,  1.21it/s]
Loading 0:  60%|█████▉    | 302/507 [00:23<02:29,  1.37it/s]
Loading 0:  61%|██████    | 307/507 [00:23<01:41,  1.98it/s]
Loading 0:  61%|██████    | 310/507 [00:24<01:19,  2.47it/s]
Loading 0:  62%|██████▏   | 315/507 [00:24<00:52,  3.63it/s]
Loading 0:  63%|██████▎   | 319/507 [00:24<00:38,  4.86it/s]
Loading 0:  64%|██████▍   | 325/507 [00:24<00:25,  7.25it/s]
Loading 0:  65%|██████▍   | 329/507 [00:24<00:19,  9.04it/s]
Loading 0:  66%|██████▌   | 335/507 [00:24<00:13, 12.90it/s]
Loading 0:  67%|██████▋   | 340/507 [00:24<00:11, 14.45it/s]
Loading 0:  68%|██████▊   | 344/507 [00:25<00:09, 16.77it/s]
Loading 0:  69%|██████▊   | 348/507 [00:25<00:08, 18.54it/s]
Loading 0:  70%|██████▉   | 354/507 [00:25<00:06, 24.50it/s]
Loading 0:  71%|███████   | 358/507 [00:25<00:05, 26.56it/s]
Loading 0:  72%|███████▏  | 363/507 [00:25<00:04, 30.71it/s]
Loading 0:  73%|███████▎  | 368/507 [00:25<00:04, 32.98it/s]
Loading 0:  74%|███████▎  | 373/507 [00:25<00:03, 34.85it/s]
Loading 0:  75%|███████▍  | 378/507 [00:25<00:03, 37.50it/s]
Loading 0:  76%|███████▌  | 383/507 [00:26<00:03, 32.07it/s]
Loading 0:  77%|███████▋  | 389/507 [00:26<00:03, 37.36it/s]
Loading 0:  78%|███████▊  | 394/507 [00:26<00:03, 34.02it/s]
Loading 0:  79%|███████▊  | 398/507 [00:26<00:03, 27.37it/s]
Loading 0:  79%|███████▉  | 402/507 [00:26<00:03, 26.84it/s]
Loading 0:  80%|████████  | 408/507 [00:26<00:03, 31.96it/s]
Loading 0:  81%|████████▏ | 412/507 [00:27<00:02, 32.16it/s]
Loading 0:  82%|████████▏ | 417/507 [00:27<00:02, 35.35it/s]
Loading 0:  83%|████████▎ | 421/507 [00:27<00:02, 33.49it/s]
Loading 0:  84%|████████▍ | 426/507 [00:27<00:02, 36.85it/s]
Loading 0:  85%|████████▍ | 430/507 [00:27<00:02, 34.22it/s]
Loading 0:  86%|████████▌ | 434/507 [00:27<00:02, 35.09it/s]
Loading 0:  86%|████████▋ | 438/507 [00:27<00:02, 30.58it/s]
Loading 0:  88%|████████▊ | 444/507 [00:27<00:01, 34.76it/s]
Loading 0:  88%|████████▊ | 448/507 [00:28<00:01, 33.71it/s]
Loading 0:  90%|████████▉ | 454/507 [00:28<00:01, 36.95it/s]
Loading 0:  90%|█████████ | 458/507 [00:30<00:08,  5.94it/s]
Loading 0:  91%|█████████ | 461/507 [00:30<00:06,  7.12it/s]
Loading 0:  92%|█████████▏| 465/507 [00:30<00:04,  9.08it/s]
Loading 0:  93%|█████████▎| 472/507 [00:30<00:02, 14.08it/s]
Loading 0:  94%|█████████▍| 476/507 [00:31<00:01, 16.49it/s]
Loading 0:  95%|█████████▍| 481/507 [00:31<00:01, 20.65it/s]
Loading 0:  96%|█████████▌| 485/507 [00:31<00:00, 23.01it/s]
Loading 0:  97%|█████████▋| 490/507 [00:31<00:00, 27.16it/s]
Loading 0:  97%|█████████▋| 494/507 [00:31<00:00, 28.66it/s]
Loading 0:  98%|█████████▊| 499/507 [00:31<00:00, 31.74it/s]
Loading 0:  99%|█████████▉| 503/507 [00:31<00:00, 32.36it/s]
                                                            
Job chaiml-ezstorytellingedi-3985-v1-mkmlizer completed after 145.88s with status: succeeded
Stopping job with name chaiml-ezstorytellingedi-3985-v1-mkmlizer
Pipeline stage MKMLizer completed in 147.33s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.16s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-ezstorytellingedi-3985-v1
Waiting for inference service chaiml-ezstorytellingedi-3985-v1 to be ready
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Inference service chaiml-ezstorytellingedi-3985-v1 ready after 175.45724892616272s
Pipeline stage MKMLDeployer completed in 176.88s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.275845527648926s
Received healthy response to inference request in 2.7734522819519043s
Received healthy response to inference request in 2.61843204498291s
Received healthy response to inference request in 2.8472323417663574s
Received healthy response to inference request in 2.8621695041656494s
5 requests
0 failed requests
5th percentile: 2.649436092376709
10th percentile: 2.680440139770508
20th percentile: 2.7424482345581054
30th percentile: 2.7882082939147947
40th percentile: 2.817720317840576
50th percentile: 2.8472323417663574
60th percentile: 2.8532072067260743
70th percentile: 2.859182071685791
80th percentile: 2.9449047088623046
90th percentile: 3.1103751182556154
95th percentile: 3.1931103229522706
99th percentile: 3.2592984867095947
mean time: 2.8754263401031492
Pipeline stage StressChecker completed in 15.80s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.47s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 2.49s
Shutdown handler de-registered
chaiml-ezstorytellingedi_3985_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3197.88s
Shutdown handler de-registered
chaiml-ezstorytellingedi_3985_v1 status is now inactive due to auto deactivation removed underperforming models