developer_uid: azuruce
submission_id: chaiml-ezstorytellingedi_9240_v3
model_name: chaiml-ezstorytellingedi_9240_v3
model_group: ChaiML/EZStorytellingEdi
status: inactive
timestamp: 2024-12-06T23:26:15+00:00
num_battles: 11682
num_wins: 5755
celo_rating: 1259.46
family_friendly_score: 0.567
family_friendly_standard_error: 0.007007296197535822
submission_type: basic
model_repo: ChaiML/EZStorytellingEditsSFT_Qi6_65convos_reward
model_architecture: MistralForCausalLM
model_num_parameters: 22247282688.0
best_of: 4
max_input_tokens: 1024
max_output_tokens: 64
latencies: [{'batch_size': 1, 'throughput': 0.3855212776916113, 'latency_mean': 2.5938096058368685, 'latency_p50': 2.6008979082107544, 'latency_p90': 2.8634013414382933}, {'batch_size': 3, 'throughput': 0.8077091564481508, 'latency_mean': 3.6981192219257353, 'latency_p50': 3.711108088493347, 'latency_p90': 4.032619953155518}, {'batch_size': 5, 'throughput': 1.0808377461969576, 'latency_mean': 4.600933209657669, 'latency_p50': 4.576558947563171, 'latency_p90': 5.263094425201416}, {'batch_size': 6, 'throughput': 1.1670140535564144, 'latency_mean': 5.106165682077408, 'latency_p50': 5.054140686988831, 'latency_p90': 5.738351821899414}, {'batch_size': 10, 'throughput': 1.3715780675540026, 'latency_mean': 7.209777995347976, 'latency_p50': 7.216475605964661, 'latency_p90': 8.210202431678772}]
gpu_counts: {'NVIDIA RTX A6000': 1}
display_name: chaiml-ezstorytellingedi_9240_v3
is_internal_developer: True
language_model: ChaiML/EZStorytellingEditsSFT_Qi6_65convos_reward
model_size: 22B
ranking_group: single
throughput_3p7s: 0.81
us_pacific_date: 2024-12-06
win_ratio: 0.49263824687553504
generation_params: {'temperature': 0.9, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 4, 'max_output_tokens': 64}
formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-ezstorytellingedi-9240-v3-mkmlizer
Waiting for job on chaiml-ezstorytellingedi-9240-v3-mkmlizer to finish
chaiml-ezstorytellingedi-9240-v3-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-ezstorytellingedi-9240-v3-mkmlizer: ║     _____            __           __                                ║
chaiml-ezstorytellingedi-9240-v3-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
chaiml-ezstorytellingedi-9240-v3-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
chaiml-ezstorytellingedi-9240-v3-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
chaiml-ezstorytellingedi-9240-v3-mkmlizer: ║       /___/                                                         ║
chaiml-ezstorytellingedi-9240-v3-mkmlizer: ║                                                                     ║
chaiml-ezstorytellingedi-9240-v3-mkmlizer: ║  Version: 0.11.12                                                   ║
chaiml-ezstorytellingedi-9240-v3-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
chaiml-ezstorytellingedi-9240-v3-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-ezstorytellingedi-9240-v3-mkmlizer: ║                                                                     ║
chaiml-ezstorytellingedi-9240-v3-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-ezstorytellingedi-9240-v3-mkmlizer: ║  belonging to:                                                      ║
chaiml-ezstorytellingedi-9240-v3-mkmlizer: ║                                                                     ║
chaiml-ezstorytellingedi-9240-v3-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-ezstorytellingedi-9240-v3-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-ezstorytellingedi-9240-v3-mkmlizer: ║  Expiration: 2025-01-15 23:59:59                                    ║
chaiml-ezstorytellingedi-9240-v3-mkmlizer: ║                                                                     ║
chaiml-ezstorytellingedi-9240-v3-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
chaiml-ezstorytellingedi-9240-v3-mkmlizer: Downloaded to shared memory in 90.572s
chaiml-ezstorytellingedi-9240-v3-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpb2rt9llj, device:0
chaiml-ezstorytellingedi-9240-v3-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-ezstorytellingedi-9240-v3-mkmlizer: quantized model in 43.175s
chaiml-ezstorytellingedi-9240-v3-mkmlizer: Processed model ChaiML/EZStorytellingEditsSFT_Qi6_65convos_reward in 133.747s
chaiml-ezstorytellingedi-9240-v3-mkmlizer: creating bucket guanaco-mkml-models
chaiml-ezstorytellingedi-9240-v3-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-ezstorytellingedi-9240-v3-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-ezstorytellingedi-9240-v3
chaiml-ezstorytellingedi-9240-v3-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-ezstorytellingedi-9240-v3/config.json
chaiml-ezstorytellingedi-9240-v3-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-ezstorytellingedi-9240-v3/special_tokens_map.json
chaiml-ezstorytellingedi-9240-v3-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-ezstorytellingedi-9240-v3/tokenizer_config.json
chaiml-ezstorytellingedi-9240-v3-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-ezstorytellingedi-9240-v3/tokenizer.json
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
chaiml-ezstorytellingedi-9240-v3-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/chaiml-ezstorytellingedi-9240-v3/flywheel_model.1.safetensors
chaiml-ezstorytellingedi-9240-v3-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-ezstorytellingedi-9240-v3/flywheel_model.0.safetensors
chaiml-ezstorytellingedi-9240-v3-mkmlizer: 
Loading 0:   0%|          | 0/507 [00:00<?, ?it/s]
Loading 0:   1%|          | 5/507 [00:00<00:19, 25.64it/s]
Loading 0:   2%|▏         | 12/507 [00:00<00:11, 41.64it/s]
Loading 0:   3%|▎         | 17/507 [00:00<00:12, 38.80it/s]
Loading 0:   4%|▍         | 22/507 [00:00<00:12, 39.59it/s]
Loading 0:   5%|▌         | 27/507 [00:00<00:11, 40.89it/s]
Loading 0:   6%|▋         | 32/507 [00:00<00:14, 32.76it/s]
Loading 0:   8%|▊         | 39/507 [00:01<00:11, 39.34it/s]
Loading 0:   9%|▊         | 44/507 [00:01<00:11, 39.47it/s]
Loading 0:  10%|▉         | 49/507 [00:01<00:13, 34.24it/s]
Loading 0:  10%|█         | 53/507 [00:01<00:17, 26.66it/s]
Loading 0:  11%|█         | 57/507 [00:01<00:17, 26.42it/s]
Loading 0:  12%|█▏        | 63/507 [00:01<00:14, 31.43it/s]
Loading 0:  13%|█▎        | 67/507 [00:02<00:14, 31.12it/s]
Loading 0:  14%|█▍        | 73/507 [00:02<00:12, 33.90it/s]
Loading 0:  16%|█▌        | 80/507 [00:02<00:11, 35.95it/s]
Loading 0:  17%|█▋        | 87/507 [00:02<00:10, 40.83it/s]
Loading 0:  18%|█▊        | 92/507 [00:02<00:10, 39.72it/s]
Loading 0:  19%|█▉        | 97/507 [00:02<00:10, 39.47it/s]
Loading 0:  20%|██        | 102/507 [00:02<00:10, 39.81it/s]
Loading 0:  21%|██        | 107/507 [00:03<00:11, 34.19it/s]
Loading 0:  22%|██▏       | 113/507 [00:03<00:12, 30.32it/s]
Loading 0:  23%|██▎       | 117/507 [00:03<00:13, 29.07it/s]
Loading 0:  24%|██▍       | 122/507 [00:03<00:13, 29.54it/s]
Loading 0:  25%|██▌       | 129/507 [00:03<00:10, 36.13it/s]
Loading 0:  26%|██▌       | 133/507 [00:03<00:10, 34.68it/s]
Loading 0:  27%|██▋       | 138/507 [00:03<00:09, 37.56it/s]
Loading 0:  28%|██▊       | 142/507 [00:04<00:10, 36.36it/s]
Loading 0:  29%|██▉       | 147/507 [00:04<00:09, 39.02it/s]
Loading 0:  30%|██▉       | 152/507 [00:04<00:09, 38.20it/s]
Loading 0:  31%|███       | 157/507 [00:04<00:09, 38.44it/s]
Loading 0:  32%|███▏      | 162/507 [00:04<00:08, 40.15it/s]
Loading 0:  33%|███▎      | 167/507 [00:04<00:08, 41.01it/s]
Loading 0:  34%|███▍      | 172/507 [00:05<00:12, 26.07it/s]
Loading 0:  35%|███▍      | 176/507 [00:05<00:12, 25.86it/s]
Loading 0:  36%|███▌      | 183/507 [00:05<00:09, 33.24it/s]
Loading 0:  37%|███▋      | 187/507 [00:05<00:09, 33.18it/s]
Loading 0:  38%|███▊      | 192/507 [00:05<00:08, 36.26it/s]
Loading 0:  39%|███▉      | 197/507 [00:05<00:08, 35.26it/s]
Loading 0:  40%|███▉      | 201/507 [00:05<00:08, 36.26it/s]
Loading 0:  40%|████      | 205/507 [00:05<00:08, 35.23it/s]
Loading 0:  41%|████▏     | 210/507 [00:06<00:07, 38.20it/s]
Loading 0:  42%|████▏     | 214/507 [00:06<00:07, 36.79it/s]
Loading 0:  43%|████▎     | 218/507 [00:06<00:07, 36.82it/s]
Loading 0:  44%|████▍     | 222/507 [00:06<00:07, 35.92it/s]
Loading 0:  45%|████▍     | 226/507 [00:06<00:10, 26.80it/s]
Loading 0:  45%|████▌     | 230/507 [00:06<00:10, 26.38it/s]
Loading 0:  47%|████▋     | 237/507 [00:06<00:07, 33.81it/s]
Loading 0:  48%|████▊     | 241/507 [00:07<00:07, 33.62it/s]
Loading 0:  49%|████▊     | 246/507 [00:07<00:07, 36.43it/s]
Loading 0:  49%|████▉     | 250/507 [00:07<00:07, 35.50it/s]
Loading 0:  50%|█████     | 255/507 [00:07<00:06, 37.89it/s]
Loading 0:  51%|█████     | 259/507 [00:07<00:06, 36.58it/s]
Loading 0:  52%|█████▏    | 264/507 [00:07<00:06, 39.15it/s]
Loading 0:  53%|█████▎    | 268/507 [00:07<00:06, 37.08it/s]
Loading 0:  54%|█████▍    | 273/507 [00:07<00:05, 39.38it/s]
Loading 0:  55%|█████▍    | 277/507 [00:07<00:06, 37.29it/s]
Loading 0:  56%|█████▌    | 283/507 [00:08<00:05, 39.04it/s]
Loading 0:  57%|█████▋    | 287/507 [00:08<00:08, 25.73it/s]
Loading 0:  58%|█████▊    | 293/507 [00:08<00:07, 28.84it/s]
Loading 0:  59%|█████▉    | 299/507 [00:23<00:07, 28.84it/s]
Loading 0:  59%|█████▉    | 300/507 [00:23<02:45,  1.25it/s]
Loading 0:  60%|█████▉    | 302/507 [00:23<02:24,  1.42it/s]
Loading 0:  61%|██████    | 307/507 [00:23<01:37,  2.04it/s]
Loading 0:  61%|██████    | 310/507 [00:23<01:17,  2.54it/s]
Loading 0:  62%|██████▏   | 314/507 [00:23<00:55,  3.48it/s]
Loading 0:  63%|██████▎   | 319/507 [00:23<00:37,  5.07it/s]
Loading 0:  64%|██████▍   | 324/507 [00:23<00:25,  7.17it/s]
Loading 0:  65%|██████▍   | 328/507 [00:24<00:19,  9.16it/s]
Loading 0:  65%|██████▌   | 332/507 [00:24<00:15, 11.61it/s]
Loading 0:  66%|██████▋   | 337/507 [00:24<00:10, 15.47it/s]
Loading 0:  67%|██████▋   | 341/507 [00:24<00:10, 15.54it/s]
Loading 0:  68%|██████▊   | 345/507 [00:24<00:09, 17.39it/s]
Loading 0:  69%|██████▉   | 349/507 [00:24<00:07, 19.90it/s]
Loading 0:  70%|██████▉   | 354/507 [00:24<00:06, 24.63it/s]
Loading 0:  71%|███████   | 358/507 [00:25<00:05, 26.54it/s]
Loading 0:  72%|███████▏  | 363/507 [00:25<00:04, 30.92it/s]
Loading 0:  72%|███████▏  | 367/507 [00:25<00:04, 31.13it/s]
Loading 0:  73%|███████▎  | 372/507 [00:25<00:03, 35.08it/s]
Loading 0:  74%|███████▍  | 376/507 [00:25<00:03, 35.02it/s]
Loading 0:  75%|███████▌  | 381/507 [00:25<00:03, 38.43it/s]
Loading 0:  76%|███████▌  | 386/507 [00:25<00:03, 38.36it/s]
Loading 0:  77%|███████▋  | 391/507 [00:25<00:03, 33.21it/s]
Loading 0:  78%|███████▊  | 395/507 [00:26<00:04, 26.47it/s]
Loading 0:  79%|███████▉  | 401/507 [00:26<00:03, 28.32it/s]
Loading 0:  80%|████████  | 408/507 [00:26<00:02, 34.82it/s]
Loading 0:  81%|████████▏ | 412/507 [00:26<00:02, 34.25it/s]
Loading 0:  82%|████████▏ | 417/507 [00:26<00:02, 36.95it/s]
Loading 0:  83%|████████▎ | 421/507 [00:26<00:02, 35.92it/s]
Loading 0:  84%|████████▍ | 426/507 [00:26<00:02, 38.79it/s]
Loading 0:  85%|████████▌ | 431/507 [00:27<00:02, 37.63it/s]
Loading 0:  86%|████████▌ | 436/507 [00:27<00:01, 38.10it/s]
Loading 0:  87%|████████▋ | 440/507 [00:27<00:01, 37.79it/s]
Loading 0:  88%|████████▊ | 445/507 [00:27<00:01, 38.80it/s]
Loading 0:  89%|████████▉ | 450/507 [00:27<00:01, 40.90it/s]
Loading 0:  90%|████████▉ | 455/507 [00:29<00:08,  6.34it/s]
Loading 0:  91%|█████████ | 459/507 [00:30<00:06,  7.84it/s]
Loading 0:  92%|█████████▏| 465/507 [00:30<00:03, 10.88it/s]
Loading 0:  93%|█████████▎| 472/507 [00:30<00:02, 15.46it/s]
Loading 0:  94%|█████████▍| 476/507 [00:30<00:01, 17.46it/s]
Loading 0:  95%|█████████▍| 481/507 [00:30<00:01, 21.19it/s]
Loading 0:  96%|█████████▌| 485/507 [00:30<00:00, 23.17it/s]
Loading 0:  97%|█████████▋| 490/507 [00:30<00:00, 27.01it/s]
Loading 0:  97%|█████████▋| 494/507 [00:30<00:00, 27.85it/s]
Loading 0:  98%|█████████▊| 499/507 [00:31<00:00, 30.45it/s]
Loading 0:  99%|█████████▉| 503/507 [00:31<00:00, 30.07it/s]
                                                            

Job chaiml-ezstorytellingedi-9240-v3-mkmlizer completed after 166.7s with status: succeeded
Stopping job with name chaiml-ezstorytellingedi-9240-v3-mkmlizer
Pipeline stage MKMLizer completed in 167.16s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.14s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-ezstorytellingedi-9240-v3
Waiting for inference service chaiml-ezstorytellingedi-9240-v3 to be ready
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Inference service chaiml-ezstorytellingedi-9240-v3 ready after 160.73394083976746s
Pipeline stage MKMLDeployer completed in 161.23s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.782653570175171s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Received healthy response to inference request in 2.470339298248291s
Received healthy response to inference request in 2.4386324882507324s
Received healthy response to inference request in 2.510728120803833s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Received healthy response to inference request in 2.3940682411193848s
5 requests
0 failed requests
5th percentile: 2.402981090545654
10th percentile: 2.411893939971924
20th percentile: 2.429719638824463
30th percentile: 2.444973850250244
40th percentile: 2.4576565742492678
50th percentile: 2.470339298248291
60th percentile: 2.4864948272705076
70th percentile: 2.5026503562927247
80th percentile: 2.5651132106781005
90th percentile: 2.673883390426636
95th percentile: 2.7282684803009034
99th percentile: 2.7717765522003175
mean time: 2.5192843437194825
Pipeline stage StressChecker completed in 13.92s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.42s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 2.39s
Shutdown handler de-registered
chaiml-ezstorytellingedi_9240_v3 status is now deployed due to DeploymentManager action
Connection pool is full, discarding connection: %s. Connection pool size: %s
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage MKMLProfilerDeleter completed in 0.13s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.11s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service chaiml-ezstorytellingedi-9240-v3-profiler
Waiting for inference service chaiml-ezstorytellingedi-9240-v3-profiler to be ready
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3158.14s
Shutdown handler de-registered
chaiml-ezstorytellingedi_9240_v3 status is now inactive due to auto deactivation removed underperforming models