chaiml-small-story-885-e_9288

developer_uid: azuruce

submission_id: chaiml-small-story-885-e_9288_v1

model_name: v2-fix-sft-text-bug

model_group: ChaiML/small_story_885-e

status: inactive

timestamp: 2024-11-27T19:51:44+00:00

num_battles: 16076

num_wins: 7767

celo_rating: 1249.91

family_friendly_score: 0.5778

family_friendly_standard_error: 0.006984943235273999

submission_type: basic

model_repo: ChaiML/small_story_885-ed10-100_true_rpg_v2_sft

model_architecture: MistralForCausalLM

model_num_parameters: 22247282688.0

best_of: 4

max_input_tokens: 1024

max_output_tokens: 64

latencies: [{'batch_size': 1, 'throughput': 0.38424934285550966, 'latency_mean': 2.6024136650562286, 'latency_p50': 2.5978853702545166, 'latency_p90': 2.8805431604385374}, {'batch_size': 3, 'throughput': 0.8142432686569486, 'latency_mean': 3.662510825395584, 'latency_p50': 3.658523201942444, 'latency_p90': 4.042251873016357}, {'batch_size': 5, 'throughput': 1.080432886435887, 'latency_mean': 4.5909256792068485, 'latency_p50': 4.578116297721863, 'latency_p90': 5.145975065231323}, {'batch_size': 6, 'throughput': 1.1629144525366468, 'latency_mean': 5.117487841844559, 'latency_p50': 5.090061902999878, 'latency_p90': 5.785319924354553}, {'batch_size': 10, 'throughput': 1.3839399464013897, 'latency_mean': 7.16365217924118, 'latency_p50': 7.188313007354736, 'latency_p90': 8.065661072731018}]

gpu_counts: {'NVIDIA RTX A6000': 1}

display_name: v2-fix-sft-text-bug

is_internal_developer: True

language_model: ChaiML/small_story_885-ed10-100_true_rpg_v2_sft

model_size: 22B

ranking_group: single

throughput_3p7s: 0.83

us_pacific_date: 2024-11-27

win_ratio: 0.48314257277929834

generation_params: {'temperature': 0.9, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.5, 'frequency_penalty': 0.5, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 4, 'max_output_tokens': 64}

formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-small-story-885-e-9288-v1-mkmlizer
Waiting for job on chaiml-small-story-885-e-9288-v1-mkmlizer to finish
chaiml-small-story-885-e-9288-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-small-story-885-e-9288-v1-mkmlizer: ║     _____            __           __                                ║
chaiml-small-story-885-e-9288-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
chaiml-small-story-885-e-9288-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
chaiml-small-story-885-e-9288-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
chaiml-small-story-885-e-9288-v1-mkmlizer: ║       /___/                                                         ║
chaiml-small-story-885-e-9288-v1-mkmlizer: ║                                                                     ║
chaiml-small-story-885-e-9288-v1-mkmlizer: ║  Version: 0.11.12                                                   ║
chaiml-small-story-885-e-9288-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
chaiml-small-story-885-e-9288-v1-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-small-story-885-e-9288-v1-mkmlizer: ║                                                                     ║
chaiml-small-story-885-e-9288-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-small-story-885-e-9288-v1-mkmlizer: ║  belonging to:                                                      ║
chaiml-small-story-885-e-9288-v1-mkmlizer: ║                                                                     ║
chaiml-small-story-885-e-9288-v1-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-small-story-885-e-9288-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-small-story-885-e-9288-v1-mkmlizer: ║  Expiration: 2025-01-15 23:59:59                                    ║
chaiml-small-story-885-e-9288-v1-mkmlizer: ║                                                                     ║
chaiml-small-story-885-e-9288-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
chaiml-small-story-885-e-9288-v1-mkmlizer: Downloaded to shared memory in 91.317s
chaiml-small-story-885-e-9288-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp5bcfzcrx, device:0
chaiml-small-story-885-e-9288-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-small-story-885-e-9288-v1-mkmlizer: quantized model in 46.016s
chaiml-small-story-885-e-9288-v1-mkmlizer: Processed model ChaiML/small_story_885-ed10-100_true_rpg_v2_sft in 137.333s
chaiml-small-story-885-e-9288-v1-mkmlizer: creating bucket guanaco-mkml-models
chaiml-small-story-885-e-9288-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-small-story-885-e-9288-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-small-story-885-e-9288-v1
chaiml-small-story-885-e-9288-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-small-story-885-e-9288-v1/config.json
chaiml-small-story-885-e-9288-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-small-story-885-e-9288-v1/special_tokens_map.json
chaiml-small-story-885-e-9288-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-small-story-885-e-9288-v1/tokenizer_config.json
chaiml-small-story-885-e-9288-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-small-story-885-e-9288-v1/tokenizer.json
chaiml-small-story-885-e-9288-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/chaiml-small-story-885-e-9288-v1/flywheel_model.1.safetensors
chaiml-small-story-885-e-9288-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-small-story-885-e-9288-v1/flywheel_model.0.safetensors
chaiml-small-story-885-e-9288-v1-mkmlizer: 
Loading 0:   0%|          | 0/507 [00:00<?, ?it/s]
Loading 0:   1%|          | 5/507 [00:00<00:21, 22.98it/s]
Loading 0:   2%|▏         | 10/507 [00:00<00:14, 33.23it/s]
Loading 0:   3%|▎         | 14/507 [00:00<00:16, 29.04it/s]
Loading 0:   4%|▎         | 19/507 [00:00<00:14, 34.84it/s]
Loading 0:   5%|▍         | 23/507 [00:00<00:15, 30.31it/s]
Loading 0:   6%|▌         | 28/507 [00:00<00:13, 34.85it/s]
Loading 0:   6%|▋         | 32/507 [00:01<00:15, 30.80it/s]
Loading 0:   7%|▋         | 37/507 [00:01<00:13, 34.70it/s]
Loading 0:   8%|▊         | 41/507 [00:01<00:15, 30.46it/s]
Loading 0:   9%|▉         | 47/507 [00:01<00:13, 35.32it/s]
Loading 0:  10%|█         | 51/507 [00:01<00:13, 33.31it/s]
Loading 0:  11%|█         | 55/507 [00:01<00:20, 22.31it/s]
Loading 0:  11%|█▏        | 58/507 [00:02<00:19, 23.15it/s]
Loading 0:  12%|█▏        | 63/507 [00:02<00:16, 26.67it/s]
Loading 0:  13%|█▎        | 67/507 [00:02<00:16, 26.92it/s]
Loading 0:  14%|█▍        | 70/507 [00:02<00:16, 27.29it/s]
Loading 0:  15%|█▍        | 75/507 [00:02<00:13, 31.22it/s]
Loading 0:  16%|█▌        | 80/507 [00:02<00:13, 30.76it/s]
Loading 0:  17%|█▋        | 85/507 [00:02<00:12, 34.99it/s]
Loading 0:  18%|█▊        | 89/507 [00:02<00:13, 30.93it/s]
Loading 0:  19%|█▉        | 96/507 [00:03<00:11, 36.91it/s]
Loading 0:  20%|█▉        | 100/507 [00:03<00:11, 33.96it/s]
Loading 0:  21%|██        | 105/507 [00:03<00:11, 35.50it/s]
Loading 0:  21%|██▏       | 109/507 [00:03<00:12, 32.56it/s]
Loading 0:  22%|██▏       | 113/507 [00:03<00:16, 24.53it/s]
Loading 0:  23%|██▎       | 116/507 [00:03<00:17, 22.69it/s]
Loading 0:  24%|██▍       | 122/507 [00:04<00:15, 25.60it/s]
Loading 0:  25%|██▌       | 127/507 [00:04<00:12, 29.90it/s]
Loading 0:  26%|██▌       | 131/507 [00:04<00:13, 27.80it/s]
Loading 0:  27%|██▋       | 136/507 [00:04<00:11, 32.12it/s]
Loading 0:  28%|██▊       | 140/507 [00:04<00:12, 29.51it/s]
Loading 0:  29%|██▊       | 145/507 [00:04<00:10, 33.85it/s]
Loading 0:  29%|██▉       | 149/507 [00:04<00:11, 30.48it/s]
Loading 0:  30%|███       | 154/507 [00:05<00:10, 34.34it/s]
Loading 0:  31%|███       | 158/507 [00:05<00:11, 30.74it/s]
Loading 0:  32%|███▏      | 164/507 [00:05<00:09, 35.15it/s]
Loading 0:  33%|███▎      | 169/507 [00:05<00:12, 27.11it/s]
Loading 0:  34%|███▍      | 173/507 [00:05<00:12, 27.66it/s]
Loading 0:  35%|███▍      | 177/507 [00:05<00:12, 26.51it/s]
Loading 0:  36%|███▌      | 181/507 [00:06<00:11, 28.94it/s]
Loading 0:  36%|███▋      | 185/507 [00:06<00:11, 27.22it/s]
Loading 0:  37%|███▋      | 190/507 [00:06<00:10, 31.65it/s]
Loading 0:  38%|███▊      | 194/507 [00:06<00:10, 28.97it/s]
Loading 0:  39%|███▉      | 199/507 [00:06<00:09, 33.19it/s]
Loading 0:  40%|████      | 203/507 [00:06<00:10, 29.76it/s]
Loading 0:  41%|████      | 208/507 [00:06<00:08, 34.32it/s]
Loading 0:  42%|████▏     | 212/507 [00:07<00:09, 30.94it/s]
Loading 0:  43%|████▎     | 218/507 [00:07<00:08, 35.71it/s]
Loading 0:  44%|████▍     | 222/507 [00:07<00:08, 35.07it/s]
Loading 0:  45%|████▍     | 226/507 [00:07<00:10, 25.68it/s]
Loading 0:  45%|████▌     | 230/507 [00:07<00:10, 25.21it/s]
Loading 0:  46%|████▋     | 235/507 [00:07<00:09, 29.71it/s]
Loading 0:  47%|████▋     | 239/507 [00:08<00:09, 27.67it/s]
Loading 0:  48%|████▊     | 244/507 [00:08<00:08, 31.89it/s]
Loading 0:  49%|████▉     | 248/507 [00:08<00:08, 29.13it/s]
Loading 0:  50%|████▉     | 253/507 [00:08<00:07, 33.16it/s]
Loading 0:  51%|█████     | 257/507 [00:08<00:08, 29.66it/s]
Loading 0:  52%|█████▏    | 262/507 [00:08<00:07, 33.67it/s]
Loading 0:  52%|█████▏    | 266/507 [00:08<00:07, 30.30it/s]
Loading 0:  53%|█████▎    | 271/507 [00:08<00:07, 33.63it/s]
Loading 0:  54%|█████▍    | 275/507 [00:09<00:07, 30.59it/s]
Loading 0:  55%|█████▌    | 280/507 [00:09<00:06, 35.00it/s]
Loading 0:  56%|█████▌    | 284/507 [00:09<00:08, 26.71it/s]
Loading 0:  57%|█████▋    | 288/507 [00:09<00:08, 26.35it/s]
Loading 0:  58%|█████▊    | 293/507 [00:09<00:07, 27.48it/s]
Loading 0:  59%|█████▉    | 299/507 [00:20<00:07, 27.48it/s]
Loading 0:  59%|█████▉    | 300/507 [00:24<03:01,  1.14it/s]
Loading 0:  60%|█████▉    | 302/507 [00:24<02:37,  1.31it/s]
Loading 0:  61%|██████    | 307/507 [00:25<01:44,  1.91it/s]
Loading 0:  61%|██████    | 310/507 [00:25<01:22,  2.40it/s]
Loading 0:  62%|██████▏   | 313/507 [00:25<01:03,  3.07it/s]
Loading 0:  62%|██████▏   | 316/507 [00:25<00:48,  3.98it/s]
Loading 0:  63%|██████▎   | 320/507 [00:25<00:34,  5.46it/s]
Loading 0:  64%|██████▍   | 325/507 [00:25<00:22,  8.09it/s]
Loading 0:  65%|██████▍   | 329/507 [00:25<00:17, 10.07it/s]
Loading 0:  66%|██████▌   | 334/507 [00:26<00:12, 13.78it/s]
Loading 0:  67%|██████▋   | 339/507 [00:26<00:09, 17.46it/s]
Loading 0:  68%|██████▊   | 343/507 [00:26<00:10, 15.51it/s]
Loading 0:  68%|██████▊   | 347/507 [00:26<00:09, 16.49it/s]
Loading 0:  69%|██████▉   | 352/507 [00:26<00:07, 20.72it/s]
Loading 0:  70%|███████   | 356/507 [00:26<00:07, 21.29it/s]
Loading 0:  71%|███████   | 361/507 [00:27<00:05, 25.52it/s]
Loading 0:  72%|███████▏  | 365/507 [00:27<00:05, 24.39it/s]
Loading 0:  73%|███████▎  | 370/507 [00:27<00:04, 28.28it/s]
Loading 0:  74%|███████▍  | 374/507 [00:27<00:04, 26.67it/s]
Loading 0:  75%|███████▍  | 379/507 [00:27<00:04, 30.96it/s]
Loading 0:  76%|███████▌  | 383/507 [00:27<00:04, 28.56it/s]
Loading 0:  77%|███████▋  | 388/507 [00:27<00:03, 32.95it/s]
Loading 0:  77%|███████▋  | 392/507 [00:28<00:03, 31.23it/s]
Loading 0:  78%|███████▊  | 396/507 [00:28<00:04, 24.26it/s]
Loading 0:  79%|███████▉  | 401/507 [00:28<00:04, 26.01it/s]
Loading 0:  80%|████████  | 406/507 [00:28<00:03, 30.35it/s]
Loading 0:  81%|████████  | 410/507 [00:28<00:03, 28.10it/s]
Loading 0:  82%|████████▏ | 415/507 [00:28<00:02, 32.37it/s]
Loading 0:  83%|████████▎ | 419/507 [00:29<00:02, 29.51it/s]
Loading 0:  84%|████████▎ | 424/507 [00:29<00:02, 33.68it/s]
Loading 0:  84%|████████▍ | 428/507 [00:29<00:02, 30.09it/s]
Loading 0:  85%|████████▌ | 433/507 [00:29<00:02, 34.34it/s]
Loading 0:  86%|████████▌ | 437/507 [00:29<00:02, 31.29it/s]
Loading 0:  88%|████████▊ | 444/507 [00:29<00:01, 38.11it/s]
Loading 0:  89%|████████▊ | 449/507 [00:29<00:01, 37.86it/s]
Loading 0:  90%|████████▉ | 454/507 [00:30<00:01, 36.24it/s]
Loading 0:  90%|█████████ | 458/507 [00:32<00:07,  6.18it/s]
Loading 0:  91%|█████████ | 461/507 [00:32<00:06,  7.32it/s]
Loading 0:  92%|█████████▏| 465/507 [00:32<00:04,  9.18it/s]
Loading 0:  93%|█████████▎| 470/507 [00:32<00:02, 12.67it/s]
Loading 0:  93%|█████████▎| 474/507 [00:32<00:02, 14.65it/s]
Loading 0:  94%|█████████▍| 479/507 [00:32<00:01, 19.10it/s]
Loading 0:  95%|█████████▌| 483/507 [00:33<00:01, 20.76it/s]
Loading 0:  96%|█████████▋| 488/507 [00:33<00:00, 25.63it/s]
Loading 0:  97%|█████████▋| 492/507 [00:33<00:00, 25.03it/s]
Loading 0:  98%|█████████▊| 499/507 [00:33<00:00, 32.30it/s]
Loading 0:  99%|█████████▉| 504/507 [00:33<00:00, 33.17it/s]
                                                            
Job chaiml-small-story-885-e-9288-v1-mkmlizer completed after 166.64s with status: succeeded
Stopping job with name chaiml-small-story-885-e-9288-v1-mkmlizer
Pipeline stage MKMLizer completed in 167.14s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.16s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-small-story-885-e-9288-v1
Waiting for inference service chaiml-small-story-885-e-9288-v1 to be ready
Inference service chaiml-small-story-885-e-9288-v1 ready after 130.48579692840576s
Pipeline stage MKMLDeployer completed in 131.01s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.1842265129089355s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Received healthy response to inference request in 2.2840945720672607s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Received healthy response to inference request in 2.278769016265869s
Received healthy response to inference request in 2.4070727825164795s
Received healthy response to inference request in 2.8830206394195557s
5 requests
0 failed requests
5th percentile: 2.2798341274261475
10th percentile: 2.280899238586426
20th percentile: 2.2830294609069823
30th percentile: 2.3086902141571044
40th percentile: 2.357881498336792
50th percentile: 2.4070727825164795
60th percentile: 2.59745192527771
70th percentile: 2.7878310680389404
80th percentile: 2.943261814117432
90th percentile: 3.0637441635131837
95th percentile: 3.1239853382110594
99th percentile: 3.17217827796936
mean time: 2.60743670463562
Pipeline stage StressChecker completed in 14.54s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.14s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 1.97s
Shutdown handler de-registered
chaiml-small-story-885-e_9288_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage MKMLProfilerDeleter completed in 0.13s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.12s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service chaiml-small-story-885-e-9288-v1-profiler
Waiting for inference service chaiml-small-story-885-e-9288-v1-profiler to be ready
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3493.68s
Shutdown handler de-registered
chaiml-small-story-885-e_9288_v1 status is now inactive due to auto deactivation removed underperforming models