chaiml-small-story-607-e_8056

developer_uid: azuruce

submission_id: chaiml-small-story-607-e_8056_v1

model_name: 3epoch

model_group: ChaiML/small_story_607-e

status: inactive

timestamp: 2024-11-27T20:53:38+00:00

num_battles: 13204

num_wins: 6505

celo_rating: 1256.03

family_friendly_score: 0.5796

family_friendly_standard_error: 0.006980885903665809

submission_type: basic

model_repo: ChaiML/small_story_607-ed10-100_true_rpg_v3_sft

model_architecture: MistralForCausalLM

model_num_parameters: 22247282688.0

best_of: 4

max_input_tokens: 1024

max_output_tokens: 64

latencies: [{'batch_size': 1, 'throughput': 0.38495785133151955, 'latency_mean': 2.5976056706905366, 'latency_p50': 2.6047037839889526, 'latency_p90': 2.8578827142715455}, {'batch_size': 3, 'throughput': 0.827776778705465, 'latency_mean': 3.610703959465027, 'latency_p50': 3.6374359130859375, 'latency_p90': 3.9694725275039673}, {'batch_size': 5, 'throughput': 1.0986881253772038, 'latency_mean': 4.530270889997483, 'latency_p50': 4.550591826438904, 'latency_p90': 5.056288838386536}, {'batch_size': 6, 'throughput': 1.1793694753135215, 'latency_mean': 5.060452281236649, 'latency_p50': 5.042963147163391, 'latency_p90': 5.621088814735413}, {'batch_size': 10, 'throughput': 1.40694490360138, 'latency_mean': 7.030594631433487, 'latency_p50': 6.969742298126221, 'latency_p90': 8.024596762657165}]

gpu_counts: {'NVIDIA RTX A6000': 1}

display_name: 3epoch

is_internal_developer: True

language_model: ChaiML/small_story_607-ed10-100_true_rpg_v3_sft

model_size: 22B

ranking_group: single

throughput_3p7s: 0.86

us_pacific_date: 2024-11-27

win_ratio: 0.49265374129051803

generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '</s>', '####', 'Bot:', 'User:', 'You:', '<|im_end|>', '<|eot_id|>'], 'max_input_tokens': 1024, 'best_of': 4, 'max_output_tokens': 64}

formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-small-story-607-e-8056-v1-mkmlizer
Waiting for job on chaiml-small-story-607-e-8056-v1-mkmlizer to finish
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
chaiml-small-story-607-e-8056-v1-mkmlizer: Downloaded to shared memory in 86.783s
chaiml-small-story-607-e-8056-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpjp22hj85, device:0
chaiml-small-story-607-e-8056-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-small-story-607-e-8056-v1-mkmlizer: creating bucket guanaco-mkml-models
chaiml-small-story-607-e-8056-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-small-story-607-e-8056-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-small-story-607-e-8056-v1
chaiml-small-story-607-e-8056-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-small-story-607-e-8056-v1/special_tokens_map.json
chaiml-small-story-607-e-8056-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-small-story-607-e-8056-v1/config.json
chaiml-small-story-607-e-8056-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-small-story-607-e-8056-v1/tokenizer_config.json
chaiml-small-story-607-e-8056-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-small-story-607-e-8056-v1/tokenizer.json
chaiml-small-story-607-e-8056-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/chaiml-small-story-607-e-8056-v1/flywheel_model.1.safetensors
chaiml-small-story-607-e-8056-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-small-story-607-e-8056-v1/flywheel_model.0.safetensors
chaiml-small-story-607-e-8056-v1-mkmlizer: 
Loading 0:   0%|          | 0/507 [00:00<?, ?it/s]
Loading 0:   1%|          | 4/507 [00:00<00:12, 39.08it/s]
Loading 0:   2%|▏         | 8/507 [00:00<00:18, 27.33it/s]
Loading 0:   2%|▏         | 12/507 [00:00<00:16, 30.35it/s]
Loading 0:   3%|▎         | 16/507 [00:00<00:16, 29.37it/s]
Loading 0:   4%|▍         | 21/507 [00:00<00:14, 32.70it/s]
Loading 0:   5%|▍         | 25/507 [00:00<00:15, 30.87it/s]
Loading 0:   6%|▌         | 29/507 [00:00<00:14, 32.28it/s]
Loading 0:   7%|▋         | 33/507 [00:01<00:17, 26.81it/s]
Loading 0:   7%|▋         | 37/507 [00:01<00:16, 28.09it/s]
Loading 0:   8%|▊         | 41/507 [00:01<00:17, 26.04it/s]
Loading 0:   9%|▉         | 46/507 [00:01<00:15, 30.35it/s]
Loading 0:  10%|▉         | 50/507 [00:01<00:16, 28.16it/s]
Loading 0:  10%|█         | 53/507 [00:02<00:24, 18.47it/s]
Loading 0:  11%|█         | 56/507 [00:02<00:25, 17.77it/s]
Loading 0:  12%|█▏        | 61/507 [00:02<00:19, 22.52it/s]
Loading 0:  13%|█▎        | 65/507 [00:02<00:19, 22.17it/s]
Loading 0:  14%|█▍        | 70/507 [00:02<00:16, 26.31it/s]
Loading 0:  14%|█▍        | 73/507 [00:02<00:16, 27.01it/s]
Loading 0:  16%|█▌        | 80/507 [00:02<00:14, 29.31it/s]
Loading 0:  17%|█▋        | 85/507 [00:03<00:13, 31.98it/s]
Loading 0:  18%|█▊        | 89/507 [00:03<00:14, 28.55it/s]
Loading 0:  19%|█▊        | 94/507 [00:03<00:13, 31.50it/s]
Loading 0:  19%|█▉        | 98/507 [00:03<00:14, 27.70it/s]
Loading 0:  20%|██        | 103/507 [00:03<00:13, 30.37it/s]
Loading 0:  21%|██        | 107/507 [00:03<00:14, 27.13it/s]
Loading 0:  22%|██▏       | 112/507 [00:04<00:13, 29.94it/s]
Loading 0:  23%|██▎       | 116/507 [00:04<00:20, 19.26it/s]
Loading 0:  24%|██▍       | 122/507 [00:04<00:17, 22.28it/s]
Loading 0:  25%|██▌       | 127/507 [00:04<00:14, 26.71it/s]
Loading 0:  26%|██▌       | 131/507 [00:04<00:14, 26.41it/s]
Loading 0:  27%|██▋       | 138/507 [00:05<00:10, 33.84it/s]
Loading 0:  28%|██▊       | 142/507 [00:05<00:10, 33.29it/s]
Loading 0:  29%|██▉       | 147/507 [00:05<00:09, 36.47it/s]
Loading 0:  30%|██▉       | 152/507 [00:05<00:10, 35.22it/s]
Loading 0:  31%|███       | 157/507 [00:05<00:09, 36.37it/s]
Loading 0:  32%|███▏      | 162/507 [00:05<00:08, 38.78it/s]
Loading 0:  33%|███▎      | 167/507 [00:05<00:08, 40.92it/s]
Loading 0:  34%|███▍      | 172/507 [00:06<00:13, 25.20it/s]
Loading 0:  35%|███▍      | 176/507 [00:06<00:13, 25.37it/s]
Loading 0:  36%|███▌      | 181/507 [00:06<00:10, 29.64it/s]
Loading 0:  36%|███▋      | 185/507 [00:06<00:11, 28.37it/s]
Loading 0:  38%|███▊      | 192/507 [00:06<00:08, 35.29it/s]
Loading 0:  39%|███▊      | 196/507 [00:06<00:09, 33.88it/s]
Loading 0:  40%|███▉      | 201/507 [00:06<00:08, 36.23it/s]
Loading 0:  40%|████      | 205/507 [00:07<00:08, 35.04it/s]
Loading 0:  41%|████▏     | 210/507 [00:07<00:08, 36.89it/s]
Loading 0:  42%|████▏     | 214/507 [00:07<00:08, 35.49it/s]
Loading 0:  43%|████▎     | 218/507 [00:07<00:08, 35.03it/s]
Loading 0:  44%|████▍     | 222/507 [00:07<00:08, 34.40it/s]
Loading 0:  45%|████▍     | 226/507 [00:07<00:11, 25.03it/s]
Loading 0:  45%|████▌     | 230/507 [00:07<00:11, 24.60it/s]
Loading 0:  46%|████▋     | 235/507 [00:08<00:09, 29.51it/s]
Loading 0:  47%|████▋     | 239/507 [00:08<00:09, 27.30it/s]
Loading 0:  49%|████▊     | 246/507 [00:08<00:07, 33.89it/s]
Loading 0:  49%|████▉     | 250/507 [00:08<00:08, 32.10it/s]
Loading 0:  50%|█████     | 255/507 [00:08<00:07, 33.56it/s]
Loading 0:  51%|█████     | 259/507 [00:08<00:07, 31.73it/s]
Loading 0:  52%|█████▏    | 264/507 [00:08<00:07, 33.56it/s]
Loading 0:  53%|█████▎    | 268/507 [00:09<00:07, 31.98it/s]
Loading 0:  54%|█████▍    | 273/507 [00:09<00:06, 34.11it/s]
Loading 0:  55%|█████▍    | 277/507 [00:09<00:07, 32.44it/s]
Loading 0:  56%|█████▌    | 282/507 [00:09<00:06, 35.78it/s]
Loading 0:  56%|█████▋    | 286/507 [00:09<00:09, 23.97it/s]
Loading 0:  57%|█████▋    | 289/507 [00:09<00:09, 23.91it/s]
Loading 0:  58%|█████▊    | 293/507 [00:10<00:09, 23.67it/s]
Loading 0:  59%|█████▉    | 298/507 [00:10<00:07, 28.99it/s]
Loading 0:  59%|█████▉    | 299/507 [00:24<00:07, 28.99it/s]
Loading 0:  59%|█████▉    | 300/507 [00:24<04:17,  1.24s/it]
Loading 0:  60%|█████▉    | 302/507 [00:24<03:28,  1.02s/it]
Loading 0:  61%|██████    | 307/507 [00:25<02:03,  1.62it/s]
Loading 0:  61%|██████    | 310/507 [00:25<01:32,  2.12it/s]
Loading 0:  62%|██████▏   | 314/507 [00:25<01:02,  3.08it/s]
Loading 0:  63%|██████▎   | 319/507 [00:25<00:40,  4.68it/s]
Loading 0:  64%|██████▎   | 323/507 [00:25<00:29,  6.33it/s]
Loading 0:  65%|██████▍   | 328/507 [00:25<00:19,  8.95it/s]
Loading 0:  65%|██████▌   | 332/507 [00:25<00:15, 11.40it/s]
Loading 0:  66%|██████▋   | 337/507 [00:25<00:11, 15.31it/s]
Loading 0:  67%|██████▋   | 341/507 [00:26<00:10, 15.29it/s]
Loading 0:  68%|██████▊   | 345/507 [00:26<00:09, 17.73it/s]
Loading 0:  69%|██████▉   | 349/507 [00:26<00:07, 20.47it/s]
Loading 0:  70%|██████▉   | 354/507 [00:26<00:06, 25.16it/s]
Loading 0:  71%|███████   | 358/507 [00:26<00:05, 26.70it/s]
Loading 0:  72%|███████▏  | 363/507 [00:26<00:04, 30.74it/s]
Loading 0:  72%|███████▏  | 367/507 [00:26<00:04, 30.75it/s]
Loading 0:  73%|███████▎  | 372/507 [00:27<00:03, 34.20it/s]
Loading 0:  74%|███████▍  | 376/507 [00:27<00:03, 33.12it/s]
Loading 0:  75%|███████▌  | 381/507 [00:27<00:03, 36.36it/s]
Loading 0:  76%|███████▌  | 385/507 [00:27<00:03, 35.16it/s]
Loading 0:  77%|███████▋  | 389/507 [00:27<00:03, 33.87it/s]
Loading 0:  78%|███████▊  | 393/507 [00:27<00:03, 33.03it/s]
Loading 0:  78%|███████▊  | 397/507 [00:27<00:04, 24.53it/s]
Loading 0:  79%|███████▉  | 401/507 [00:28<00:04, 24.57it/s]
Loading 0:  80%|████████  | 408/507 [00:28<00:03, 31.78it/s]
Loading 0:  81%|████████▏ | 412/507 [00:28<00:02, 31.68it/s]
Loading 0:  82%|████████▏ | 417/507 [00:28<00:02, 34.38it/s]
Loading 0:  83%|████████▎ | 421/507 [00:28<00:02, 33.62it/s]
Loading 0:  84%|████████▍ | 426/507 [00:28<00:02, 36.60it/s]
Loading 0:  85%|████████▍ | 430/507 [00:28<00:02, 35.69it/s]
Loading 0:  86%|████████▌ | 435/507 [00:28<00:01, 38.35it/s]
Loading 0:  87%|████████▋ | 439/507 [00:29<00:01, 37.32it/s]
Loading 0:  88%|████████▊ | 444/507 [00:29<00:01, 40.11it/s]
Loading 0:  89%|████████▊ | 449/507 [00:29<00:01, 40.51it/s]
Loading 0:  90%|████████▉ | 454/507 [00:29<00:01, 39.92it/s]
Loading 0:  91%|█████████ | 459/507 [00:31<00:07,  6.23it/s]
Loading 0:  92%|█████████▏| 465/507 [00:31<00:04,  8.68it/s]
Loading 0:  93%|█████████▎| 472/507 [00:32<00:02, 12.46it/s]
Loading 0:  94%|█████████▍| 476/507 [00:32<00:02, 14.27it/s]
Loading 0:  95%|█████████▍| 481/507 [00:32<00:01, 17.48it/s]
Loading 0:  96%|█████████▌| 485/507 [00:32<00:01, 19.31it/s]
Loading 0:  97%|█████████▋| 490/507 [00:32<00:00, 22.82it/s]
Loading 0:  97%|█████████▋| 494/507 [00:32<00:00, 24.09it/s]
Loading 0:  98%|█████████▊| 499/507 [00:32<00:00, 27.41it/s]
Loading 0:  99%|█████████▉| 503/507 [00:33<00:00, 27.81it/s]
                                                            
Job chaiml-small-story-607-e-8056-v1-mkmlizer completed after 165.35s with status: succeeded
Stopping job with name chaiml-small-story-607-e-8056-v1-mkmlizer
Pipeline stage MKMLizer completed in 165.87s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.18s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-small-story-607-e-8056-v1
Waiting for inference service chaiml-small-story-607-e-8056-v1 to be ready
Failed to get response for submission function_pifab_2024-11-27: max() arg is an empty sequence
Connection pool is full, discarding connection: %s. Connection pool size: %s
Inference service chaiml-small-story-607-e-8056-v1 ready after 120.44624519348145s
Pipeline stage MKMLDeployer completed in 121.01s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.013676881790161s
Received healthy response to inference request in 2.7142868041992188s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Received healthy response to inference request in 2.2320518493652344s
Received healthy response to inference request in 2.3927597999572754s
Received healthy response to inference request in 2.8576443195343018s
5 requests
0 failed requests
5th percentile: 2.2641934394836425
10th percentile: 2.2963350296020506
20th percentile: 2.3606182098388673
30th percentile: 2.457065200805664
40th percentile: 2.5856760025024412
50th percentile: 2.7142868041992188
60th percentile: 2.771629810333252
70th percentile: 2.828972816467285
80th percentile: 2.8888508319854735
90th percentile: 2.9512638568878176
95th percentile: 2.9824703693389893
99th percentile: 3.0074355792999268
mean time: 2.642083930969238
Pipeline stage StressChecker completed in 14.76s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.58s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 2.71s
Shutdown handler de-registered
chaiml-small-story-607-e_8056_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3548.85s
Shutdown handler de-registered
chaiml-small-story-607-e_8056_v1 status is now inactive due to auto deactivation removed underperforming models