chaiml-small-story-607-_8056

developer_uid: azuruce

submission_id: chaiml-small-story-607-_8056_v11

model_name: chaiml-small-story-607-_8056_v11

model_group: ChaiML/small_story_607-e

status: inactive

timestamp: 2024-11-27T22:08:22+00:00

num_battles: 10679

num_wins: 5090

celo_rating: 1241.93

family_friendly_score: 0.5851999999999999

family_friendly_standard_error: 0.006967653263474008

submission_type: basic

model_repo: ChaiML/small_story_607-ed10-100_true_rpg_v3_sft

model_architecture: MistralForCausalLM

model_num_parameters: 22247282688.0

best_of: 4

max_input_tokens: 1024

max_output_tokens: 64

latencies: [{'batch_size': 1, 'throughput': 0.3831823956723429, 'latency_mean': 2.609660427570343, 'latency_p50': 2.6135021448135376, 'latency_p90': 2.8756196975708006}, {'batch_size': 3, 'throughput': 0.8141043096495885, 'latency_mean': 3.6656322157382966, 'latency_p50': 3.671618938446045, 'latency_p90': 4.022702479362488}, {'batch_size': 5, 'throughput': 1.0764237124956644, 'latency_mean': 4.611770391464233, 'latency_p50': 4.59838593006134, 'latency_p90': 5.161775064468384}, {'batch_size': 6, 'throughput': 1.1547970058884156, 'latency_mean': 5.150208444595337, 'latency_p50': 5.169514179229736, 'latency_p90': 5.802527904510498}, {'batch_size': 10, 'throughput': 1.3782285932273022, 'latency_mean': 7.177010060548782, 'latency_p50': 7.194514632225037, 'latency_p90': 8.242707586288452}]

gpu_counts: {'NVIDIA RTX A6000': 1}

display_name: chaiml-small-story-607-_8056_v11

is_internal_developer: True

language_model: ChaiML/small_story_607-ed10-100_true_rpg_v3_sft

model_size: 22B

ranking_group: single

throughput_3p7s: 0.83

us_pacific_date: 2024-11-27

win_ratio: 0.4766363891750164

generation_params: {'temperature': 0.8, 'top_p': 1.0, 'min_p': 0.1, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.1, 'stopping_words': ['\n', '</s>', '####', 'Bot:', 'User:', 'You:', '<|im_end|>', '<|eot_id|>'], 'max_input_tokens': 1024, 'best_of': 4, 'max_output_tokens': 64}

formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-small-story-607-8056-v11-mkmlizer
Waiting for job on chaiml-small-story-607-8056-v11-mkmlizer to finish
chaiml-small-story-607-8056-v11-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-small-story-607-8056-v11-mkmlizer: ║     _____            __           __                                ║
chaiml-small-story-607-8056-v11-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
chaiml-small-story-607-8056-v11-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
chaiml-small-story-607-8056-v11-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
chaiml-small-story-607-8056-v11-mkmlizer: ║       /___/                                                         ║
chaiml-small-story-607-8056-v11-mkmlizer: ║                                                                     ║
chaiml-small-story-607-8056-v11-mkmlizer: ║  Version: 0.11.12                                                   ║
chaiml-small-story-607-8056-v11-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
chaiml-small-story-607-8056-v11-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-small-story-607-8056-v11-mkmlizer: ║                                                                     ║
chaiml-small-story-607-8056-v11-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-small-story-607-8056-v11-mkmlizer: ║  belonging to:                                                      ║
chaiml-small-story-607-8056-v11-mkmlizer: ║                                                                     ║
chaiml-small-story-607-8056-v11-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-small-story-607-8056-v11-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-small-story-607-8056-v11-mkmlizer: ║  Expiration: 2025-01-15 23:59:59                                    ║
chaiml-small-story-607-8056-v11-mkmlizer: ║                                                                     ║
chaiml-small-story-607-8056-v11-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Failed to get response for submission function_doteb_2024-11-27: no entry with id "fake_submission_ifd_for_testing" found on database!
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Failed to get response for submission function_nupus_2024-11-27: no entry with id "fake_submission_id_for_testing" found on database!
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
chaiml-small-story-607-8056-v11-mkmlizer: Downloaded to shared memory in 54.997s
chaiml-small-story-607-8056-v11-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpscvhgxcw, device:0
chaiml-small-story-607-8056-v11-mkmlizer: Saving flywheel model at /dev/shm/model_cache
Failed to get response for submission function_sosuf_2024-11-27: no entry with id "fake_submission_ifd_for_testing" found on database!
chaiml-small-story-607-8056-v11-mkmlizer: quantized model in 46.110s
chaiml-small-story-607-8056-v11-mkmlizer: Processed model ChaiML/small_story_607-ed10-100_true_rpg_v3_sft in 101.108s
chaiml-small-story-607-8056-v11-mkmlizer: creating bucket guanaco-mkml-models
chaiml-small-story-607-8056-v11-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-small-story-607-8056-v11-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-small-story-607-8056-v11
chaiml-small-story-607-8056-v11-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-small-story-607-8056-v11/special_tokens_map.json
chaiml-small-story-607-8056-v11-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-small-story-607-8056-v11/config.json
chaiml-small-story-607-8056-v11-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-small-story-607-8056-v11/tokenizer_config.json
chaiml-small-story-607-8056-v11-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-small-story-607-8056-v11/tokenizer.json
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
chaiml-small-story-607-8056-v11-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/chaiml-small-story-607-8056-v11/flywheel_model.1.safetensors
chaiml-small-story-607-8056-v11-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-small-story-607-8056-v11/flywheel_model.0.safetensors
chaiml-small-story-607-8056-v11-mkmlizer: 
Loading 0:   0%|          | 0/507 [00:00<?, ?it/s]
Loading 0:   1%|          | 5/507 [00:00<00:24, 20.48it/s]
Loading 0:   2%|▏         | 10/507 [00:00<00:17, 29.05it/s]
Loading 0:   3%|▎         | 14/507 [00:00<00:19, 25.27it/s]
Loading 0:   4%|▎         | 19/507 [00:00<00:16, 30.00it/s]
Loading 0:   5%|▍         | 23/507 [00:00<00:17, 26.94it/s]
Loading 0:   6%|▌         | 30/507 [00:00<00:13, 34.49it/s]
Loading 0:   7%|▋         | 34/507 [00:01<00:14, 33.44it/s]
Loading 0:   8%|▊         | 39/507 [00:01<00:13, 35.95it/s]
Loading 0:   8%|▊         | 43/507 [00:01<00:14, 33.03it/s]
Loading 0:   9%|▉         | 47/507 [00:01<00:15, 30.36it/s]
Loading 0:  10%|█         | 51/507 [00:01<00:16, 27.26it/s]
Loading 0:  11%|█         | 54/507 [00:02<00:23, 19.44it/s]
Loading 0:  11%|█         | 57/507 [00:02<00:22, 19.90it/s]
Loading 0:  12%|█▏        | 61/507 [00:02<00:19, 22.89it/s]
Loading 0:  13%|█▎        | 65/507 [00:02<00:21, 20.19it/s]
Loading 0:  14%|█▍        | 70/507 [00:02<00:18, 23.85it/s]
Loading 0:  14%|█▍        | 73/507 [00:02<00:17, 24.15it/s]
Loading 0:  16%|█▌        | 79/507 [00:02<00:13, 30.71it/s]
Loading 0:  16%|█▋        | 83/507 [00:03<00:14, 28.53it/s]
Loading 0:  17%|█▋        | 87/507 [00:03<00:14, 28.66it/s]
Loading 0:  18%|█▊        | 91/507 [00:03<00:14, 28.48it/s]
Loading 0:  19%|█▊        | 94/507 [00:03<00:15, 27.52it/s]
Loading 0:  19%|█▉        | 98/507 [00:03<00:16, 24.62it/s]
Loading 0:  20%|██        | 103/507 [00:03<00:13, 29.60it/s]
Loading 0:  21%|██        | 107/507 [00:03<00:14, 27.82it/s]
Loading 0:  22%|██▏       | 112/507 [00:04<00:12, 31.51it/s]
Loading 0:  23%|██▎       | 116/507 [00:04<00:18, 20.59it/s]
Loading 0:  24%|██▍       | 122/507 [00:04<00:16, 22.87it/s]
Loading 0:  25%|██▌       | 127/507 [00:04<00:14, 26.65it/s]
Loading 0:  26%|██▌       | 131/507 [00:04<00:14, 25.13it/s]
Loading 0:  27%|██▋       | 136/507 [00:05<00:12, 28.62it/s]
Loading 0:  28%|██▊       | 140/507 [00:05<00:13, 26.91it/s]
Loading 0:  29%|██▊       | 145/507 [00:05<00:11, 30.65it/s]
Loading 0:  29%|██▉       | 149/507 [00:05<00:12, 27.71it/s]
Loading 0:  30%|███       | 154/507 [00:05<00:11, 32.07it/s]
Loading 0:  31%|███       | 158/507 [00:05<00:11, 29.70it/s]
Loading 0:  32%|███▏      | 164/507 [00:05<00:09, 34.60it/s]
Loading 0:  33%|███▎      | 168/507 [00:06<00:09, 35.37it/s]
Loading 0:  34%|███▍      | 172/507 [00:06<00:13, 24.22it/s]
Loading 0:  35%|███▍      | 176/507 [00:06<00:13, 24.00it/s]
Loading 0:  36%|███▌      | 181/507 [00:06<00:11, 28.37it/s]
Loading 0:  36%|███▋      | 185/507 [00:06<00:12, 26.62it/s]
Loading 0:  37%|███▋      | 190/507 [00:06<00:10, 30.64it/s]
Loading 0:  38%|███▊      | 194/507 [00:07<00:11, 28.31it/s]
Loading 0:  40%|███▉      | 201/507 [00:07<00:08, 35.01it/s]
Loading 0:  40%|████      | 205/507 [00:07<00:09, 32.72it/s]
Loading 0:  41%|████      | 209/507 [00:07<00:08, 33.39it/s]
Loading 0:  42%|████▏     | 213/507 [00:07<00:10, 28.62it/s]
Loading 0:  43%|████▎     | 217/507 [00:07<00:09, 30.13it/s]
Loading 0:  44%|████▎     | 221/507 [00:07<00:09, 29.34it/s]
Loading 0:  44%|████▍     | 225/507 [00:08<00:12, 23.27it/s]
Loading 0:  45%|████▌     | 230/507 [00:08<00:11, 25.01it/s]
Loading 0:  46%|████▋     | 235/507 [00:08<00:09, 29.74it/s]
Loading 0:  47%|████▋     | 239/507 [00:08<00:10, 26.71it/s]
Loading 0:  48%|████▊     | 244/507 [00:08<00:08, 30.05it/s]
Loading 0:  49%|████▉     | 248/507 [00:08<00:09, 27.42it/s]
Loading 0:  50%|█████     | 255/507 [00:09<00:07, 34.36it/s]
Loading 0:  51%|█████     | 259/507 [00:09<00:07, 33.02it/s]
Loading 0:  52%|█████▏    | 263/507 [00:09<00:07, 33.70it/s]
Loading 0:  53%|█████▎    | 267/507 [00:09<00:08, 29.52it/s]
Loading 0:  53%|█████▎    | 271/507 [00:09<00:07, 31.04it/s]
Loading 0:  54%|█████▍    | 275/507 [00:09<00:07, 29.04it/s]
Loading 0:  56%|█████▌    | 282/507 [00:09<00:05, 37.50it/s]
Loading 0:  57%|█████▋    | 287/507 [00:10<00:09, 23.08it/s]
Loading 0:  58%|█████▊    | 293/507 [00:10<00:08, 25.05it/s]
Loading 0:  59%|█████▉    | 298/507 [00:10<00:07, 28.83it/s]
Loading 0:  59%|█████▉    | 299/507 [00:25<00:07, 28.83it/s]
Loading 0:  59%|█████▉    | 300/507 [00:25<03:46,  1.10s/it]
Loading 0:  60%|█████▉    | 302/507 [00:25<03:09,  1.08it/s]
Loading 0:  61%|██████    | 307/507 [00:26<01:58,  1.69it/s]
Loading 0:  61%|██████    | 310/507 [00:26<01:30,  2.18it/s]
Loading 0:  62%|██████▏   | 313/507 [00:26<01:07,  2.86it/s]
Loading 0:  63%|██████▎   | 318/507 [00:26<00:42,  4.44it/s]
Loading 0:  64%|██████▎   | 322/507 [00:26<00:30,  6.00it/s]
Loading 0:  64%|██████▍   | 327/507 [00:26<00:20,  8.66it/s]
Loading 0:  65%|██████▌   | 331/507 [00:26<00:16, 10.90it/s]
Loading 0:  66%|██████▌   | 335/507 [00:26<00:12, 13.76it/s]
Loading 0:  67%|██████▋   | 340/507 [00:27<00:10, 15.26it/s]
Loading 0:  68%|██████▊   | 344/507 [00:27<00:09, 17.84it/s]
Loading 0:  69%|██████▊   | 348/507 [00:27<00:08, 19.50it/s]
Loading 0:  70%|██████▉   | 354/507 [00:27<00:05, 25.70it/s]
Loading 0:  71%|███████   | 358/507 [00:27<00:05, 27.34it/s]
Loading 0:  72%|███████▏  | 363/507 [00:27<00:04, 31.73it/s]
Loading 0:  72%|███████▏  | 367/507 [00:27<00:04, 32.13it/s]
Loading 0:  73%|███████▎  | 372/507 [00:28<00:03, 36.07it/s]
Loading 0:  74%|███████▍  | 377/507 [00:28<00:03, 37.40it/s]
Loading 0:  75%|███████▌  | 382/507 [00:28<00:03, 38.72it/s]
Loading 0:  76%|███████▋  | 387/507 [00:28<00:02, 40.96it/s]
Loading 0:  77%|███████▋  | 392/507 [00:28<00:03, 35.06it/s]
Loading 0:  78%|███████▊  | 396/507 [00:28<00:04, 26.36it/s]
Loading 0:  79%|███████▉  | 401/507 [00:28<00:03, 27.13it/s]
Loading 0:  80%|████████  | 408/507 [00:29<00:02, 33.66it/s]
Loading 0:  81%|████████▏ | 412/507 [00:29<00:02, 33.00it/s]
Loading 0:  82%|████████▏ | 417/507 [00:29<00:02, 35.56it/s]
Loading 0:  83%|████████▎ | 421/507 [00:29<00:02, 34.03it/s]
Loading 0:  84%|████████▍ | 426/507 [00:29<00:02, 35.73it/s]
Loading 0:  85%|████████▍ | 430/507 [00:29<00:02, 34.34it/s]
Loading 0:  86%|████████▌ | 435/507 [00:29<00:01, 37.17it/s]
Loading 0:  87%|████████▋ | 439/507 [00:29<00:01, 36.72it/s]
Loading 0:  88%|████████▊ | 444/507 [00:30<00:01, 40.03it/s]
Loading 0:  89%|████████▊ | 449/507 [00:30<00:01, 40.23it/s]
Loading 0:  90%|████████▉ | 454/507 [00:30<00:01, 38.86it/s]
Loading 0:  90%|█████████ | 458/507 [00:32<00:08,  5.94it/s]
Loading 0:  91%|█████████ | 461/507 [00:32<00:06,  7.16it/s]
Loading 0:  92%|█████████▏| 465/507 [00:32<00:04,  9.15it/s]
Loading 0:  93%|█████████▎| 470/507 [00:32<00:02, 12.68it/s]
Loading 0:  93%|█████████▎| 474/507 [00:33<00:02, 14.71it/s]
Loading 0:  95%|█████████▍| 481/507 [00:33<00:01, 21.33it/s]
Loading 0:  96%|█████████▌| 485/507 [00:33<00:00, 23.14it/s]
Loading 0:  97%|█████████▋| 490/507 [00:33<00:00, 26.77it/s]
Loading 0:  97%|█████████▋| 494/507 [00:33<00:00, 28.20it/s]
Loading 0:  98%|█████████▊| 499/507 [00:33<00:00, 31.53it/s]
Loading 0:  99%|█████████▉| 503/507 [00:33<00:00, 31.71it/s]
                                                            
Failed to get response for submission function_nupus_2024-11-27: no entry with id "fake_submission_id_for_testing" found on database!
Job chaiml-small-story-607-8056-v11-mkmlizer completed after 135.4s with status: succeeded
Stopping job with name chaiml-small-story-607-8056-v11-mkmlizer
Pipeline stage MKMLizer completed in 135.92s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.17s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-small-story-607-8056-v11
Waiting for inference service chaiml-small-story-607-8056-v11 to be ready
Failed to get response for submission function_sosuf_2024-11-27: no entry with id "fake_submission_ifd_for_testing" found on database!
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Inference service chaiml-small-story-607-8056-v11 ready after 130.5059916973114s
Pipeline stage MKMLDeployer completed in 131.13s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.7072551250457764s
Received healthy response to inference request in 2.4157660007476807s
Failed to get response for submission function_nupus_2024-11-27: no entry with id "fake_submission_id_for_testing" found on database!
Received healthy response to inference request in 2.644564390182495s
Failed to get response for submission function_nupus_2024-11-27: no entry with id "fake_submission_id_for_testing" found on database!
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Received healthy response to inference request in 2.744811534881592s
Received healthy response to inference request in 2.288985013961792s
5 requests
0 failed requests
5th percentile: 2.3143412113189696
10th percentile: 2.3396974086761473
20th percentile: 2.390409803390503
30th percentile: 2.4615256786346436
40th percentile: 2.5530450344085693
50th percentile: 2.644564390182495
60th percentile: 2.6696406841278075
70th percentile: 2.69471697807312
80th percentile: 2.7147664070129394
90th percentile: 2.729788970947266
95th percentile: 2.737300252914429
99th percentile: 2.743309278488159
mean time: 2.560276412963867
Pipeline stage StressChecker completed in 14.34s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.47s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 2.37s
Shutdown handler de-registered
chaiml-small-story-607-_8056_v11 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3470.84s
Shutdown handler de-registered
chaiml-small-story-607-_8056_v11 status is now inactive due to auto deactivation removed underperforming models