stark2000s-axolotl-test-calm

developer_uid: stark2000s

submission_id: stark2000s-axolotl-test-calm_v1

model_name: stark2000s-axolotl-test-calm_v1

model_group: stark2000s/axolotl_test_

status: inactive

timestamp: 2024-11-17T10:16:13+00:00

num_battles: 17506

num_wins: 8103

celo_rating: 1220.97

family_friendly_score: 0.5614

family_friendly_standard_error: 0.007017549999821875

submission_type: basic

model_repo: stark2000s/axolotl_test_calm

model_architecture: LlamaForCausalLM

model_num_parameters: 8030261248.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

latencies: [{'batch_size': 1, 'throughput': 0.8649655333175954, 'latency_mean': 1.1560502767562866, 'latency_p50': 1.1511247158050537, 'latency_p90': 1.2785160064697265}, {'batch_size': 4, 'throughput': 1.8785579821442748, 'latency_mean': 2.118537974357605, 'latency_p50': 2.10570228099823, 'latency_p90': 2.365116310119629}, {'batch_size': 5, 'throughput': 2.01394584745198, 'latency_mean': 2.4699926602840425, 'latency_p50': 2.4606616497039795, 'latency_p90': 2.8105325222015383}, {'batch_size': 8, 'throughput': 2.217566932219917, 'latency_mean': 3.5807330775260926, 'latency_p50': 3.557698965072632, 'latency_p90': 4.058151960372925}, {'batch_size': 10, 'throughput': 2.314107083022445, 'latency_mean': 4.294499279260635, 'latency_p50': 4.305296063423157, 'latency_p90': 4.852603936195373}, {'batch_size': 12, 'throughput': 2.309298246759627, 'latency_mean': 5.145713949203492, 'latency_p50': 5.153296709060669, 'latency_p90': 5.738737297058106}, {'batch_size': 15, 'throughput': 2.352280680124158, 'latency_mean': 6.29185992717743, 'latency_p50': 6.351991772651672, 'latency_p90': 7.014225387573242}]

gpu_counts: {'NVIDIA RTX A5000': 1}

display_name: stark2000s-axolotl-test-calm_v1

is_internal_developer: False

language_model: stark2000s/axolotl_test_calm

model_size: 8B

ranking_group: single

throughput_3p7s: 2.25

us_pacific_date: 2024-11-17

win_ratio: 0.4628698731863361

generation_params: {'temperature': 1.15, 'top_p': 1.0, 'min_p': 0.06, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.1, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name stark2000s-axolotl-test-calm-v1-mkmlizer
Waiting for job on stark2000s-axolotl-test-calm-v1-mkmlizer to finish
stark2000s-axolotl-test-calm-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
stark2000s-axolotl-test-calm-v1-mkmlizer: ║     _____            __           __                                ║
stark2000s-axolotl-test-calm-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
stark2000s-axolotl-test-calm-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
stark2000s-axolotl-test-calm-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
stark2000s-axolotl-test-calm-v1-mkmlizer: ║       /___/                                                         ║
stark2000s-axolotl-test-calm-v1-mkmlizer: ║                                                                     ║
stark2000s-axolotl-test-calm-v1-mkmlizer: ║  Version: 0.11.12                                                   ║
stark2000s-axolotl-test-calm-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
stark2000s-axolotl-test-calm-v1-mkmlizer: ║  https://mk1.ai                                                     ║
stark2000s-axolotl-test-calm-v1-mkmlizer: ║                                                                     ║
stark2000s-axolotl-test-calm-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
stark2000s-axolotl-test-calm-v1-mkmlizer: ║  belonging to:                                                      ║
stark2000s-axolotl-test-calm-v1-mkmlizer: ║                                                                     ║
stark2000s-axolotl-test-calm-v1-mkmlizer: ║  Chai Research Corp.                                                ║
stark2000s-axolotl-test-calm-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
stark2000s-axolotl-test-calm-v1-mkmlizer: ║  Expiration: 2025-01-15 23:59:59                                    ║
stark2000s-axolotl-test-calm-v1-mkmlizer: ║                                                                     ║
stark2000s-axolotl-test-calm-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
stark2000s-axolotl-test-calm-v1-mkmlizer: quantized model in 26.818s
stark2000s-axolotl-test-calm-v1-mkmlizer: Processed model stark2000s/axolotl_test_calm in 59.541s
stark2000s-axolotl-test-calm-v1-mkmlizer: creating bucket guanaco-mkml-models
stark2000s-axolotl-test-calm-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
stark2000s-axolotl-test-calm-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/stark2000s-axolotl-test-calm-v1
stark2000s-axolotl-test-calm-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/stark2000s-axolotl-test-calm-v1/special_tokens_map.json
stark2000s-axolotl-test-calm-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/stark2000s-axolotl-test-calm-v1/config.json
stark2000s-axolotl-test-calm-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/stark2000s-axolotl-test-calm-v1/tokenizer_config.json
stark2000s-axolotl-test-calm-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/stark2000s-axolotl-test-calm-v1/tokenizer.json
stark2000s-axolotl-test-calm-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/stark2000s-axolotl-test-calm-v1/flywheel_model.0.safetensors
stark2000s-axolotl-test-calm-v1-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 4/291 [00:00<00:07, 38.36it/s]
Loading 0:   4%|▍         | 13/291 [00:00<00:04, 67.45it/s]
Loading 0:   8%|▊         | 22/291 [00:00<00:03, 74.53it/s]
Loading 0:  11%|█         | 31/291 [00:00<00:03, 75.64it/s]
Loading 0:  14%|█▎        | 40/291 [00:00<00:03, 77.59it/s]
Loading 0:  17%|█▋        | 49/291 [00:00<00:03, 80.10it/s]
Loading 0:  20%|█▉        | 58/291 [00:00<00:02, 81.85it/s]
Loading 0:  23%|██▎       | 67/291 [00:00<00:02, 81.19it/s]
Loading 0:  26%|██▌       | 76/291 [00:00<00:02, 80.41it/s]
Loading 0:  29%|██▉       | 85/291 [00:02<00:10, 20.56it/s]
Loading 0:  32%|███▏      | 94/291 [00:02<00:07, 26.51it/s]
Loading 0:  35%|███▌      | 103/291 [00:02<00:05, 33.42it/s]
Loading 0:  38%|███▊      | 112/291 [00:02<00:04, 40.34it/s]
Loading 0:  42%|████▏     | 121/291 [00:02<00:03, 48.32it/s]
Loading 0:  45%|████▍     | 130/291 [00:02<00:02, 56.04it/s]
Loading 0:  48%|████▊     | 139/291 [00:02<00:02, 63.03it/s]
Loading 0:  51%|█████     | 148/291 [00:02<00:02, 67.07it/s]
Loading 0:  54%|█████▍    | 157/291 [00:03<00:01, 70.23it/s]
Loading 0:  57%|█████▋    | 166/291 [00:03<00:01, 74.83it/s]
Loading 0:  60%|██████    | 175/291 [00:03<00:01, 77.65it/s]
Loading 0:  63%|██████▎   | 184/291 [00:03<00:01, 77.96it/s]
Loading 0:  66%|██████▋   | 193/291 [00:04<00:04, 20.40it/s]
Loading 0:  69%|██████▉   | 202/291 [00:04<00:03, 26.43it/s]
Loading 0:  73%|███████▎  | 211/291 [00:04<00:02, 33.24it/s]
Loading 0:  76%|███████▌  | 220/291 [00:04<00:01, 40.16it/s]
Loading 0:  79%|███████▊  | 229/291 [00:04<00:01, 46.99it/s]
Loading 0:  82%|████████▏ | 238/291 [00:05<00:01, 52.22it/s]
Loading 0:  85%|████████▍ | 247/291 [00:05<00:00, 58.20it/s]
Loading 0:  88%|████████▊ | 256/291 [00:05<00:00, 63.48it/s]
Loading 0:  91%|█████████ | 265/291 [00:05<00:00, 66.93it/s]
Loading 0:  94%|█████████▍| 274/291 [00:05<00:00, 69.16it/s]
Loading 0:  97%|█████████▋| 283/291 [00:05<00:00, 73.90it/s]
Loading 0: 100%|██████████| 291/291 [00:11<00:00,  5.04it/s]
                                                            
Job stark2000s-axolotl-test-calm-v1-mkmlizer completed after 84.33s with status: succeeded
Stopping job with name stark2000s-axolotl-test-calm-v1-mkmlizer
Pipeline stage MKMLizer completed in 84.94s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.21s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service stark2000s-axolotl-test-calm-v1
Waiting for inference service stark2000s-axolotl-test-calm-v1 to be ready
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Inference service stark2000s-axolotl-test-calm-v1 ready after 210.86481976509094s
Pipeline stage MKMLDeployer completed in 211.47s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.045600175857544s
Received healthy response to inference request in 1.4919805526733398s
Received healthy response to inference request in 1.2772016525268555s
Received healthy response to inference request in 2.597132444381714s
Received healthy response to inference request in 1.2473697662353516s
5 requests
0 failed requests
5th percentile: 1.2533361434936523
10th percentile: 1.259302520751953
20th percentile: 1.2712352752685547
30th percentile: 1.3201574325561523
40th percentile: 1.406068992614746
50th percentile: 1.4919805526733398
60th percentile: 1.7134284019470214
70th percentile: 1.934876251220703
80th percentile: 2.155906629562378
90th percentile: 2.376519536972046
95th percentile: 2.4868259906768797
99th percentile: 2.575071153640747
mean time: 1.731856918334961
Pipeline stage StressChecker completed in 10.28s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.46s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 2.18s
Shutdown handler de-registered
stark2000s-axolotl-test-calm_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2309.07s
Shutdown handler de-registered
stark2000s-axolotl-test-calm_v1 status is now inactive due to auto deactivation removed underperforming models