chaiml-elo-alignment-run-3

developer_uid: jxlu90

submission_id: chaiml-elo-alignment-run-3_v65

model_name: chaiml-elo-alignment-run-3_v65

model_group: ChaiML/elo_alignment_run

status: inactive

timestamp: 2024-12-17T23:05:18+00:00

num_battles: 11695

num_wins: 5881

celo_rating: 1265.44

family_friendly_score: 0.589

family_friendly_standard_error: 0.006958146304871722

submission_type: basic

model_repo: ChaiML/elo_alignment_run_3

model_architecture: LlamaForCausalLM

model_num_parameters: 8030261248.0

best_of: 16

max_input_tokens: 2048

max_output_tokens: 64

latencies: [{'batch_size': 1, 'throughput': 0.6460024605580781, 'latency_mean': 1.5478919327259064, 'latency_p50': 1.5379490852355957, 'latency_p90': 1.7138404607772828}, {'batch_size': 3, 'throughput': 0.9583893856613362, 'latency_mean': 3.1227367413043976, 'latency_p50': 3.123727560043335, 'latency_p90': 3.4677834033966066}, {'batch_size': 5, 'throughput': 1.0408516564076942, 'latency_mean': 4.783289910554886, 'latency_p50': 4.776329517364502, 'latency_p90': 5.3343566179275514}, {'batch_size': 6, 'throughput': 1.0485489264667187, 'latency_mean': 5.694643957614899, 'latency_p50': 5.664154887199402, 'latency_p90': 6.501932168006897}, {'batch_size': 10, 'throughput': 1.05458768235621, 'latency_mean': 9.382622717618942, 'latency_p50': 9.537584662437439, 'latency_p90': 10.315816926956177}]

gpu_counts: {'NVIDIA RTX A5000': 1}

display_name: chaiml-elo-alignment-run-3_v65

is_internal_developer: True

language_model: ChaiML/elo_alignment_run_3

model_size: 8B

ranking_group: single

throughput_3p7s: 1.0

us_pacific_date: 2024-12-17

win_ratio: 0.5028644719965797

generation_params: {'temperature': 0.95, 'top_p': 1.0, 'min_p': 0.08, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|eot_id|>'], 'max_input_tokens': 2048, 'best_of': 16, 'max_output_tokens': 64}

formatter: {'memory_template': '{memory}\n####\n', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-elo-alignment-run-3-v65-mkmlizer
Waiting for job on chaiml-elo-alignment-run-3-v65-mkmlizer to finish
chaiml-elo-alignment-run-3-v65-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-elo-alignment-run-3-v65-mkmlizer: ║     _____            __           __                                ║
chaiml-elo-alignment-run-3-v65-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
chaiml-elo-alignment-run-3-v65-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
chaiml-elo-alignment-run-3-v65-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
chaiml-elo-alignment-run-3-v65-mkmlizer: ║       /___/                                                         ║
chaiml-elo-alignment-run-3-v65-mkmlizer: ║                                                                     ║
chaiml-elo-alignment-run-3-v65-mkmlizer: ║  Version: 0.11.12                                                   ║
chaiml-elo-alignment-run-3-v65-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
chaiml-elo-alignment-run-3-v65-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-elo-alignment-run-3-v65-mkmlizer: ║                                                                     ║
chaiml-elo-alignment-run-3-v65-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-elo-alignment-run-3-v65-mkmlizer: ║  belonging to:                                                      ║
chaiml-elo-alignment-run-3-v65-mkmlizer: ║                                                                     ║
chaiml-elo-alignment-run-3-v65-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-elo-alignment-run-3-v65-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-elo-alignment-run-3-v65-mkmlizer: ║  Expiration: 2025-01-15 23:59:59                                    ║
chaiml-elo-alignment-run-3-v65-mkmlizer: ║                                                                     ║
chaiml-elo-alignment-run-3-v65-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
chaiml-elo-alignment-run-3-v65-mkmlizer: Downloaded to shared memory in 41.911s
chaiml-elo-alignment-run-3-v65-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpqaop8rm9, device:0
chaiml-elo-alignment-run-3-v65-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-elo-alignment-run-3-v65-mkmlizer: quantized model in 29.964s
chaiml-elo-alignment-run-3-v65-mkmlizer: Processed model ChaiML/elo_alignment_run_3 in 71.876s
chaiml-elo-alignment-run-3-v65-mkmlizer: creating bucket guanaco-mkml-models
chaiml-elo-alignment-run-3-v65-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-elo-alignment-run-3-v65-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-elo-alignment-run-3-v65
chaiml-elo-alignment-run-3-v65-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-elo-alignment-run-3-v65/config.json
chaiml-elo-alignment-run-3-v65-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-elo-alignment-run-3-v65/special_tokens_map.json
chaiml-elo-alignment-run-3-v65-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-elo-alignment-run-3-v65/tokenizer_config.json
chaiml-elo-alignment-run-3-v65-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-elo-alignment-run-3-v65/tokenizer.json
chaiml-elo-alignment-run-3-v65-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-elo-alignment-run-3-v65/flywheel_model.0.safetensors
chaiml-elo-alignment-run-3-v65-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 5/291 [00:00<00:11, 24.29it/s]
Loading 0:   4%|▍         | 12/291 [00:00<00:08, 34.69it/s]
Loading 0:   5%|▌         | 16/291 [00:00<00:08, 31.71it/s]
Loading 0:   7%|▋         | 21/291 [00:00<00:07, 34.20it/s]
Loading 0:   9%|▊         | 25/291 [00:00<00:08, 32.22it/s]
Loading 0:  11%|█         | 31/291 [00:00<00:06, 38.73it/s]
Loading 0:  12%|█▏        | 36/291 [00:01<00:11, 22.09it/s]
Loading 0:  14%|█▍        | 41/291 [00:01<00:10, 23.82it/s]
Loading 0:  16%|█▋        | 48/291 [00:01<00:08, 30.30it/s]
Loading 0:  18%|█▊        | 52/291 [00:01<00:07, 29.90it/s]
Loading 0:  20%|█▉        | 57/291 [00:01<00:07, 31.53it/s]
Loading 0:  21%|██        | 61/291 [00:02<00:07, 30.78it/s]
Loading 0:  23%|██▎       | 66/291 [00:02<00:06, 33.37it/s]
Loading 0:  24%|██▍       | 70/291 [00:02<00:06, 32.51it/s]
Loading 0:  25%|██▌       | 74/291 [00:02<00:06, 33.15it/s]
Loading 0:  27%|██▋       | 78/291 [00:02<00:06, 32.39it/s]
Loading 0:  28%|██▊       | 82/291 [00:02<00:09, 22.62it/s]
Loading 0:  29%|██▉       | 85/291 [00:02<00:08, 23.53it/s]
Loading 0:  31%|███       | 90/291 [00:03<00:07, 27.19it/s]
Loading 0:  32%|███▏      | 94/291 [00:03<00:07, 27.90it/s]
Loading 0:  34%|███▍      | 99/291 [00:03<00:06, 31.47it/s]
Loading 0:  35%|███▌      | 103/291 [00:03<00:06, 29.95it/s]
Loading 0:  37%|███▋      | 108/291 [00:03<00:05, 33.12it/s]
Loading 0:  38%|███▊      | 112/291 [00:03<00:05, 32.26it/s]
Loading 0:  40%|███▉      | 116/291 [00:03<00:05, 33.09it/s]
Loading 0:  42%|████▏     | 122/291 [00:03<00:04, 37.36it/s]
Loading 0:  44%|████▎     | 127/291 [00:04<00:04, 35.29it/s]
Loading 0:  46%|████▌     | 133/291 [00:04<00:05, 30.49it/s]
Loading 0:  47%|████▋     | 137/291 [00:04<00:05, 29.31it/s]
Loading 0:  48%|████▊     | 141/291 [00:04<00:05, 27.82it/s]
Loading 0:  51%|█████     | 147/291 [00:04<00:04, 33.19it/s]
Loading 0:  52%|█████▏    | 151/291 [00:04<00:04, 32.55it/s]
Loading 0:  54%|█████▎    | 156/291 [00:05<00:03, 35.43it/s]
Loading 0:  55%|█████▍    | 160/291 [00:05<00:03, 33.66it/s]
Loading 0:  57%|█████▋    | 165/291 [00:05<00:03, 36.92it/s]
Loading 0:  58%|█████▊    | 169/291 [00:05<00:03, 35.49it/s]
Loading 0:  60%|█████▉    | 174/291 [00:05<00:03, 38.40it/s]
Loading 0:  61%|██████    | 178/291 [00:05<00:03, 35.00it/s]
Loading 0:  64%|██████▎   | 185/291 [00:05<00:02, 38.97it/s]
Loading 0:  65%|██████▍   | 189/291 [00:06<00:04, 25.19it/s]
Loading 0:  67%|██████▋   | 194/291 [00:06<00:03, 26.65it/s]
Loading 0:  69%|██████▉   | 201/291 [00:06<00:02, 33.78it/s]
Loading 0:  70%|███████   | 205/291 [00:06<00:02, 33.21it/s]
Loading 0:  72%|███████▏  | 210/291 [00:06<00:02, 36.06it/s]
Loading 0:  74%|███████▎  | 214/291 [00:06<00:02, 34.08it/s]
Loading 0:  75%|███████▌  | 219/291 [00:06<00:01, 36.56it/s]
Loading 0:  77%|███████▋  | 223/291 [00:07<00:02, 33.46it/s]
Loading 0:  78%|███████▊  | 227/291 [00:07<00:01, 33.79it/s]
Loading 0:  79%|███████▉  | 231/291 [00:07<00:01, 33.54it/s]
Loading 0:  81%|████████  | 235/291 [00:07<00:02, 25.07it/s]
Loading 0:  82%|████████▏ | 239/291 [00:07<00:02, 24.33it/s]
Loading 0:  85%|████████▍ | 246/291 [00:07<00:01, 31.16it/s]
Loading 0:  86%|████████▌ | 250/291 [00:08<00:01, 30.48it/s]
Loading 0:  88%|████████▊ | 255/291 [00:08<00:01, 33.51it/s]
Loading 0:  89%|████████▉ | 259/291 [00:08<00:01, 31.52it/s]
Loading 0:  91%|█████████ | 264/291 [00:08<00:00, 34.09it/s]
Loading 0:  92%|█████████▏| 268/291 [00:08<00:00, 31.81it/s]
Loading 0:  94%|█████████▍| 273/291 [00:08<00:00, 34.09it/s]
Loading 0:  95%|█████████▌| 277/291 [00:08<00:00, 32.68it/s]
Loading 0:  97%|█████████▋| 281/291 [00:08<00:00, 33.30it/s]
Loading 0:  98%|█████████▊| 286/291 [00:14<00:01,  2.53it/s]
Loading 0:  99%|█████████▉| 289/291 [00:14<00:00,  3.14it/s]
                                                            
Job chaiml-elo-alignment-run-3-v65-mkmlizer completed after 94.4s with status: succeeded
Stopping job with name chaiml-elo-alignment-run-3-v65-mkmlizer
Pipeline stage MKMLizer completed in 94.94s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.17s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-elo-alignment-run-3-v65
Waiting for inference service chaiml-elo-alignment-run-3-v65 to be ready
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Inference service chaiml-elo-alignment-run-3-v65 ready after 220.9696364402771s
Pipeline stage MKMLDeployer completed in 221.54s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.0216193199157715s
Received healthy response to inference request in 1.3365538120269775s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 9.94466233253479s
Received healthy response to inference request in 1.2217800617218018s
5 requests
1 failed requests
5th percentile: 1.244734811782837
10th percentile: 1.2676895618438722
20th percentile: 1.3135990619659423
30th percentile: 1.4735669136047362
40th percentile: 1.747593116760254
50th percentile: 2.0216193199157715
60th percentile: 5.1908365249633786
70th percentile: 8.360053730010986
80th percentile: 11.984169816970827
90th percentile: 16.063184785842896
95th percentile: 18.10269227027893
99th percentile: 19.73429825782776
mean time: 6.933363056182861
%s, retrying in %s seconds...
Received healthy response to inference request in 1.6078705787658691s
Received healthy response to inference request in 1.4248876571655273s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Received healthy response to inference request in 1.3825149536132812s
Received healthy response to inference request in 3.5885393619537354s
Received healthy response to inference request in 1.415825605392456s
5 requests
0 failed requests
5th percentile: 1.3891770839691162
10th percentile: 1.3958392143249512
20th percentile: 1.409163475036621
30th percentile: 1.4176380157470703
40th percentile: 1.421262836456299
50th percentile: 1.4248876571655273
60th percentile: 1.4980808258056642
70th percentile: 1.5712739944458007
80th percentile: 2.004004335403443
90th percentile: 2.796271848678589
95th percentile: 3.1924056053161616
99th percentile: 3.5093126106262207
mean time: 1.8839276313781739
Pipeline stage StressChecker completed in 46.79s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
Connection pool is full, discarding connection: %s. Connection pool size: %s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.42s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
Connection pool is full, discarding connection: %s. Connection pool size: %s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 2.32s
Shutdown handler de-registered
chaiml-elo-alignment-run-3_v65 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage MKMLProfilerDeleter completed in 0.15s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.12s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service chaiml-elo-alignment-run-3-v65-profiler
Waiting for inference service chaiml-elo-alignment-run-3-v65-profiler to be ready
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3083.99s
Shutdown handler de-registered
chaiml-elo-alignment-run-3_v65 status is now inactive due to auto deactivation removed underperforming models