valentin87-haunted-stori_7811

submission_id: valentin87-haunted-stori_7811_v1

developer_uid: valentin87

alignment_samples: 11068

alignment_score: 0.29812699757086714

best_of: 1

celo_rating: 1181.32

display_name: valentin87-haunted-stori_7811_v1

formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 100, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 1, 'max_output_tokens': 64}

gpu_counts: {'NVIDIA RTX A5000': 1}

is_internal_developer: False

language_model: valentin87/haunted_stories_r64

latencies: [{'batch_size': 1, 'throughput': 0.7527382859256218, 'latency_mean': 1.3284224331378938, 'latency_p50': 1.34418523311615, 'latency_p90': 1.4567408561706543}, {'batch_size': 5, 'throughput': 2.4497155522183798, 'latency_mean': 2.0308967781066896, 'latency_p50': 2.032694935798645, 'latency_p90': 2.2426842927932737}, {'batch_size': 10, 'throughput': 3.487124845318895, 'latency_mean': 2.8351994729042054, 'latency_p50': 2.8272026777267456, 'latency_p90': 3.248555898666382}, {'batch_size': 15, 'throughput': 4.03109207579578, 'latency_mean': 3.6749217641353606, 'latency_p50': 3.6613805294036865, 'latency_p90': 4.2501136302948}, {'batch_size': 20, 'throughput': 4.338750057398413, 'latency_mean': 4.556251538991928, 'latency_p50': 4.542709589004517, 'latency_p90': 5.2919070482254025}, {'batch_size': 25, 'throughput': 4.543095844830316, 'latency_mean': 5.411470793485641, 'latency_p50': 5.403368949890137, 'latency_p90': 6.212262868881225}]

max_input_tokens: 512

max_output_tokens: 64

model_architecture: MistralForCausalLM

model_group: valentin87/haunted_stori

model_name: valentin87-haunted-stori_7811_v1

model_num_parameters: 12772070400.0

model_repo: valentin87/haunted_stories_r64

model_size: 13B

num_battles: 11068

num_wins: 4545

propriety_score: 0.7635782747603834

propriety_total_count: 939.0

ranking_group: single

status: inactive

submission_type: basic

throughput_3p7s: 4.09

timestamp: 2024-09-13T14:23:36+00:00

us_pacific_date: 2024-09-13

win_ratio: 0.41064329598843513

Download Preference Data

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name valentin87-haunted-stori-7811-v1-mkmlizer
Waiting for job on valentin87-haunted-stori-7811-v1-mkmlizer to finish
valentin87-haunted-stori-7811-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
valentin87-haunted-stori-7811-v1-mkmlizer: ║     _____            __           __                                ║
valentin87-haunted-stori-7811-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
valentin87-haunted-stori-7811-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
valentin87-haunted-stori-7811-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
valentin87-haunted-stori-7811-v1-mkmlizer: ║       /___/                                                         ║
valentin87-haunted-stori-7811-v1-mkmlizer: ║                                                                     ║
valentin87-haunted-stori-7811-v1-mkmlizer: ║  Version: 0.10.1                                                    ║
valentin87-haunted-stori-7811-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
valentin87-haunted-stori-7811-v1-mkmlizer: ║  https://mk1.ai                                                     ║
valentin87-haunted-stori-7811-v1-mkmlizer: ║                                                                     ║
valentin87-haunted-stori-7811-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
valentin87-haunted-stori-7811-v1-mkmlizer: ║  belonging to:                                                      ║
valentin87-haunted-stori-7811-v1-mkmlizer: ║                                                                     ║
valentin87-haunted-stori-7811-v1-mkmlizer: ║  Chai Research Corp.                                                ║
valentin87-haunted-stori-7811-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
valentin87-haunted-stori-7811-v1-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
valentin87-haunted-stori-7811-v1-mkmlizer: ║                                                                     ║
valentin87-haunted-stori-7811-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
valentin87-haunted-stori-7811-v1-mkmlizer: Downloaded to shared memory in 46.158s
valentin87-haunted-stori-7811-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpvnjwk6fi, device:0
valentin87-haunted-stori-7811-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
valentin87-haunted-stori-7811-v1-mkmlizer: quantized model in 34.743s
valentin87-haunted-stori-7811-v1-mkmlizer: Processed model valentin87/haunted_stories_r64 in 80.901s
valentin87-haunted-stori-7811-v1-mkmlizer: creating bucket guanaco-mkml-models
valentin87-haunted-stori-7811-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
valentin87-haunted-stori-7811-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/valentin87-haunted-stori-7811-v1
valentin87-haunted-stori-7811-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/valentin87-haunted-stori-7811-v1/config.json
valentin87-haunted-stori-7811-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/valentin87-haunted-stori-7811-v1/special_tokens_map.json
valentin87-haunted-stori-7811-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/valentin87-haunted-stori-7811-v1/tokenizer_config.json
valentin87-haunted-stori-7811-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/valentin87-haunted-stori-7811-v1/tokenizer.json
valentin87-haunted-stori-7811-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/valentin87-haunted-stori-7811-v1/flywheel_model.0.safetensors
valentin87-haunted-stori-7811-v1-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:11, 30.20it/s]
Loading 0:   4%|▎         | 13/363 [00:00<00:06, 51.43it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:07, 46.73it/s]
Loading 0:   7%|▋         | 24/363 [00:00<00:07, 45.30it/s]
Loading 0:   9%|▊         | 31/363 [00:00<00:06, 51.11it/s]
Loading 0:  10%|█         | 37/363 [00:00<00:07, 44.36it/s]
Loading 0:  12%|█▏        | 42/363 [00:00<00:07, 44.18it/s]
Loading 0:  13%|█▎        | 49/363 [00:01<00:06, 49.98it/s]
Loading 0:  15%|█▌        | 55/363 [00:01<00:06, 47.14it/s]
Loading 0:  17%|█▋        | 61/363 [00:01<00:08, 35.57it/s]
Loading 0:  18%|█▊        | 66/363 [00:01<00:08, 36.80it/s]
Loading 0:  20%|█▉        | 72/363 [00:01<00:07, 41.03it/s]
Loading 0:  21%|██▏       | 78/363 [00:01<00:06, 41.71it/s]
Loading 0:  23%|██▎       | 83/363 [00:01<00:06, 42.30it/s]
Loading 0:  25%|██▍       | 90/363 [00:02<00:05, 46.97it/s]
Loading 0:  26%|██▋       | 96/363 [00:02<00:05, 45.22it/s]
Loading 0:  28%|██▊       | 101/363 [00:02<00:05, 43.89it/s]
Loading 0:  30%|██▉       | 108/363 [00:02<00:05, 50.29it/s]
Loading 0:  31%|███▏      | 114/363 [00:02<00:05, 45.78it/s]
Loading 0:  33%|███▎      | 119/363 [00:02<00:05, 44.78it/s]
Loading 0:  35%|███▍      | 126/363 [00:02<00:04, 49.75it/s]
Loading 0:  36%|███▋      | 132/363 [00:02<00:04, 47.24it/s]
Loading 0:  38%|███▊      | 137/363 [00:03<00:04, 46.44it/s]
Loading 0:  39%|███▉      | 142/363 [00:03<00:06, 36.59it/s]
Loading 0:  40%|████      | 147/363 [00:03<00:05, 37.66it/s]
Loading 0:  42%|████▏     | 152/363 [00:03<00:05, 39.94it/s]
Loading 0:  43%|████▎     | 157/363 [00:03<00:04, 42.39it/s]
Loading 0:  45%|████▍     | 163/363 [00:03<00:04, 42.05it/s]
Loading 0:  46%|████▋     | 168/363 [00:03<00:04, 42.71it/s]
Loading 0:  48%|████▊     | 175/363 [00:03<00:03, 47.93it/s]
Loading 0:  50%|████▉     | 181/363 [00:04<00:04, 45.06it/s]
Loading 0:  51%|█████     | 186/363 [00:04<00:03, 44.48it/s]
Loading 0:  53%|█████▎    | 193/363 [00:04<00:03, 49.46it/s]
Loading 0:  55%|█████▍    | 199/363 [00:04<00:03, 46.80it/s]
Loading 0:  56%|█████▌    | 204/363 [00:04<00:03, 45.43it/s]
Loading 0:  58%|█████▊    | 211/363 [00:04<00:03, 50.33it/s]
Loading 0:  60%|█████▉    | 217/363 [00:04<00:03, 47.12it/s]
Loading 0:  61%|██████▏   | 223/363 [00:05<00:03, 37.36it/s]
Loading 0:  63%|██████▎   | 228/363 [00:05<00:03, 37.98it/s]
Loading 0:  64%|██████▍   | 233/363 [00:05<00:03, 39.98it/s]
Loading 0:  66%|██████▌   | 238/363 [00:05<00:02, 42.13it/s]
Loading 0:  67%|██████▋   | 243/363 [00:05<00:02, 43.63it/s]
Loading 0:  68%|██████▊   | 248/363 [00:05<00:03, 37.16it/s]
Loading 0:  70%|███████   | 255/363 [00:05<00:02, 44.02it/s]
Loading 0:  72%|███████▏  | 260/363 [00:05<00:02, 43.71it/s]
Loading 0:  73%|███████▎  | 265/363 [00:06<00:02, 44.68it/s]
Loading 0:  75%|███████▍  | 271/363 [00:06<00:02, 43.70it/s]
Loading 0:  76%|███████▌  | 276/363 [00:06<00:02, 43.00it/s]
Loading 0:  78%|███████▊  | 283/363 [00:06<00:01, 48.30it/s]
Loading 0:  80%|███████▉  | 289/363 [00:06<00:01, 46.35it/s]
Loading 0:  81%|████████  | 294/363 [00:06<00:01, 44.96it/s]
Loading 0:  83%|████████▎ | 301/363 [00:06<00:01, 51.22it/s]
Loading 0:  85%|████████▍ | 307/363 [00:13<00:19,  2.84it/s]
Loading 0:  86%|████████▌ | 312/363 [00:13<00:13,  3.76it/s]
Loading 0:  88%|████████▊ | 320/363 [00:13<00:07,  5.83it/s]
Loading 0:  90%|████████▉ | 326/363 [00:13<00:04,  7.74it/s]
Loading 0:  91%|█████████ | 331/363 [00:14<00:03,  9.79it/s]
Loading 0:  93%|█████████▎| 338/363 [00:14<00:01, 13.75it/s]
Loading 0:  95%|█████████▍| 344/363 [00:14<00:01, 17.09it/s]
Loading 0:  96%|█████████▌| 349/363 [00:14<00:00, 20.14it/s]
Loading 0:  98%|█████████▊| 356/363 [00:14<00:00, 26.32it/s]
Loading 0: 100%|█████████▉| 362/363 [00:14<00:00, 29.21it/s]
                                                            
Job valentin87-haunted-stori-7811-v1-mkmlizer completed after 105.5s with status: succeeded
Stopping job with name valentin87-haunted-stori-7811-v1-mkmlizer
Pipeline stage MKMLizer completed in 106.79s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.10s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service valentin87-haunted-stori-7811-v1
Waiting for inference service valentin87-haunted-stori-7811-v1 to be ready
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Inference service valentin87-haunted-stori-7811-v1 ready after 180.95308923721313s
Pipeline stage MKMLDeployer completed in 181.47s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.3254263401031494s
Received healthy response to inference request in 0.808251142501831s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Received healthy response to inference request in 1.2890315055847168s
Received healthy response to inference request in 2.953681230545044s
Received healthy response to inference request in 1.439845323562622s
5 requests
0 failed requests
5th percentile: 0.9044072151184082
10th percentile: 1.0005632877349853
20th percentile: 1.1928754329681397
30th percentile: 1.3191942691802978
40th percentile: 1.37951979637146
50th percentile: 1.439845323562622
60th percentile: 1.7940777301788329
70th percentile: 2.1483101367950437
80th percentile: 2.4510773181915284
90th percentile: 2.702379274368286
95th percentile: 2.828030252456665
99th percentile: 2.9285510349273682
mean time: 1.7632471084594727
Pipeline stage StressChecker completed in 9.59s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 6.83s
Shutdown handler de-registered
valentin87-haunted-stori_7811_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage MKMLProfilerDeleter completed in 0.13s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.11s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service valentin87-haunted-stori-7811-v1-profiler
Waiting for inference service valentin87-haunted-stori-7811-v1-profiler to be ready
Inference service valentin87-haunted-stori-7811-v1-profiler ready after 170.41290616989136s
Pipeline stage MKMLProfilerDeployer completed in 170.79s
run pipeline stage %s
Running pipeline stage MKMLProfilerRunner
kubectl cp /code/guanaco/guanaco_inference_services/src/inference_scripts tenant-chaiml-guanaco/valentin87-haunted-s11c881799857d849419bc1b3ff91a588-deplowd58p:/code/chaiverse_profiler_1726237940 --namespace tenant-chaiml-guanaco
kubectl exec -it valentin87-haunted-s11c881799857d849419bc1b3ff91a588-deplowd58p --namespace tenant-chaiml-guanaco -- sh -c 'cd /code/chaiverse_profiler_1726237940 && python profiles.py profile --best_of_n 1 --auto_batch 5 --batches 1,5,10,15,20,25,30,35,40,45,50,55,60,65,70,75,80,85,90,95,100,105,110,115,120,125,130,135,140,145,150,155,160,165,170,175,180,185,190,195 --samples 200 --input_tokens 512 --output_tokens 64 --summary /code/chaiverse_profiler_1726237940/summary.json'
kubectl exec -it valentin87-haunted-s11c881799857d849419bc1b3ff91a588-deplowd58p --namespace tenant-chaiml-guanaco -- bash -c 'cat /code/chaiverse_profiler_1726237940/summary.json'
Pipeline stage MKMLProfilerRunner completed in 549.53s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Checking if service valentin87-haunted-stori-7811-v1-profiler is running
Tearing down inference service valentin87-haunted-stori-7811-v1-profiler
Service valentin87-haunted-stori-7811-v1-profiler has been torndown
Pipeline stage MKMLProfilerDeleter completed in 2.15s
Shutdown handler de-registered
valentin87-haunted-stori_7811_v1 status is now inactive due to auto deactivation removed underperforming models