submission_id: rica40325-llama3_v2
developer_uid: rica40325
alignment_samples: 14870
alignment_score: -1.727198305403792
best_of: 16
celo_rating: 1212.73
display_name: rica40325-llama3_v1
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}
is_internal_developer: False
language_model: rica40325/llama3
max_input_tokens: 512
max_output_tokens: 64
model_architecture: LlamaForCausalLM
model_group: rica40325/llama3
model_name: rica40325-llama3_v1
model_num_parameters: 8030261248.0
model_repo: rica40325/llama3
model_size: 8B
num_battles: 14870
num_wins: 6848
propriety_score: 0.7177726926010679
propriety_total_count: 1311.0
ranking_group: single
status: inactive
submission_type: basic
timestamp: 2024-09-04T12:01:28+00:00
us_pacific_date: 2024-09-04
win_ratio: 0.4605245460659045
Download Preference Data
Resubmit model
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name rica40325-llama3-v2-mkmlizer
Waiting for job on rica40325-llama3-v2-mkmlizer to finish
rica40325-llama3-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
rica40325-llama3-v2-mkmlizer: ║     _____            __           __                                ║
rica40325-llama3-v2-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
rica40325-llama3-v2-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
rica40325-llama3-v2-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
rica40325-llama3-v2-mkmlizer: ║       /___/                                                         ║
rica40325-llama3-v2-mkmlizer: ║                                                                     ║
rica40325-llama3-v2-mkmlizer: ║  Version: 0.10.1                                                    ║
rica40325-llama3-v2-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
rica40325-llama3-v2-mkmlizer: ║  https://mk1.ai                                                     ║
rica40325-llama3-v2-mkmlizer: ║                                                                     ║
rica40325-llama3-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
rica40325-llama3-v2-mkmlizer: ║  belonging to:                                                      ║
rica40325-llama3-v2-mkmlizer: ║                                                                     ║
rica40325-llama3-v2-mkmlizer: ║  Chai Research Corp.                                                ║
rica40325-llama3-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
rica40325-llama3-v2-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
rica40325-llama3-v2-mkmlizer: ║                                                                     ║
rica40325-llama3-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
rica40325-llama3-v2-mkmlizer: Downloaded to shared memory in 37.496s
rica40325-llama3-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpnyv1sn55, device:0
rica40325-llama3-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
rica40325-llama3-v2-mkmlizer: quantized model in 25.099s
rica40325-llama3-v2-mkmlizer: Processed model rica40325/llama3 in 62.595s
rica40325-llama3-v2-mkmlizer: creating bucket guanaco-mkml-models
rica40325-llama3-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
rica40325-llama3-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/rica40325-llama3-v2
rica40325-llama3-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/rica40325-llama3-v2/config.json
rica40325-llama3-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/rica40325-llama3-v2/special_tokens_map.json
rica40325-llama3-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/rica40325-llama3-v2/tokenizer_config.json
rica40325-llama3-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/rica40325-llama3-v2/tokenizer.json
rica40325-llama3-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/rica40325-llama3-v2/flywheel_model.0.safetensors
rica40325-llama3-v2-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 5/291 [00:00<00:07, 37.05it/s]
Loading 0:   5%|▍         | 14/291 [00:00<00:05, 51.31it/s]
Loading 0:   8%|▊         | 23/291 [00:00<00:04, 55.37it/s]
Loading 0:  11%|█         | 32/291 [00:00<00:04, 56.90it/s]
Loading 0:  14%|█▍        | 41/291 [00:00<00:04, 57.60it/s]
Loading 0:  17%|█▋        | 50/291 [00:00<00:04, 57.46it/s]
Loading 0:  20%|██        | 59/291 [00:01<00:04, 57.05it/s]
Loading 0:  23%|██▎       | 68/291 [00:01<00:03, 56.82it/s]
Loading 0:  26%|██▌       | 76/291 [00:01<00:03, 61.77it/s]
Loading 0:  29%|██▊       | 83/291 [00:01<00:04, 43.63it/s]
Loading 0:  31%|███       | 89/291 [00:01<00:04, 44.77it/s]
Loading 0:  33%|███▎      | 95/291 [00:01<00:04, 42.24it/s]
Loading 0:  35%|███▌      | 103/291 [00:02<00:03, 49.10it/s]
Loading 0:  37%|███▋      | 109/291 [00:02<00:03, 48.43it/s]
Loading 0:  40%|███▉      | 115/291 [00:02<00:03, 50.30it/s]
Loading 0:  42%|████▏     | 121/291 [00:02<00:03, 52.43it/s]
Loading 0:  44%|████▎     | 127/291 [00:02<00:03, 50.67it/s]
Loading 0:  46%|████▌     | 133/291 [00:02<00:03, 52.34it/s]
Loading 0:  48%|████▊     | 140/291 [00:02<00:03, 50.01it/s]
Loading 0:  51%|█████     | 148/291 [00:02<00:02, 57.44it/s]
Loading 0:  53%|█████▎    | 154/291 [00:02<00:02, 51.64it/s]
Loading 0:  55%|█████▍    | 160/291 [00:03<00:02, 52.04it/s]
Loading 0:  57%|█████▋    | 166/291 [00:03<00:02, 53.77it/s]
Loading 0:  59%|█████▉    | 172/291 [00:03<00:02, 50.99it/s]
Loading 0:  62%|██████▏   | 179/291 [00:03<00:02, 54.49it/s]
Loading 0:  64%|██████▎   | 185/291 [00:03<00:01, 55.82it/s]
Loading 0:  66%|██████▌   | 191/291 [00:03<00:02, 36.12it/s]
Loading 0:  67%|██████▋   | 196/291 [00:03<00:02, 37.92it/s]
Loading 0:  70%|██████▉   | 203/291 [00:04<00:02, 39.70it/s]
Loading 0:  73%|███████▎  | 211/291 [00:04<00:01, 48.22it/s]
Loading 0:  75%|███████▍  | 217/291 [00:04<00:01, 48.61it/s]
Loading 0:  77%|███████▋  | 223/291 [00:04<00:01, 51.10it/s]
Loading 0:  79%|███████▉  | 230/291 [00:04<00:01, 47.52it/s]
Loading 0:  82%|████████▏ | 238/291 [00:04<00:00, 55.00it/s]
Loading 0:  84%|████████▍ | 244/291 [00:04<00:00, 51.45it/s]
Loading 0:  86%|████████▌ | 250/291 [00:04<00:00, 52.93it/s]
Loading 0:  88%|████████▊ | 257/291 [00:05<00:00, 49.35it/s]
Loading 0:  91%|█████████▏| 266/291 [00:05<00:00, 51.72it/s]
Loading 0:  95%|█████████▍| 275/291 [00:05<00:00, 53.13it/s]
Loading 0:  97%|█████████▋| 282/291 [00:05<00:00, 51.06it/s]
Loading 0:  99%|█████████▉| 288/291 [00:10<00:00,  4.22it/s]
                                                            

Job rica40325-llama3-v2-mkmlizer completed after 83.8s with status: succeeded
Stopping job with name rica40325-llama3-v2-mkmlizer
Pipeline stage MKMLizer completed in 85.21s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.12s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service rica40325-llama3-v2
Waiting for inference service rica40325-llama3-v2 to be ready
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Inference service rica40325-llama3-v2 ready after 140.59524178504944s
Pipeline stage MKMLDeployer completed in 141.18s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.3395116329193115s
Received healthy response to inference request in 2.3010237216949463s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Received healthy response to inference request in 2.3631277084350586s
Received healthy response to inference request in 1.3248224258422852s
Received healthy response to inference request in 1.3856196403503418s
5 requests
0 failed requests
5th percentile: 1.3369818687438966
10th percentile: 1.3491413116455078
20th percentile: 1.3734601974487304
30th percentile: 1.5687004566192626
40th percentile: 1.9348620891571047
50th percentile: 2.3010237216949463
60th percentile: 2.3164188861846924
70th percentile: 2.3318140506744385
80th percentile: 2.344234848022461
90th percentile: 2.35368127822876
95th percentile: 2.3584044933319093
99th percentile: 2.362183065414429
mean time: 1.9428210258483887
Pipeline stage StressChecker completed in 10.64s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
starting trigger_guanaco_pipeline %s
Pipeline stage TriggerMKMLProfilingPipeline completed in 5.41s
rica40325-llama3_v2 status is now deployed due to DeploymentManager action
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage MKMLProfilerDeleter completed in 0.11s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.10s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service rica40325-llama3-v2-profiler
Waiting for inference service rica40325-llama3-v2-profiler to be ready
Inference service rica40325-llama3-v2-profiler ready after 140.3332715034485s
Pipeline stage MKMLProfilerDeployer completed in 140.69s
run pipeline stage %s
Running pipeline stage MKMLProfilerRunner
kubectl cp /code/guanaco/guanaco_inference_services/src/inference_scripts tenant-chaiml-guanaco/rica40325-llama3-v2-profiler-predictor-00001-deployment-fbc8wg6:/code/chaiverse_profiler_1725451720 --namespace tenant-chaiml-guanaco
kubectl exec -it rica40325-llama3-v2-profiler-predictor-00001-deployment-fbc8wg6 --namespace tenant-chaiml-guanaco -- sh -c 'cd /code/chaiverse_profiler_1725451720 && python profiles.py profile --best_of_n 16 --auto_batch 5 --batches 1,5,10,15,20,25,30,35,40,45,50,55,60,65,70,75,80,85,90,95,100,105,110,115,120,125,130,135,140,145,150,155,160,165,170,175,180,185,190,195 --samples 200 --input_tokens 512 --output_tokens 64 --summary /code/chaiverse_profiler_1725451720/summary.json'
Received signal 15, running shutdown handler
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Checking if service rica40325-llama3-v2-profiler is running
Tearing down inference service rica40325-llama3-v2-profiler
Service rica40325-llama3-v2-profiler has been torndown
Pipeline stage MKMLProfilerDeleter completed in 1.56s
Shutdown handler unregistered and original handlers restored
rica40325-llama3_v2 status is now inactive due to auto deactivation removed underperforming models

Usage Metrics

Latency Metrics