submission_id: rica40325-feedback-and-d_8178_v3
developer_uid: rica40325
alignment_samples: 12125
alignment_score: -0.13707097977332444
best_of: 16
celo_rating: 1237.27
display_name: rica40325-feedbac_8178_v2
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.15, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}
is_internal_developer: False
language_model: rica40325/feedback-and-dpo3kstep
max_input_tokens: 512
max_output_tokens: 64
model_architecture: LlamaForCausalLM
model_group: rica40325/feedback-and-d
model_name: rica40325-feedbac_8178_v2
model_num_parameters: 8030261248.0
model_repo: rica40325/feedback-and-dpo3kstep
model_size: 8B
num_battles: 12125
num_wins: 6149
propriety_score: 0.75
propriety_total_count: 1012.0
ranking_group: single
status: inactive
submission_type: basic
timestamp: 2024-09-01T08:31:30+00:00
us_pacific_date: 2024-09-01
win_ratio: 0.5071340206185567
Download Preference Data
Resubmit model
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name rica40325-feedback-and-d-8178-v3-mkmlizer
Waiting for job on rica40325-feedback-and-d-8178-v3-mkmlizer to finish
Stopping job with name rica40325-feedback-and-d-8178-v3-mkmlizer
%s, retrying in %s seconds...
Starting job with name rica40325-feedback-and-d-8178-v3-mkmlizer
Waiting for job on rica40325-feedback-and-d-8178-v3-mkmlizer to finish
rica40325-feedback-and-d-8178-v3-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
rica40325-feedback-and-d-8178-v3-mkmlizer: ║     _____            __           __                                ║
rica40325-feedback-and-d-8178-v3-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
rica40325-feedback-and-d-8178-v3-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
rica40325-feedback-and-d-8178-v3-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
rica40325-feedback-and-d-8178-v3-mkmlizer: ║       /___/                                                         ║
rica40325-feedback-and-d-8178-v3-mkmlizer: ║                                                                     ║
rica40325-feedback-and-d-8178-v3-mkmlizer: ║  Version: 0.10.1                                                    ║
rica40325-feedback-and-d-8178-v3-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
rica40325-feedback-and-d-8178-v3-mkmlizer: ║  https://mk1.ai                                                     ║
rica40325-feedback-and-d-8178-v3-mkmlizer: ║                                                                     ║
rica40325-feedback-and-d-8178-v3-mkmlizer: ║  The license key for the current software has been verified as      ║
rica40325-feedback-and-d-8178-v3-mkmlizer: ║  belonging to:                                                      ║
rica40325-feedback-and-d-8178-v3-mkmlizer: ║                                                                     ║
rica40325-feedback-and-d-8178-v3-mkmlizer: ║  Chai Research Corp.                                                ║
rica40325-feedback-and-d-8178-v3-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
rica40325-feedback-and-d-8178-v3-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
rica40325-feedback-and-d-8178-v3-mkmlizer: ║                                                                     ║
rica40325-feedback-and-d-8178-v3-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
rica40325-feedback-and-d-8178-v3-mkmlizer: Downloaded to shared memory in 32.139s
rica40325-feedback-and-d-8178-v3-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpkrj85mr3, device:0
rica40325-feedback-and-d-8178-v3-mkmlizer: Saving flywheel model at /dev/shm/model_cache
rica40325-feedback-and-d-8178-v3-mkmlizer: quantized model in 25.571s
rica40325-feedback-and-d-8178-v3-mkmlizer: Processed model rica40325/feedback-and-dpo3kstep in 57.711s
rica40325-feedback-and-d-8178-v3-mkmlizer: creating bucket guanaco-mkml-models
rica40325-feedback-and-d-8178-v3-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/rica40325-feedback-and-d-8178-v3/config.json
rica40325-feedback-and-d-8178-v3-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/rica40325-feedback-and-d-8178-v3/special_tokens_map.json
rica40325-feedback-and-d-8178-v3-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/rica40325-feedback-and-d-8178-v3/tokenizer_config.json
rica40325-feedback-and-d-8178-v3-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/rica40325-feedback-and-d-8178-v3/tokenizer.json
rica40325-feedback-and-d-8178-v3-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/rica40325-feedback-and-d-8178-v3/flywheel_model.0.safetensors
rica40325-feedback-and-d-8178-v3-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 4/291 [00:00<00:07, 37.14it/s]
Loading 0:   4%|▍         | 13/291 [00:00<00:04, 65.36it/s]
Loading 0:   8%|▊         | 22/291 [00:00<00:03, 75.59it/s]
Loading 0:  11%|█         | 31/291 [00:00<00:03, 73.70it/s]
Loading 0:  14%|█▎        | 40/291 [00:00<00:03, 78.84it/s]
Loading 0:  17%|█▋        | 49/291 [00:00<00:02, 81.94it/s]
Loading 0:  21%|██        | 61/291 [00:00<00:02, 82.00it/s]
Loading 0:  24%|██▍       | 70/291 [00:00<00:02, 78.60it/s]
Loading 0:  27%|██▋       | 79/291 [00:01<00:02, 77.78it/s]
Loading 0:  30%|██▉       | 87/291 [00:02<00:09, 21.94it/s]
Loading 0:  32%|███▏      | 94/291 [00:02<00:07, 26.62it/s]
Loading 0:  35%|███▌      | 103/291 [00:02<00:05, 33.90it/s]
Loading 0:  38%|███▊      | 112/291 [00:02<00:04, 41.96it/s]
Loading 0:  42%|████▏     | 121/291 [00:02<00:03, 49.99it/s]
Loading 0:  45%|████▍     | 130/291 [00:02<00:02, 57.04it/s]
Loading 0:  48%|████▊     | 139/291 [00:02<00:02, 60.37it/s]
Loading 0:  51%|█████     | 148/291 [00:02<00:02, 63.42it/s]
Loading 0:  54%|█████▍    | 157/291 [00:02<00:01, 67.86it/s]
Loading 0:  58%|█████▊    | 168/291 [00:03<00:01, 78.19it/s]
Loading 0:  61%|██████    | 177/291 [00:03<00:01, 81.17it/s]
Loading 0:  64%|██████▍   | 186/291 [00:03<00:01, 79.19it/s]
Loading 0:  67%|██████▋   | 195/291 [00:04<00:04, 22.82it/s]
Loading 0:  69%|██████▉   | 202/291 [00:04<00:03, 27.18it/s]
Loading 0:  73%|███████▎  | 213/291 [00:04<00:02, 37.10it/s]
Loading 0:  76%|███████▌  | 221/291 [00:04<00:01, 43.17it/s]
Loading 0:  79%|███████▊  | 229/291 [00:04<00:01, 49.23it/s]
Loading 0:  82%|████████▏ | 238/291 [00:04<00:00, 55.99it/s]
Loading 0:  85%|████████▍ | 247/291 [00:05<00:00, 61.83it/s]
Loading 0:  88%|████████▊ | 256/291 [00:05<00:00, 63.36it/s]
Loading 0:  91%|█████████ | 265/291 [00:05<00:00, 66.81it/s]
Loading 0:  95%|█████████▍| 276/291 [00:05<00:00, 77.09it/s]
Loading 0:  98%|█████████▊| 285/291 [00:05<00:00, 80.30it/s]
                                                            

Job rica40325-feedback-and-d-8178-v3-mkmlizer completed after 76.49s with status: succeeded
Stopping job with name rica40325-feedback-and-d-8178-v3-mkmlizer
Pipeline stage MKMLizer completed in 77.89s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.50s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service rica40325-feedback-and-d-8178-v3
Waiting for inference service rica40325-feedback-and-d-8178-v3 to be ready
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Inference service rica40325-feedback-and-d-8178-v3 ready after 190.80471849441528s
Pipeline stage MKMLDeployer completed in 191.29s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.997446298599243s
Received healthy response to inference request in 2.0584828853607178s
Received healthy response to inference request in 2.1402194499969482s
Received healthy response to inference request in 2.1762351989746094s
Received healthy response to inference request in 1.6541597843170166s
5 requests
0 failed requests
5th percentile: 1.7350244045257568
10th percentile: 1.815889024734497
20th percentile: 1.9776182651519776
30th percentile: 2.074830198287964
40th percentile: 2.1075248241424562
50th percentile: 2.1402194499969482
60th percentile: 2.154625749588013
70th percentile: 2.1690320491790773
80th percentile: 2.3404774188995363
90th percentile: 2.6689618587493897
95th percentile: 2.8332040786743162
99th percentile: 2.964597854614258
mean time: 2.205308723449707
Pipeline stage StressChecker completed in 11.85s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
starting trigger_guanaco_pipeline %s
Pipeline stage TriggerMKMLProfilingPipeline completed in 2.72s
rica40325-feedback-and-d_8178_v3 status is now deployed due to DeploymentManager action
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.13s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service rica40325-feedback-and-d-8178-v3-profiler
Waiting for inference service rica40325-feedback-and-d-8178-v3-profiler to be ready
Inference service rica40325-feedback-and-d-8178-v3-profiler ready after 180.40830993652344s
Pipeline stage MKMLProfilerDeployer completed in 180.77s
run pipeline stage %s
Running pipeline stage MKMLProfilerRunner
script pods %s
Pipeline stage MKMLProfilerRunner completed in 0.34s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Checking if service rica40325-feedback-and-d-8178-v3-profiler is running
Tearing down inference service rica40325-feedback-and-d-8178-v3-profiler
Service rica40325-feedback-and-d-8178-v3-profiler has been torndown
Pipeline stage MKMLProfilerDeleter completed in 1.77s
rica40325-feedback-and-d_8178_v3 status is now inactive due to auto deactivation removed underperforming models

Usage Metrics

Latency Metrics