submission_id: riverise-model-trained-o_9186_v3
developer_uid: Riverise
alignment_samples: 11627
alignment_score: 0.16759046426048405
best_of: 16
celo_rating: 1228.63
display_name: riverise-model-_daily_v1
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.15, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}
is_internal_developer: False
language_model: Riverise/model_trained_on_sonnet_and_daily
max_input_tokens: 512
max_output_tokens: 64
model_architecture: LlamaForCausalLM
model_group: Riverise/model_trained_o
model_name: riverise-model-_daily_v1
model_num_parameters: 8030261248.0
model_repo: Riverise/model_trained_on_sonnet_and_daily
model_size: 8B
num_battles: 11627
num_wins: 5718
propriety_score: 0.7433888344760039
propriety_total_count: 1021.0
ranking_group: single
status: inactive
submission_type: basic
timestamp: 2024-08-31T07:29:32+00:00
us_pacific_date: 2024-08-31
win_ratio: 0.4917863593360282
Download Preference Data
Resubmit model
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name riverise-model-trained-o-9186-v3-mkmlizer
Waiting for job on riverise-model-trained-o-9186-v3-mkmlizer to finish
riverise-model-trained-o-9186-v3-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
riverise-model-trained-o-9186-v3-mkmlizer: ║     _____            __           __                                ║
riverise-model-trained-o-9186-v3-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
riverise-model-trained-o-9186-v3-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
riverise-model-trained-o-9186-v3-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
riverise-model-trained-o-9186-v3-mkmlizer: ║       /___/                                                         ║
riverise-model-trained-o-9186-v3-mkmlizer: ║                                                                     ║
riverise-model-trained-o-9186-v3-mkmlizer: ║  Version: 0.10.1                                                    ║
riverise-model-trained-o-9186-v3-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
riverise-model-trained-o-9186-v3-mkmlizer: ║  https://mk1.ai                                                     ║
riverise-model-trained-o-9186-v3-mkmlizer: ║                                                                     ║
riverise-model-trained-o-9186-v3-mkmlizer: ║  The license key for the current software has been verified as      ║
riverise-model-trained-o-9186-v3-mkmlizer: ║  belonging to:                                                      ║
riverise-model-trained-o-9186-v3-mkmlizer: ║                                                                     ║
riverise-model-trained-o-9186-v3-mkmlizer: ║  Chai Research Corp.                                                ║
riverise-model-trained-o-9186-v3-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
riverise-model-trained-o-9186-v3-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
riverise-model-trained-o-9186-v3-mkmlizer: ║                                                                     ║
riverise-model-trained-o-9186-v3-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
riverise-model-trained-o-9186-v3-mkmlizer: Downloaded to shared memory in 43.626s
riverise-model-trained-o-9186-v3-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp478oxwbd, device:0
riverise-model-trained-o-9186-v3-mkmlizer: Saving flywheel model at /dev/shm/model_cache
riverise-model-trained-o-9186-v3-mkmlizer: quantized model in 29.087s
riverise-model-trained-o-9186-v3-mkmlizer: Processed model Riverise/model_trained_on_sonnet_and_daily in 72.713s
riverise-model-trained-o-9186-v3-mkmlizer: creating bucket guanaco-mkml-models
riverise-model-trained-o-9186-v3-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
riverise-model-trained-o-9186-v3-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/riverise-model-trained-o-9186-v3
riverise-model-trained-o-9186-v3-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/riverise-model-trained-o-9186-v3/config.json
riverise-model-trained-o-9186-v3-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/riverise-model-trained-o-9186-v3/special_tokens_map.json
riverise-model-trained-o-9186-v3-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/riverise-model-trained-o-9186-v3/tokenizer_config.json
riverise-model-trained-o-9186-v3-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/riverise-model-trained-o-9186-v3/tokenizer.json
riverise-model-trained-o-9186-v3-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/riverise-model-trained-o-9186-v3/flywheel_model.0.safetensors
riverise-model-trained-o-9186-v3-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 5/291 [00:00<00:10, 26.69it/s]
Loading 0:   4%|▍         | 12/291 [00:00<00:07, 37.58it/s]
Loading 0:   5%|▌         | 16/291 [00:00<00:07, 35.25it/s]
Loading 0:   7%|▋         | 21/291 [00:00<00:07, 38.40it/s]
Loading 0:   9%|▊         | 25/291 [00:00<00:07, 35.45it/s]
Loading 0:  11%|█         | 32/291 [00:00<00:06, 39.02it/s]
Loading 0:  12%|█▏        | 36/291 [00:01<00:10, 24.81it/s]
Loading 0:  14%|█▍        | 41/291 [00:01<00:09, 26.56it/s]
Loading 0:  16%|█▋        | 48/291 [00:01<00:07, 33.14it/s]
Loading 0:  18%|█▊        | 52/291 [00:01<00:07, 32.70it/s]
Loading 0:  20%|█▉        | 57/291 [00:01<00:06, 35.40it/s]
Loading 0:  21%|██        | 61/291 [00:01<00:06, 33.39it/s]
Loading 0:  23%|██▎       | 66/291 [00:01<00:06, 34.78it/s]
Loading 0:  24%|██▍       | 70/291 [00:02<00:06, 33.07it/s]
Loading 0:  25%|██▌       | 74/291 [00:02<00:06, 34.06it/s]
Loading 0:  27%|██▋       | 78/291 [00:02<00:06, 33.95it/s]
Loading 0:  28%|██▊       | 82/291 [00:02<00:08, 24.35it/s]
Loading 0:  29%|██▉       | 85/291 [00:02<00:08, 25.37it/s]
Loading 0:  31%|███       | 90/291 [00:02<00:06, 30.07it/s]
Loading 0:  32%|███▏      | 94/291 [00:02<00:06, 30.31it/s]
Loading 0:  34%|███▍      | 99/291 [00:03<00:05, 34.16it/s]
Loading 0:  35%|███▌      | 103/291 [00:03<00:05, 32.58it/s]
Loading 0:  37%|███▋      | 108/291 [00:03<00:05, 34.51it/s]
Loading 0:  38%|███▊      | 112/291 [00:03<00:05, 32.53it/s]
Loading 0:  40%|███▉      | 116/291 [00:03<00:05, 32.42it/s]
Loading 0:  42%|████▏     | 122/291 [00:03<00:04, 37.62it/s]
Loading 0:  44%|████▎     | 127/291 [00:03<00:04, 34.89it/s]
Loading 0:  46%|████▌     | 133/291 [00:04<00:05, 30.10it/s]
Loading 0:  47%|████▋     | 137/291 [00:04<00:05, 29.42it/s]
Loading 0:  48%|████▊     | 141/291 [00:04<00:05, 27.23it/s]
Loading 0:  51%|█████     | 147/291 [00:04<00:04, 32.43it/s]
Loading 0:  52%|█████▏    | 151/291 [00:04<00:04, 30.91it/s]
Loading 0:  54%|█████▎    | 156/291 [00:04<00:04, 33.14it/s]
Loading 0:  55%|█████▍    | 160/291 [00:05<00:04, 32.22it/s]
Loading 0:  57%|█████▋    | 165/291 [00:05<00:03, 33.69it/s]
Loading 0:  58%|█████▊    | 169/291 [00:05<00:03, 31.17it/s]
Loading 0:  60%|█████▉    | 174/291 [00:05<00:03, 32.46it/s]
Loading 0:  61%|██████    | 178/291 [00:05<00:03, 30.79it/s]
Loading 0:  63%|██████▎   | 183/291 [00:05<00:03, 34.19it/s]
Loading 0:  64%|██████▍   | 187/291 [00:05<00:04, 24.62it/s]
Loading 0:  65%|██████▌   | 190/291 [00:06<00:04, 24.23it/s]
Loading 0:  67%|██████▋   | 194/291 [00:06<00:03, 24.49it/s]
Loading 0:  69%|██████▉   | 201/291 [00:06<00:02, 32.88it/s]
Loading 0:  70%|███████   | 205/291 [00:06<00:02, 31.50it/s]
Loading 0:  72%|███████▏  | 210/291 [00:06<00:02, 34.55it/s]
Loading 0:  74%|███████▎  | 214/291 [00:06<00:02, 33.99it/s]
Loading 0:  75%|███████▌  | 219/291 [00:06<00:01, 36.99it/s]
Loading 0:  77%|███████▋  | 223/291 [00:07<00:01, 34.63it/s]
Loading 0:  78%|███████▊  | 227/291 [00:07<00:01, 34.65it/s]
Loading 0:  79%|███████▉  | 231/291 [00:07<00:01, 34.47it/s]
Loading 0:  81%|████████  | 235/291 [00:07<00:02, 24.92it/s]
Loading 0:  82%|████████▏ | 239/291 [00:07<00:02, 24.07it/s]
Loading 0:  85%|████████▍ | 246/291 [00:07<00:01, 31.36it/s]
Loading 0:  86%|████████▌ | 250/291 [00:07<00:01, 31.64it/s]
Loading 0:  88%|████████▊ | 255/291 [00:08<00:01, 35.04it/s]
Loading 0:  89%|████████▉ | 259/291 [00:08<00:00, 33.74it/s]
Loading 0:  91%|█████████ | 264/291 [00:08<00:00, 37.05it/s]
Loading 0:  92%|█████████▏| 268/291 [00:08<00:00, 35.26it/s]
Loading 0:  94%|█████████▍| 273/291 [00:08<00:00, 37.62it/s]
Loading 0:  95%|█████████▌| 277/291 [00:08<00:00, 34.26it/s]
Loading 0:  97%|█████████▋| 281/291 [00:08<00:00, 34.26it/s]
Loading 0:  98%|█████████▊| 286/291 [00:14<00:01,  2.60it/s]
Loading 0:  99%|█████████▉| 289/291 [00:14<00:00,  3.22it/s]
                                                            

Job riverise-model-trained-o-9186-v3-mkmlizer completed after 94.83s with status: succeeded
Stopping job with name riverise-model-trained-o-9186-v3-mkmlizer
Pipeline stage MKMLizer completed in 96.15s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.11s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service riverise-model-trained-o-9186-v3
Waiting for inference service riverise-model-trained-o-9186-v3 to be ready
Failed to get response for submission blend_berib_2024-08-16: ('http://zonemercy-graft-cogent-v-7573-v6-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '{"error":"ValueError : [TypeError(\\"\'numpy.int64\' object is not iterable\\"), TypeError(\'vars() argument must have __dict__ attribute\')]"}')
Inference service riverise-model-trained-o-9186-v3 ready after 190.73693823814392s
Pipeline stage MKMLDeployer completed in 191.21s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.502110719680786s
Received healthy response to inference request in 1.9549040794372559s
Received healthy response to inference request in 1.5458648204803467s
Received healthy response to inference request in 1.811842679977417s
Received healthy response to inference request in 2.0789756774902344s
5 requests
0 failed requests
5th percentile: 1.5990603923797608
10th percentile: 1.6522559642791748
20th percentile: 1.7586471080780028
30th percentile: 1.8404549598693847
40th percentile: 1.8976795196533203
50th percentile: 1.9549040794372559
60th percentile: 2.004532718658447
70th percentile: 2.0541613578796385
80th percentile: 2.163602685928345
90th percentile: 2.3328567028045653
95th percentile: 2.4174837112426757
99th percentile: 2.4851853179931642
mean time: 1.978739595413208
Pipeline stage StressChecker completed in 10.93s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
starting trigger_guanaco_pipeline %s
Pipeline stage TriggerMKMLProfilingPipeline completed in 5.68s
riverise-model-trained-o_9186_v3 status is now deployed due to DeploymentManager action
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service riverise-model-trained-o-9186-v3-profiler
Waiting for inference service riverise-model-trained-o-9186-v3-profiler to be ready
Inference service riverise-model-trained-o-9186-v3-profiler ready after 190.45914578437805s
Pipeline stage MKMLProfilerDeployer completed in 190.82s
run pipeline stage %s
Running pipeline stage MKMLProfilerRunner
script pods %s
Pipeline stage MKMLProfilerRunner completed in 0.37s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Checking if service riverise-model-trained-o-9186-v3-profiler is running
Tearing down inference service riverise-model-trained-o-9186-v3-profiler
Service riverise-model-trained-o-9186-v3-profiler has been torndown
Pipeline stage MKMLProfilerDeleter completed in 1.95s
riverise-model-trained-o_9186_v3 status is now inactive due to auto deactivation removed underperforming models

Usage Metrics

Latency Metrics