developer_uid: noumanjavaid
submission_id: riverise-lima-safe-sft_v4
model_name: riverise-lima-safe-sft_v1
model_group: Riverise/lima_safe_sft
status: torndown
timestamp: 2025-02-02T14:04:49+00:00
num_battles: 6860
num_wins: 3236
celo_rating: 1245.79
family_friendly_score: 0.5820000000000001
family_friendly_standard_error: 0.006975327949279518
submission_type: basic
model_repo: Riverise/lima_safe_sft
model_architecture: MistralForCausalLM
model_num_parameters: 12772070400.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
display_name: riverise-lima-safe-sft_v1
is_internal_developer: False
language_model: Riverise/lima_safe_sft
model_size: 13B
ranking_group: single
us_pacific_date: 2025-02-02
win_ratio: 0.4717201166180758
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name riverise-lima-safe-sft-v4-mkmlizer
Waiting for job on riverise-lima-safe-sft-v4-mkmlizer to finish
riverise-lima-safe-sft-v4-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
riverise-lima-safe-sft-v4-mkmlizer: ║     _____            __           __                                ║
riverise-lima-safe-sft-v4-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
riverise-lima-safe-sft-v4-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
riverise-lima-safe-sft-v4-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
riverise-lima-safe-sft-v4-mkmlizer: ║       /___/                                                         ║
riverise-lima-safe-sft-v4-mkmlizer: ║                                                                     ║
riverise-lima-safe-sft-v4-mkmlizer: ║  Version: 0.11.12                                                   ║
riverise-lima-safe-sft-v4-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
riverise-lima-safe-sft-v4-mkmlizer: ║  https://mk1.ai                                                     ║
riverise-lima-safe-sft-v4-mkmlizer: ║                                                                     ║
riverise-lima-safe-sft-v4-mkmlizer: ║  The license key for the current software has been verified as      ║
riverise-lima-safe-sft-v4-mkmlizer: ║  belonging to:                                                      ║
riverise-lima-safe-sft-v4-mkmlizer: ║                                                                     ║
riverise-lima-safe-sft-v4-mkmlizer: ║  Chai Research Corp.                                                ║
riverise-lima-safe-sft-v4-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
riverise-lima-safe-sft-v4-mkmlizer: ║  Expiration: 2025-04-15 23:59:59                                    ║
riverise-lima-safe-sft-v4-mkmlizer: ║                                                                     ║
riverise-lima-safe-sft-v4-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
riverise-lima-safe-sft-v4-mkmlizer: Downloaded to shared memory in 97.517s
riverise-lima-safe-sft-v4-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpcb5jjsky, device:0
riverise-lima-safe-sft-v4-mkmlizer: Saving flywheel model at /dev/shm/model_cache
riverise-lima-safe-sft-v4-mkmlizer: quantized model in 43.926s
riverise-lima-safe-sft-v4-mkmlizer: Processed model Riverise/lima_safe_sft in 141.443s
riverise-lima-safe-sft-v4-mkmlizer: creating bucket guanaco-mkml-models
riverise-lima-safe-sft-v4-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
riverise-lima-safe-sft-v4-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/riverise-lima-safe-sft-v4
riverise-lima-safe-sft-v4-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/riverise-lima-safe-sft-v4/config.json
riverise-lima-safe-sft-v4-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/riverise-lima-safe-sft-v4/special_tokens_map.json
riverise-lima-safe-sft-v4-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/riverise-lima-safe-sft-v4/tokenizer_config.json
riverise-lima-safe-sft-v4-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/riverise-lima-safe-sft-v4/tokenizer.json
riverise-lima-safe-sft-v4-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/riverise-lima-safe-sft-v4/flywheel_model.0.safetensors
riverise-lima-safe-sft-v4-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:16, 21.45it/s]
Loading 0:   3%|▎         | 10/363 [00:00<00:13, 26.48it/s]
Loading 0:   4%|▍         | 14/363 [00:00<00:14, 23.47it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:11, 29.69it/s]
Loading 0:   6%|▋         | 23/363 [00:01<00:17, 19.15it/s]
Loading 0:   7%|▋         | 26/363 [00:01<00:19, 17.67it/s]
Loading 0:   9%|▊         | 31/363 [00:01<00:14, 23.28it/s]
Loading 0:  10%|▉         | 35/363 [00:01<00:13, 24.49it/s]
Loading 0:  11%|█         | 39/363 [00:01<00:12, 25.69it/s]
Loading 0:  12%|█▏        | 42/363 [00:01<00:13, 23.56it/s]
Loading 0:  13%|█▎        | 46/363 [00:01<00:11, 26.56it/s]
Loading 0:  13%|█▎        | 49/363 [00:02<00:11, 27.34it/s]
Loading 0:  14%|█▍        | 52/363 [00:02<00:12, 25.64it/s]
Loading 0:  15%|█▌        | 55/363 [00:02<00:11, 26.33it/s]
Loading 0:  17%|█▋        | 60/363 [00:02<00:10, 27.79it/s]
Loading 0:  17%|█▋        | 63/363 [00:02<00:15, 19.77it/s]
Loading 0:  18%|█▊        | 66/363 [00:02<00:14, 20.43it/s]
Loading 0:  19%|█▉        | 69/363 [00:02<00:13, 22.23it/s]
Loading 0:  20%|█▉        | 72/363 [00:03<00:13, 20.91it/s]
Loading 0:  21%|██        | 77/363 [00:03<00:12, 22.06it/s]
Loading 0:  23%|██▎       | 82/363 [00:03<00:10, 27.16it/s]
Loading 0:  24%|██▎       | 86/363 [00:03<00:11, 24.33it/s]
Loading 0:  25%|██▌       | 91/363 [00:03<00:09, 28.91it/s]
Loading 0:  26%|██▌       | 95/363 [00:03<00:10, 24.89it/s]
Loading 0:  28%|██▊       | 100/363 [00:04<00:08, 29.33it/s]
Loading 0:  29%|██▊       | 104/363 [00:04<00:14, 18.25it/s]
Loading 0:  30%|███       | 109/363 [00:04<00:11, 22.76it/s]
Loading 0:  31%|███       | 113/363 [00:04<00:11, 21.36it/s]
Loading 0:  33%|███▎      | 118/363 [00:04<00:09, 25.78it/s]
Loading 0:  34%|███▎      | 122/363 [00:05<00:10, 23.31it/s]
Loading 0:  35%|███▍      | 127/363 [00:05<00:08, 27.78it/s]
Loading 0:  36%|███▌      | 131/363 [00:05<00:09, 24.68it/s]
Loading 0:  37%|███▋      | 136/363 [00:05<00:07, 28.93it/s]
Loading 0:  39%|███▉      | 141/363 [00:05<00:07, 29.01it/s]
Loading 0:  40%|███▉      | 145/363 [00:06<00:10, 20.39it/s]
Loading 0:  41%|████      | 149/363 [00:06<00:10, 20.02it/s]
Loading 0:  42%|████▏     | 154/363 [00:06<00:08, 24.24it/s]
Loading 0:  44%|████▎     | 158/363 [00:06<00:09, 21.98it/s]
Loading 0:  45%|████▍     | 163/363 [00:06<00:07, 26.32it/s]
Loading 0:  46%|████▌     | 167/363 [00:07<00:08, 23.84it/s]
Loading 0:  47%|████▋     | 172/363 [00:07<00:06, 28.30it/s]
Loading 0:  48%|████▊     | 176/363 [00:07<00:07, 24.94it/s]
Loading 0:  50%|████▉     | 181/363 [00:07<00:06, 28.80it/s]
Loading 0:  51%|█████     | 185/363 [00:07<00:09, 18.82it/s]
Loading 0:  52%|█████▏    | 190/363 [00:07<00:07, 22.91it/s]
Loading 0:  53%|█████▎    | 194/363 [00:08<00:07, 21.73it/s]
Loading 0:  55%|█████▍    | 199/363 [00:08<00:06, 25.54it/s]
Loading 0:  56%|█████▌    | 203/363 [00:08<00:06, 23.24it/s]
Loading 0:  57%|█████▋    | 208/363 [00:08<00:05, 27.71it/s]
Loading 0:  58%|█████▊    | 212/363 [00:08<00:06, 24.46it/s]
Loading 0:  60%|█████▉    | 217/363 [00:08<00:05, 28.86it/s]
Loading 0:  61%|██████    | 222/363 [00:09<00:04, 29.77it/s]
Loading 0:  62%|██████▏   | 226/363 [00:09<00:06, 20.73it/s]
Loading 0:  63%|██████▎   | 230/363 [00:09<00:06, 20.35it/s]
Loading 0:  65%|██████▍   | 235/363 [00:09<00:05, 24.84it/s]
Loading 0:  66%|██████▌   | 239/363 [00:09<00:05, 22.83it/s]
Loading 0:  67%|██████▋   | 244/363 [00:10<00:04, 27.37it/s]
Loading 0:  68%|██████▊   | 248/363 [00:10<00:04, 24.25it/s]
Loading 0:  70%|██████▉   | 253/363 [00:10<00:03, 28.99it/s]
Loading 0:  71%|███████   | 257/363 [00:10<00:04, 25.24it/s]
Loading 0:  72%|███████▏  | 262/363 [00:10<00:03, 29.34it/s]
Loading 0:  73%|███████▎  | 266/363 [00:11<00:04, 19.53it/s]
Loading 0:  75%|███████▍  | 271/363 [00:11<00:03, 24.32it/s]
Loading 0:  76%|███████▌  | 275/363 [00:11<00:03, 22.45it/s]
Loading 0:  77%|███████▋  | 280/363 [00:11<00:03, 27.28it/s]
Loading 0:  78%|███████▊  | 284/363 [00:11<00:03, 24.10it/s]
Loading 0:  80%|███████▉  | 289/363 [00:11<00:02, 28.87it/s]
Loading 0:  81%|████████  | 293/363 [00:12<00:02, 25.26it/s]
Loading 0:  82%|████████▏ | 298/363 [00:12<00:02, 29.21it/s]
Loading 0:  83%|████████▎ | 303/363 [00:12<00:02, 29.37it/s]
Loading 0:  85%|████████▍ | 307/363 [00:12<00:02, 20.48it/s]
Loading 0:  86%|████████▌ | 311/363 [00:12<00:02, 19.98it/s]
Loading 0:  87%|████████▋ | 316/363 [00:13<00:01, 24.47it/s]
Loading 0:  88%|████████▊ | 320/363 [00:13<00:01, 22.12it/s]
Loading 0:  90%|████████▉ | 325/363 [00:13<00:01, 26.60it/s]
Loading 0:  91%|█████████ | 329/363 [00:13<00:01, 23.09it/s]
Loading 0:  92%|█████████▏| 334/363 [00:13<00:01, 27.37it/s]
Loading 0:  93%|█████████▎| 338/363 [00:13<00:01, 24.24it/s]
Loading 0:  94%|█████████▍| 343/363 [00:14<00:00, 28.73it/s]
Loading 0:  96%|█████████▌| 347/363 [00:21<00:08,  1.96it/s]
Loading 0:  96%|█████████▋| 350/363 [00:21<00:05,  2.48it/s]
Loading 0:  97%|█████████▋| 353/363 [00:21<00:03,  3.16it/s]
Loading 0:  98%|█████████▊| 357/363 [00:21<00:01,  4.28it/s]
Loading 0: 100%|█████████▉| 362/363 [00:21<00:00,  6.40it/s]
                                                            

Job riverise-lima-safe-sft-v4-mkmlizer completed after 175.25s with status: succeeded
Stopping job with name riverise-lima-safe-sft-v4-mkmlizer
Pipeline stage MKMLizer completed in 175.70s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service riverise-lima-safe-sft-v4
Waiting for inference service riverise-lima-safe-sft-v4 to be ready
Inference service riverise-lima-safe-sft-v4 ready after 180.641015291214s
Pipeline stage MKMLDeployer completed in 181.06s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.0862529277801514s
Received healthy response to inference request in 1.971837043762207s
Received healthy response to inference request in 1.6516973972320557s
Received healthy response to inference request in 1.9291081428527832s
5 requests
1 failed requests
5th percentile: 1.7071795463562012
10th percentile: 1.7626616954803467
20th percentile: 1.8736259937286377
30th percentile: 1.9376539230346679
40th percentile: 1.9547454833984375
50th percentile: 1.971837043762207
60th percentile: 2.0176033973693848
70th percentile: 2.0633697509765625
80th percentile: 5.696083211898807
90th percentile: 12.915743780136111
95th percentile: 16.52557406425476
99th percentile: 19.41343829154968
mean time: 5.554859972000122
%s, retrying in %s seconds...
Received healthy response to inference request in 1.6294171810150146s
Received healthy response to inference request in 1.8547992706298828s
Received healthy response to inference request in 1.6108601093292236s
Received healthy response to inference request in 1.7064883708953857s
Received healthy response to inference request in 1.6052443981170654s
5 requests
0 failed requests
5th percentile: 1.606367540359497
10th percentile: 1.6074906826019286
20th percentile: 1.609736967086792
30th percentile: 1.6145715236663818
40th percentile: 1.6219943523406983
50th percentile: 1.6294171810150146
60th percentile: 1.660245656967163
70th percentile: 1.6910741329193115
80th percentile: 1.7361505508422852
90th percentile: 1.795474910736084
95th percentile: 1.8251370906829834
99th percentile: 1.8488668346405028
mean time: 1.6813618659973144
Pipeline stage StressChecker completed in 38.67s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.60s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.70s
Shutdown handler de-registered
riverise-lima-safe-sft_v4 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3919.00s
Shutdown handler de-registered
riverise-lima-safe-sft_v4 status is now inactive due to auto deactivation removed underperforming models
riverise-lima-safe-sft_v4 status is now torndown due to DeploymentManager action
ChatRequest
Generation Params
Prompt Formatter
Chat History
ChatMessage 1