chaiml-lexical-nemo-v4-1k1e5

submission_id: chaiml-lexical-nemo-v4-1k1e5_v16

developer_uid: valentin

best_of: 1

celo_rating: 1210.56

display_name: lexical_nemo_best_of_1

family_friendly_score: 0.5762

family_friendly_standard_error: 0.006988469932681975

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

generation_params: {'temperature': 0.9, 'top_p': 1.0, 'min_p': 0.05, 'top_k': 80, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 1, 'max_output_tokens': 64}

is_internal_developer: True

language_model: ChaiML/Lexical-Nemo-v4-1k1e5

max_input_tokens: 1024

max_output_tokens: 64

model_architecture: MistralForCausalLM

model_group: ChaiML/Lexical-Nemo-v4-1

model_name: lexical_nemo_best_of_1

model_num_parameters: 12772070400.0

model_repo: ChaiML/Lexical-Nemo-v4-1k1e5

model_size: 13B

num_battles: 7883

num_wins: 3602

ranking_group: single

status: inactive

submission_type: basic

timestamp: 2024-10-29T08:56:04+00:00

us_pacific_date: 2024-10-29

win_ratio: 0.4569326398579221

Download Preference Data

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-lexical-nemo-v4-1k1e5-v16-mkmlizer
Waiting for job on chaiml-lexical-nemo-v4-1k1e5-v16-mkmlizer to finish
chaiml-lexical-nemo-v4-1k1e5-v16-mkmlizer: Downloaded to shared memory in 129.900s
chaiml-lexical-nemo-v4-1k1e5-v16-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp4pgwe4sl, device:0
chaiml-lexical-nemo-v4-1k1e5-v16-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-lexical-nemo-v4-1k1e5-v16-mkmlizer: quantized model in 42.070s
chaiml-lexical-nemo-v4-1k1e5-v16-mkmlizer: Processed model ChaiML/Lexical-Nemo-v4-1k1e5 in 171.971s
chaiml-lexical-nemo-v4-1k1e5-v16-mkmlizer: creating bucket guanaco-mkml-models
chaiml-lexical-nemo-v4-1k1e5-v16-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-lexical-nemo-v4-1k1e5-v16-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-lexical-nemo-v4-1k1e5-v16
chaiml-lexical-nemo-v4-1k1e5-v16-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-lexical-nemo-v4-1k1e5-v16/config.json
chaiml-lexical-nemo-v4-1k1e5-v16-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-lexical-nemo-v4-1k1e5-v16/special_tokens_map.json
chaiml-lexical-nemo-v4-1k1e5-v16-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-lexical-nemo-v4-1k1e5-v16/tokenizer_config.json
chaiml-lexical-nemo-v4-1k1e5-v16-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-lexical-nemo-v4-1k1e5-v16/tokenizer.json
chaiml-lexical-nemo-v4-1k1e5-v16-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-lexical-nemo-v4-1k1e5-v16/flywheel_model.0.safetensors
chaiml-lexical-nemo-v4-1k1e5-v16-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|▏         | 5/363 [00:00<00:16, 22.26it/s]
Loading 0:   3%|▎         | 10/363 [00:00<00:12, 27.71it/s]
Loading 0:   4%|▍         | 14/363 [00:00<00:14, 23.96it/s]
Loading 0:   5%|▌         | 19/363 [00:00<00:11, 30.26it/s]
Loading 0:   6%|▋         | 23/363 [00:00<00:14, 23.66it/s]
Loading 0:   7%|▋         | 26/363 [00:01<00:16, 20.87it/s]
Loading 0:   9%|▊         | 31/363 [00:01<00:12, 26.75it/s]
Loading 0:  10%|▉         | 35/363 [00:01<00:11, 28.56it/s]
Loading 0:  11%|█         | 39/363 [00:01<00:10, 29.96it/s]
Loading 0:  12%|█▏        | 43/363 [00:01<00:10, 29.26it/s]
Loading 0:  13%|█▎        | 48/363 [00:01<00:09, 32.13it/s]
Loading 0:  14%|█▍        | 52/363 [00:01<00:10, 30.05it/s]
Loading 0:  15%|█▌        | 56/363 [00:02<00:10, 30.07it/s]
Loading 0:  17%|█▋        | 61/363 [00:02<00:12, 24.37it/s]
Loading 0:  18%|█▊        | 64/363 [00:02<00:14, 20.08it/s]
Loading 0:  20%|█▉        | 71/363 [00:02<00:10, 26.89it/s]
Loading 0:  21%|██        | 75/363 [00:02<00:10, 26.88it/s]
Loading 0:  21%|██▏       | 78/363 [00:02<00:11, 24.81it/s]
Loading 0:  23%|██▎       | 84/363 [00:03<00:09, 29.44it/s]
Loading 0:  24%|██▍       | 88/363 [00:03<00:09, 28.20it/s]
Loading 0:  26%|██▌       | 93/363 [00:03<00:09, 29.82it/s]
Loading 0:  27%|██▋       | 97/363 [00:03<00:09, 28.21it/s]
Loading 0:  28%|██▊       | 101/363 [00:03<00:11, 22.12it/s]
Loading 0:  29%|██▊       | 104/363 [00:04<00:13, 19.85it/s]
Loading 0:  31%|███       | 111/363 [00:04<00:09, 26.49it/s]
Loading 0:  31%|███▏      | 114/363 [00:04<00:10, 24.07it/s]
Loading 0:  33%|███▎      | 118/363 [00:04<00:09, 26.86it/s]
Loading 0:  34%|███▎      | 122/363 [00:04<00:10, 23.80it/s]
Loading 0:  35%|███▍      | 127/363 [00:04<00:08, 28.69it/s]
Loading 0:  36%|███▌      | 131/363 [00:05<00:09, 24.74it/s]
Loading 0:  37%|███▋      | 136/363 [00:05<00:07, 29.60it/s]
Loading 0:  39%|███▉      | 141/363 [00:05<00:07, 29.79it/s]
Loading 0:  40%|███▉      | 145/363 [00:05<00:10, 20.19it/s]
Loading 0:  41%|████      | 148/363 [00:05<00:09, 21.77it/s]
Loading 0:  42%|████▏     | 151/363 [00:05<00:09, 22.16it/s]
Loading 0:  43%|████▎     | 156/363 [00:06<00:08, 25.51it/s]
Loading 0:  44%|████▍     | 159/363 [00:06<00:08, 23.54it/s]
Loading 0:  45%|████▍     | 163/363 [00:06<00:07, 26.65it/s]
Loading 0:  46%|████▌     | 167/363 [00:06<00:08, 23.85it/s]
Loading 0:  47%|████▋     | 172/363 [00:06<00:06, 29.05it/s]
Loading 0:  48%|████▊     | 176/363 [00:06<00:07, 25.32it/s]
Loading 0:  50%|████▉     | 181/363 [00:06<00:06, 30.20it/s]
Loading 0:  51%|█████     | 185/363 [00:07<00:09, 19.11it/s]
Loading 0:  52%|█████▏    | 190/363 [00:07<00:07, 23.62it/s]
Loading 0:  53%|█████▎    | 194/363 [00:07<00:07, 22.57it/s]
Loading 0:  55%|█████▌    | 201/363 [00:07<00:05, 29.23it/s]
Loading 0:  56%|█████▋    | 205/363 [00:07<00:05, 29.02it/s]
Loading 0:  58%|█████▊    | 210/363 [00:08<00:04, 31.43it/s]
Loading 0:  59%|█████▉    | 214/363 [00:08<00:04, 30.44it/s]
Loading 0:  60%|██████    | 218/363 [00:08<00:04, 31.06it/s]
Loading 0:  61%|██████▏   | 223/363 [00:08<00:05, 26.68it/s]
Loading 0:  62%|██████▏   | 226/363 [00:08<00:05, 25.05it/s]
Loading 0:  63%|██████▎   | 230/363 [00:08<00:05, 23.69it/s]
Loading 0:  65%|██████▌   | 237/363 [00:09<00:04, 30.47it/s]
Loading 0:  66%|██████▋   | 241/363 [00:09<00:04, 29.96it/s]
Loading 0:  68%|██████▊   | 246/363 [00:09<00:03, 32.73it/s]
Loading 0:  69%|██████▉   | 250/363 [00:09<00:03, 30.81it/s]
Loading 0:  70%|███████   | 255/363 [00:09<00:03, 32.77it/s]
Loading 0:  71%|███████▏  | 259/363 [00:09<00:03, 30.91it/s]
Loading 0:  72%|███████▏  | 263/363 [00:09<00:04, 24.50it/s]
Loading 0:  73%|███████▎  | 266/363 [00:10<00:04, 21.64it/s]
Loading 0:  75%|███████▌  | 273/363 [00:10<00:03, 28.82it/s]
Loading 0:  76%|███████▋  | 277/363 [00:10<00:03, 27.86it/s]
Loading 0:  78%|███████▊  | 282/363 [00:10<00:02, 30.58it/s]
Loading 0:  79%|███████▉  | 286/363 [00:10<00:02, 29.48it/s]
Loading 0:  80%|████████  | 291/363 [00:10<00:02, 31.54it/s]
Loading 0:  81%|████████▏ | 295/363 [00:11<00:02, 29.97it/s]
Loading 0:  82%|████████▏ | 299/363 [00:11<00:02, 30.68it/s]
Loading 0:  84%|████████▎ | 304/363 [00:11<00:02, 26.41it/s]
Loading 0:  85%|████████▍ | 307/363 [00:11<00:02, 24.75it/s]
Loading 0:  86%|████████▌ | 311/363 [00:11<00:02, 23.75it/s]
Loading 0:  88%|████████▊ | 318/363 [00:11<00:01, 30.29it/s]
Loading 0:  89%|████████▊ | 322/363 [00:12<00:01, 29.16it/s]
Loading 0:  90%|█████████ | 327/363 [00:12<00:01, 31.88it/s]
Loading 0:  91%|█████████ | 331/363 [00:12<00:01, 29.90it/s]
Loading 0:  93%|█████████▎| 336/363 [00:12<00:00, 31.39it/s]
Loading 0:  94%|█████████▎| 340/363 [00:12<00:00, 29.82it/s]
Loading 0:  95%|█████████▍| 344/363 [00:19<00:09,  1.93it/s]
Loading 0:  96%|█████████▌| 348/363 [00:19<00:05,  2.59it/s]
Loading 0:  97%|█████████▋| 353/363 [00:20<00:02,  3.78it/s]
Loading 0:  98%|█████████▊| 357/363 [00:20<00:01,  4.92it/s]
                                                            
Job chaiml-lexical-nemo-v4-1k1e5-v16-mkmlizer completed after 194.68s with status: succeeded
Stopping job with name chaiml-lexical-nemo-v4-1k1e5-v16-mkmlizer
Pipeline stage MKMLizer completed in 195.25s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.18s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-lexical-nemo-v4-1k1e5-v16
Waiting for inference service chaiml-lexical-nemo-v4-1k1e5-v16 to be ready
Inference service chaiml-lexical-nemo-v4-1k1e5-v16 ready after 120.42863464355469s
Pipeline stage MKMLDeployer completed in 121.04s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.8965468406677246s
Received healthy response to inference request in 1.498913049697876s
Received healthy response to inference request in 1.9478373527526855s
Received healthy response to inference request in 1.5129358768463135s
Received healthy response to inference request in 1.4883098602294922s
5 requests
0 failed requests
5th percentile: 1.4904304981231689
10th percentile: 1.4925511360168457
20th percentile: 1.4967924118041993
30th percentile: 1.5017176151275635
40th percentile: 1.5073267459869384
50th percentile: 1.5129358768463135
60th percentile: 1.666380262374878
70th percentile: 1.8198246479034423
80th percentile: 1.9068049430847167
90th percentile: 1.9273211479187011
95th percentile: 1.9375792503356934
99th percentile: 1.945785732269287
mean time: 1.6689085960388184
Pipeline stage StressChecker completed in 9.98s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.84s
Shutdown handler de-registered
chaiml-lexical-nemo-v4-1k1e5_v16 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2016.06s
Shutdown handler de-registered
chaiml-lexical-nemo-v4-1k1e5_v16 status is now inactive due to auto deactivation removed underperforming models