epiculous-novaspark

developer_uid: Epiculous

submission_id: epiculous-novaspark_v1

model_name: NovaSpark-ChatML

model_group: Epiculous/NovaSpark

status: torndown

timestamp: 2024-10-14T13:56:50+00:00

num_battles: 8073

num_wins: 3906

celo_rating: 1241.48

family_friendly_score: 0.5852312390924956

family_friendly_standard_error: 0.0056119156075897245

submission_type: basic

model_repo: Epiculous/NovaSpark

model_architecture: LlamaForCausalLM

model_num_parameters: 8030261248.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

display_name: NovaSpark-ChatML

is_internal_developer: False

language_model: Epiculous/NovaSpark

model_size: 8B

ranking_group: single

us_pacific_date: 2024-10-14

win_ratio: 0.483835005574136

generation_params: {'temperature': 1.22, 'top_p': 1.0, 'min_p': 0.075, 'top_k': 50, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|im_end|>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name epiculous-novaspark-v1-mkmlizer
Waiting for job on epiculous-novaspark-v1-mkmlizer to finish
epiculous-novaspark-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
epiculous-novaspark-v1-mkmlizer: ║     _____            __           __                                ║
epiculous-novaspark-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
epiculous-novaspark-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
epiculous-novaspark-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
epiculous-novaspark-v1-mkmlizer: ║       /___/                                                         ║
epiculous-novaspark-v1-mkmlizer: ║                                                                     ║
epiculous-novaspark-v1-mkmlizer: ║  Version: 0.11.12                                                   ║
epiculous-novaspark-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
epiculous-novaspark-v1-mkmlizer: ║  https://mk1.ai                                                     ║
epiculous-novaspark-v1-mkmlizer: ║                                                                     ║
epiculous-novaspark-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
epiculous-novaspark-v1-mkmlizer: ║  belonging to:                                                      ║
epiculous-novaspark-v1-mkmlizer: ║                                                                     ║
epiculous-novaspark-v1-mkmlizer: ║  Chai Research Corp.                                                ║
epiculous-novaspark-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
epiculous-novaspark-v1-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
epiculous-novaspark-v1-mkmlizer: ║                                                                     ║
epiculous-novaspark-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
epiculous-novaspark-v1-mkmlizer: Downloaded to shared memory in 35.390s
epiculous-novaspark-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp8c5yc_ko, device:0
epiculous-novaspark-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
epiculous-novaspark-v1-mkmlizer: quantized model in 26.923s
epiculous-novaspark-v1-mkmlizer: Processed model Epiculous/NovaSpark in 62.314s
epiculous-novaspark-v1-mkmlizer: creating bucket guanaco-mkml-models
epiculous-novaspark-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
epiculous-novaspark-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/epiculous-novaspark-v1
epiculous-novaspark-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/epiculous-novaspark-v1/config.json
epiculous-novaspark-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/epiculous-novaspark-v1/special_tokens_map.json
epiculous-novaspark-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/epiculous-novaspark-v1/tokenizer_config.json
epiculous-novaspark-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/epiculous-novaspark-v1/tokenizer.json
epiculous-novaspark-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/epiculous-novaspark-v1/flywheel_model.0.safetensors
epiculous-novaspark-v1-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 5/291 [00:00<00:08, 32.77it/s]
Loading 0:   5%|▍         | 14/291 [00:00<00:06, 43.26it/s]
Loading 0:   7%|▋         | 21/291 [00:00<00:05, 51.79it/s]
Loading 0:   9%|▉         | 27/291 [00:00<00:05, 52.31it/s]
Loading 0:  11%|█▏        | 33/291 [00:00<00:05, 45.79it/s]
Loading 0:  14%|█▍        | 41/291 [00:00<00:05, 46.84it/s]
Loading 0:  17%|█▋        | 50/291 [00:01<00:04, 49.56it/s]
Loading 0:  20%|█▉        | 58/291 [00:01<00:04, 56.18it/s]
Loading 0:  22%|██▏       | 64/291 [00:01<00:04, 52.03it/s]
Loading 0:  24%|██▍       | 70/291 [00:01<00:04, 52.61it/s]
Loading 0:  26%|██▌       | 76/291 [00:01<00:04, 53.75it/s]
Loading 0:  28%|██▊       | 82/291 [00:01<00:04, 48.16it/s]
Loading 0:  30%|██▉       | 87/291 [00:01<00:06, 30.89it/s]
Loading 0:  32%|███▏      | 94/291 [00:02<00:05, 37.26it/s]
Loading 0:  34%|███▍      | 100/291 [00:02<00:04, 38.86it/s]
Loading 0:  36%|███▌      | 105/291 [00:02<00:04, 39.89it/s]
Loading 0:  38%|███▊      | 111/291 [00:02<00:04, 44.04it/s]
Loading 0:  40%|███▉      | 116/291 [00:02<00:03, 44.17it/s]
Loading 0:  42%|████▏     | 122/291 [00:02<00:04, 40.84it/s]
Loading 0:  45%|████▍     | 130/291 [00:02<00:03, 48.08it/s]
Loading 0:  47%|████▋     | 136/291 [00:03<00:03, 44.02it/s]
Loading 0:  48%|████▊     | 141/291 [00:03<00:03, 42.33it/s]
Loading 0:  51%|█████     | 148/291 [00:03<00:02, 47.87it/s]
Loading 0:  53%|█████▎    | 154/291 [00:03<00:03, 44.04it/s]
Loading 0:  55%|█████▍    | 159/291 [00:03<00:03, 41.92it/s]
Loading 0:  57%|█████▋    | 165/291 [00:03<00:02, 45.25it/s]
Loading 0:  58%|█████▊    | 170/291 [00:03<00:02, 44.04it/s]
Loading 0:  60%|██████    | 175/291 [00:03<00:02, 44.91it/s]
Loading 0:  62%|██████▏   | 180/291 [00:04<00:02, 43.15it/s]
Loading 0:  64%|██████▎   | 185/291 [00:04<00:02, 43.68it/s]
Loading 0:  65%|██████▌   | 190/291 [00:04<00:03, 26.41it/s]
Loading 0:  67%|██████▋   | 194/291 [00:04<00:03, 28.43it/s]
Loading 0:  69%|██████▉   | 202/291 [00:04<00:02, 38.28it/s]
Loading 0:  71%|███████▏  | 208/291 [00:04<00:02, 38.91it/s]
Loading 0:  73%|███████▎  | 213/291 [00:04<00:01, 39.50it/s]
Loading 0:  76%|███████▌  | 220/291 [00:05<00:01, 45.72it/s]
Loading 0:  78%|███████▊  | 226/291 [00:05<00:01, 43.62it/s]
Loading 0:  79%|███████▉  | 231/291 [00:05<00:01, 39.97it/s]
Loading 0:  82%|████████▏ | 238/291 [00:05<00:01, 45.95it/s]
Loading 0:  84%|████████▍ | 244/291 [00:05<00:01, 45.64it/s]
Loading 0:  86%|████████▌ | 249/291 [00:05<00:00, 44.92it/s]
Loading 0:  88%|████████▊ | 256/291 [00:05<00:00, 49.94it/s]
Loading 0:  90%|█████████ | 262/291 [00:06<00:00, 45.70it/s]
Loading 0:  92%|█████████▏| 267/291 [00:06<00:00, 45.44it/s]
Loading 0:  94%|█████████▍| 273/291 [00:06<00:00, 48.41it/s]
Loading 0:  96%|█████████▌| 278/291 [00:06<00:00, 48.28it/s]
Loading 0:  97%|█████████▋| 283/291 [00:06<00:00, 41.75it/s]
Loading 0:  99%|█████████▉| 288/291 [00:11<00:00,  3.06it/s]
                                                            
Job epiculous-novaspark-v1-mkmlizer completed after 83.82s with status: succeeded
Stopping job with name epiculous-novaspark-v1-mkmlizer
Pipeline stage MKMLizer completed in 84.40s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.16s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service epiculous-novaspark-v1
Waiting for inference service epiculous-novaspark-v1 to be ready
Inference service epiculous-novaspark-v1 ready after 180.63041925430298s
Pipeline stage MKMLDeployer completed in 181.18s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.048804521560669s
Received healthy response to inference request in 1.3939740657806396s
Received healthy response to inference request in 1.2712981700897217s
Received healthy response to inference request in 1.5146515369415283s
Received healthy response to inference request in 1.3759126663208008s
5 requests
0 failed requests
5th percentile: 1.2922210693359375
10th percentile: 1.3131439685821533
20th percentile: 1.354989767074585
30th percentile: 1.3795249462127686
40th percentile: 1.386749505996704
50th percentile: 1.3939740657806396
60th percentile: 1.4422450542449952
70th percentile: 1.4905160427093507
80th percentile: 1.6214821338653564
90th percentile: 1.8351433277130127
95th percentile: 1.9419739246368408
99th percentile: 2.0274384021759033
mean time: 1.520928192138672
Pipeline stage StressChecker completed in 9.08s
Shutdown handler de-registered
epiculous-novaspark_v1 status is now deployed due to DeploymentManager action
epiculous-novaspark_v1 status is now inactive due to auto deactivation removed underperforming models
epiculous-novaspark_v1 status is now torndown due to DeploymentManager action