deverdever-heavenly-mouse-v19

developer_uid: clover0103

submission_id: deverdever-heavenly-mouse-v19_v1

model_name: deverdever-heavenly-mouse-v19_v1

model_group: DeverDever/heavenly-mous

status: torndown

timestamp: 2024-09-03T12:20:31+00:00

num_battles: 11540

num_wins: 5695

celo_rating: 1232.32

family_friendly_score: 0.0

submission_type: basic

model_repo: DeverDever/heavenly-mouse-v19

model_architecture: LlamaForCausalLM

model_num_parameters: 8030261248.0

best_of: 16

max_input_tokens: 512

max_output_tokens: 64

display_name: deverdever-heavenly-mouse-v19_v1

is_internal_developer: False

language_model: DeverDever/heavenly-mouse-v19

model_size: 8B

ranking_group: single

us_pacific_date: 2024-09-03

win_ratio: 0.4935008665511265

generation_params: {'temperature': 0.9, 'top_p': 0.75, 'min_p': 0.2, 'top_k': 30, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|end_header_id|>', '<|eot_id|>'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}

formatter: {'memory_template': "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n{bot_name}'s Persona: {memory}\n\n", 'prompt_template': '{prompt}<|eot_id|>', 'bot_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}: {message}<|eot_id|>', 'user_template': '<|start_header_id|>user<|end_header_id|>\n\n{user_name}: {message}<|eot_id|>', 'response_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}:', 'truncate_by_message': False}

Resubmit model

run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name deverdever-heavenly-mouse-v19-v1-mkmlizer
Waiting for job on deverdever-heavenly-mouse-v19-v1-mkmlizer to finish
deverdever-heavenly-mouse-v19-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
deverdever-heavenly-mouse-v19-v1-mkmlizer: ║     _____            __           __                                ║
deverdever-heavenly-mouse-v19-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
deverdever-heavenly-mouse-v19-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
deverdever-heavenly-mouse-v19-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
deverdever-heavenly-mouse-v19-v1-mkmlizer: ║       /___/                                                         ║
deverdever-heavenly-mouse-v19-v1-mkmlizer: ║                                                                     ║
deverdever-heavenly-mouse-v19-v1-mkmlizer: ║  Version: 0.10.1                                                    ║
deverdever-heavenly-mouse-v19-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
deverdever-heavenly-mouse-v19-v1-mkmlizer: ║  https://mk1.ai                                                     ║
deverdever-heavenly-mouse-v19-v1-mkmlizer: ║                                                                     ║
deverdever-heavenly-mouse-v19-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
deverdever-heavenly-mouse-v19-v1-mkmlizer: ║  belonging to:                                                      ║
deverdever-heavenly-mouse-v19-v1-mkmlizer: ║                                                                     ║
deverdever-heavenly-mouse-v19-v1-mkmlizer: ║  Chai Research Corp.                                                ║
deverdever-heavenly-mouse-v19-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
deverdever-heavenly-mouse-v19-v1-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
deverdever-heavenly-mouse-v19-v1-mkmlizer: ║                                                                     ║
deverdever-heavenly-mouse-v19-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
deverdever-heavenly-mouse-v19-v1-mkmlizer: Downloaded to shared memory in 42.888s
deverdever-heavenly-mouse-v19-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp2avsb4es, device:0
deverdever-heavenly-mouse-v19-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
deverdever-heavenly-mouse-v19-v1-mkmlizer: quantized model in 26.173s
deverdever-heavenly-mouse-v19-v1-mkmlizer: Processed model DeverDever/heavenly-mouse-v19 in 69.061s
deverdever-heavenly-mouse-v19-v1-mkmlizer: creating bucket guanaco-mkml-models
deverdever-heavenly-mouse-v19-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
deverdever-heavenly-mouse-v19-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/deverdever-heavenly-mouse-v19-v1
deverdever-heavenly-mouse-v19-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/deverdever-heavenly-mouse-v19-v1/config.json
deverdever-heavenly-mouse-v19-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/deverdever-heavenly-mouse-v19-v1/special_tokens_map.json
deverdever-heavenly-mouse-v19-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/deverdever-heavenly-mouse-v19-v1/tokenizer_config.json
deverdever-heavenly-mouse-v19-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/deverdever-heavenly-mouse-v19-v1/tokenizer.json
deverdever-heavenly-mouse-v19-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/deverdever-heavenly-mouse-v19-v1/flywheel_model.0.safetensors
deverdever-heavenly-mouse-v19-v1-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   1%|          | 2/291 [00:04<11:28,  2.38s/it]
Loading 0:   2%|▏         | 6/291 [00:04<03:03,  1.55it/s]
Loading 0:   4%|▍         | 13/291 [00:05<01:05,  4.24it/s]
Loading 0:   7%|▋         | 20/291 [00:05<00:35,  7.56it/s]
Loading 0:   9%|▊         | 25/291 [00:05<00:27,  9.54it/s]
Loading 0:  11%|█         | 32/291 [00:05<00:18, 14.36it/s]
Loading 0:  13%|█▎        | 38/291 [00:05<00:13, 18.44it/s]
Loading 0:  15%|█▍        | 43/291 [00:05<00:11, 22.10it/s]
Loading 0:  17%|█▋        | 49/291 [00:05<00:08, 27.42it/s]
Loading 0:  19%|█▊        | 54/291 [00:05<00:07, 30.90it/s]
Loading 0:  21%|██        | 60/291 [00:06<00:06, 33.15it/s]
Loading 0:  24%|██▎       | 69/291 [00:06<00:05, 39.78it/s]
Loading 0:  26%|██▋       | 77/291 [00:06<00:04, 47.58it/s]
Loading 0:  29%|██▊       | 83/291 [00:06<00:05, 36.60it/s]
Loading 0:  30%|███       | 88/291 [00:06<00:05, 38.25it/s]
Loading 0:  33%|███▎      | 95/291 [00:06<00:04, 44.72it/s]
Loading 0:  35%|███▍      | 101/291 [00:07<00:04, 43.13it/s]
Loading 0:  36%|███▋      | 106/291 [00:07<00:04, 42.09it/s]
Loading 0:  38%|███▊      | 112/291 [00:07<00:03, 46.17it/s]
Loading 0:  40%|████      | 117/291 [00:07<00:03, 46.58it/s]
Loading 0:  42%|████▏     | 123/291 [00:07<00:03, 43.98it/s]
Loading 0:  45%|████▌     | 132/291 [00:07<00:03, 46.30it/s]
Loading 0:  48%|████▊     | 140/291 [00:07<00:02, 52.07it/s]
Loading 0:  50%|█████     | 146/291 [00:08<00:03, 38.07it/s]
Loading 0:  52%|█████▏    | 151/291 [00:08<00:03, 38.68it/s]
Loading 0:  54%|█████▍    | 158/291 [00:08<00:02, 45.10it/s]
Loading 0:  56%|█████▋    | 164/291 [00:08<00:02, 44.58it/s]
Loading 0:  58%|█████▊    | 169/291 [00:08<00:02, 43.62it/s]
Loading 0:  60%|██████    | 176/291 [00:08<00:02, 48.06it/s]
Loading 0:  63%|██████▎   | 182/291 [00:08<00:02, 44.93it/s]
Loading 0:  64%|██████▍   | 187/291 [00:08<00:02, 43.56it/s]
Loading 0:  66%|██████▋   | 193/291 [00:09<00:02, 46.69it/s]
Loading 0:  68%|██████▊   | 198/291 [00:09<00:01, 46.74it/s]
Loading 0:  70%|██████▉   | 203/291 [00:09<00:02, 37.16it/s]
Loading 0:  72%|███████▏  | 209/291 [00:09<00:02, 36.81it/s]
Loading 0:  73%|███████▎  | 213/291 [00:09<00:02, 35.43it/s]
Loading 0:  76%|███████▌  | 221/291 [00:09<00:01, 45.62it/s]
Loading 0:  78%|███████▊  | 227/291 [00:09<00:01, 46.11it/s]
Loading 0:  80%|███████▉  | 232/291 [00:10<00:01, 44.30it/s]
Loading 0:  82%|████████▏ | 239/291 [00:10<00:01, 49.35it/s]
Loading 0:  84%|████████▍ | 245/291 [00:10<00:01, 45.81it/s]
Loading 0:  86%|████████▌ | 250/291 [00:10<00:00, 44.10it/s]
Loading 0:  88%|████████▊ | 257/291 [00:10<00:00, 49.11it/s]
Loading 0:  90%|█████████ | 263/291 [00:10<00:00, 46.94it/s]
Loading 0:  92%|█████████▏| 268/291 [00:10<00:00, 35.95it/s]
Loading 0:  95%|█████████▍| 276/291 [00:11<00:00, 39.89it/s]
Loading 0:  98%|█████████▊| 284/291 [00:11<00:00, 46.82it/s]
Loading 0: 100%|█████████▉| 290/291 [00:11<00:00, 42.99it/s]
                                                            
Job deverdever-heavenly-mouse-v19-v1-mkmlizer completed after 95.46s with status: succeeded
Stopping job with name deverdever-heavenly-mouse-v19-v1-mkmlizer
Pipeline stage MKMLizer completed in 96.63s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.11s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service deverdever-heavenly-mouse-v19-v1
Waiting for inference service deverdever-heavenly-mouse-v19-v1 to be ready
Inference service deverdever-heavenly-mouse-v19-v1 ready after 150.94542121887207s
Pipeline stage MKMLDeployer completed in 151.47s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.076876163482666s
Received healthy response to inference request in 1.98378586769104s
Received healthy response to inference request in 1.9758117198944092s
Received healthy response to inference request in 1.6053872108459473s
Received healthy response to inference request in 1.7967283725738525s
5 requests
0 failed requests
5th percentile: 1.6436554431915282
10th percentile: 1.6819236755371094
20th percentile: 1.7584601402282716
30th percentile: 1.8325450420379639
40th percentile: 1.9041783809661865
50th percentile: 1.9758117198944092
60th percentile: 1.9790013790130616
70th percentile: 1.9821910381317138
80th percentile: 2.0024039268493654
90th percentile: 2.0396400451660157
95th percentile: 2.0582581043243406
99th percentile: 2.0731525516510008
mean time: 1.887717866897583
Pipeline stage StressChecker completed in 10.34s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
starting trigger_guanaco_pipeline %s
Pipeline stage TriggerMKMLProfilingPipeline completed in 6.13s
deverdever-heavenly-mouse-v19_v1 status is now deployed due to DeploymentManager action
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage MKMLProfilerDeleter completed in 0.12s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.12s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service deverdever-heavenly-mouse-v19-v1-profiler
Waiting for inference service deverdever-heavenly-mouse-v19-v1-profiler to be ready
Inference service deverdever-heavenly-mouse-v19-v1-profiler ready after 150.3466980457306s
Pipeline stage MKMLProfilerDeployer completed in 150.78s
run pipeline stage %s
Running pipeline stage MKMLProfilerRunner
kubectl cp /code/guanaco/guanaco_inference_services/src/inference_scripts tenant-chaiml-guanaco/deverdever-heavenly-e058319a1add08213ae86cfa15aab439-deploth4xf:/code/chaiverse_profiler_1725366506 --namespace tenant-chaiml-guanaco
kubectl exec -it deverdever-heavenly-e058319a1add08213ae86cfa15aab439-deploth4xf --namespace tenant-chaiml-guanaco -- sh -c 'cd /code/chaiverse_profiler_1725366506 && chmod +x profiles.py && python profiles.py profile --best_of_n 16 --batches 1,5,10,15,20,25,30,35,40,45,50,55,60,65,70,75,80,85,90,95,100,105,110,115,120,125,130,135,140,145,150,155,160,165,170,175,180,185,190,195 --samples 200 --input_tokens 512 --output_tokens 64 --summary /code/chaiverse_profiler_1725366506/summary.json'
deverdever-heavenly-mouse-v19_v1 status is now inactive due to auto deactivation removed underperforming models
deverdever-heavenly-mouse-v19_v1 status is now torndown due to DeploymentManager action