developer_uid: Locutusque
submission_id: locutusque-apollo-2-0-ll_3599_v2
model_name: locutusque-apollo-2-0-ll_3599_v2
model_group: Locutusque/Apollo-2.0-Ll
status: torndown
timestamp: 2024-09-01T03:36:14+00:00
num_battles: 12202
num_wins: 5936
celo_rating: 1226.93
family_friendly_score: 0.0
submission_type: basic
model_repo: Locutusque/Apollo-2.0-Llama-3.1-8B
model_architecture: LlamaForCausalLM
model_num_parameters: 8030261248.0
best_of: 6
max_input_tokens: 1024
max_output_tokens: 64
display_name: locutusque-apollo-2-0-ll_3599_v2
is_internal_developer: False
language_model: Locutusque/Apollo-2.0-Llama-3.1-8B
model_size: 8B
ranking_group: single
us_pacific_date: 2024-08-31
win_ratio: 0.4864776266185871
generation_params: {'temperature': 0.8, 'top_p': 0.95, 'min_p': 0.05, 'top_k': 40, 'presence_penalty': 0.05, 'frequency_penalty': 0.0, 'stopping_words': ['<|im_end|>', '<|im_start|>'], 'max_input_tokens': 1024, 'best_of': 6, 'max_output_tokens': 64}
formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': False}
Resubmit model
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name locutusque-apollo-2-0-ll-3599-v2-mkmlizer
Waiting for job on locutusque-apollo-2-0-ll-3599-v2-mkmlizer to finish
Stopping job with name locutusque-apollo-2-0-ll-3599-v2-mkmlizer
%s, retrying in %s seconds...
Starting job with name locutusque-apollo-2-0-ll-3599-v2-mkmlizer
Waiting for job on locutusque-apollo-2-0-ll-3599-v2-mkmlizer to finish
locutusque-apollo-2-0-ll-3599-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
locutusque-apollo-2-0-ll-3599-v2-mkmlizer: ║     _____            __           __                                ║
locutusque-apollo-2-0-ll-3599-v2-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
locutusque-apollo-2-0-ll-3599-v2-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
locutusque-apollo-2-0-ll-3599-v2-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
locutusque-apollo-2-0-ll-3599-v2-mkmlizer: ║       /___/                                                         ║
locutusque-apollo-2-0-ll-3599-v2-mkmlizer: ║                                                                     ║
locutusque-apollo-2-0-ll-3599-v2-mkmlizer: ║  Version: 0.10.1                                                    ║
locutusque-apollo-2-0-ll-3599-v2-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
locutusque-apollo-2-0-ll-3599-v2-mkmlizer: ║  https://mk1.ai                                                     ║
locutusque-apollo-2-0-ll-3599-v2-mkmlizer: ║                                                                     ║
locutusque-apollo-2-0-ll-3599-v2-mkmlizer: ║  The license key for the current software has been verified as      ║
locutusque-apollo-2-0-ll-3599-v2-mkmlizer: ║  belonging to:                                                      ║
locutusque-apollo-2-0-ll-3599-v2-mkmlizer: ║                                                                     ║
locutusque-apollo-2-0-ll-3599-v2-mkmlizer: ║  Chai Research Corp.                                                ║
locutusque-apollo-2-0-ll-3599-v2-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
locutusque-apollo-2-0-ll-3599-v2-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
locutusque-apollo-2-0-ll-3599-v2-mkmlizer: ║                                                                     ║
locutusque-apollo-2-0-ll-3599-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Failed to get response for submission blend_jerun_2024-08-22: ('http://neversleep-noromaid-v0-8068-v150-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', '{"error":"ValueError : [TypeError(\\"\'numpy.int64\' object is not iterable\\"), TypeError(\'vars() argument must have __dict__ attribute\')]"}')
locutusque-apollo-2-0-ll-3599-v2-mkmlizer: Downloaded to shared memory in 32.332s
locutusque-apollo-2-0-ll-3599-v2-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp2e93_1y8, device:0
locutusque-apollo-2-0-ll-3599-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
locutusque-apollo-2-0-ll-3599-v2-mkmlizer: quantized model in 25.529s
locutusque-apollo-2-0-ll-3599-v2-mkmlizer: Processed model Locutusque/Apollo-2.0-Llama-3.1-8B in 57.861s
locutusque-apollo-2-0-ll-3599-v2-mkmlizer: creating bucket guanaco-mkml-models
locutusque-apollo-2-0-ll-3599-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
locutusque-apollo-2-0-ll-3599-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/locutusque-apollo-2-0-ll-3599-v2
locutusque-apollo-2-0-ll-3599-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/locutusque-apollo-2-0-ll-3599-v2/special_tokens_map.json
locutusque-apollo-2-0-ll-3599-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/locutusque-apollo-2-0-ll-3599-v2/config.json
locutusque-apollo-2-0-ll-3599-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/locutusque-apollo-2-0-ll-3599-v2/tokenizer_config.json
locutusque-apollo-2-0-ll-3599-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/locutusque-apollo-2-0-ll-3599-v2/tokenizer.json
locutusque-apollo-2-0-ll-3599-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/locutusque-apollo-2-0-ll-3599-v2/flywheel_model.0.safetensors
locutusque-apollo-2-0-ll-3599-v2-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 5/291 [00:00<00:07, 38.38it/s]
Loading 0:   4%|▍         | 13/291 [00:00<00:04, 58.79it/s]
Loading 0:   7%|▋         | 20/291 [00:00<00:04, 55.37it/s]
Loading 0:   9%|▉         | 26/291 [00:00<00:04, 56.01it/s]
Loading 0:  11%|█         | 32/291 [00:00<00:05, 48.46it/s]
Loading 0:  14%|█▍        | 42/291 [00:00<00:04, 55.01it/s]
Loading 0:  16%|█▋        | 48/291 [00:01<00:05, 41.16it/s]
Loading 0:  19%|█▉        | 55/291 [00:01<00:05, 45.10it/s]
Loading 0:  21%|██        | 61/291 [00:01<00:04, 48.00it/s]
Loading 0:  23%|██▎       | 67/291 [00:01<00:04, 49.49it/s]
Loading 0:  25%|██▌       | 73/291 [00:01<00:04, 44.95it/s]
Loading 0:  27%|██▋       | 79/291 [00:01<00:04, 47.99it/s]
Loading 0:  29%|██▉       | 85/291 [00:01<00:04, 50.65it/s]
Loading 0:  31%|███▏      | 91/291 [00:01<00:04, 45.58it/s]
Loading 0:  33%|███▎      | 97/291 [00:01<00:03, 48.91it/s]
Loading 0:  35%|███▌      | 103/291 [00:02<00:03, 51.58it/s]
Loading 0:  37%|███▋      | 109/291 [00:02<00:04, 36.99it/s]
Loading 0:  39%|███▉      | 114/291 [00:02<00:04, 38.99it/s]
Loading 0:  41%|████      | 120/291 [00:02<00:03, 43.46it/s]
Loading 0:  43%|████▎     | 126/291 [00:02<00:03, 47.05it/s]
Loading 0:  45%|████▌     | 132/291 [00:02<00:03, 44.20it/s]
Loading 0:  48%|████▊     | 139/291 [00:02<00:03, 50.41it/s]
Loading 0:  50%|████▉     | 145/291 [00:03<00:03, 44.27it/s]
Loading 0:  52%|█████▏    | 150/291 [00:03<00:03, 45.24it/s]
Loading 0:  54%|█████▍    | 157/291 [00:03<00:02, 51.28it/s]
Loading 0:  56%|█████▌    | 163/291 [00:03<00:02, 48.88it/s]
Loading 0:  58%|█████▊    | 169/291 [00:03<00:03, 39.10it/s]
Loading 0:  60%|██████    | 175/291 [00:03<00:02, 43.07it/s]
Loading 0:  62%|██████▏   | 181/291 [00:03<00:02, 41.41it/s]
Loading 0:  64%|██████▍   | 186/291 [00:04<00:02, 42.39it/s]
Loading 0:  66%|██████▋   | 193/291 [00:04<00:02, 47.73it/s]
Loading 0:  68%|██████▊   | 199/291 [00:04<00:02, 45.99it/s]
Loading 0:  70%|███████   | 204/291 [00:04<00:01, 46.20it/s]
Loading 0:  73%|███████▎  | 211/291 [00:04<00:01, 51.69it/s]
Loading 0:  75%|███████▍  | 217/291 [00:04<00:01, 47.91it/s]
Loading 0:  76%|███████▋  | 222/291 [00:04<00:01, 47.12it/s]
Loading 0:  78%|███████▊  | 227/291 [00:04<00:01, 38.88it/s]
Loading 0:  80%|███████▉  | 232/291 [00:05<00:01, 39.96it/s]
Loading 0:  82%|████████▏ | 238/291 [00:05<00:01, 43.39it/s]
Loading 0:  84%|████████▍ | 244/291 [00:05<00:01, 42.35it/s]
Loading 0:  86%|████████▌ | 249/291 [00:05<00:00, 42.42it/s]
Loading 0:  88%|████████▊ | 256/291 [00:05<00:00, 47.80it/s]
Loading 0:  90%|████████▉ | 261/291 [00:05<00:00, 48.06it/s]
Loading 0:  91%|█████████▏| 266/291 [00:05<00:00, 40.48it/s]
Loading 0:  94%|█████████▍| 274/291 [00:05<00:00, 48.33it/s]
Loading 0:  96%|█████████▌| 280/291 [00:06<00:00, 44.66it/s]
Loading 0:  98%|█████████▊| 285/291 [00:06<00:00, 45.34it/s]
Loading 0: 100%|█████████▉| 290/291 [00:11<00:00,  3.35it/s]
                                                            

Job locutusque-apollo-2-0-ll-3599-v2-mkmlizer completed after 122.03s with status: succeeded
Stopping job with name locutusque-apollo-2-0-ll-3599-v2-mkmlizer
Pipeline stage MKMLizer completed in 127.85s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.11s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service locutusque-apollo-2-0-ll-3599-v2
Waiting for inference service locutusque-apollo-2-0-ll-3599-v2 to be ready
Connection pool is full, discarding connection: %s. Connection pool size: %s
Connection pool is full, discarding connection: %s. Connection pool size: %s
Inference service locutusque-apollo-2-0-ll-3599-v2 ready after 191.10109043121338s
Pipeline stage MKMLDeployer completed in 191.52s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.8141608238220215s
Received healthy response to inference request in 1.2889888286590576s
Received healthy response to inference request in 1.715489149093628s
Received healthy response to inference request in 2.0189290046691895s
Received healthy response to inference request in 1.6207306385040283s
5 requests
0 failed requests
5th percentile: 1.3553371906280518
10th percentile: 1.421685552597046
20th percentile: 1.5543822765350341
30th percentile: 1.6396823406219483
40th percentile: 1.677585744857788
50th percentile: 1.715489149093628
60th percentile: 1.7549578189849853
70th percentile: 1.7944264888763428
80th percentile: 1.855114459991455
90th percentile: 1.9370217323303223
95th percentile: 1.9779753684997559
99th percentile: 2.0107382774353026
mean time: 1.691659688949585
Pipeline stage StressChecker completed in 9.38s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
starting trigger_guanaco_pipeline %s
Pipeline stage TriggerMKMLProfilingPipeline completed in 4.76s
locutusque-apollo-2-0-ll_3599_v2 status is now deployed due to DeploymentManager action
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.15s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service locutusque-apollo-2-0-ll-3599-v2-profiler
Waiting for inference service locutusque-apollo-2-0-ll-3599-v2-profiler to be ready
Inference service locutusque-apollo-2-0-ll-3599-v2-profiler ready after 190.46710586547852s
Pipeline stage MKMLProfilerDeployer completed in 190.88s
run pipeline stage %s
Running pipeline stage MKMLProfilerRunner
script pods %s
Pipeline stage MKMLProfilerRunner completed in 0.38s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Checking if service locutusque-apollo-2-0-ll-3599-v2-profiler is running
Tearing down inference service locutusque-apollo-2-0-ll-3599-v2-profiler
Service locutusque-apollo-2-0-ll-3599-v2-profiler has been torndown
Pipeline stage MKMLProfilerDeleter completed in 1.96s
locutusque-apollo-2-0-ll_3599_v2 status is now inactive due to auto deactivation removed underperforming models
locutusque-apollo-2-0-ll_3599_v2 status is now torndown due to DeploymentManager action