stark2000s-gpt3

developer_uid: stark2000s

submission_id: stark2000s-gpt3_v1

model_name: stark2000s-gpt3_v1

model_group: stark2000s/gpt3

status: torndown

timestamp: 2024-08-30T08:24:31+00:00

num_battles: 11792

num_wins: 3927

celo_rating: 1111.36

family_friendly_score: 0.0

submission_type: basic

model_repo: stark2000s/gpt3

model_architecture: LlamaForCausalLM

model_num_parameters: 8030261248.0

best_of: 1

max_input_tokens: 512

max_output_tokens: 64

display_name: stark2000s-gpt3_v1

is_internal_developer: False

language_model: stark2000s/gpt3

model_size: 8B

ranking_group: single

us_pacific_date: 2024-08-30

win_ratio: 0.33302238805970147

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 1, 'max_output_tokens': 64}

formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}

Resubmit model

Running pipeline stage MKMLizer
Starting job with name stark2000s-gpt3-v1-mkmlizer
Waiting for job on stark2000s-gpt3-v1-mkmlizer to finish
Stopping job with name stark2000s-gpt3-v1-mkmlizer
%s, retrying in %s seconds...
Starting job with name stark2000s-gpt3-v1-mkmlizer
Waiting for job on stark2000s-gpt3-v1-mkmlizer to finish
stark2000s-gpt3-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
stark2000s-gpt3-v1-mkmlizer: ║     _____            __           __                                ║
stark2000s-gpt3-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
stark2000s-gpt3-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
stark2000s-gpt3-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
stark2000s-gpt3-v1-mkmlizer: ║       /___/                                                         ║
stark2000s-gpt3-v1-mkmlizer: ║                                                                     ║
stark2000s-gpt3-v1-mkmlizer: ║  Version: 0.10.1                                                    ║
stark2000s-gpt3-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
stark2000s-gpt3-v1-mkmlizer: ║  https://mk1.ai                                                     ║
stark2000s-gpt3-v1-mkmlizer: ║                                                                     ║
stark2000s-gpt3-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
stark2000s-gpt3-v1-mkmlizer: ║  belonging to:                                                      ║
stark2000s-gpt3-v1-mkmlizer: ║                                                                     ║
stark2000s-gpt3-v1-mkmlizer: ║  Chai Research Corp.                                                ║
stark2000s-gpt3-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
stark2000s-gpt3-v1-mkmlizer: ║  Expiration: 2024-10-15 23:59:59                                    ║
stark2000s-gpt3-v1-mkmlizer: ║                                                                     ║
stark2000s-gpt3-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
stark2000s-gpt3-v1-mkmlizer: Downloaded to shared memory in 35.707s
stark2000s-gpt3-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmpbd7rnxaz, device:0
stark2000s-gpt3-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
stark2000s-gpt3-v1-mkmlizer: quantized model in 26.393s
stark2000s-gpt3-v1-mkmlizer: Processed model stark2000s/gpt3 in 62.099s
stark2000s-gpt3-v1-mkmlizer: creating bucket guanaco-mkml-models
stark2000s-gpt3-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
stark2000s-gpt3-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/stark2000s-gpt3-v1
stark2000s-gpt3-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/stark2000s-gpt3-v1/config.json
stark2000s-gpt3-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/stark2000s-gpt3-v1/special_tokens_map.json
stark2000s-gpt3-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/stark2000s-gpt3-v1/tokenizer_config.json
stark2000s-gpt3-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/stark2000s-gpt3-v1/tokenizer.json
stark2000s-gpt3-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/stark2000s-gpt3-v1/flywheel_model.0.safetensors
stark2000s-gpt3-v1-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   2%|▏         | 7/291 [00:00<00:05, 48.22it/s]
Loading 0:   8%|▊         | 22/291 [00:00<00:03, 76.30it/s]
Loading 0:  11%|█         | 31/291 [00:00<00:03, 80.31it/s]
Loading 0:  14%|█▎        | 40/291 [00:00<00:03, 78.48it/s]
Loading 0:  17%|█▋        | 49/291 [00:00<00:02, 81.08it/s]
Loading 0:  21%|██        | 61/291 [00:00<00:02, 80.72it/s]
Loading 0:  24%|██▍       | 71/291 [00:00<00:02, 85.42it/s]
Loading 0:  27%|██▋       | 80/291 [00:00<00:02, 85.92it/s]
Loading 0:  31%|███       | 89/291 [00:02<00:09, 21.68it/s]
Loading 0:  33%|███▎      | 97/291 [00:02<00:07, 26.97it/s]
Loading 0:  36%|███▋      | 106/291 [00:02<00:05, 34.13it/s]
Loading 0:  40%|███▉      | 116/291 [00:02<00:04, 43.32it/s]
Loading 0:  45%|████▍     | 130/291 [00:02<00:02, 54.46it/s]
Loading 0:  48%|████▊     | 139/291 [00:02<00:02, 57.04it/s]
Loading 0:  51%|█████     | 148/291 [00:02<00:02, 61.67it/s]
Loading 0:  54%|█████▍    | 157/291 [00:02<00:02, 66.53it/s]
Loading 0:  57%|█████▋    | 166/291 [00:03<00:01, 70.73it/s]
Loading 0:  60%|██████    | 175/291 [00:03<00:01, 73.92it/s]
Loading 0:  64%|██████▎   | 185/291 [00:03<00:01, 80.54it/s]
Loading 0:  67%|██████▋   | 194/291 [00:04<00:04, 21.19it/s]
Loading 0:  69%|██████▉   | 202/291 [00:04<00:03, 25.99it/s]
Loading 0:  73%|███████▎  | 211/291 [00:04<00:02, 32.36it/s]
Loading 0:  76%|███████▌  | 220/291 [00:04<00:01, 38.88it/s]
Loading 0:  79%|███████▊  | 229/291 [00:04<00:01, 46.00it/s]
Loading 0:  82%|████████▏ | 240/291 [00:05<00:00, 57.48it/s]
Loading 0:  86%|████████▌ | 250/291 [00:05<00:00, 60.20it/s]
Loading 0:  89%|████████▉ | 259/291 [00:05<00:00, 60.78it/s]
Loading 0:  92%|█████████▏| 268/291 [00:05<00:00, 63.72it/s]
Loading 0:  95%|█████████▌| 277/291 [00:05<00:00, 67.76it/s]
Loading 0:  98%|█████████▊| 286/291 [00:05<00:00, 72.67it/s]
                                                            
Job stark2000s-gpt3-v1-mkmlizer completed after 83.89s with status: succeeded
Stopping job with name stark2000s-gpt3-v1-mkmlizer
Pipeline stage MKMLizer completed in 85.35s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.08s
Running pipeline stage ISVCDeployer
Creating inference service stark2000s-gpt3-v1
Waiting for inference service stark2000s-gpt3-v1 to be ready
Failed to get response for submission neversleep-noromaid-v0_8068_v133: ('http://neversleep-noromaid-v0-8068-v133-predictor.tenant-chaiml-guanaco.k.chaiverse.com/v1/models/GPT-J-6B-lit-v2:predict', 'read tcp 127.0.0.1:51470->127.0.0.1:8080: read: connection reset by peer\n')
Inference service stark2000s-gpt3-v1 ready after 181.59304022789001s
Pipeline stage ISVCDeployer completed in 182.30s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.5006821155548096s
Received healthy response to inference request in 1.1572496891021729s
Received healthy response to inference request in 1.2662262916564941s
Received healthy response to inference request in 0.6658813953399658s
Received healthy response to inference request in 2.086796998977661s
5 requests
0 failed requests
5th percentile: 0.7641550540924072
10th percentile: 0.8624287128448487
20th percentile: 1.0589760303497315
30th percentile: 1.1790450096130372
40th percentile: 1.2226356506347655
50th percentile: 1.2662262916564941
60th percentile: 1.3600086212158202
70th percentile: 1.4537909507751465
80th percentile: 1.61790509223938
90th percentile: 1.8523510456085206
95th percentile: 1.9695740222930906
99th percentile: 2.063352403640747
mean time: 1.3353672981262208
Pipeline stage StressChecker completed in 8.24s
stark2000s-gpt3_v1 status is now deployed due to DeploymentManager action
stark2000s-gpt3_v1 status is now inactive due to auto deactivation removed underperforming models
stark2000s-gpt3_v1 status is now torndown due to DeploymentManager action