rirv938-llama-8b-mcl-old-1950

developer_uid: robert_irvine

submission_id: rirv938-llama-8b-mcl-old-1950_v1

model_name: rirv938-llama-8b-mcl-old-1950_v1

model_group: rirv938/llama_8b_mcl_old

status: torndown

timestamp: 2024-10-24T21:07:52+00:00

num_battles: 10616

num_wins: 4801

celo_rating: 1198.73

family_friendly_score: 0.5766

family_friendly_standard_error: 0.00698759529452014

submission_type: basic

model_repo: rirv938/llama_8b_mcl_old_1950

model_architecture: LlamaForSequenceClassification

model_num_parameters: 8030261248.0

best_of: 1

max_input_tokens: 256

max_output_tokens: 1

display_name: rirv938-llama-8b-mcl-old-1950_v1

ineligible_reason: max_output_tokens!=64

is_internal_developer: True

language_model: rirv938/llama_8b_mcl_old_1950

model_size: 8B

ranking_group: single

us_pacific_date: 2024-10-24

win_ratio: 0.45224189902034667

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 256, 'best_of': 1, 'max_output_tokens': 1}

formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '', 'truncate_by_message': False}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name rirv938-llama-8b-mcl-old-1950-v1-mkmlizer
Waiting for job on rirv938-llama-8b-mcl-old-1950-v1-mkmlizer to finish
rirv938-llama-8b-mcl-old-1950-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
rirv938-llama-8b-mcl-old-1950-v1-mkmlizer: ║     _____            __           __                                ║
rirv938-llama-8b-mcl-old-1950-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
rirv938-llama-8b-mcl-old-1950-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
rirv938-llama-8b-mcl-old-1950-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
rirv938-llama-8b-mcl-old-1950-v1-mkmlizer: ║       /___/                                                         ║
rirv938-llama-8b-mcl-old-1950-v1-mkmlizer: ║                                                                     ║
rirv938-llama-8b-mcl-old-1950-v1-mkmlizer: ║  Version: 0.11.12                                                   ║
rirv938-llama-8b-mcl-old-1950-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
rirv938-llama-8b-mcl-old-1950-v1-mkmlizer: ║  https://mk1.ai                                                     ║
rirv938-llama-8b-mcl-old-1950-v1-mkmlizer: ║                                                                     ║
rirv938-llama-8b-mcl-old-1950-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
rirv938-llama-8b-mcl-old-1950-v1-mkmlizer: ║  belonging to:                                                      ║
rirv938-llama-8b-mcl-old-1950-v1-mkmlizer: ║                                                                     ║
rirv938-llama-8b-mcl-old-1950-v1-mkmlizer: ║  Chai Research Corp.                                                ║
rirv938-llama-8b-mcl-old-1950-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
rirv938-llama-8b-mcl-old-1950-v1-mkmlizer: ║  Expiration: 2025-01-15 23:59:59                                    ║
rirv938-llama-8b-mcl-old-1950-v1-mkmlizer: ║                                                                     ║
rirv938-llama-8b-mcl-old-1950-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
rirv938-llama-8b-mcl-old-1950-v1-mkmlizer: Downloaded to shared memory in 35.803s
rirv938-llama-8b-mcl-old-1950-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:t0, folder:/tmp/tmpf7ezp8pj, device:0
rirv938-llama-8b-mcl-old-1950-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
rirv938-llama-8b-mcl-old-1950-v1-mkmlizer: quantized model in 84.688s
rirv938-llama-8b-mcl-old-1950-v1-mkmlizer: Processed model rirv938/llama_8b_mcl_old_1950 in 120.492s
rirv938-llama-8b-mcl-old-1950-v1-mkmlizer: creating bucket guanaco-mkml-models
rirv938-llama-8b-mcl-old-1950-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
rirv938-llama-8b-mcl-old-1950-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/rirv938-llama-8b-mcl-old-1950-v1
rirv938-llama-8b-mcl-old-1950-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/rirv938-llama-8b-mcl-old-1950-v1/config.json
rirv938-llama-8b-mcl-old-1950-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/rirv938-llama-8b-mcl-old-1950-v1/special_tokens_map.json
rirv938-llama-8b-mcl-old-1950-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/rirv938-llama-8b-mcl-old-1950-v1/tokenizer_config.json
rirv938-llama-8b-mcl-old-1950-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/rirv938-llama-8b-mcl-old-1950-v1/tokenizer.json
Job rirv938-llama-8b-mcl-old-1950-v1-mkmlizer completed after 145.35s with status: succeeded
Stopping job with name rirv938-llama-8b-mcl-old-1950-v1-mkmlizer
Pipeline stage MKMLizer completed in 146.13s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.20s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service rirv938-llama-8b-mcl-old-1950-v1
Waiting for inference service rirv938-llama-8b-mcl-old-1950-v1 to be ready
Connection pool is full, discarding connection: %s. Connection pool size: %s
Inference service rirv938-llama-8b-mcl-old-1950-v1 ready after 120.85937833786011s
Pipeline stage MKMLDeployer completed in 121.43s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.3728349208831787s
Received healthy response to inference request in 2.256239414215088s
Received healthy response to inference request in 4.776138067245483s
Received healthy response to inference request in 2.2932868003845215s
5 requests
1 failed requests
5th percentile: 2.2636488914489745
10th percentile: 2.271058368682861
20th percentile: 2.285877323150635
30th percentile: 2.309196424484253
40th percentile: 2.341015672683716
50th percentile: 2.3728349208831787
60th percentile: 3.3341561794281005
70th percentile: 4.295477437973022
80th percentile: 7.8518860340118435
90th percentile: 14.003381967544557
95th percentile: 17.07912993431091
99th percentile: 19.539728307723998
mean time: 6.370675420761108
%s, retrying in %s seconds...
Received healthy response to inference request in 3.639519214630127s
Received healthy response to inference request in 4.2997071743011475s
Received healthy response to inference request in 2.382596492767334s
Received healthy response to inference request in 2.2321689128875732s
Received healthy response to inference request in 3.7958884239196777s
5 requests
0 failed requests
5th percentile: 2.2622544288635256
10th percentile: 2.2923399448394775
20th percentile: 2.3525109767913817
30th percentile: 2.6339810371398924
40th percentile: 3.13675012588501
50th percentile: 3.639519214630127
60th percentile: 3.7020668983459473
70th percentile: 3.7646145820617676
80th percentile: 3.896652173995972
90th percentile: 4.098179674148559
95th percentile: 4.198943424224853
99th percentile: 4.279554424285888
mean time: 3.269976043701172
%s, retrying in %s seconds...
Received healthy response to inference request in 2.7563605308532715s
Received healthy response to inference request in 4.4535651206970215s
Received healthy response to inference request in 3.5726382732391357s
Received healthy response to inference request in 1.9921255111694336s
Received healthy response to inference request in 2.667006731033325s
5 requests
0 failed requests
5th percentile: 2.127101755142212
10th percentile: 2.2620779991149904
20th percentile: 2.532030487060547
30th percentile: 2.6848774909973145
40th percentile: 2.720619010925293
50th percentile: 2.7563605308532715
60th percentile: 3.082871627807617
70th percentile: 3.409382724761963
80th percentile: 3.748823642730713
90th percentile: 4.101194381713867
95th percentile: 4.277379751205444
99th percentile: 4.418328046798706
mean time: 3.0883392333984374
Pipeline stage StressChecker completed in 68.19s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 5.60s
Shutdown handler de-registered
rirv938-llama-8b-mcl-old-1950_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3199.47s
Shutdown handler de-registered
rirv938-llama-8b-mcl-old-1950_v1 status is now inactive due to auto deactivation removed underperforming models
rirv938-llama-8b-mcl-old-1950_v1 status is now torndown due to DeploymentManager action