developer_uid: cycy233
submission_id: cycy233-model0_v1
model_name: 486
model_group: cycy233/model0
status: torndown
timestamp: 2025-03-07T09:41:27+00:00
num_battles: 10071
num_wins: 4531
celo_rating: 1242.41
family_friendly_score: 0.5054000000000001
family_friendly_standard_error: 0.007070655415164849
submission_type: basic
model_repo: cycy233/model0
model_architecture: LlamaForCausalLM
model_num_parameters: 8030261248.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
reward_model: default
latencies: [{'batch_size': 1, 'throughput': 0.8437236069145171, 'latency_mean': 1.1851199460029602, 'latency_p50': 1.1798627376556396, 'latency_p90': 1.3190314292907714}, {'batch_size': 4, 'throughput': 1.7791746543354818, 'latency_mean': 2.2402419257164, 'latency_p50': 2.2520097494125366, 'latency_p90': 2.4845108509063722}, {'batch_size': 5, 'throughput': 1.9176216528909056, 'latency_mean': 2.5965245997905733, 'latency_p50': 2.597972273826599, 'latency_p90': 2.9065876483917235}, {'batch_size': 8, 'throughput': 2.1255554197885473, 'latency_mean': 3.7339341390132903, 'latency_p50': 3.744287610054016, 'latency_p90': 4.18029637336731}, {'batch_size': 10, 'throughput': 2.1773395565327367, 'latency_mean': 4.557828276157379, 'latency_p50': 4.609998106956482, 'latency_p90': 5.084023976325989}, {'batch_size': 12, 'throughput': 2.1984814212287462, 'latency_mean': 5.416722635030746, 'latency_p50': 5.395032644271851, 'latency_p90': 6.060475707054138}, {'batch_size': 15, 'throughput': 2.2209473381127225, 'latency_mean': 6.662063980102539, 'latency_p50': 6.6944698095321655, 'latency_p90': 7.485851836204529}]
gpu_counts: {'NVIDIA RTX A5000': 1}
display_name: 486
is_internal_developer: False
language_model: cycy233/model0
model_size: 8B
ranking_group: single
throughput_3p7s: 2.14
us_pacific_date: 2025-03-07
win_ratio: 0.44990566974481183
generation_params: {'temperature': 0.95, 'top_p': 1.0, 'min_p': 0.08, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|eot_id|>'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': "<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\n{bot_name}'s Persona: {memory}\n\n", 'prompt_template': '{prompt}<|eot_id|>', 'bot_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}: {message}<|eot_id|>', 'user_template': '<|start_header_id|>user<|end_header_id|>\n\n{user_name}: {message}<|eot_id|>', 'response_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}:', 'truncate_by_message': False}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name cycy233-model0-v1-mkmlizer
Waiting for job on cycy233-model0-v1-mkmlizer to finish
cycy233-model0-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
cycy233-model0-v1-mkmlizer: ║     _____            __           __                                ║
cycy233-model0-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
cycy233-model0-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
cycy233-model0-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
cycy233-model0-v1-mkmlizer: ║       /___/                                                         ║
cycy233-model0-v1-mkmlizer: ║                                                                     ║
cycy233-model0-v1-mkmlizer: ║  Version: 0.12.8                                                    ║
cycy233-model0-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
cycy233-model0-v1-mkmlizer: ║  https://mk1.ai                                                     ║
cycy233-model0-v1-mkmlizer: ║                                                                     ║
cycy233-model0-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
cycy233-model0-v1-mkmlizer: ║  belonging to:                                                      ║
cycy233-model0-v1-mkmlizer: ║                                                                     ║
cycy233-model0-v1-mkmlizer: ║  Chai Research Corp.                                                ║
cycy233-model0-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
cycy233-model0-v1-mkmlizer: ║  Expiration: 2025-04-15 23:59:59                                    ║
cycy233-model0-v1-mkmlizer: ║                                                                     ║
cycy233-model0-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
cycy233-model0-v1-mkmlizer: Traceback (most recent call last):
cycy233-model0-v1-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_http.py", line 406, in hf_raise_for_status
cycy233-model0-v1-mkmlizer:     response.raise_for_status()
cycy233-model0-v1-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/requests/models.py", line 1024, in raise_for_status
cycy233-model0-v1-mkmlizer:     raise HTTPError(http_error_msg, response=self)
cycy233-model0-v1-mkmlizer: requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/cycy233/model0/resolve/9517dd4dcfa415c0e48389d80d61300d5b445054/model-00002-of-00004.safetensors
cycy233-model0-v1-mkmlizer: The above exception was the direct cause of the following exception:
cycy233-model0-v1-mkmlizer: Traceback (most recent call last):
cycy233-model0-v1-mkmlizer:   File "/code/uploading/mkmlize.py", line 193, in <module>
cycy233-model0-v1-mkmlizer:     cli()
cycy233-model0-v1-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
cycy233-model0-v1-mkmlizer:     return self.main(*args, **kwargs)
cycy233-model0-v1-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1078, in main
cycy233-model0-v1-mkmlizer:     rv = self.invoke(ctx)
cycy233-model0-v1-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
cycy233-model0-v1-mkmlizer:     return _process_result(sub_ctx.command.invoke(sub_ctx))
cycy233-model0-v1-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
cycy233-model0-v1-mkmlizer:     return ctx.invoke(self.callback, **ctx.params)
cycy233-model0-v1-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 783, in invoke
cycy233-model0-v1-mkmlizer:     return __callback(*args, **kwargs)
cycy233-model0-v1-mkmlizer:   File "/code/uploading/mkmlize.py", line 39, in quantize
cycy233-model0-v1-mkmlizer:     temp_folder = download_to_shared_memory(repo_id, revision, hf_auth_token)
cycy233-model0-v1-mkmlizer:   File "/code/uploading/mkmlize.py", line 86, in download_to_shared_memory
cycy233-model0-v1-mkmlizer:     snapshot_download(
cycy233-model0-v1-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
cycy233-model0-v1-mkmlizer:     return fn(*args, **kwargs)
cycy233-model0-v1-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/_snapshot_download.py", line 294, in snapshot_download
cycy233-model0-v1-mkmlizer:     _inner_hf_hub_download(file)
cycy233-model0-v1-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/_snapshot_download.py", line 270, in _inner_hf_hub_download
cycy233-model0-v1-mkmlizer:     return hf_hub_download(
cycy233-model0-v1-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
cycy233-model0-v1-mkmlizer:     return fn(*args, **kwargs)
cycy233-model0-v1-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 840, in hf_hub_download
cycy233-model0-v1-mkmlizer:     return _hf_hub_download_to_local_dir(
cycy233-model0-v1-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1089, in _hf_hub_download_to_local_dir
cycy233-model0-v1-mkmlizer:     _raise_on_head_call_error(head_call_error, force_download, local_files_only)
cycy233-model0-v1-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1482, in _raise_on_head_call_error
cycy233-model0-v1-mkmlizer:     raise head_call_error
cycy233-model0-v1-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1374, in _get_metadata_or_catch_error
cycy233-model0-v1-mkmlizer:     metadata = get_hf_file_metadata(
cycy233-model0-v1-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
cycy233-model0-v1-mkmlizer:     return fn(*args, **kwargs)
cycy233-model0-v1-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1294, in get_hf_file_metadata
cycy233-model0-v1-mkmlizer:     r = _request_wrapper(
cycy233-model0-v1-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 278, in _request_wrapper
cycy233-model0-v1-mkmlizer:     response = _request_wrapper(
cycy233-model0-v1-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 302, in _request_wrapper
cycy233-model0-v1-mkmlizer:     hf_raise_for_status(response)
cycy233-model0-v1-mkmlizer:   File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_http.py", line 454, in hf_raise_for_status
cycy233-model0-v1-mkmlizer:     raise _format(RepositoryNotFoundError, message, response) from e
cycy233-model0-v1-mkmlizer: huggingface_hub.errors.RepositoryNotFoundError: 404 Client Error. (Request ID: Root=1-67cabf5a-02849f11731d27c96f115406;85ef487e-b0df-4256-960f-4ba95df32513)
cycy233-model0-v1-mkmlizer: Repository Not Found for url: https://huggingface.co/cycy233/model0/resolve/9517dd4dcfa415c0e48389d80d61300d5b445054/model-00002-of-00004.safetensors.
cycy233-model0-v1-mkmlizer: Please make sure you specified the correct `repo_id` and `repo_type`.
cycy233-model0-v1-mkmlizer: If you are trying to access a private or gated repo, make sure you are authenticated.
Job cycy233-model0-v1-mkmlizer completed after 34.88s with status: failed
Stopping job with name cycy233-model0-v1-mkmlizer
%s, retrying in %s seconds...
Starting job with name cycy233-model0-v1-mkmlizer
Waiting for job on cycy233-model0-v1-mkmlizer to finish
cycy233-model0-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
cycy233-model0-v1-mkmlizer: ║     _____            __           __                                ║
cycy233-model0-v1-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
cycy233-model0-v1-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
cycy233-model0-v1-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
cycy233-model0-v1-mkmlizer: ║       /___/                                                         ║
cycy233-model0-v1-mkmlizer: ║                                                                     ║
cycy233-model0-v1-mkmlizer: ║  Version: 0.12.8                                                    ║
cycy233-model0-v1-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
cycy233-model0-v1-mkmlizer: ║  https://mk1.ai                                                     ║
cycy233-model0-v1-mkmlizer: ║                                                                     ║
cycy233-model0-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
cycy233-model0-v1-mkmlizer: ║  belonging to:                                                      ║
cycy233-model0-v1-mkmlizer: ║                                                                     ║
cycy233-model0-v1-mkmlizer: ║  Chai Research Corp.                                                ║
cycy233-model0-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
cycy233-model0-v1-mkmlizer: ║  Expiration: 2025-04-15 23:59:59                                    ║
cycy233-model0-v1-mkmlizer: ║                                                                     ║
cycy233-model0-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
cycy233-model0-v1-mkmlizer: Downloaded to shared memory in 20.626s
cycy233-model0-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp1tdb0bor, device:0
cycy233-model0-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
cycy233-model0-v1-mkmlizer: quantized model in 27.000s
cycy233-model0-v1-mkmlizer: Processed model cycy233/model0 in 47.632s
cycy233-model0-v1-mkmlizer: creating bucket guanaco-mkml-models
cycy233-model0-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
cycy233-model0-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/cycy233-model0-v1
cycy233-model0-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/cycy233-model0-v1/config.json
cycy233-model0-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/cycy233-model0-v1/special_tokens_map.json
cycy233-model0-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/cycy233-model0-v1/tokenizer_config.json
cycy233-model0-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/cycy233-model0-v1/tokenizer.json
cycy233-model0-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/cycy233-model0-v1/flywheel_model.0.safetensors
cycy233-model0-v1-mkmlizer: 
Loading 0:   0%|          | 0/291 [00:00<?, ?it/s]
Loading 0:   1%|          | 2/291 [00:04<11:31,  2.39s/it]
Loading 0:   2%|▏         | 6/291 [00:04<03:04,  1.54it/s]
Loading 0:   4%|▍         | 11/291 [00:05<01:21,  3.42it/s]
Loading 0:   5%|▌         | 15/291 [00:05<00:52,  5.26it/s]
Loading 0:   8%|▊         | 22/291 [00:05<00:27,  9.71it/s]
Loading 0:   9%|▉         | 27/291 [00:05<00:19, 13.26it/s]
Loading 0:  11%|█         | 32/291 [00:05<00:14, 17.38it/s]
Loading 0:  13%|█▎        | 38/291 [00:05<00:11, 21.75it/s]
Loading 0:  15%|█▍        | 43/291 [00:05<00:09, 25.17it/s]
Loading 0:  17%|█▋        | 49/291 [00:05<00:07, 31.23it/s]
Loading 0:  19%|█▊        | 54/291 [00:05<00:07, 33.57it/s]
Loading 0:  20%|██        | 59/291 [00:06<00:06, 35.30it/s]
Loading 0:  22%|██▏       | 64/291 [00:06<00:08, 26.26it/s]
Loading 0:  23%|██▎       | 68/291 [00:06<00:07, 28.23it/s]
Loading 0:  25%|██▍       | 72/291 [00:06<00:07, 30.19it/s]
Loading 0:  26%|██▌       | 76/291 [00:06<00:06, 32.24it/s]
Loading 0:  27%|██▋       | 80/291 [00:06<00:06, 32.61it/s]
Loading 0:  29%|██▉       | 85/291 [00:06<00:05, 36.00it/s]
Loading 0:  31%|███       | 89/291 [00:07<00:05, 34.57it/s]
Loading 0:  32%|███▏      | 94/291 [00:07<00:05, 37.73it/s]
Loading 0:  34%|███▎      | 98/291 [00:07<00:05, 36.88it/s]
Loading 0:  35%|███▌      | 103/291 [00:07<00:04, 39.77it/s]
Loading 0:  37%|███▋      | 108/291 [00:07<00:04, 40.01it/s]
Loading 0:  39%|███▉      | 113/291 [00:07<00:04, 40.05it/s]
Loading 0:  41%|████      | 118/291 [00:07<00:04, 42.52it/s]
Loading 0:  42%|████▏     | 123/291 [00:07<00:04, 36.17it/s]
Loading 0:  45%|████▍     | 130/291 [00:08<00:03, 43.65it/s]
Loading 0:  46%|████▋     | 135/291 [00:08<00:03, 42.89it/s]
Loading 0:  48%|████▊     | 140/291 [00:08<00:03, 42.79it/s]
Loading 0:  50%|████▉     | 145/291 [00:08<00:03, 43.68it/s]
Loading 0:  52%|█████▏    | 150/291 [00:08<00:03, 39.81it/s]
Loading 0:  54%|█████▍    | 158/291 [00:08<00:02, 49.28it/s]
Loading 0:  56%|█████▋    | 164/291 [00:08<00:02, 48.69it/s]
Loading 0:  58%|█████▊    | 170/291 [00:09<00:03, 34.27it/s]
Loading 0:  61%|██████    | 177/291 [00:09<00:03, 36.11it/s]
Loading 0:  64%|██████▎   | 185/291 [00:09<00:02, 44.31it/s]
Loading 0:  66%|██████▌   | 191/291 [00:09<00:02, 42.67it/s]
Loading 0:  67%|██████▋   | 196/291 [00:09<00:02, 43.69it/s]
Loading 0:  70%|███████   | 204/291 [00:09<00:01, 44.21it/s]
Loading 0:  73%|███████▎  | 212/291 [00:09<00:01, 50.20it/s]
Loading 0:  75%|███████▍  | 218/291 [00:10<00:01, 43.99it/s]
Loading 0:  77%|███████▋  | 223/291 [00:10<00:01, 43.33it/s]
Loading 0:  79%|███████▉  | 230/291 [00:10<00:01, 48.06it/s]
Loading 0:  81%|████████  | 236/291 [00:10<00:01, 44.71it/s]
Loading 0:  83%|████████▎ | 241/291 [00:10<00:01, 42.57it/s]
Loading 0:  85%|████████▍ | 247/291 [00:10<00:00, 45.91it/s]
Loading 0:  87%|████████▋ | 252/291 [00:10<00:00, 45.29it/s]
Loading 0:  88%|████████▊ | 257/291 [00:10<00:00, 43.62it/s]
Loading 0:  90%|█████████ | 262/291 [00:11<00:00, 43.95it/s]
Loading 0:  92%|█████████▏| 267/291 [00:11<00:00, 26.73it/s]
Loading 0:  94%|█████████▍| 274/291 [00:11<00:00, 34.45it/s]
Loading 0:  96%|█████████▌| 279/291 [00:11<00:00, 36.52it/s]
Loading 0:  98%|█████████▊| 284/291 [00:11<00:00, 38.83it/s]
Loading 0: 100%|█████████▉| 290/291 [00:11<00:00, 37.57it/s]
                                                            

Job cycy233-model0-v1-mkmlizer completed after 85.71s with status: succeeded
Stopping job with name cycy233-model0-v1-mkmlizer
Pipeline stage MKMLizer completed in 121.47s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.16s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service cycy233-model0-v1
Waiting for inference service cycy233-model0-v1 to be ready
Inference service cycy233-model0-v1 ready after 100.36901068687439s
Pipeline stage MKMLDeployer completed in 100.94s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.114110231399536s
Received healthy response to inference request in 1.2943027019500732s
Received healthy response to inference request in 1.2515804767608643s
Received healthy response to inference request in 1.2255895137786865s
Received healthy response to inference request in 1.398331880569458s
5 requests
0 failed requests
5th percentile: 1.230787706375122
10th percentile: 1.2359858989715575
20th percentile: 1.2463822841644288
30th percentile: 1.260124921798706
40th percentile: 1.2772138118743896
50th percentile: 1.2943027019500732
60th percentile: 1.3359143733978271
70th percentile: 1.377526044845581
80th percentile: 1.5414875507354737
90th percentile: 1.827798891067505
95th percentile: 1.9709545612335204
99th percentile: 2.085479097366333
mean time: 1.4567829608917235
Pipeline stage StressChecker completed in 9.06s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.72s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 2.71s
Shutdown handler de-registered
cycy233-model0_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 3686.11s
Shutdown handler de-registered
cycy233-model0_v1 status is now inactive due to auto deactivation removed underperforming models
cycy233-model0_v1 status is now torndown due to DeploymentManager action