submission_id: ayalexf-graftmaxx-l3-8b_v2
developer_uid: Ayalexf
status: inactive
model_repo: Ayalexf/Graftmaxx-L3-8B
reward_repo: ChaiML/reward_gpt2_medium_preference_24m_e2
generation_params: {'temperature': 1.12, 'top_p': 1.0, 'min_p': 0.075, 'top_k': 50, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 4, 'max_output_tokens': 64}
formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
reward_formatter: {'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'bot_template': '{bot_name}: {message}\n', 'user_template': '{user_name}: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': False}
timestamp: 2024-06-16T16:18:57+00:00
model_name: ayalexf-graftmaxx-l3-8b_v2
model_eval_status: success
model_group: Ayalexf/Graftmaxx-L3-8B
num_battles: 23886
num_wins: 12251
celo_rating: 1182.26
propriety_score: 0.7099785867237688
propriety_total_count: 11675.0
submission_type: basic
model_architecture: LlamaForCausalLM
model_num_parameters: 8030261248.0
best_of: 4
max_input_tokens: 512
max_output_tokens: 64
display_name: ayalexf-graftmaxx-l3-8b_v2
ineligible_reason: None
language_model: Ayalexf/Graftmaxx-L3-8B
model_size: 8B
reward_model: ChaiML/reward_gpt2_medium_preference_24m_e2
us_pacific_date: 2024-06-16
win_ratio: 0.5128945826006865
Resubmit model
Running pipeline stage MKMLizer
Starting job with name ayalexf-graftmaxx-l3-8b-v2-mkmlizer
Waiting for job on ayalexf-graftmaxx-l3-8b-v2-mkmlizer to finish
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ _____ __ __ ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ /___/ ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ Version: 0.8.14 ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ https://mk1.ai ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ The license key for the current software has been verified as ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ belonging to: ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ Chai Research Corp. ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ Expiration: 2024-07-15 23:59:59 ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: /opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_deprecation.py:131: FutureWarning: 'list_files_info' (from 'huggingface_hub.hf_api') is deprecated and will be removed from version '0.23'. Use `list_repo_tree` and `get_paths_info` instead.
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: warnings.warn(warning_message, FutureWarning)
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: Downloaded to shared memory in 52.082s
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: quantizing model to /dev/shm/model_cache
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: Loading 0: 0%| | 0/291 [00:00<?, ?it/s] Loading 0: 1%| | 2/291 [00:05<12:30, 2.60s/it] Loading 0: 4%|▍ | 13/291 [00:05<01:24, 3.30it/s] Loading 0: 8%|▊ | 23/291 [00:05<00:39, 6.81it/s] Loading 0: 11%|█▏ | 33/291 [00:05<00:22, 11.29it/s] Loading 0: 14%|█▍ | 42/291 [00:05<00:15, 16.41it/s] Loading 0: 18%|█▊ | 51/291 [00:05<00:10, 22.64it/s] Loading 0: 21%|██ | 60/291 [00:06<00:10, 22.72it/s] Loading 0: 24%|██▍ | 70/291 [00:06<00:07, 30.84it/s] Loading 0: 27%|██▋ | 79/291 [00:06<00:05, 38.54it/s] Loading 0: 31%|███ | 90/291 [00:06<00:04, 49.33it/s] Loading 0: 34%|███▍ | 99/291 [00:06<00:03, 56.60it/s] Loading 0: 37%|███▋ | 109/291 [00:06<00:02, 64.56it/s] Loading 0: 41%|████ | 119/291 [00:06<00:02, 72.33it/s] Loading 0: 45%|████▍ | 130/291 [00:06<00:02, 79.09it/s] Loading 0: 48%|████▊ | 140/291 [00:06<00:01, 81.13it/s] Loading 0: 52%|█████▏ | 150/291 [00:07<00:01, 76.57it/s] Loading 0: 55%|█████▍ | 159/291 [00:07<00:01, 78.90it/s] Loading 0: 58%|█████▊ | 168/291 [00:07<00:02, 48.42it/s] Loading 0: 61%|██████ | 177/291 [00:07<00:02, 55.84it/s] Loading 0: 64%|██████▍ | 186/291 [00:07<00:01, 62.45it/s] Loading 0: 67%|██████▋ | 195/291 [00:07<00:01, 67.64it/s] Loading 0: 71%|███████▏ | 208/291 [00:08<00:01, 82.56it/s] Loading 0: 75%|███████▍ | 218/291 [00:08<00:00, 86.04it/s] Loading 0: 78%|███████▊ | 228/291 [00:08<00:00, 88.44it/s] Loading 0: 82%|████████▏ | 238/291 [00:08<00:00, 84.45it/s] Loading 0: 85%|████████▍ | 247/291 [00:08<00:00, 85.71it/s] Loading 0: 88%|████████▊ | 256/291 [00:08<00:00, 85.70it/s] Loading 0: 91%|█████████ | 265/291 [00:08<00:00, 85.31it/s] Loading 0: 94%|█████████▍| 274/291 [00:09<00:00, 48.12it/s] Loading 0: 98%|█████████▊| 284/291 [00:09<00:00, 55.46it/s] Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: quantized model in 25.843s
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: Processed model Ayalexf/Graftmaxx-L3-8B in 80.775s
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: creating bucket guanaco-mkml-models
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/ayalexf-graftmaxx-l3-8b-v2
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/ayalexf-graftmaxx-l3-8b-v2/config.json
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/ayalexf-graftmaxx-l3-8b-v2/special_tokens_map.json
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/ayalexf-graftmaxx-l3-8b-v2/tokenizer_config.json
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/ayalexf-graftmaxx-l3-8b-v2/tokenizer.json
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/ayalexf-graftmaxx-l3-8b-v2/flywheel_model.0.safetensors
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: loading reward model from ChaiML/reward_gpt2_medium_preference_24m_e2
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:913: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: warnings.warn(
Job ayalexf-graftmaxx-l3-8b-v2-mkmlizer completed after 113.43s with status: failed
Stopping job with name ayalexf-graftmaxx-l3-8b-v2-mkmlizer
%s, retrying in %s seconds...
Starting job with name ayalexf-graftmaxx-l3-8b-v2-mkmlizer
Waiting for job on ayalexf-graftmaxx-l3-8b-v2-mkmlizer to finish
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ _____ __ __ ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ /___/ ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ Version: 0.8.14 ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ https://mk1.ai ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ The license key for the current software has been verified as ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ belonging to: ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ Chai Research Corp. ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ Expiration: 2024-07-15 23:59:59 ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: /opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_deprecation.py:131: FutureWarning: 'list_files_info' (from 'huggingface_hub.hf_api') is deprecated and will be removed from version '0.23'. Use `list_repo_tree` and `get_paths_info` instead.
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: warnings.warn(warning_message, FutureWarning)
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: Traceback (most recent call last):
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 509, in http_get
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: hf_transfer.download(
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: Exception: Error while downloading: reqwest::Error { kind: Request, url: Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("cdn-lfs-us-1.huggingface.co")), port: None, path: "/repos/c5/4f/700277821c9d0ce252dd9/be2875d74d4b46a3d19c5adfba9c3aea3a74b8afdc80b345dbf05e75feff0acf", query: Some("response-content-disposition=inline%3B+filename*%3DUTF-8%27%27model-00004-of-00004.safetensors%3B+filename%3D%22model-00004-of-00004.safetensors%22%3B&Expires=1718814113&Policy=eyJTdGF0ZW1lbnQiOlt7IkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTcxODgxNDExM319LCJSZXNvdXJjZSI6Imh0dHBzOi8vY2RuLWxmcy11cy0xLmh1Z2dpbmdmYWNlLmNvL3JlcG9zL2M1LzRmL2M1NGY2MmI5ZWExNzI2MjUyODJhODEwNWE3M2RhYzdjODEwMDIwNmYwMGM3MDAyNzc4MjFjOWQwY2UyNTJkZDkvYmUyODc1ZDc0ZDRiNDZhM2QxOWM1YWRmYmE5YzNhZWEzYTc0YjhhZmRjODBiMzQ1ZGJmMDVlNzVmZWZmMGFjZj9yZXNwb25zZS1jb250ZW50LWRpc3Bvc2l0aW9uPSoifV19&Signature=o3x4dWlybYXVrxUkJoJw9qEKxpXrRK7el4waYLgHokTJN750uU75gk0XigtSLFE6f-S0cHD3tYuK3PvtkL8g76We5%7EO7BeVecKWu33Z2NestYihFnWpl1BRLVAYr0JVfilgVtkCwQZ5SOrtA2e-Zrv6bWDrBP9oqx9HSop6EsPNvpwLS2H%7EBHYNNn7zhz7nT7sedxskM5i9nsnNLml-mzumNkes1M3eaAHJbUcmhyCM-ObBMEA3eQyMRTcbWjLFkruiRwIvgV0gEVjXhbFg8PFbQPYlJn85M3r4vO9E8tKrfE1YQGHmQlZGmZCGop%7E9TuIOD-EpM2CDI9U5q2gwktg__&Key-Pair-Id=K2FPYV99P2N66Q"), fragment: None }, source: hyper::Error(Connect, ConnectError("dns error", Custom { kind: Uncategorized, error: "failed to lookup address information: Temporary failure in name resolution" })) }
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: The above exception was the direct cause of the following exception:
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: Traceback (most recent call last):
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: File "/code/uploading/mkmlize.py", line 151, in <module>
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: cli()
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1128, in __call__
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: return self.main(*args, **kwargs)
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1053, in main
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: rv = self.invoke(ctx)
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1659, in invoke
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: return _process_result(sub_ctx.command.invoke(sub_ctx))
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1395, in invoke
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: return ctx.invoke(self.callback, **ctx.params)
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 754, in invoke
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: return __callback(*args, **kwargs)
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: File "/code/uploading/mkmlize.py", line 38, in quantize
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: temp_folder = download_to_shared_memory(repo_id, revision, hf_auth_token)
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: File "/code/uploading/mkmlize.py", line 65, in download_to_shared_memory
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: snapshot_download(
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 119, in _inner_fn
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: return fn(*args, **kwargs)
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/_snapshot_download.py", line 314, in snapshot_download
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: _inner_hf_hub_download(file)
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/_snapshot_download.py", line 290, in _inner_hf_hub_download
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: return hf_hub_download(
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 119, in _inner_fn
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: return fn(*args, **kwargs)
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1492, in hf_hub_download
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: http_get(
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 520, in http_get
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: raise RuntimeError(
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: RuntimeError: An error occurred while downloading using `hf_transfer`. Consider disabling HF_HUB_ENABLE_HF_TRANSFER for better error handling.
Job ayalexf-graftmaxx-l3-8b-v2-mkmlizer completed after 73.56s with status: failed
Stopping job with name ayalexf-graftmaxx-l3-8b-v2-mkmlizer
%s, retrying in %s seconds...
Starting job with name ayalexf-graftmaxx-l3-8b-v2-mkmlizer
Waiting for job on ayalexf-graftmaxx-l3-8b-v2-mkmlizer to finish
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ _____ __ __ ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ /___/ ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ Version: 0.8.14 ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ https://mk1.ai ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ The license key for the current software has been verified as ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ belonging to: ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ Chai Research Corp. ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ Expiration: 2024-07-15 23:59:59 ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ║ ║
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: /opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_deprecation.py:131: FutureWarning: 'list_files_info' (from 'huggingface_hub.hf_api') is deprecated and will be removed from version '0.23'. Use `list_repo_tree` and `get_paths_info` instead.
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: warnings.warn(warning_message, FutureWarning)
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: Downloaded to shared memory in 187.417s
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: quantizing model to /dev/shm/model_cache
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: Loading 0: 0%| | 0/291 [00:00<?, ?it/s] Loading 0: 1%| | 2/291 [00:05<12:07, 2.52s/it] Loading 0: 4%|▍ | 13/291 [00:05<01:21, 3.40it/s] Loading 0: 8%|▊ | 23/291 [00:05<00:38, 6.98it/s] Loading 0: 11%|█▏ | 33/291 [00:05<00:22, 11.57it/s] Loading 0: 15%|█▌ | 45/291 [00:05<00:13, 18.74it/s] Loading 0: 20%|█▉ | 58/291 [00:05<00:08, 28.18it/s] Loading 0: 23%|██▎ | 68/291 [00:06<00:08, 27.09it/s] Loading 0: 27%|██▋ | 78/291 [00:06<00:06, 34.04it/s] Loading 0: 31%|███ | 90/291 [00:06<00:04, 44.79it/s] Loading 0: 35%|███▌ | 103/291 [00:06<00:03, 57.67it/s] Loading 0: 39%|███▉ | 114/291 [00:06<00:02, 65.41it/s] Loading 0: 45%|████▍ | 130/291 [00:06<00:01, 82.64it/s] Loading 0: 49%|████▉ | 142/291 [00:06<00:01, 87.53it/s] Loading 0: 53%|█████▎ | 153/291 [00:06<00:01, 92.06it/s] Loading 0: 56%|█████▋ | 164/291 [00:06<00:01, 95.89it/s] Loading 0: 60%|██████ | 175/291 [00:07<00:01, 59.93it/s] Loading 0: 64%|██████▎ | 185/291 [00:07<00:01, 64.24it/s] Loading 0: 67%|██████▋ | 195/291 [00:07<00:01, 67.77it/s] Loading 0: 71%|███████ | 206/291 [00:07<00:01, 76.46it/s] Loading 0: 74%|███████▍ | 216/291 [00:07<00:00, 80.74it/s] Loading 0: 78%|███████▊ | 226/291 [00:07<00:00, 84.43it/s] Loading 0: 82%|████████▏ | 238/291 [00:07<00:00, 91.05it/s] Loading 0: 85%|████████▌ | 248/291 [00:08<00:00, 90.70it/s] Loading 0: 89%|████████▊ | 258/291 [00:08<00:00, 87.84it/s] Loading 0: 92%|█████████▏| 268/291 [00:08<00:00, 51.50it/s] Loading 0: 96%|█████████▌| 280/291 [00:08<00:00, 63.21it/s] Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: quantized model in 25.536s
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: Processed model Ayalexf/Graftmaxx-L3-8B in 215.631s
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: creating bucket guanaco-mkml-models
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/ayalexf-graftmaxx-l3-8b-v2
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/ayalexf-graftmaxx-l3-8b-v2/config.json
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/ayalexf-graftmaxx-l3-8b-v2/special_tokens_map.json
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/ayalexf-graftmaxx-l3-8b-v2/tokenizer_config.json
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/ayalexf-graftmaxx-l3-8b-v2/tokenizer.json
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/ayalexf-graftmaxx-l3-8b-v2/flywheel_model.0.safetensors
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: loading reward model from ChaiML/reward_gpt2_medium_preference_24m_e2
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:913: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: warnings.warn(
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:757: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: warnings.warn(
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:468: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: warnings.warn(
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: /opt/conda/lib/python3.10/site-packages/torch/_utils.py:831: UserWarning: TypedStorage is deprecated. It will be removed in the future and UntypedStorage will be the only storage class. This should only matter to you if you are using storages directly. To access UntypedStorage directly, use tensor.untyped_storage() instead of tensor.storage()
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: return self.fget.__get__(instance, owner)()
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: Saving duration: 0.509s
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: Processed model ChaiML/reward_gpt2_medium_preference_24m_e2 in 22.698s
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: creating bucket guanaco-reward-models
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: Bucket 's3://guanaco-reward-models/' created
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/ayalexf-graftmaxx-l3-8b-v2_reward
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/ayalexf-graftmaxx-l3-8b-v2_reward/special_tokens_map.json
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/ayalexf-graftmaxx-l3-8b-v2_reward/tokenizer_config.json
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/ayalexf-graftmaxx-l3-8b-v2_reward/vocab.json
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/ayalexf-graftmaxx-l3-8b-v2_reward/config.json
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/ayalexf-graftmaxx-l3-8b-v2_reward/merges.txt
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/ayalexf-graftmaxx-l3-8b-v2_reward/tokenizer.json
ayalexf-graftmaxx-l3-8b-v2-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/ayalexf-graftmaxx-l3-8b-v2_reward/reward.tensors
Job ayalexf-graftmaxx-l3-8b-v2-mkmlizer completed after 278.7s with status: succeeded
Stopping job with name ayalexf-graftmaxx-l3-8b-v2-mkmlizer
Pipeline stage MKMLizer completed in 468.57s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.11s
Running pipeline stage ISVCDeployer
Creating inference service ayalexf-graftmaxx-l3-8b-v2
Waiting for inference service ayalexf-graftmaxx-l3-8b-v2 to be ready
Inference service ayalexf-graftmaxx-l3-8b-v2 ready after 583.5204935073853s
Pipeline stage ISVCDeployer completed in 590.40s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.1231601238250732s
Received healthy response to inference request in 1.1161479949951172s
Received healthy response to inference request in 1.1201484203338623s
Received healthy response to inference request in 1.1096510887145996s
Received healthy response to inference request in 1.5677807331085205s
5 requests
0 failed requests
5th percentile: 1.1109504699707031
10th percentile: 1.1122498512268066
20th percentile: 1.1148486137390137
30th percentile: 1.1169480800628662
40th percentile: 1.1185482501983643
50th percentile: 1.1201484203338623
60th percentile: 1.2992013454437255
70th percentile: 1.4782542705535888
80th percentile: 1.678856611251831
90th percentile: 1.9010083675384521
95th percentile: 2.0120842456817627
99th percentile: 2.1009449481964113
mean time: 1.4073776721954345
Pipeline stage StressChecker completed in 7.97s
Running pipeline stage DaemonicModelEvalScorer
Pipeline stage DaemonicModelEvalScorer completed in 0.05s
Running pipeline stage DaemonicSafetyScorer
Running M-Eval for topic stay_in_character
Pipeline stage DaemonicSafetyScorer completed in 0.04s
M-Eval Dataset for topic stay_in_character is loaded
ayalexf-graftmaxx-l3-8b_v2 status is now deployed due to DeploymentManager action
ayalexf-graftmaxx-l3-8b_v2 status is now inactive due to auto deactivation removed underperforming models

Usage Metrics

Latency Metrics