submission_id: undi95-meta-llama-3-70b-_6209_v4
developer_uid: chai_backend_admin
status: inactive
model_repo: Undi95/Meta-Llama-3-70B-Instruct-hf
reward_repo: rirv938/gpt2_ties_merge_preference_plus_classic_e2_density_99
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 50, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n', '<|im_end|>', '<|im_start|>', '\n\n'], 'max_input_tokens': 512, 'best_of': 2, 'max_output_tokens': 64}
formatter: {'memory_template': "<|im_start|>system\n{bot_name}'s Persona: {memory}<|im_end|>\n", 'prompt_template': '<|im_start|>system\n{prompt}<|im_end|>\n', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': False}
reward_formatter: {'memory_template': 'Memory: {memory}\n', 'prompt_template': '{prompt}\n', 'bot_template': 'Bot: {message}\n', 'user_template': 'User: {message}\n', 'response_template': 'Bot:', 'truncate_by_message': False}
timestamp: 2024-06-18T22:14:55+00:00
model_name: undi95-meta-llama-3-70b-_6209_v4
model_group: Undi95/Meta-Llama-3-70B-
num_battles: 54089
num_wins: 27655
celo_rating: 1191.51
propriety_score: 0.7272187710204566
propriety_total_count: 25273.0
submission_type: basic
model_architecture: LlamaForCausalLM
model_num_parameters: 70553706496.0
best_of: 2
max_input_tokens: 512
max_output_tokens: 64
display_name: undi95-meta-llama-3-70b-_6209_v4
ineligible_reason: None
language_model: Undi95/Meta-Llama-3-70B-Instruct-hf
model_size: 71B
reward_model: rirv938/gpt2_ties_merge_preference_plus_classic_e2_density_99
us_pacific_date: 2024-06-18
win_ratio: 0.5112869529848952
Resubmit model
Running pipeline stage MKMLizer
Starting job with name undi95-meta-llama-3-70b-6209-v4-mkmlizer
Waiting for job on undi95-meta-llama-3-70b-6209-v4-mkmlizer to finish
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ _____ __ __ ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ /___/ ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ Version: 0.8.14 ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ https://mk1.ai ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ The license key for the current software has been verified as ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ belonging to: ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ Chai Research Corp. ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ Expiration: 2024-07-15 23:59:59 ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
undi95-meta-llama-3-70b-6209-v4-mkmlizer: Traceback (most recent call last):
undi95-meta-llama-3-70b-6209-v4-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/urllib3/connection.py", line 174, in _new_conn
undi95-meta-llama-3-70b-6209-v4-mkmlizer: conn = connection.create_connection(
undi95-meta-llama-3-70b-6209-v4-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/urllib3/util/connection.py", line 72, in create_connection
undi95-meta-llama-3-70b-6209-v4-mkmlizer: for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
undi95-meta-llama-3-70b-6209-v4-mkmlizer: File "/opt/conda/lib/python3.10/socket.py", line 955, in getaddrinfo
undi95-meta-llama-3-70b-6209-v4-mkmlizer: for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
undi95-meta-llama-3-70b-6209-v4-mkmlizer: socket.gaierror: [Errno -3] Temporary failure in name resolution
undi95-meta-llama-3-70b-6209-v4-mkmlizer: During handling of the above exception, another exception occurred:
undi95-meta-llama-3-70b-6209-v4-mkmlizer: Traceback (most recent call last):
undi95-meta-llama-3-70b-6209-v4-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/urllib3/connectionpool.py", line 714, in urlopen
undi95-meta-llama-3-70b-6209-v4-mkmlizer: httplib_response = self._make_request(
undi95-meta-llama-3-70b-6209-v4-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/urllib3/connectionpool.py", line 403, in _make_request
undi95-meta-llama-3-70b-6209-v4-mkmlizer: self._validate_conn(conn)
undi95-meta-llama-3-70b-6209-v4-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1053, in _validate_conn
undi95-meta-llama-3-70b-6209-v4-mkmlizer: conn.connect()
undi95-meta-llama-3-70b-6209-v4-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/urllib3/connection.py", line 363, in connect
undi95-meta-llama-3-70b-6209-v4-mkmlizer: self.sock = conn = self._new_conn()
undi95-meta-llama-3-70b-6209-v4-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/urllib3/connection.py", line 186, in _new_conn
undi95-meta-llama-3-70b-6209-v4-mkmlizer: raise NewConnectionError(
undi95-meta-llama-3-70b-6209-v4-mkmlizer: urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f1f6555a860>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution
HTTP Request: %s %s "%s %d %s"
undi95-meta-llama-3-70b-6209-v4-mkmlizer: During handling of the above exception, another exception occurred:
undi95-meta-llama-3-70b-6209-v4-mkmlizer: Traceback (most recent call last):
undi95-meta-llama-3-70b-6209-v4-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/requests/adapters.py", line 486, in send
undi95-meta-llama-3-70b-6209-v4-mkmlizer: resp = conn.urlopen(
undi95-meta-llama-3-70b-6209-v4-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/urllib3/connectionpool.py", line 798, in urlopen
undi95-meta-llama-3-70b-6209-v4-mkmlizer: retries = retries.increment(
undi95-meta-llama-3-70b-6209-v4-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/urllib3/util/retry.py", line 592, in increment
undi95-meta-llama-3-70b-6209-v4-mkmlizer: raise MaxRetryError(_pool, url, error or ResponseError(cause))
undi95-meta-llama-3-70b-6209-v4-mkmlizer: urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='cdn-lfs-us-1.huggingface.co', port=443): Max retries exceeded with url: /repos/9a/b1/ce4ad7d9fc667e6a80768/dc1959f112e1737a364d25e839a2553b76372028644346f94ad51dc15d9fbbca?response-content-disposition=inline%3B+filename*%3DUTF-8%27%27consolidated.04.pth%3B+filename%3D%22consolidated.04.pth%22%3B&Expires=1719008960&Policy=eyJTdGF0ZW1lbnQiOlt7IkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTcxOTAwODk2MH19LCJSZXNvdXJjZSI6Imh0dHBzOi8vY2RuLWxmcy11cy0xLmh1Z2dpbmdmYWNlLmNvL3JlcG9zLzlhL2IxLzlhYjE1ZjcyYjE4YzgwMWYyY2MyMWFkMDEwZTM5YmZmODEwOWYyMTY4NDljZTRhZDdkOWZjNjY3ZTZhODA3NjgvZGMxOTU5ZjExMmUxNzM3YTM2NGQyNWU4MzlhMjU1M2I3NjM3MjAyODY0NDM0NmY5NGFkNTFkYzE1ZDlmYmJjYT9yZXNwb25zZS1jb250ZW50LWRpc3Bvc2l0aW9uPSoifV19&Signature=mpufjyjw5jKlrNBUwUIFF13ZqUbtpD6qI6UXx7jIMl7CwKvdcZXAeXXwfplr-bSzreSic0~Y-62qS0FZthMwdGMGTmDhUKG9ioF8stY5XpEY400A0XcKKNTEbarEyqO22RF2gCN4kPFJjX4Ee7dMC6d8yE00pDHoldoGd05fnunSBNxW7-AMcyVdB-TPQVZB~Ysh3-px1OhXwSKE3fEge0XpfAdAfbT~Vu3z24Ct8T87Lbwrve1vJj6Txtds9o5cBJ4ubJty~tGjsIiSAyRwcYermKEZNp2fq4h1uBZbBaK20NCz5eI8erZNZeZwPki1QXHDstKPBr97lnaBdyZo8w__&Key-Pair-Id=K2FPYV99P2N66Q (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f1f6555a860>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))
undi95-meta-llama-3-70b-6209-v4-mkmlizer: During handling of the above exception, another exception occurred:
undi95-meta-llama-3-70b-6209-v4-mkmlizer: Traceback (most recent call last):
undi95-meta-llama-3-70b-6209-v4-mkmlizer: File "/code/uploading/mkmlize.py", line 151, in <module>
undi95-meta-llama-3-70b-6209-v4-mkmlizer: cli()
undi95-meta-llama-3-70b-6209-v4-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1128, in __call__
undi95-meta-llama-3-70b-6209-v4-mkmlizer: return self.main(*args, **kwargs)
undi95-meta-llama-3-70b-6209-v4-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1053, in main
undi95-meta-llama-3-70b-6209-v4-mkmlizer: rv = self.invoke(ctx)
undi95-meta-llama-3-70b-6209-v4-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1659, in invoke
undi95-meta-llama-3-70b-6209-v4-mkmlizer: return _process_result(sub_ctx.command.invoke(sub_ctx))
undi95-meta-llama-3-70b-6209-v4-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1395, in invoke
undi95-meta-llama-3-70b-6209-v4-mkmlizer: return ctx.invoke(self.callback, **ctx.params)
undi95-meta-llama-3-70b-6209-v4-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 754, in invoke
undi95-meta-llama-3-70b-6209-v4-mkmlizer: return __callback(*args, **kwargs)
undi95-meta-llama-3-70b-6209-v4-mkmlizer: File "/code/uploading/mkmlize.py", line 38, in quantize
undi95-meta-llama-3-70b-6209-v4-mkmlizer: temp_folder = download_to_shared_memory(repo_id, revision, hf_auth_token)
undi95-meta-llama-3-70b-6209-v4-mkmlizer: File "/code/uploading/mkmlize.py", line 65, in download_to_shared_memory
undi95-meta-llama-3-70b-6209-v4-mkmlizer: snapshot_download(
undi95-meta-llama-3-70b-6209-v4-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
undi95-meta-llama-3-70b-6209-v4-mkmlizer: return fn(*args, **kwargs)
undi95-meta-llama-3-70b-6209-v4-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/_snapshot_download.py", line 292, in snapshot_download
undi95-meta-llama-3-70b-6209-v4-mkmlizer: _inner_hf_hub_download(file)
undi95-meta-llama-3-70b-6209-v4-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/_snapshot_download.py", line 268, in _inner_hf_hub_download
undi95-meta-llama-3-70b-6209-v4-mkmlizer: return hf_hub_download(
undi95-meta-llama-3-70b-6209-v4-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
undi95-meta-llama-3-70b-6209-v4-mkmlizer: return fn(*args, **kwargs)
undi95-meta-llama-3-70b-6209-v4-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1202, in hf_hub_download
undi95-meta-llama-3-70b-6209-v4-mkmlizer: return _hf_hub_download_to_local_dir(
undi95-meta-llama-3-70b-6209-v4-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1487, in _hf_hub_download_to_local_dir
undi95-meta-llama-3-70b-6209-v4-mkmlizer: _download_to_tmp_and_move(
undi95-meta-llama-3-70b-6209-v4-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1884, in _download_to_tmp_and_move
undi95-meta-llama-3-70b-6209-v4-mkmlizer: http_get(
undi95-meta-llama-3-70b-6209-v4-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 459, in http_get
undi95-meta-llama-3-70b-6209-v4-mkmlizer: r = _request_wrapper(
undi95-meta-llama-3-70b-6209-v4-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 395, in _request_wrapper
undi95-meta-llama-3-70b-6209-v4-mkmlizer: response = get_session().request(method=method, url=url, **params)
undi95-meta-llama-3-70b-6209-v4-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/requests/sessions.py", line 589, in request
undi95-meta-llama-3-70b-6209-v4-mkmlizer: resp = self.send(prep, **send_kwargs)
undi95-meta-llama-3-70b-6209-v4-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/requests/sessions.py", line 703, in send
undi95-meta-llama-3-70b-6209-v4-mkmlizer: r = adapter.send(request, **kwargs)
undi95-meta-llama-3-70b-6209-v4-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_http.py", line 66, in send
undi95-meta-llama-3-70b-6209-v4-mkmlizer: return super().send(request, *args, **kwargs)
undi95-meta-llama-3-70b-6209-v4-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/requests/adapters.py", line 519, in send
undi95-meta-llama-3-70b-6209-v4-mkmlizer: raise ConnectionError(e, request=request)
undi95-meta-llama-3-70b-6209-v4-mkmlizer: requests.exceptions.ConnectionError: (MaxRetryError("HTTPSConnectionPool(host='cdn-lfs-us-1.huggingface.co', port=443): Max retries exceeded with url: /repos/9a/b1/ce4ad7d9fc667e6a80768/dc1959f112e1737a364d25e839a2553b76372028644346f94ad51dc15d9fbbca?response-content-disposition=inline%3B+filename*%3DUTF-8%27%27consolidated.04.pth%3B+filename%3D%22consolidated.04.pth%22%3B&Expires=1719008960&Policy=eyJTdGF0ZW1lbnQiOlt7IkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTcxOTAwODk2MH19LCJSZXNvdXJjZSI6Imh0dHBzOi8vY2RuLWxmcy11cy0xLmh1Z2dpbmdmYWNlLmNvL3JlcG9zLzlhL2IxLzlhYjE1ZjcyYjE4YzgwMWYyY2MyMWFkMDEwZTM5YmZmODEwOWYyMTY4NDljZTRhZDdkOWZjNjY3ZTZhODA3NjgvZGMxOTU5ZjExMmUxNzM3YTM2NGQyNWU4MzlhMjU1M2I3NjM3MjAyODY0NDM0NmY5NGFkNTFkYzE1ZDlmYmJjYT9yZXNwb25zZS1jb250ZW50LWRpc3Bvc2l0aW9uPSoifV19&Signature=mpufjyjw5jKlrNBUwUIFF13ZqUbtpD6qI6UXx7jIMl7CwKvdcZXAeXXwfplr-bSzreSic0~Y-62qS0FZthMwdGMGTmDhUKG9ioF8stY5XpEY400A0XcKKNTEbarEyqO22RF2gCN4kPFJjX4Ee7dMC6d8yE00pDHoldoGd05fnunSBNxW7-AMcyVdB-TPQVZB~Ysh3-px1OhXwSKE3fEge0XpfAdAfbT~Vu3z24Ct8T87Lbwrve1vJj6Txtds9o5cBJ4ubJty~tGjsIiSAyRwcYermKEZNp2fq4h1uBZbBaK20NCz5eI8erZNZeZwPki1QXHDstKPBr97lnaBdyZo8w__&Key-Pair-Id=K2FPYV99P2N66Q (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f1f6555a860>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))"), '(Request ID: 9be0058e-9bc6-49bb-8674-e520ba94983d)')
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ _____ __ __ ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ /___/ ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ Version: 0.8.14 ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ https://mk1.ai ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ The license key for the current software has been verified as ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ belonging to: ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ Chai Research Corp. ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ Expiration: 2024-07-15 23:59:59 ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Job undi95-meta-llama-3-70b-6209-v4-mkmlizer completed after 941.8s with status: failed
Stopping job with name undi95-meta-llama-3-70b-6209-v4-mkmlizer
%s, retrying in %s seconds...
Starting job with name undi95-meta-llama-3-70b-6209-v4-mkmlizer
Waiting for job on undi95-meta-llama-3-70b-6209-v4-mkmlizer to finish
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ _____ __ __ ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ /___/ ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ Version: 0.8.14 ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ https://mk1.ai ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ The license key for the current software has been verified as ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ belonging to: ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ Chai Research Corp. ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ Expiration: 2024-07-15 23:59:59 ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ║ ║
undi95-meta-llama-3-70b-6209-v4-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
undi95-meta-llama-3-70b-6209-v4-mkmlizer: Downloaded to shared memory in 343.187s
undi95-meta-llama-3-70b-6209-v4-mkmlizer: quantizing model to /dev/shm/model_cache
undi95-meta-llama-3-70b-6209-v4-mkmlizer: Saving flywheel model at /dev/shm/model_cache
undi95-meta-llama-3-70b-6209-v4-mkmlizer: Loading 0: 0%| | 0/723 [00:00<?, ?it/s] Loading 0: 0%| | 3/723 [00:00<01:48, 6.66it/s] Loading 0: 1%| | 4/723 [00:00<02:50, 4.22it/s] Loading 0: 1%| | 5/723 [00:01<03:31, 3.40it/s] Loading 0: 1%| | 8/723 [00:01<01:49, 6.55it/s] Loading 0: 1%|▏ | 10/723 [00:01<01:27, 8.14it/s] Loading 0: 2%|▏ | 12/723 [00:02<01:50, 6.46it/s] Loading 0: 2%|▏ | 13/723 [00:02<01:46, 6.64it/s] Loading 0: 2%|▏ | 14/723 [00:02<01:42, 6.91it/s] Loading 0: 2%|▏ | 16/723 [00:02<01:26, 8.19it/s] Loading 0: 2%|▏ | 17/723 [00:02<02:13, 5.29it/s] Loading 0: 2%|▏ | 18/723 [00:03<02:53, 4.06it/s] Loading 0: 3%|▎ | 21/723 [00:03<02:17, 5.12it/s] Loading 0: 3%|▎ | 22/723 [00:04<02:49, 4.15it/s] Loading 0: 3%|▎ | 23/723 [00:04<03:20, 3.49it/s] Loading 0: 4%|▎ | 26/723 [00:04<01:59, 5.82it/s] Loading 0: 4%|▍ | 30/723 [00:05<01:24, 8.18it/s] Loading 0: 4%|▍ | 32/723 [00:05<01:36, 7.20it/s] Loading 0: 5%|▍ | 36/723 [00:05<01:03, 10.86it/s] Loading 0: 6%|▌ | 40/723 [00:05<00:46, 14.58it/s] Loading 0: 6%|▌ | 43/723 [00:05<00:54, 12.45it/s] Loading 0: 6%|▌ | 45/723 [00:06<01:08, 9.88it/s] Loading 0: 7%|▋ | 48/723 [00:06<00:59, 11.33it/s] Loading 0: 7%|▋ | 50/723 [00:06<01:13, 9.13it/s] Loading 0: 7%|▋ | 54/723 [00:06<00:51, 12.88it/s] Loading 0: 8%|▊ | 57/723 [00:07<00:48, 13.79it/s] Loading 0: 8%|▊ | 59/723 [00:07<01:03, 10.41it/s] Loading 0: 9%|▊ | 63/723 [00:07<00:46, 14.25it/s] Loading 0: 9%|▉ | 68/723 [00:07<00:39, 16.53it/s] Loading 0: 10%|▉ | 71/723 [00:08<00:59, 10.88it/s] Loading 0: 10%|█ | 75/723 [00:08<00:52, 12.40it/s] Loading 0: 11%|█ | 77/723 [00:09<01:04, 10.06it/s] Loading 0: 11%|█ | 81/723 [00:09<00:47, 13.44it/s] Loading 0: 12%|█▏ | 84/723 [00:09<00:45, 14.19it/s] Loading 0: 12%|█▏ | 86/723 [00:09<00:59, 10.72it/s] Loading 0: 12%|█▏ | 90/723 [00:09<00:43, 14.48it/s] Loading 0: 13%|█▎ | 93/723 [00:10<00:52, 11.91it/s] Loading 0: 13%|█▎ | 95/723 [00:10<01:05, 9.59it/s] Loading 0: 14%|█▎ | 99/723 [00:10<00:47, 13.19it/s] Loading 0: 14%|█▍ | 102/723 [00:10<00:44, 14.03it/s] Loading 0: 14%|█▍ | 104/723 [00:11<00:58, 10.62it/s] Loading 0: 15%|█▍ | 108/723 [00:11<00:42, 14.50it/s] Loading 0: 15%|█▌ | 111/723 [00:11<00:51, 11.93it/s] Loading 0: 16%|█▌ | 114/723 [00:11<00:43, 14.13it/s] Loading 0: 16%|█▌ | 116/723 [00:11<00:44, 13.63it/s] Loading 0: 16%|█▋ | 118/723 [00:12<00:47, 12.82it/s] Loading 0: 17%|█▋ | 120/723 [00:12<00:48, 12.38it/s] Loading 0: 17%|█▋ | 122/723 [00:12<01:04, 9.27it/s] Loading 0: 17%|█▋ | 125/723 [00:24<15:06, 1.52s/it] Loading 0: 18%|█▊ | 129/723 [00:24<09:06, 1.09it/s] Loading 0: 18%|█▊ | 131/723 [00:25<07:26, 1.33it/s] Loading 0: 19%|█▊ | 135/723 [00:25<04:35, 2.13it/s] Loading 0: 19%|█▉ | 137/723 [00:25<03:46, 2.59it/s] Loading 0: 19%|█▉ | 140/723 [00:25<02:39, 3.64it/s] Loading 0: 20%|█▉ | 142/723 [00:25<02:17, 4.23it/s] Loading 0: 20%|█▉ | 144/723 [00:26<02:11, 4.41it/s] Loading 0: 20%|██ | 147/723 [00:26<01:36, 5.94it/s] Loading 0: 21%|██ | 149/723 [00:26<01:37, 5.86it/s] Loading 0: 21%|██ | 153/723 [00:26<01:03, 9.00it/s] Loading 0: 22%|██▏ | 156/723 [00:27<00:54, 10.49it/s] Loading 0: 22%|██▏ | 158/723 [00:27<01:04, 8.82it/s] Loading 0: 22%|██▏ | 162/723 [00:27<00:44, 12.54it/s] Loading 0: 23%|██▎ | 166/723 [00:27<00:34, 16.36it/s] Loading 0: 23%|██▎ | 169/723 [00:27<00:41, 13.43it/s] Loading 0: 24%|██▍ | 172/723 [00:28<00:48, 11.44it/s] Loading 0: 24%|██▍ | 174/723 [00:28<00:47, 11.49it/s] Loading 0: 24%|██▍ | 176/723 [00:28<00:59, 9.23it/s] Loading 0: 25%|██▍ | 180/723 [00:28<00:41, 13.11it/s] Loading 0: 25%|██▌ | 183/723 [00:29<00:38, 13.93it/s] Loading 0: 26%|██▌ | 185/723 [00:29<00:51, 10.45it/s] Loading 0: 26%|██▌ | 189/723 [00:29<00:37, 14.35it/s] Loading 0: 27%|██▋ | 194/723 [00:29<00:31, 16.75it/s] Loading 0: 27%|██▋ | 197/723 [00:30<00:47, 11.05it/s] Loading 0: 28%|██▊ | 201/723 [00:30<00:41, 12.50it/s] Loading 0: 28%|██▊ | 203/723 [00:30<00:51, 10.18it/s] Loading 0: 29%|██▊ | 207/723 [00:31<00:37, 13.62it/s] Loading 0: 29%|██▉ | 210/723 [00:31<00:35, 14.27it/s] Loading 0: 29%|██▉ | 212/723 [00:31<00:47, 10.79it/s] Loading 0: 30%|██▉ | 216/723 [00:31<00:34, 14.57it/s] Loading 0: 30%|███ | 219/723 [00:32<00:41, 12.27it/s] Loading 0: 31%|███ | 221/723 [00:32<00:51, 9.79it/s] Loading 0: 31%|███ | 225/723 [00:32<00:36, 13.53it/s] Loading 0: 32%|███▏ | 228/723 [00:32<00:34, 14.35it/s] Loading 0: 32%|███▏ | 230/723 [00:33<00:45, 10.80it/s] Loading 0: 32%|███▏ | 234/723 [00:33<00:33, 14.71it/s] Loading 0: 33%|███▎ | 237/723 [00:33<00:40, 12.07it/s] Loading 0: 33%|███▎ | 240/723 [00:33<00:33, 14.37it/s] Loading 0: 34%|███▎ | 243/723 [00:33<00:39, 12.20it/s] Loading 0: 34%|███▍ | 246/723 [00:34<00:36, 13.22it/s] Loading 0: 34%|███▍ | 248/723 [00:34<00:47, 9.94it/s] Loading 0: 35%|███▍ | 252/723 [00:34<00:34, 13.74it/s] Loading 0: 35%|███▌ | 255/723 [00:34<00:32, 14.25it/s] Loading 0: 36%|███▌ | 257/723 [00:35<00:45, 10.32it/s] Loading 0: 36%|███▌ | 261/723 [00:35<00:32, 14.09it/s] Loading 0: 37%|███▋ | 264/723 [00:47<09:39, 1.26s/it] Loading 0: 37%|███▋ | 266/723 [00:47<07:37, 1.00s/it] Loading 0: 37%|███▋ | 268/723 [00:48<06:01, 1.26it/s] Loading 0: 37%|███▋ | 270/723 [00:48<04:52, 1.55it/s] Loading 0: 38%|███▊ | 273/723 [00:48<03:17, 2.27it/s] Loading 0: 38%|███▊ | 275/723 [00:49<02:47, 2.67it/s] Loading 0: 39%|███▊ | 279/723 [00:49<01:42, 4.35it/s] Loading 0: 39%|███▉ | 282/723 [00:49<01:18, 5.64it/s] Loading 0: 39%|███▉ | 284/723 [00:49<01:17, 5.64it/s] Loading 0: 40%|███▉ | 288/723 [00:49<00:51, 8.45it/s] Loading 0: 40%|████ | 292/723 [00:49<00:36, 11.66it/s] Loading 0: 41%|████ | 295/723 [00:50<00:39, 10.73it/s] Loading 0: 41%|████ | 298/723 [00:50<00:42, 9.94it/s] Loading 0: 41%|████▏ | 300/723 [00:50<00:41, 10.20it/s] Loading 0: 42%|████▏ | 302/723 [00:51<00:49, 8.55it/s] Loading 0: 42%|████▏ | 306/723 [00:51<00:33, 12.28it/s] Loading 0: 43%|████▎ | 309/723 [00:51<00:31, 13.34it/s] Loading 0: 43%|████▎ | 311/723 [00:51<00:42, 9.80it/s] Loading 0: 44%|████▎ | 315/723 [00:52<00:29, 13.68it/s] Loading 0: 44%|████▍ | 320/723 [00:52<00:24, 16.36it/s] Loading 0: 45%|████▍ | 323/723 [00:52<00:36, 10.85it/s] Loading 0: 45%|████▌ | 327/723 [00:53<00:32, 12.34it/s] Loading 0: 46%|████▌ | 329/723 [00:53<00:39, 10.05it/s] Loading 0: 46%|████▌ | 333/723 [00:53<00:28, 13.46it/s] Loading 0: 46%|████▋ | 336/723 [00:53<00:27, 14.22it/s] Loading 0: 47%|████▋ | 338/723 [00:54<00:35, 10.83it/s] Loading 0: 47%|████▋ | 342/723 [00:54<00:25, 14.66it/s] Loading 0: 48%|████▊ | 345/723 [00:54<00:30, 12.40it/s] Loading 0: 48%|████▊ | 347/723 [00:54<00:38, 9.82it/s] Loading 0: 49%|████▊ | 351/723 [00:54<00:27, 13.55it/s] Loading 0: 49%|████▉ | 354/723 [00:55<00:25, 14.32it/s] Loading 0: 49%|████▉ | 356/723 [00:55<00:34, 10.72it/s] Loading 0: 50%|████▉ | 360/723 [00:55<00:24, 14.57it/s] Loading 0: 50%|█████ | 363/723 [00:55<00:30, 11.63it/s] Loading 0: 51%|█████ | 366/723 [00:56<00:25, 13.84it/s] Loading 0: 51%|█████ | 368/723 [00:56<00:26, 13.31it/s] Loading 0: 51%|█████ | 370/723 [00:56<00:27, 12.66it/s] Loading 0: 51%|█████▏ | 372/723 [00:56<00:28, 12.33it/s] Loading 0: 52%|█████▏ | 374/723 [00:56<00:37, 9.29it/s] Loading 0: 52%|█████▏ | 378/723 [00:57<00:25, 13.62it/s] Loading 0: 53%|█████▎ | 381/723 [00:57<00:23, 14.47it/s] Loading 0: 53%|█████▎ | 383/723 [00:57<00:32, 10.60it/s] Loading 0: 54%|█████▎ | 387/723 [00:57<00:22, 14.74it/s] Loading 0: 54%|█████▍ | 390/723 [00:57<00:22, 15.05it/s] Loading 0: 54%|█████▍ | 392/723 [00:58<00:20, 15.91it/s] Loading 0: 54%|█████▍ | 394/723 [00:58<00:22, 14.95it/s] Loading 0: 55%|█████▍ | 396/723 [00:58<00:31, 10.49it/s] Loading 0: 55%|█████▌ | 399/723 [00:58<00:27, 11.99it/s] Loading 0: 55%|█████▌ | 400/723 [01:10<00:26, 11.99it/s] Loading 0: 55%|█████▌ | 401/723 [01:10<08:29, 1.58s/it] Loading 0: 56%|█████▌ | 406/723 [01:10<04:26, 1.19it/s] Loading 0: 57%|█████▋ | 409/723 [01:11<03:18, 1.59it/s] Loading 0: 57%|█████▋ | 412/723 [01:11<02:24, 2.15it/s] Loading 0: 57%|█████▋ | 415/723 [01:11<01:44, 2.96it/s] Loading 0: 58%|█████▊ | 418/723 [01:11<01:15, 4.02it/s] Loading 0: 58%|█████▊ | 421/723 [01:12<01:05, 4.58it/s] Loading 0: 59%|█████▊ | 423/723 [01:12<01:02, 4.77it/s] Loading 0: 59%|█████▉ | 426/723 [01:12<00:47, 6.19it/s] Loading 0: 59%|█████▉ | 428/723 [01:12<00:48, 6.06it/s] Loading 0: 60%|█████▉ | 432/723 [01:13<00:32, 9.08it/s] Loading 0: 60%|██████ | 435/723 [01:13<00:27, 10.48it/s] Loading 0: 60%|██████ | 437/723 [01:13<00:32, 8.82it/s] Loading 0: 61%|██████ | 441/723 [01:13<00:22, 12.47it/s] Loading 0: 62%|██████▏ | 446/723 [01:13<00:18, 15.28it/s] Loading 0: 62%|██████▏ | 449/723 [01:14<00:26, 10.29it/s] Loading 0: 63%|██████▎ | 453/723 [01:14<00:23, 11.66it/s] Loading 0: 63%|██████▎ | 455/723 [01:15<00:28, 9.37it/s] Loading 0: 63%|██████▎ | 459/723 [01:15<00:21, 12.54it/s] Loading 0: 64%|██████▍ | 462/723 [01:15<00:19, 13.07it/s] Loading 0: 64%|██████▍ | 464/723 [01:15<00:25, 9.98it/s] Loading 0: 65%|██████▍ | 468/723 [01:15<00:18, 13.48it/s] Loading 0: 65%|██████▌ | 470/723 [01:16<00:19, 12.82it/s] Loading 0: 65%|██████▌ | 472/723 [01:16<00:26, 9.62it/s] Loading 0: 66%|██████▌ | 474/723 [01:16<00:25, 9.63it/s] Loading 0: 66%|██████▌ | 477/723 [01:16<00:19, 12.33it/s] Loading 0: 66%|██████▋ | 480/723 [01:17<00:18, 13.27it/s] Loading 0: 67%|██████▋ | 482/723 [01:17<00:25, 9.64it/s] Loading 0: 67%|██████▋ | 486/723 [01:17<00:17, 13.55it/s] Loading 0: 67%|██████▋ | 488/723 [01:17<00:18, 12.70it/s] Loading 0: 68%|██████▊ | 490/723 [01:17<00:19, 12.00it/s] Loading 0: 68%|██████▊ | 492/723 [01:18<00:17, 13.25it/s] Loading 0: 68%|██████▊ | 494/723 [01:18<00:17, 12.79it/s] Loading 0: 69%|██████▊ | 496/723 [01:18<00:18, 12.09it/s] Loading 0: 69%|██████▉ | 498/723 [01:18<00:19, 11.83it/s] Loading 0: 69%|██████▉ | 500/723 [01:18<00:25, 8.89it/s] Loading 0: 70%|██████▉ | 504/723 [01:19<00:16, 13.46it/s] Loading 0: 70%|███████ | 507/723 [01:19<00:15, 14.36it/s] Loading 0: 70%|███████ | 509/723 [01:19<00:20, 10.45it/s] Loading 0: 71%|███████ | 513/723 [01:19<00:14, 14.60it/s] Loading 0: 71%|███████▏ | 516/723 [01:19<00:13, 14.88it/s] Loading 0: 72%|███████▏ | 518/723 [01:19<00:13, 15.76it/s] Loading 0: 72%|███████▏ | 520/723 [01:20<00:13, 14.69it/s] Loading 0: 72%|███████▏ | 522/723 [01:20<00:20, 10.02it/s] Loading 0: 73%|███████▎ | 525/723 [01:20<00:17, 11.55it/s] Loading 0: 73%|███████▎ | 527/723 [01:21<00:21, 9.07it/s] Loading 0: 73%|███████▎ | 531/723 [01:21<00:14, 13.22it/s] Loading 0: 74%|███████▍ | 534/723 [01:21<00:13, 14.07it/s] Loading 0: 74%|███████▍ | 536/723 [01:21<00:18, 10.28it/s] Loading 0: 75%|███████▍ | 540/723 [01:21<00:12, 14.28it/s] Loading 0: 75%|███████▍ | 540/723 [01:33<00:12, 14.28it/s] Loading 0: 75%|███████▍ | 541/723 [01:33<04:40, 1.54s/it] Loading 0: 75%|███████▌ | 544/723 [01:33<03:02, 1.02s/it] Loading 0: 76%|███████▌ | 547/723 [01:34<02:08, 1.37it/s] Loading 0: 76%|███████▌ | 549/723 [01:34<01:44, 1.66it/s] Loading 0: 76%|███████▋ | 552/723 [01:34<01:11, 2.39it/s] Loading 0: 77%|███████▋ | 554/723 [01:35<01:00, 2.78it/s] Loading 0: 77%|███████▋ | 558/723 [01:35<00:36, 4.48it/s] Loading 0: 78%|███████▊ | 561/723 [01:35<00:28, 5.76it/s] Loading 0: 78%|███████▊ | 563/723 [01:35<00:27, 5.74it/s] Loading 0: 78%|███████▊ | 567/723 [01:36<00:18, 8.56it/s] Loading 0: 79%|███████▉ | 572/723 [01:36<00:13, 11.53it/s] Loading 0: 80%|███████▉ | 575/723 [01:36<00:16, 9.01it/s] Loading 0: 80%|████████ | 579/723 [01:37<00:13, 10.74it/s] Loading 0: 80%|████████ | 581/723 [01:37<00:15, 9.14it/s] Loading 0: 81%|████████ | 585/723 [01:37<00:11, 12.43it/s] Loading 0: 81%|████████▏ | 588/723 [01:37<00:10, 13.29it/s] Loading 0: 82%|████████▏ | 590/723 [01:38<00:13, 9.94it/s] Loading 0: 82%|████████▏ | 594/723 [01:38<00:09, 13.27it/s] Loading 0: 82%|████████▏ | 596/723 [01:38<00:09, 13.20it/s] Loading 0: 83%|████████▎ | 598/723 [01:38<00:12, 9.95it/s] Loading 0: 83%|████████▎ | 600/723 [01:38<00:12, 10.06it/s] Loading 0: 83%|████████▎ | 603/723 [01:39<00:09, 12.86it/s] Loading 0: 84%|████████▍ | 606/723 [01:39<00:08, 13.93it/s] Loading 0: 84%|████████▍ | 608/723 [01:39<00:11, 10.15it/s] Loading 0: 85%|████████▍ | 612/723 [01:39<00:07, 14.27it/s] Loading 0: 85%|████████▍ | 614/723 [01:39<00:08, 13.38it/s] Loading 0: 85%|████████▌ | 616/723 [01:40<00:08, 12.50it/s] Loading 0: 85%|████████▌ | 618/723 [01:40<00:07, 13.76it/s] Loading 0: 86%|████████▌ | 620/723 [01:40<00:07, 13.35it/s] Loading 0: 86%|████████▌ | 622/723 [01:40<00:08, 11.95it/s] Loading 0: 86%|████████▋ | 624/723 [01:40<00:08, 11.33it/s] Loading 0: 87%|████████▋ | 626/723 [01:41<00:11, 8.28it/s] Loading 0: 87%|████████▋ | 630/723 [01:41<00:07, 12.24it/s] Loading 0: 88%|████████▊ | 633/723 [01:41<00:07, 12.15it/s] Loading 0: 88%|████████▊ | 635/723 [01:41<00:09, 8.89it/s] Loading 0: 88%|████████▊ | 639/723 [01:42<00:06, 12.52it/s] Loading 0: 89%|████████▊ | 641/723 [01:42<00:06, 11.94it/s] Loading 0: 89%|████████▉ | 644/723 [01:42<00:05, 14.40it/s] Loading 0: 89%|████████▉ | 646/723 [01:42<00:05, 13.44it/s] Loading 0: 90%|████████▉ | 648/723 [01:42<00:07, 9.49it/s] Loading 0: 90%|█████████ | 651/723 [01:43<00:06, 10.99it/s] Loading 0: 90%|█████████ | 653/723 [01:43<00:07, 8.75it/s] Loading 0: 91%|█████████ | 657/723 [01:43<00:05, 12.73it/s] Loading 0: 91%|█████████▏| 660/723 [01:43<00:04, 13.47it/s] Loading 0: 92%|█████████▏| 662/723 [01:44<00:06, 10.08it/s] Loading 0: 92%|█████████▏| 666/723 [01:44<00:04, 13.88it/s] Loading 0: 93%|█████████▎| 670/723 [01:44<00:03, 17.64it/s] Loading 0: 93%|█████████▎| 673/723 [01:44<00:03, 12.65it/s] Loading 0: 93%|█████████▎| 675/723 [01:45<00:04, 9.85it/s] Loading 0: 94%|█████████▍| 678/723 [01:45<00:04, 11.22it/s] Loading 0: 94%|█████████▍| 678/723 [01:57<00:04, 11.22it/s] Loading 0: 94%|█████████▍| 679/723 [01:57<01:10, 1.61s/it] Loading 0: 94%|█████████▍| 680/723 [01:57<00:59, 1.39s/it] Loading 0: 95%|█████████▍| 684/723 [01:57<00:29, 1.34it/s] Loading 0: 95%|█████████▌| 687/723 [01:57<00:18, 1.93it/s] Loading 0: 95%|█████████▌| 689/723 [01:58<00:14, 2.30it/s] Loading 0: 96%|█████████▌| 693/723 [01:58<00:08, 3.71it/s] Loading 0: 97%|█████████▋| 698/723 [01:58<00:04, 5.56it/s] Loading 0: 97%|█████████▋| 700/723 [01:58<00:04, 5.42it/s] Loading 0: 97%|█████████▋| 702/723 [01:59<00:03, 6.05it/s] Loading 0: 98%|█████████▊| 705/723 [01:59<00:02, 7.29it/s] Loading 0: 98%|█████████▊| 707/723 [01:59<00:02, 6.80it/s] Loading 0: 98%|█████████▊| 711/723 [01:59<00:01, 9.94it/s] Loading 0: 99%|█████████▉| 714/723 [02:00<00:00, 11.35it/s] Loading 0: 99%|█████████▉| 716/723 [02:00<00:00, 9.12it/s] Loading 0: 100%|█████████▉| 720/723 [02:00<00:00, 12.62it/s] Loading 0: 100%|█████████▉| 722/723 [02:11<00:00, 12.62it/s] Loading 0: 100%|██████████| 723/723 [02:11<00:00, 1.12s/it] Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
undi95-meta-llama-3-70b-6209-v4-mkmlizer: quantized model in 146.546s
undi95-meta-llama-3-70b-6209-v4-mkmlizer: Processed model Undi95/Meta-Llama-3-70B-Instruct-hf in 489.733s
undi95-meta-llama-3-70b-6209-v4-mkmlizer: creating bucket guanaco-mkml-models
undi95-meta-llama-3-70b-6209-v4-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
undi95-meta-llama-3-70b-6209-v4-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/undi95-meta-llama-3-70b-6209-v4
undi95-meta-llama-3-70b-6209-v4-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/undi95-meta-llama-3-70b-6209-v4/config.json
undi95-meta-llama-3-70b-6209-v4-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/undi95-meta-llama-3-70b-6209-v4/special_tokens_map.json
undi95-meta-llama-3-70b-6209-v4-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/undi95-meta-llama-3-70b-6209-v4/tokenizer_config.json
undi95-meta-llama-3-70b-6209-v4-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/undi95-meta-llama-3-70b-6209-v4/tokenizer.json
undi95-meta-llama-3-70b-6209-v4-mkmlizer: cp /dev/shm/model_cache/flywheel_model.5.safetensors s3://guanaco-mkml-models/undi95-meta-llama-3-70b-6209-v4/flywheel_model.5.safetensors
undi95-meta-llama-3-70b-6209-v4-mkmlizer: cp /dev/shm/model_cache/flywheel_model.2.safetensors s3://guanaco-mkml-models/undi95-meta-llama-3-70b-6209-v4/flywheel_model.2.safetensors
undi95-meta-llama-3-70b-6209-v4-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/undi95-meta-llama-3-70b-6209-v4/flywheel_model.0.safetensors
undi95-meta-llama-3-70b-6209-v4-mkmlizer: cp /dev/shm/model_cache/flywheel_model.4.safetensors s3://guanaco-mkml-models/undi95-meta-llama-3-70b-6209-v4/flywheel_model.4.safetensors
undi95-meta-llama-3-70b-6209-v4-mkmlizer: cp /dev/shm/model_cache/flywheel_model.3.safetensors s3://guanaco-mkml-models/undi95-meta-llama-3-70b-6209-v4/flywheel_model.3.safetensors
undi95-meta-llama-3-70b-6209-v4-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/undi95-meta-llama-3-70b-6209-v4/flywheel_model.1.safetensors
undi95-meta-llama-3-70b-6209-v4-mkmlizer: loading reward model from rirv938/gpt2_ties_merge_preference_plus_classic_e2_density_99
undi95-meta-llama-3-70b-6209-v4-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:919: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
undi95-meta-llama-3-70b-6209-v4-mkmlizer: warnings.warn(
undi95-meta-llama-3-70b-6209-v4-mkmlizer: /opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py:1132: FutureWarning: `resume_download` is deprecated and will be removed in version 1.0.0. Downloads always resume when possible. If you want to force a new download, use `force_download=True`.
undi95-meta-llama-3-70b-6209-v4-mkmlizer: warnings.warn(
undi95-meta-llama-3-70b-6209-v4-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:769: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
undi95-meta-llama-3-70b-6209-v4-mkmlizer: warnings.warn(
undi95-meta-llama-3-70b-6209-v4-mkmlizer: Downloading shards: 0%| | 0/1 [00:00<?, ?it/s] Downloading shards: 100%|██████████| 1/1 [00:02<00:00, 2.93s/it] Downloading shards: 100%|██████████| 1/1 [00:02<00:00, 2.93s/it]
undi95-meta-llama-3-70b-6209-v4-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
undi95-meta-llama-3-70b-6209-v4-mkmlizer: Saving duration: 0.123s
undi95-meta-llama-3-70b-6209-v4-mkmlizer: Processed model rirv938/gpt2_ties_merge_preference_plus_classic_e2_density_99 in 5.806s
undi95-meta-llama-3-70b-6209-v4-mkmlizer: creating bucket guanaco-reward-models
undi95-meta-llama-3-70b-6209-v4-mkmlizer: Bucket 's3://guanaco-reward-models/' created
undi95-meta-llama-3-70b-6209-v4-mkmlizer: uploading /tmp/reward_cache to s3://guanaco-reward-models/undi95-meta-llama-3-70b-6209-v4_reward
undi95-meta-llama-3-70b-6209-v4-mkmlizer: cp /tmp/reward_cache/config.json s3://guanaco-reward-models/undi95-meta-llama-3-70b-6209-v4_reward/config.json
undi95-meta-llama-3-70b-6209-v4-mkmlizer: cp /tmp/reward_cache/special_tokens_map.json s3://guanaco-reward-models/undi95-meta-llama-3-70b-6209-v4_reward/special_tokens_map.json
undi95-meta-llama-3-70b-6209-v4-mkmlizer: cp /tmp/reward_cache/tokenizer_config.json s3://guanaco-reward-models/undi95-meta-llama-3-70b-6209-v4_reward/tokenizer_config.json
undi95-meta-llama-3-70b-6209-v4-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/undi95-meta-llama-3-70b-6209-v4_reward/merges.txt
undi95-meta-llama-3-70b-6209-v4-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/undi95-meta-llama-3-70b-6209-v4_reward/vocab.json
undi95-meta-llama-3-70b-6209-v4-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/undi95-meta-llama-3-70b-6209-v4_reward/tokenizer.json
undi95-meta-llama-3-70b-6209-v4-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/undi95-meta-llama-3-70b-6209-v4_reward/reward.tensors
Job undi95-meta-llama-3-70b-6209-v4-mkmlizer completed after 589.57s with status: succeeded
Stopping job with name undi95-meta-llama-3-70b-6209-v4-mkmlizer
Pipeline stage MKMLizer completed in 1537.43s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.52s
Running pipeline stage ISVCDeployer
Creating inference service undi95-meta-llama-3-70b-6209-v4
Waiting for inference service undi95-meta-llama-3-70b-6209-v4 to be ready
Inference service undi95-meta-llama-3-70b-6209-v4 ready after 156.2346272468567s
Pipeline stage ISVCDeployer completed in 160.00s
Running pipeline stage StressChecker
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 5.913599967956543s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 5.632144927978516s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 5.652750730514526s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 5.7990100383758545s
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 4.704571008682251s
5 requests
0 failed requests
5th percentile: 4.8900857925415036
10th percentile: 5.075600576400757
20th percentile: 5.446630144119263
30th percentile: 5.636266088485717
40th percentile: 5.644508409500122
50th percentile: 5.652750730514526
60th percentile: 5.711254453659057
70th percentile: 5.769758176803589
80th percentile: 5.821928024291992
90th percentile: 5.867763996124268
95th percentile: 5.890681982040405
99th percentile: 5.909016370773315
mean time: 5.540415334701538
%s, retrying in %s seconds...
HTTP Request: %s %s "%s %d %s"
Received healthy response to inference request in 4.781125783920288s
Shutting down server chaiverse_console.server.app.
HTTP Request: %s %s "%s %d %s"

Usage Metrics

Latency Metrics