Running pipeline stage MKMLizer
Starting job with name mistralai-mixtral-8x7b-3473-v117-mkmlizer
Waiting for job on mistralai-mixtral-8x7b-3473-v117-mkmlizer to finish
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ║ _____ __ __ ║
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ║ /___/ ║
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ║ ║
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ║ Version: 0.9.9 ║
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ║ https://mk1.ai ║
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ║ ║
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ║ The license key for the current software has been verified as ║
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ║ belonging to: ║
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ║ ║
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ║ Chai Research Corp. ║
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ║ ║
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
mistralai-mixtral-8x7b-3473-v117-mkmlizer: Traceback (most recent call last):
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/urllib3/connectionpool.py", line 467, in _make_request
mistralai-mixtral-8x7b-3473-v117-mkmlizer: self._validate_conn(conn)
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1096, in _validate_conn
mistralai-mixtral-8x7b-3473-v117-mkmlizer: conn.connect()
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/urllib3/connection.py", line 642, in connect
mistralai-mixtral-8x7b-3473-v117-mkmlizer: sock_and_verified = _ssl_wrap_socket_and_match_hostname(
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/urllib3/connection.py", line 782, in _ssl_wrap_socket_and_match_hostname
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ssl_sock = ssl_wrap_socket(
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 470, in ssl_wrap_socket
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls, server_hostname)
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/urllib3/util/ssl_.py", line 514, in _ssl_wrap_socket_impl
mistralai-mixtral-8x7b-3473-v117-mkmlizer: return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/ssl.py", line 513, in wrap_socket
mistralai-mixtral-8x7b-3473-v117-mkmlizer: return self.sslsocket_class._create(
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/ssl.py", line 1104, in _create
mistralai-mixtral-8x7b-3473-v117-mkmlizer: self.do_handshake()
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/ssl.py", line 1375, in do_handshake
mistralai-mixtral-8x7b-3473-v117-mkmlizer: self._sslobj.do_handshake()
mistralai-mixtral-8x7b-3473-v117-mkmlizer: TimeoutError: _ssl.c:990: The handshake operation timed out
mistralai-mixtral-8x7b-3473-v117-mkmlizer: The above exception was the direct cause of the following exception:
mistralai-mixtral-8x7b-3473-v117-mkmlizer: Traceback (most recent call last):
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/requests/adapters.py", line 667, in send
mistralai-mixtral-8x7b-3473-v117-mkmlizer: resp = conn.urlopen(
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/urllib3/connectionpool.py", line 844, in urlopen
mistralai-mixtral-8x7b-3473-v117-mkmlizer: retries = retries.increment(
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/urllib3/util/retry.py", line 470, in increment
mistralai-mixtral-8x7b-3473-v117-mkmlizer: raise reraise(type(error), error, _stacktrace)
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/urllib3/util/util.py", line 39, in reraise
mistralai-mixtral-8x7b-3473-v117-mkmlizer: raise value
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/urllib3/connectionpool.py", line 790, in urlopen
mistralai-mixtral-8x7b-3473-v117-mkmlizer: response = self._make_request(
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/urllib3/connectionpool.py", line 491, in _make_request
mistralai-mixtral-8x7b-3473-v117-mkmlizer: raise new_e
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/urllib3/connectionpool.py", line 469, in _make_request
mistralai-mixtral-8x7b-3473-v117-mkmlizer: self._raise_timeout(err=e, url=url, timeout_value=conn.timeout)
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/urllib3/connectionpool.py", line 370, in _raise_timeout
mistralai-mixtral-8x7b-3473-v117-mkmlizer: raise ReadTimeoutError(
mistralai-mixtral-8x7b-3473-v117-mkmlizer: urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='cdn-lfs-us-1.huggingface.co', port=443): Read timed out. (read timeout=10)
mistralai-mixtral-8x7b-3473-v117-mkmlizer: During handling of the above exception, another exception occurred:
mistralai-mixtral-8x7b-3473-v117-mkmlizer: Traceback (most recent call last):
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/code/uploading/mkmlize.py", line 151, in <module>
mistralai-mixtral-8x7b-3473-v117-mkmlizer: cli()
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
mistralai-mixtral-8x7b-3473-v117-mkmlizer: return self.main(*args, **kwargs)
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1078, in main
mistralai-mixtral-8x7b-3473-v117-mkmlizer: rv = self.invoke(ctx)
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
mistralai-mixtral-8x7b-3473-v117-mkmlizer: return _process_result(sub_ctx.command.invoke(sub_ctx))
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
mistralai-mixtral-8x7b-3473-v117-mkmlizer: return ctx.invoke(self.callback, **ctx.params)
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 783, in invoke
mistralai-mixtral-8x7b-3473-v117-mkmlizer: return __callback(*args, **kwargs)
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/code/uploading/mkmlize.py", line 38, in quantize
mistralai-mixtral-8x7b-3473-v117-mkmlizer: temp_folder = download_to_shared_memory(repo_id, revision, hf_auth_token)
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/code/uploading/mkmlize.py", line 65, in download_to_shared_memory
mistralai-mixtral-8x7b-3473-v117-mkmlizer: snapshot_download(
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
mistralai-mixtral-8x7b-3473-v117-mkmlizer: return fn(*args, **kwargs)
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/_snapshot_download.py", line 297, in snapshot_download
mistralai-mixtral-8x7b-3473-v117-mkmlizer: _inner_hf_hub_download(file)
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/_snapshot_download.py", line 273, in _inner_hf_hub_download
mistralai-mixtral-8x7b-3473-v117-mkmlizer: return hf_hub_download(
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_deprecation.py", line 101, in inner_f
mistralai-mixtral-8x7b-3473-v117-mkmlizer: return f(*args, **kwargs)
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
mistralai-mixtral-8x7b-3473-v117-mkmlizer: return fn(*args, **kwargs)
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1220, in hf_hub_download
mistralai-mixtral-8x7b-3473-v117-mkmlizer: return _hf_hub_download_to_local_dir(
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1515, in _hf_hub_download_to_local_dir
mistralai-mixtral-8x7b-3473-v117-mkmlizer: _download_to_tmp_and_move(
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1915, in _download_to_tmp_and_move
mistralai-mixtral-8x7b-3473-v117-mkmlizer: http_get(
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 463, in http_get
mistralai-mixtral-8x7b-3473-v117-mkmlizer: r = _request_wrapper(
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 399, in _request_wrapper
mistralai-mixtral-8x7b-3473-v117-mkmlizer: response = get_session().request(method=method, url=url, **params)
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/requests/sessions.py", line 589, in request
mistralai-mixtral-8x7b-3473-v117-mkmlizer: resp = self.send(prep, **send_kwargs)
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/requests/sessions.py", line 703, in send
mistralai-mixtral-8x7b-3473-v117-mkmlizer: r = adapter.send(request, **kwargs)
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_http.py", line 66, in send
mistralai-mixtral-8x7b-3473-v117-mkmlizer: return super().send(request, *args, **kwargs)
mistralai-mixtral-8x7b-3473-v117-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/requests/adapters.py", line 713, in send
mistralai-mixtral-8x7b-3473-v117-mkmlizer: raise ReadTimeout(e, request=request)
mistralai-mixtral-8x7b-3473-v117-mkmlizer: requests.exceptions.ReadTimeout: (ReadTimeoutError("HTTPSConnectionPool(host='cdn-lfs-us-1.huggingface.co', port=443): Read timed out. (read timeout=10)"), '(Request ID: e5a60f21-ccf8-4865-97cc-8cd6f41cdd42)')
Job mistralai-mixtral-8x7b-3473-v117-mkmlizer completed after 190.83s with status: failed
Stopping job with name mistralai-mixtral-8x7b-3473-v117-mkmlizer
%s, retrying in %s seconds...
Starting job with name mistralai-mixtral-8x7b-3473-v117-mkmlizer
Waiting for job on mistralai-mixtral-8x7b-3473-v117-mkmlizer to finish
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ║ _____ __ __ ║
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ║ /___/ ║
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ║ ║
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ║ Version: 0.9.9 ║
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ║ https://mk1.ai ║
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ║ ║
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ║ The license key for the current software has been verified as ║
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ║ belonging to: ║
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ║ ║
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ║ Chai Research Corp. ║
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ║ Expiration: 2024-10-15 23:59:59 ║
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ║ ║
mistralai-mixtral-8x7b-3473-v117-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
mistralai-mixtral-8x7b-3473-v117-mkmlizer: Downloaded to shared memory in 183.661s
mistralai-mixtral-8x7b-3473-v117-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp42l41022, device:0
mistralai-mixtral-8x7b-3473-v117-mkmlizer: Saving flywheel model at /dev/shm/model_cache
mistralai-mixtral-8x7b-3473-v117-mkmlizer: quantized model in 90.643s
mistralai-mixtral-8x7b-3473-v117-mkmlizer: Processed model mistralai/Mixtral-8x7B-Instruct-v0.1 in 274.305s
mistralai-mixtral-8x7b-3473-v117-mkmlizer: creating bucket guanaco-mkml-models
mistralai-mixtral-8x7b-3473-v117-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
mistralai-mixtral-8x7b-3473-v117-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v117
mistralai-mixtral-8x7b-3473-v117-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v117/config.json
mistralai-mixtral-8x7b-3473-v117-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v117/tokenizer_config.json
mistralai-mixtral-8x7b-3473-v117-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v117/special_tokens_map.json
mistralai-mixtral-8x7b-3473-v117-mkmlizer: cp /dev/shm/model_cache/tokenizer.model s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v117/tokenizer.model
mistralai-mixtral-8x7b-3473-v117-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v117/tokenizer.json
mistralai-mixtral-8x7b-3473-v117-mkmlizer: cp /dev/shm/model_cache/flywheel_model.3.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v117/flywheel_model.3.safetensors
mistralai-mixtral-8x7b-3473-v117-mkmlizer: cp /dev/shm/model_cache/flywheel_model.2.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v117/flywheel_model.2.safetensors
mistralai-mixtral-8x7b-3473-v117-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v117/flywheel_model.1.safetensors
mistralai-mixtral-8x7b-3473-v117-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/mistralai-mixtral-8x7b-3473-v117/flywheel_model.0.safetensors
mistralai-mixtral-8x7b-3473-v117-mkmlizer: loading reward model from rirv938/reward_gpt2_preference_24m_e2
mistralai-mixtral-8x7b-3473-v117-mkmlizer:
Loading 0: 0%| | 0/995 [00:00<?, ?it/s]
Loading 0: 0%| | 4/995 [00:00<00:29, 33.50it/s]
Loading 0: 1%| | 8/995 [00:00<00:33, 29.90it/s]
Loading 0: 1%| | 12/995 [00:00<00:36, 26.85it/s]
Loading 0: 2%|▏ | 15/995 [00:00<00:36, 26.55it/s]
Loading 0: 2%|▏ | 18/995 [00:00<00:37, 26.23it/s]
Loading 0: 2%|▏ | 21/995 [00:00<00:37, 26.21it/s]
Loading 0: 2%|▏ | 24/995 [00:00<00:36, 26.28it/s]
Loading 0: 3%|▎ | 32/995 [00:01<00:26, 36.62it/s]
Loading 0: 4%|▎ | 36/995 [00:01<00:27, 34.52it/s]
Loading 0: 4%|▍ | 40/995 [00:01<00:28, 33.41it/s]
Loading 0: 4%|▍ | 44/995 [00:01<00:33, 28.69it/s]
Loading 0: 5%|▌ | 51/995 [00:01<00:25, 37.06it/s]
Loading 0: 6%|▌ | 55/995 [00:01<00:41, 22.92it/s]
Loading 0: 6%|▌ | 59/995 [00:02<00:38, 24.01it/s]
Loading 0: 6%|▋ | 64/995 [00:02<00:32, 28.22it/s]
Loading 0: 7%|▋ | 68/995 [00:02<00:32, 28.95it/s]
Loading 0: 7%|▋ | 72/995 [00:02<00:35, 26.25it/s]
Loading 0: 8%|▊ | 75/995 [00:02<00:35, 26.25it/s]
Loading 0: 8%|▊ | 78/995 [00:02<00:34, 26.31it/s]
Loading 0: 8%|▊ | 81/995 [00:02<00:35, 25.46it/s]
Loading 0: 8%|▊ | 84/995 [00:03<00:35, 25.65it/s]
Loading 0: 9%|▊ | 87/995 [00:03<00:35, 25.61it/s]
Loading 0: 10%|▉ | 96/995 [00:03<00:22, 40.78it/s]
Loading 0: 10%|█ | 101/995 [00:03<00:27, 32.80it/s]
Loading 0: 11%|█ | 107/995 [00:03<00:31, 28.44it/s]
Loading 0: 11%|█ | 111/995 [00:03<00:33, 26.17it/s]
Loading 0: 11%|█▏ | 114/995 [00:04<00:33, 26.34it/s]
Loading 0: 12%|█▏ | 117/995 [00:04<00:33, 26.38it/s]
Loading 0: 12%|█▏ | 120/995 [00:04<00:33, 26.28it/s]
Loading 0: 12%|█▏ | 123/995 [00:04<00:34, 25.58it/s]
Loading 0: 13%|█▎ | 128/995 [00:04<00:28, 30.31it/s]
Loading 0: 13%|█▎ | 132/995 [00:04<00:29, 29.60it/s]
Loading 0: 14%|█▎ | 136/995 [00:04<00:29, 29.57it/s]
Loading 0: 14%|█▍ | 140/995 [00:04<00:33, 25.90it/s]
Loading 0: 14%|█▍ | 143/995 [00:05<00:32, 26.10it/s]
Loading 0: 15%|█▍ | 146/995 [00:05<00:32, 26.15it/s]
Loading 0: 15%|█▍ | 149/995 [00:05<00:32, 26.24it/s]
Loading 0: 16%|█▌ | 161/995 [00:05<00:18, 45.09it/s]
Loading 0: 17%|█▋ | 166/995 [00:05<00:28, 29.11it/s]
Loading 0: 17%|█▋ | 170/995 [00:05<00:31, 26.45it/s]
Loading 0: 17%|█▋ | 174/995 [00:06<00:31, 26.41it/s]
Loading 0: 18%|█▊ | 177/995 [00:06<00:31, 26.20it/s]
Loading 0: 18%|█▊ | 180/995 [00:06<00:31, 25.88it/s]
Loading 0: 18%|█▊ | 183/995 [00:06<00:31, 26.00it/s]
Loading 0: 19%|█▉ | 188/995 [00:06<00:26, 30.26it/s]
Loading 0: 19%|█▉ | 192/995 [00:06<00:26, 30.39it/s]
Loading 0: 20%|█▉ | 196/995 [00:06<00:30, 26.55it/s]
Loading 0: 20%|██ | 199/995 [00:07<00:30, 26.34it/s]
Loading 0: 20%|██ | 202/995 [00:07<00:30, 26.19it/s]
Loading 0: 21%|██ | 209/995 [00:07<00:21, 35.86it/s]
Loading 0: 21%|██▏ | 213/995 [00:07<00:35, 21.88it/s]
Loading 0: 22%|██▏ | 216/995 [00:07<00:34, 22.78it/s]
Loading 0: 22%|██▏ | 221/995 [00:07<00:28, 27.25it/s]
Loading 0: 23%|██▎ | 225/995 [00:08<00:27, 27.72it/s]
Loading 0: 23%|██▎ | 229/995 [00:08<00:26, 28.69it/s]
Loading 0: 23%|██▎ | 233/995 [00:08<00:29, 25.52it/s]
Loading 0: 24%|██▎ | 236/995 [00:08<00:30, 24.93it/s]
Loading 0: 24%|██▍ | 239/995 [00:08<00:30, 24.90it/s]
Loading 0: 24%|██▍ | 242/995 [00:08<00:30, 25.06it/s]
Loading 0: 25%|██▌ | 251/995 [00:08<00:18, 39.80it/s]
Loading 0: 26%|██▌ | 256/995 [00:09<00:22, 32.32it/s]
Loading 0: 26%|██▌ | 260/995 [00:09<00:22, 32.70it/s]
Loading 0: 27%|██▋ | 265/995 [00:09<00:27, 26.27it/s]
Loading 0: 27%|██▋ | 269/995 [00:09<00:25, 28.63it/s]
Loading 0: 28%|██▊ | 274/995 [00:09<00:24, 29.01it/s]
Loading 0: 28%|██▊ | 278/995 [00:09<00:24, 28.95it/s]
Loading 0: 28%|██▊ | 282/995 [00:10<00:24, 28.80it/s]
Loading 0: 28%|██▊ | 282/995 [00:20<00:24, 28.80it/s]
Loading 0: 28%|██▊ | 283/995 [00:24<15:47, 1.33s/it]
Loading 0: 29%|██▊ | 285/995 [00:24<12:41, 1.07s/it]
Loading 0: 29%|██▉ | 288/995 [00:24<08:56, 1.32it/s]
Loading 0: 29%|██▉ | 291/995 [00:24<06:23, 1.84it/s]
Loading 0: 30%|██▉ | 294/995 [00:24<04:34, 2.55it/s]
Loading 0: 30%|██▉ | 297/995 [00:25<03:18, 3.51it/s]
Loading 0: 30%|███ | 300/995 [00:25<02:26, 4.75it/s]
Loading 0: 30%|███ | 303/995 [00:25<01:49, 6.29it/s]
Loading 0: 31%|███ | 306/995 [00:25<01:24, 8.16it/s]
Loading 0: 31%|███ | 309/995 [00:25<01:06, 10.34it/s]
Loading 0: 31%|███▏ | 312/995 [00:25<00:53, 12.68it/s]
Loading 0: 32%|███▏ | 320/995 [00:25<00:37, 17.81it/s]
Loading 0: 32%|███▏ | 323/995 [00:26<00:34, 19.23it/s]
Loading 0: 33%|███▎ | 326/995 [00:26<00:32, 20.59it/s]
Loading 0: 33%|███▎ | 329/995 [00:26<00:30, 21.69it/s]
Loading 0: 33%|███▎ | 332/995 [00:26<00:29, 22.72it/s]
Loading 0: 34%|███▎ | 335/995 [00:26<00:28, 23.56it/s]
Loading 0: 34%|███▍ | 338/995 [00:26<00:27, 23.98it/s]
Loading 0: 34%|███▍ | 341/995 [00:26<00:25, 25.44it/s]
Loading 0: 35%|███▍ | 345/995 [00:26<00:23, 27.70it/s]
Loading 0: 35%|███▍ | 348/995 [00:26<00:23, 26.98it/s]
Loading 0: 35%|███▌ | 351/995 [00:27<00:24, 26.52it/s]
Loading 0: 36%|███▌ | 354/995 [00:27<00:24, 26.27it/s]
Loading 0: 36%|███▌ | 357/995 [00:27<00:24, 26.42it/s]
Loading 0: 36%|███▌ | 360/995 [00:27<00:24, 26.45it/s]
Loading 0: 37%|███▋ | 367/995 [00:27<00:16, 37.28it/s]
Loading 0: 37%|███▋ | 371/995 [00:27<00:28, 21.74it/s]
Loading 0: 38%|███▊ | 376/995 [00:28<00:23, 26.05it/s]
Loading 0: 38%|███▊ | 380/995 [00:28<00:22, 26.84it/s]
Loading 0: 39%|███▊ | 384/995 [00:28<00:22, 27.31it/s]
Loading 0: 39%|███▉ | 388/995 [00:28<00:24, 24.59it/s]
Loading 0: 39%|███▉ | 391/995 [00:28<00:24, 24.86it/s]
Loading 0: 40%|███▉ | 394/995 [00:28<00:23, 25.17it/s]
Loading 0: 40%|███▉ | 397/995 [00:28<00:23, 25.42it/s]
Loading 0: 41%|████ | 405/995 [00:28<00:15, 38.58it/s]
Loading 0: 41%|████ | 410/995 [00:29<00:19, 30.59it/s]
Loading 0: 42%|████▏ | 414/995 [00:29<00:19, 29.82it/s]
Loading 0: 42%|████▏ | 418/995 [00:29<00:18, 30.58it/s]
Loading 0: 43%|████▎ | 423/995 [00:29<00:22, 25.09it/s]
Loading 0: 43%|████▎ | 426/995 [00:29<00:22, 25.11it/s]
Loading 0: 43%|████▎ | 429/995 [00:29<00:22, 25.07it/s]
Loading 0: 43%|████▎ | 432/995 [00:30<00:22, 25.21it/s]
Loading 0: 44%|████▍ | 436/995 [00:30<00:19, 28.53it/s]
Loading 0: 44%|████▍ | 440/995 [00:30<00:19, 27.79it/s]
Loading 0: 45%|████▍ | 443/995 [00:30<00:20, 27.52it/s]
Loading 0: 45%|████▍ | 446/995 [00:30<00:20, 27.39it/s]
Loading 0: 45%|████▌ | 449/995 [00:30<00:20, 27.02it/s]
Loading 0: 45%|████▌ | 452/995 [00:30<00:20, 26.57it/s]
Loading 0: 46%|████▌ | 455/995 [00:30<00:20, 26.09it/s]
Loading 0: 46%|████▌ | 458/995 [00:31<00:20, 26.25it/s]
Loading 0: 47%|████▋ | 466/995 [00:31<00:14, 36.23it/s]
Loading 0: 47%|████▋ | 470/995 [00:31<00:15, 34.11it/s]
Loading 0: 48%|████▊ | 475/995 [00:31<00:13, 37.60it/s]
Loading 0: 48%|████▊ | 479/995 [00:31<00:19, 26.45it/s]
Loading 0: 49%|████▊ | 483/995 [00:31<00:20, 24.74it/s]
Loading 0: 49%|████▉ | 486/995 [00:32<00:20, 24.63it/s]
Loading 0: 49%|████▉ | 489/995 [00:32<00:20, 24.95it/s]
Loading 0: 49%|████▉ | 492/995 [00:32<00:20, 25.10it/s]
Loading 0: 50%|████▉ | 495/995 [00:32<00:19, 25.46it/s]
Loading 0: 50%|█████ | 500/995 [00:32<00:16, 30.65it/s]
Loading 0: 51%|█████ | 504/995 [00:32<00:16, 30.07it/s]
Loading 0: 51%|█████ | 508/995 [00:32<00:15, 30.45it/s]
Loading 0: 51%|█████▏ | 512/995 [00:32<00:17, 27.01it/s]
Loading 0: 52%|█████▏ | 515/995 [00:33<00:18, 26.54it/s]
Loading 0: 52%|█████▏ | 518/995 [00:33<00:17, 26.70it/s]
Loading 0: 53%|█████▎ | 525/995 [00:33<00:12, 36.80it/s]
Loading 0: 53%|█████▎ | 529/995 [00:33<00:18, 25.75it/s]
Loading 0: 54%|█████▎ | 533/995 [00:33<00:17, 27.11it/s]
Loading 0: 54%|█████▍ | 537/995 [00:33<00:18, 25.14it/s]
Loading 0: 54%|█████▍ | 540/995 [00:33<00:17, 25.42it/s]
Loading 0: 55%|█████▍ | 543/995 [00:34<00:17, 25.80it/s]
Loading 0: 55%|█████▍ | 546/995 [00:34<00:17, 26.06it/s]
Loading 0: 55%|█████▌ | 549/995 [00:34<00:17, 25.88it/s]
Loading 0: 55%|█████▌ | 552/995 [00:34<00:17, 25.59it/s]
Loading 0: 56%|█████▋ | 561/995 [00:34<00:10, 41.05it/s]
Loading 0: 56%|█████▋ | 561/995 [00:49<00:10, 41.05it/s]
Loading 0: 56%|█████▋ | 562/995 [00:49<08:34, 1.19s/it]
Loading 0: 57%|█████▋ | 564/995 [00:49<07:02, 1.02it/s]
Loading 0: 57%|█████▋ | 568/995 [00:49<04:44, 1.50it/s]
Loading 0: 57%|█████▋ | 571/995 [00:49<03:30, 2.01it/s]
Loading 0: 58%|█████▊ | 574/995 [00:49<02:35, 2.71it/s]
Loading 0: 58%|█████▊ | 581/995 [00:50<01:27, 4.73it/s]
Loading 0: 59%|█████▊ | 584/995 [00:50<01:10, 5.81it/s]
Loading 0: 59%|█████▉ | 587/995 [00:50<00:56, 7.17it/s]
Loading 0: 59%|█████▉ | 592/995 [00:50<00:38, 10.36it/s]
Loading 0: 60%|█████▉ | 596/995 [00:50<00:32, 12.17it/s]
Loading 0: 60%|██████ | 599/995 [00:50<00:28, 13.81it/s]
Loading 0: 61%|██████ | 602/995 [00:50<00:24, 15.78it/s]
Loading 0: 61%|██████ | 605/995 [00:51<00:22, 17.64it/s]
Loading 0: 61%|██████ | 608/995 [00:51<00:19, 19.47it/s]
Loading 0: 61%|██████▏ | 611/995 [00:51<00:18, 20.88it/s]
Loading 0: 62%|██████▏ | 614/995 [00:51<00:17, 22.36it/s]
Loading 0: 63%|██████▎ | 623/995 [00:51<00:09, 37.31it/s]
Loading 0: 63%|██████▎ | 628/995 [00:51<00:11, 30.80it/s]
Loading 0: 64%|██████▎ | 633/995 [00:51<00:10, 34.56it/s]
Loading 0: 64%|██████▍ | 638/995 [00:52<00:14, 23.99it/s]
Loading 0: 65%|██████▍ | 642/995 [00:52<00:14, 25.18it/s]
Loading 0: 65%|██████▍ | 646/995 [00:52<00:13, 26.42it/s]
Loading 0: 65%|██████▌ | 650/995 [00:52<00:14, 24.29it/s]
Loading 0: 66%|██████▌ | 655/995 [00:52<00:11, 28.48it/s]
Loading 0: 66%|██████▌ | 659/995 [00:52<00:11, 28.52it/s]
Loading 0: 67%|██████▋ | 663/995 [00:53<00:11, 28.86it/s]
Loading 0: 67%|██████▋ | 667/995 [00:53<00:12, 25.44it/s]
Loading 0: 67%|██████▋ | 670/995 [00:53<00:12, 25.59it/s]
Loading 0: 68%|██████▊ | 673/995 [00:53<00:12, 25.54it/s]
Loading 0: 68%|██████▊ | 676/995 [00:53<00:12, 25.20it/s]
Loading 0: 69%|██████▉ | 685/995 [00:53<00:07, 40.24it/s]
Loading 0: 69%|██████▉ | 691/995 [00:53<00:10, 29.30it/s]
Loading 0: 70%|██████▉ | 695/995 [00:54<00:10, 29.55it/s]
Loading 0: 70%|███████ | 699/995 [00:54<00:09, 29.86it/s]
Loading 0: 71%|███████ | 703/995 [00:54<00:10, 26.68it/s]
Loading 0: 71%|███████ | 706/995 [00:54<00:10, 26.67it/s]
Loading 0: 71%|███████▏ | 709/995 [00:54<00:10, 26.61it/s]
Loading 0: 72%|███████▏ | 712/995 [00:54<00:10, 26.44it/s]
Loading 0: 72%|███████▏ | 717/995 [00:54<00:08, 31.32it/s]
Loading 0: 72%|███████▏ | 721/995 [00:55<00:09, 30.16it/s]
Loading 0: 73%|███████▎ | 725/995 [00:55<00:08, 30.15it/s]
Loading 0: 73%|███████▎ | 729/995 [00:55<00:09, 26.84it/s]
Loading 0: 74%|███████▎ | 732/995 [00:55<00:09, 26.76it/s]
Loading 0: 74%|███████▍ | 739/995 [00:55<00:10, 25.15it/s]
Loading 0: 75%|███████▍ | 742/995 [00:55<00:09, 25.37it/s]
Loading 0: 75%|███████▌ | 747/995 [00:55<00:08, 29.56it/s]
Loading 0: 75%|███████▌ | 751/995 [00:56<00:09, 26.18it/s]
Loading 0: 76%|███████▌ | 754/995 [00:56<00:09, 26.19it/s]
Loading 0: 76%|███████▌ | 757/995 [00:56<00:09, 26.12it/s]
Loading 0: 76%|███████▋ | 760/995 [00:56<00:08, 26.27it/s]
Loading 0: 77%|███████▋ | 763/995 [00:56<00:08, 26.45it/s]
Loading 0: 77%|███████▋ | 766/995 [00:56<00:08, 26.48it/s]
Loading 0: 77%|███████▋ | 769/995 [00:56<00:08, 26.15it/s]
Loading 0: 78%|███████▊ | 778/995 [00:56<00:05, 41.96it/s]
Loading 0: 79%|███████▊ | 783/995 [00:57<00:06, 34.21it/s]
Loading 0: 79%|███████▉ | 787/995 [00:57<00:06, 32.82it/s]
Loading 0: 80%|███████▉ | 793/995 [00:57<00:05, 35.31it/s]
Loading 0: 80%|████████ | 797/995 [00:57<00:08, 23.79it/s]
Loading 0: 80%|████████ | 800/995 [00:57<00:08, 24.01it/s]
Loading 0: 81%|████████ | 803/995 [00:58<00:07, 24.56it/s]
Loading 0: 81%|████████ | 808/995 [00:58<00:06, 28.56it/s]
Loading 0: 82%|████████▏ | 812/995 [00:58<00:06, 29.00it/s]
Loading 0: 82%|████████▏ | 816/995 [00:58<00:06, 25.71it/s]
Loading 0: 82%|████████▏ | 819/995 [00:58<00:06, 25.70it/s]
Loading 0: 83%|████████▎ | 822/995 [00:58<00:06, 25.64it/s]
Loading 0: 83%|████████▎ | 825/995 [00:58<00:06, 25.16it/s]
Loading 0: 83%|████████▎ | 828/995 [00:58<00:06, 25.18it/s]
Loading 0: 84%|████████▎ | 831/995 [00:59<00:06, 25.02it/s]
Loading 0: 84%|████████▍ | 840/995 [00:59<00:03, 39.99it/s]
Loading 0: 85%|████████▍ | 842/995 [01:13<00:03, 39.99it/s]
Loading 0: 85%|████████▍ | 843/995 [01:13<02:36, 1.03s/it]
Loading 0: 85%|████████▌ | 848/995 [01:13<01:42, 1.43it/s]
Loading 0: 86%|████████▌ | 853/995 [01:14<01:11, 1.99it/s]
Loading 0: 86%|████████▌ | 857/995 [01:14<00:52, 2.65it/s]
Loading 0: 87%|████████▋ | 861/995 [01:14<00:38, 3.49it/s]
Loading 0: 87%|████████▋ | 864/995 [01:14<00:30, 4.35it/s]
Loading 0: 87%|████████▋ | 867/995 [01:15<00:23, 5.46it/s]
Loading 0: 88%|████████▊ | 872/995 [01:15<00:15, 8.07it/s]
Loading 0: 88%|████████▊ | 876/995 [01:15<00:11, 10.24it/s]
Loading 0: 88%|████████▊ | 880/995 [01:15<00:09, 12.77it/s]
Loading 0: 89%|████████▉ | 884/995 [01:15<00:07, 14.40it/s]
Loading 0: 89%|████████▉ | 887/995 [01:15<00:06, 16.03it/s]
Loading 0: 89%|████████▉ | 890/995 [01:15<00:05, 17.72it/s]
Loading 0: 90%|█████████ | 897/995 [01:16<00:04, 20.13it/s]
Loading 0: 90%|█████████ | 900/995 [01:16<00:04, 21.67it/s]
Loading 0: 91%|█████████ | 903/995 [01:16<00:04, 22.46it/s]
Loading 0: 91%|█████████ | 906/995 [01:16<00:03, 22.93it/s]
Loading 0: 91%|█████████▏| 909/995 [01:16<00:03, 23.44it/s]
Loading 0: 92%|█████████▏| 912/995 [01:16<00:03, 24.10it/s]
Loading 0: 92%|█████████▏| 915/995 [01:16<00:03, 24.62it/s]
Loading 0: 92%|█████████▏| 918/995 [01:16<00:03, 25.01it/s]
Loading 0: 93%|█████████▎| 921/995 [01:17<00:03, 24.61it/s]
Loading 0: 93%|█████████▎| 924/995 [01:17<00:02, 24.36it/s]
Loading 0: 94%|█████████▍| 933/995 [01:17<00:01, 39.76it/s]
Loading 0: 94%|█████████▍| 938/995 [01:17<00:01, 32.06it/s]
Loading 0: 95%|█████████▍| 942/995 [01:17<00:01, 31.27it/s]
Loading 0: 95%|█████████▌| 946/995 [01:17<00:01, 26.83it/s]
Loading 0: 96%|█████████▌| 952/995 [01:19<00:05, 8.43it/s]
Loading 0: 96%|█████████▌| 955/995 [01:19<00:04, 9.76it/s]
Loading 0: 96%|█████████▋| 958/995 [01:19<00:03, 11.30it/s]
Loading 0: 97%|█████████▋| 961/995 [01:19<00:02, 12.98it/s]
Loading 0: 97%|█████████▋| 966/995 [01:19<00:01, 17.42it/s]
Loading 0: 97%|█████████▋| 969/995 [01:20<00:01, 18.73it/s]
Loading 0: 98%|█████████▊| 972/995 [01:20<00:01, 19.98it/s]
Loading 0: 98%|█████████▊| 975/995 [01:20<00:00, 20.82it/s]
Loading 0: 98%|█████████▊| 978/995 [01:20<00:00, 21.90it/s]
Loading 0: 99%|█████████▊| 981/995 [01:20<00:00, 22.78it/s]
Loading 0: 99%|█████████▉| 984/995 [01:20<00:00, 22.94it/s]
Loading 0: 99%|█████████▉| 987/995 [01:20<00:00, 23.68it/s]
/opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:957: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mistralai-mixtral-8x7b-3473-v117-mkmlizer: warnings.warn(
mistralai-mixtral-8x7b-3473-v117-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:785: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mistralai-mixtral-8x7b-3473-v117-mkmlizer: warnings.warn(
mistralai-mixtral-8x7b-3473-v117-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:469: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
mistralai-mixtral-8x7b-3473-v117-mkmlizer: warnings.warn(
mistralai-mixtral-8x7b-3473-v117-mkmlizer: Saving model to /tmp/reward_cache/reward.tensors
mistralai-mixtral-8x7b-3473-v117-mkmlizer: Saving duration: 0.126s
mistralai-mixtral-8x7b-3473-v117-mkmlizer: Processed model rirv938/reward_gpt2_preference_24m_e2 in 5.075s
mistralai-mixtral-8x7b-3473-v117-mkmlizer: creating bucket guanaco-reward-models
mistralai-mixtral-8x7b-3473-v117-mkmlizer: cp /tmp/reward_cache/merges.txt s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v117_reward/merges.txt
mistralai-mixtral-8x7b-3473-v117-mkmlizer: cp /tmp/reward_cache/vocab.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v117_reward/vocab.json
mistralai-mixtral-8x7b-3473-v117-mkmlizer: cp /tmp/reward_cache/tokenizer.json s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v117_reward/tokenizer.json
mistralai-mixtral-8x7b-3473-v117-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/mistralai-mixtral-8x7b-3473-v117_reward/reward.tensors
Job mistralai-mixtral-8x7b-3473-v117-mkmlizer completed after 322.41s with status: succeeded
Stopping job with name mistralai-mixtral-8x7b-3473-v117-mkmlizer
Pipeline stage MKMLizer completed in 514.89s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.13s
Running pipeline stage ISVCDeployer
Creating inference service mistralai-mixtral-8x7b-3473-v117
Waiting for inference service mistralai-mixtral-8x7b-3473-v117 to be ready
Inference service mistralai-mixtral-8x7b-3473-v117 ready after 181.48370218276978s
Pipeline stage ISVCDeployer completed in 184.03s
Running pipeline stage StressChecker
Received healthy response to inference request in 1.7801523208618164s
Received healthy response to inference request in 1.0075623989105225s
Received healthy response to inference request in 0.8280165195465088s
Received healthy response to inference request in 0.7649879455566406s
Received healthy response to inference request in 0.7351024150848389s
5 requests
0 failed requests
5th percentile: 0.7410795211791992
10th percentile: 0.7470566272735596
20th percentile: 0.7590108394622803
30th percentile: 0.7775936603546143
40th percentile: 0.8028050899505615
50th percentile: 0.8280165195465088
60th percentile: 0.8998348712921143
70th percentile: 0.9716532230377197
80th percentile: 1.1620803833007813
90th percentile: 1.4711163520812989
95th percentile: 1.6256343364715575
99th percentile: 1.7492487239837646
mean time: 1.0231643199920655
Pipeline stage StressChecker completed in 6.59s
mistralai-mixtral-8x7b_3473_v117 status is now deployed due to DeploymentManager action
mistralai-mixtral-8x7b_3473_v117 status is now inactive due to auto deactivation removed underperforming models
mistralai-mixtral-8x7b_3473_v117 status is now torndown due to DeploymentManager action