Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name randomforest1024-v25-v13-mkmlizer
Waiting for job on randomforest1024-v25-v13-mkmlizer to finish
randomforest1024-v25-v13-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
randomforest1024-v25-v13-mkmlizer: ║ _____ __ __ ║
randomforest1024-v25-v13-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
randomforest1024-v25-v13-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
randomforest1024-v25-v13-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
randomforest1024-v25-v13-mkmlizer: ║ /___/ ║
randomforest1024-v25-v13-mkmlizer: ║ ║
randomforest1024-v25-v13-mkmlizer: ║ Version: 0.11.12 ║
randomforest1024-v25-v13-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
randomforest1024-v25-v13-mkmlizer: ║ https://mk1.ai ║
randomforest1024-v25-v13-mkmlizer: ║ ║
randomforest1024-v25-v13-mkmlizer: ║ The license key for the current software has been verified as ║
randomforest1024-v25-v13-mkmlizer: ║ belonging to: ║
randomforest1024-v25-v13-mkmlizer: ║ ║
randomforest1024-v25-v13-mkmlizer: ║ Chai Research Corp. ║
randomforest1024-v25-v13-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
randomforest1024-v25-v13-mkmlizer: ║ Expiration: 2025-04-15 23:59:59 ║
randomforest1024-v25-v13-mkmlizer: ║ ║
randomforest1024-v25-v13-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
randomforest1024-v25-v13-mkmlizer: Traceback (most recent call last):
randomforest1024-v25-v13-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/urllib3/connectionpool.py", line 536, in _make_request
randomforest1024-v25-v13-mkmlizer: response = conn.getresponse()
randomforest1024-v25-v13-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/urllib3/connection.py", line 461, in getresponse
randomforest1024-v25-v13-mkmlizer: httplib_response = super().getresponse()
randomforest1024-v25-v13-mkmlizer: File "/opt/conda/lib/python3.10/http/client.py", line 1375, in getresponse
randomforest1024-v25-v13-mkmlizer: response.begin()
randomforest1024-v25-v13-mkmlizer: File "/opt/conda/lib/python3.10/http/client.py", line 318, in begin
randomforest1024-v25-v13-mkmlizer: version, status, reason = self._read_status()
randomforest1024-v25-v13-mkmlizer: File "/opt/conda/lib/python3.10/http/client.py", line 279, in _read_status
randomforest1024-v25-v13-mkmlizer: line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
randomforest1024-v25-v13-mkmlizer: File "/opt/conda/lib/python3.10/socket.py", line 705, in readinto
randomforest1024-v25-v13-mkmlizer: return self._sock.recv_into(b)
randomforest1024-v25-v13-mkmlizer: File "/opt/conda/lib/python3.10/ssl.py", line 1307, in recv_into
randomforest1024-v25-v13-mkmlizer: return self.read(nbytes, buffer)
randomforest1024-v25-v13-mkmlizer: File "/opt/conda/lib/python3.10/ssl.py", line 1163, in read
randomforest1024-v25-v13-mkmlizer: return self._sslobj.read(len, buffer)
randomforest1024-v25-v13-mkmlizer: TimeoutError: The read operation timed out
randomforest1024-v25-v13-mkmlizer: The above exception was the direct cause of the following exception:
randomforest1024-v25-v13-mkmlizer: Traceback (most recent call last):
randomforest1024-v25-v13-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/requests/adapters.py", line 667, in send
randomforest1024-v25-v13-mkmlizer: resp = conn.urlopen(
randomforest1024-v25-v13-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/urllib3/connectionpool.py", line 844, in urlopen
randomforest1024-v25-v13-mkmlizer: retries = retries.increment(
randomforest1024-v25-v13-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/urllib3/util/retry.py", line 470, in increment
randomforest1024-v25-v13-mkmlizer: raise reraise(type(error), error, _stacktrace)
randomforest1024-v25-v13-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/urllib3/util/util.py", line 39, in reraise
randomforest1024-v25-v13-mkmlizer: raise value
randomforest1024-v25-v13-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/urllib3/connectionpool.py", line 790, in urlopen
randomforest1024-v25-v13-mkmlizer: response = self._make_request(
randomforest1024-v25-v13-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/urllib3/connectionpool.py", line 538, in _make_request
randomforest1024-v25-v13-mkmlizer: self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
randomforest1024-v25-v13-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/urllib3/connectionpool.py", line 370, in _raise_timeout
randomforest1024-v25-v13-mkmlizer: raise ReadTimeoutError(
randomforest1024-v25-v13-mkmlizer: urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='huggingface.co', port=443): Read timed out. (read timeout=10)
randomforest1024-v25-v13-mkmlizer: During handling of the above exception, another exception occurred:
randomforest1024-v25-v13-mkmlizer: Traceback (most recent call last):
randomforest1024-v25-v13-mkmlizer: File "/code/uploading/mkmlize.py", line 151, in <module>
randomforest1024-v25-v13-mkmlizer: cli()
randomforest1024-v25-v13-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
randomforest1024-v25-v13-mkmlizer: return self.main(*args, **kwargs)
randomforest1024-v25-v13-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1078, in main
randomforest1024-v25-v13-mkmlizer: rv = self.invoke(ctx)
randomforest1024-v25-v13-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
randomforest1024-v25-v13-mkmlizer: return _process_result(sub_ctx.command.invoke(sub_ctx))
randomforest1024-v25-v13-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
randomforest1024-v25-v13-mkmlizer: return ctx.invoke(self.callback, **ctx.params)
randomforest1024-v25-v13-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/click/core.py", line 783, in invoke
randomforest1024-v25-v13-mkmlizer: return __callback(*args, **kwargs)
randomforest1024-v25-v13-mkmlizer: File "/code/uploading/mkmlize.py", line 38, in quantize
randomforest1024-v25-v13-mkmlizer: temp_folder = download_to_shared_memory(repo_id, revision, hf_auth_token)
randomforest1024-v25-v13-mkmlizer: File "/code/uploading/mkmlize.py", line 65, in download_to_shared_memory
randomforest1024-v25-v13-mkmlizer: snapshot_download(
randomforest1024-v25-v13-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
randomforest1024-v25-v13-mkmlizer: return fn(*args, **kwargs)
randomforest1024-v25-v13-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/_snapshot_download.py", line 292, in snapshot_download
randomforest1024-v25-v13-mkmlizer: _inner_hf_hub_download(file)
randomforest1024-v25-v13-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/_snapshot_download.py", line 268, in _inner_hf_hub_download
randomforest1024-v25-v13-mkmlizer: return hf_hub_download(
randomforest1024-v25-v13-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_validators.py", line 114, in _inner_fn
randomforest1024-v25-v13-mkmlizer: return fn(*args, **kwargs)
randomforest1024-v25-v13-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1202, in hf_hub_download
randomforest1024-v25-v13-mkmlizer: return _hf_hub_download_to_local_dir(
randomforest1024-v25-v13-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1487, in _hf_hub_download_to_local_dir
randomforest1024-v25-v13-mkmlizer: _download_to_tmp_and_move(
randomforest1024-v25-v13-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 1884, in _download_to_tmp_and_move
randomforest1024-v25-v13-mkmlizer: http_get(
randomforest1024-v25-v13-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 459, in http_get
randomforest1024-v25-v13-mkmlizer: r = _request_wrapper(
randomforest1024-v25-v13-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/file_download.py", line 395, in _request_wrapper
randomforest1024-v25-v13-mkmlizer: response = get_session().request(method=method, url=url, **params)
randomforest1024-v25-v13-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/requests/sessions.py", line 589, in request
randomforest1024-v25-v13-mkmlizer: resp = self.send(prep, **send_kwargs)
randomforest1024-v25-v13-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/requests/sessions.py", line 703, in send
randomforest1024-v25-v13-mkmlizer: r = adapter.send(request, **kwargs)
randomforest1024-v25-v13-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_http.py", line 66, in send
randomforest1024-v25-v13-mkmlizer: return super().send(request, *args, **kwargs)
randomforest1024-v25-v13-mkmlizer: File "/opt/conda/lib/python3.10/site-packages/requests/adapters.py", line 713, in send
randomforest1024-v25-v13-mkmlizer: raise ReadTimeout(e, request=request)
randomforest1024-v25-v13-mkmlizer: requests.exceptions.ReadTimeout: (ReadTimeoutError("HTTPSConnectionPool(host='huggingface.co', port=443): Read timed out. (read timeout=10)"), '(Request ID: bb35bc56-a2a8-46af-ad0b-9fb2a95205cc)')
Job randomforest1024-v25-v13-mkmlizer completed after 67.8s with status: failed
Stopping job with name randomforest1024-v25-v13-mkmlizer
%s, retrying in %s seconds...
Starting job with name randomforest1024-v25-v13-mkmlizer
Waiting for job on randomforest1024-v25-v13-mkmlizer to finish
randomforest1024-v25-v13-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
randomforest1024-v25-v13-mkmlizer: ║ _____ __ __ ║
randomforest1024-v25-v13-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
randomforest1024-v25-v13-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
randomforest1024-v25-v13-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
randomforest1024-v25-v13-mkmlizer: ║ /___/ ║
randomforest1024-v25-v13-mkmlizer: ║ ║
randomforest1024-v25-v13-mkmlizer: ║ Version: 0.11.12 ║
randomforest1024-v25-v13-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
randomforest1024-v25-v13-mkmlizer: ║ https://mk1.ai ║
randomforest1024-v25-v13-mkmlizer: ║ ║
randomforest1024-v25-v13-mkmlizer: ║ The license key for the current software has been verified as ║
randomforest1024-v25-v13-mkmlizer: ║ belonging to: ║
randomforest1024-v25-v13-mkmlizer: ║ ║
randomforest1024-v25-v13-mkmlizer: ║ Chai Research Corp. ║
randomforest1024-v25-v13-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
randomforest1024-v25-v13-mkmlizer: ║ Expiration: 2025-04-15 23:59:59 ║
randomforest1024-v25-v13-mkmlizer: ║ ║
randomforest1024-v25-v13-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
randomforest1024-v25-v13-mkmlizer: Downloaded to shared memory in 35.939s
randomforest1024-v25-v13-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp5rgs_hiy, device:0
randomforest1024-v25-v13-mkmlizer: Saving flywheel model at /dev/shm/model_cache
randomforest1024-v25-v13-mkmlizer: quantized model in 36.194s
randomforest1024-v25-v13-mkmlizer: Processed model RandomForest1024/V25 in 72.133s
randomforest1024-v25-v13-mkmlizer: creating bucket guanaco-mkml-models
randomforest1024-v25-v13-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
randomforest1024-v25-v13-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/randomforest1024-v25-v13
randomforest1024-v25-v13-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/randomforest1024-v25-v13/config.json
randomforest1024-v25-v13-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/randomforest1024-v25-v13/special_tokens_map.json
randomforest1024-v25-v13-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/randomforest1024-v25-v13/tokenizer_config.json
randomforest1024-v25-v13-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/randomforest1024-v25-v13/tokenizer.json
randomforest1024-v25-v13-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/randomforest1024-v25-v13/flywheel_model.0.safetensors
randomforest1024-v25-v13-mkmlizer:
Loading 0: 0%| | 0/363 [00:00<?, ?it/s]
Loading 0: 1%|▏ | 5/363 [00:00<00:10, 35.04it/s]
Loading 0: 4%|▎ | 13/363 [00:00<00:06, 53.26it/s]
Loading 0: 5%|▌ | 19/363 [00:00<00:08, 41.57it/s]
Loading 0: 7%|▋ | 24/363 [00:00<00:08, 39.37it/s]
Loading 0: 8%|▊ | 30/363 [00:00<00:07, 44.85it/s]
Loading 0: 10%|▉ | 35/363 [00:00<00:07, 44.33it/s]
Loading 0: 11%|█ | 40/363 [00:00<00:07, 44.69it/s]
Loading 0: 13%|█▎ | 46/363 [00:01<00:07, 41.19it/s]
Loading 0: 14%|█▍ | 51/363 [00:01<00:07, 39.60it/s]
Loading 0: 15%|█▌ | 56/363 [00:01<00:07, 40.37it/s]
Loading 0: 17%|█▋ | 61/363 [00:01<00:11, 26.46it/s]
Loading 0: 18%|█▊ | 65/363 [00:01<00:10, 27.40it/s]
Loading 0: 20%|█▉ | 71/363 [00:01<00:08, 33.48it/s]
Loading 0: 21%|██ | 75/363 [00:02<00:08, 33.70it/s]
Loading 0: 22%|██▏ | 80/363 [00:02<00:07, 36.60it/s]
Loading 0: 23%|██▎ | 85/363 [00:02<00:07, 36.62it/s]
Loading 0: 25%|██▍ | 89/363 [00:02<00:07, 36.54it/s]
Loading 0: 26%|██▌ | 93/363 [00:02<00:07, 35.95it/s]
Loading 0: 27%|██▋ | 98/363 [00:02<00:06, 39.51it/s]
Loading 0: 28%|██▊ | 103/363 [00:02<00:06, 39.41it/s]
Loading 0: 30%|██▉ | 108/363 [00:02<00:06, 41.05it/s]
Loading 0: 31%|███ | 113/363 [00:03<00:07, 34.57it/s]
Loading 0: 33%|███▎ | 118/363 [00:03<00:07, 34.37it/s]
Loading 0: 34%|███▍ | 123/363 [00:03<00:06, 37.34it/s]
Loading 0: 35%|███▍ | 127/363 [00:03<00:06, 35.36it/s]
Loading 0: 37%|███▋ | 134/363 [00:03<00:05, 42.75it/s]
Loading 0: 38%|███▊ | 139/363 [00:03<00:05, 41.93it/s]
Loading 0: 40%|███▉ | 144/363 [00:04<00:09, 22.54it/s]
Loading 0: 41%|████ | 149/363 [00:04<00:08, 25.07it/s]
Loading 0: 43%|████▎ | 156/363 [00:04<00:06, 31.78it/s]
Loading 0: 44%|████▍ | 161/363 [00:04<00:05, 34.63it/s]
Loading 0: 46%|████▌ | 166/363 [00:04<00:05, 37.49it/s]
Loading 0: 47%|████▋ | 172/363 [00:04<00:05, 37.45it/s]
Loading 0: 49%|████▉ | 177/363 [00:04<00:04, 37.66it/s]
Loading 0: 50%|█████ | 183/363 [00:05<00:04, 40.67it/s]
Loading 0: 52%|█████▏ | 188/363 [00:05<00:04, 41.79it/s]
Loading 0: 53%|█████▎ | 193/363 [00:05<00:03, 43.08it/s]
Loading 0: 55%|█████▍ | 199/363 [00:05<00:03, 41.46it/s]
Loading 0: 56%|█████▌ | 204/363 [00:05<00:03, 41.10it/s]
Loading 0: 58%|█████▊ | 211/363 [00:05<00:03, 46.04it/s]
Loading 0: 60%|█████▉ | 217/363 [00:05<00:03, 44.12it/s]
Loading 0: 61%|██████ | 222/363 [00:05<00:03, 45.36it/s]
Loading 0: 63%|██████▎ | 227/363 [00:06<00:04, 28.56it/s]
Loading 0: 64%|██████▎ | 231/363 [00:06<00:04, 29.19it/s]
Loading 0: 65%|██████▌ | 237/363 [00:06<00:03, 33.01it/s]
Loading 0: 66%|██████▋ | 241/363 [00:06<00:03, 34.14it/s]
Loading 0: 68%|██████▊ | 246/363 [00:06<00:03, 37.61it/s]
Loading 0: 69%|██████▉ | 251/363 [00:06<00:02, 37.62it/s]
Loading 0: 71%|███████ | 256/363 [00:06<00:02, 38.62it/s]
Loading 0: 72%|███████▏ | 262/363 [00:07<00:02, 38.11it/s]
Loading 0: 73%|███████▎ | 266/363 [00:07<00:02, 36.63it/s]
Loading 0: 75%|███████▌ | 273/363 [00:07<00:02, 43.63it/s]
Loading 0: 77%|███████▋ | 278/363 [00:07<00:01, 44.14it/s]
Loading 0: 78%|███████▊ | 283/363 [00:07<00:01, 43.04it/s]
Loading 0: 79%|███████▉ | 288/363 [00:07<00:01, 40.62it/s]
Loading 0: 81%|████████ | 293/363 [00:07<00:02, 34.86it/s]
Loading 0: 82%|████████▏ | 299/363 [00:08<00:01, 39.84it/s]
Loading 0: 84%|████████▎ | 304/363 [00:14<00:23, 2.48it/s]
Loading 0: 85%|████████▍ | 307/363 [00:14<00:18, 3.02it/s]
Loading 0: 86%|████████▌ | 312/363 [00:15<00:11, 4.27it/s]
Loading 0: 88%|████████▊ | 319/363 [00:15<00:06, 6.71it/s]
Loading 0: 89%|████████▉ | 324/363 [00:15<00:04, 8.83it/s]
Loading 0: 91%|█████████ | 329/363 [00:15<00:02, 11.48it/s]
Loading 0: 92%|█████████▏| 335/363 [00:15<00:01, 14.80it/s]
Loading 0: 94%|█████████▎| 340/363 [00:15<00:01, 17.95it/s]
Loading 0: 96%|█████████▌| 347/363 [00:15<00:00, 23.97it/s]
Loading 0: 97%|█████████▋| 353/363 [00:15<00:00, 26.68it/s]
Loading 0: 99%|█████████▊| 358/363 [00:16<00:00, 29.10it/s]
Job randomforest1024-v25-v13-mkmlizer completed after 131.88s with status: succeeded
Stopping job with name randomforest1024-v25-v13-mkmlizer
Pipeline stage MKMLizer completed in 200.66s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.16s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service randomforest1024-v25-v13
Waiting for inference service randomforest1024-v25-v13 to be ready
Inference service randomforest1024-v25-v13 ready after 190.6673765182495s
Pipeline stage MKMLDeployer completed in 191.27s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 3.006361246109009s
Received healthy response to inference request in 1.9224486351013184s
Received healthy response to inference request in 2.0163943767547607s
Received healthy response to inference request in 1.8834717273712158s
Received healthy response to inference request in 1.6753363609313965s
5 requests
0 failed requests
5th percentile: 1.7169634342193603
10th percentile: 1.7585905075073243
20th percentile: 1.841844654083252
30th percentile: 1.8912671089172364
40th percentile: 1.9068578720092773
50th percentile: 1.9224486351013184
60th percentile: 1.9600269317626953
70th percentile: 1.9976052284240722
80th percentile: 2.2143877506256104
90th percentile: 2.61037449836731
95th percentile: 2.808367872238159
99th percentile: 2.9667625713348387
mean time: 2.10080246925354
Pipeline stage StressChecker completed in 12.23s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.75s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.74s
Shutdown handler de-registered
randomforest1024-v25_v13 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeleter
Skipping teardown as no inference service was successfully deployed
Pipeline stage MKMLProfilerDeleter completed in 0.15s
run pipeline stage %s
Running pipeline stage MKMLProfilerTemplater
Pipeline stage MKMLProfilerTemplater completed in 0.13s
run pipeline stage %s
Running pipeline stage MKMLProfilerDeployer
Creating inference service randomforest1024-v25-v13-profiler
Waiting for inference service randomforest1024-v25-v13-profiler to be ready
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 4002.63s
Shutdown handler de-registered
randomforest1024-v25_v13 status is now inactive due to auto deactivation removed underperforming models
randomforest1024-v25_v13 status is now torndown due to DeploymentManager action
randomforest1024-v25_v13 status is now torndown due to DeploymentManager action
randomforest1024-v25_v13 status is now torndown due to DeploymentManager action