submission_id: arlineka-nyannyan_v2
developer_uid: arlineka
formatter: {'memory_template': "<|start_header_id|>system<|end_header_id|>\n\n{bot_name}'s Persona: {memory}\n\n", 'prompt_template': "{prompt}<|eot_id|>', 'bot_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}: {message}<|eot_id|>", 'bot_template': '{bot_name}: {message}\n', 'user_template': '<|start_header_id|>user<|end_header_id|>\n\n{user_name}: {message}<|eot_id|>', 'response_template': '<|start_header_id|>assistant<|end_header_id|>\n\n{bot_name}:', 'truncate_by_message': False}
generation_params: {'temperature': 1.1, 'top_p': 1.0, 'min_p': 0.08, 'top_k': 100, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 512, 'best_of': 16, 'max_output_tokens': 64}
model_name: arlineka-nyannyan_v2
model_repo: arlineka/NyanNyan
reward_formatter: {'bot_template': '{bot_name}: {message}\n', 'memory_template': "{bot_name}'s Persona: {memory}\n####\n", 'prompt_template': '{prompt}\n<START>\n', 'response_template': '{bot_name}:', 'truncate_by_message': False, 'user_template': '{user_name}: {message}\n'}
reward_repo: ChaiML/reward_gpt2_medium_preference_24m_e2
status: torndown
timestamp: 2024-05-31T18:11:49+00:00
Resubmit model
Running pipeline stage MKMLizer
Starting job with name arlineka-nyannyan-v2-mkmlizer
Waiting for job on arlineka-nyannyan-v2-mkmlizer to finish
arlineka-nyannyan-v2-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
arlineka-nyannyan-v2-mkmlizer: ║ _____ __ __ ║
arlineka-nyannyan-v2-mkmlizer: ║ / _/ /_ ___ __/ / ___ ___ / / ║
arlineka-nyannyan-v2-mkmlizer: ║ / _/ / // / |/|/ / _ \/ -_) -_) / ║
arlineka-nyannyan-v2-mkmlizer: ║ /_//_/\_, /|__,__/_//_/\__/\__/_/ ║
arlineka-nyannyan-v2-mkmlizer: ║ /___/ ║
arlineka-nyannyan-v2-mkmlizer: ║ ║
arlineka-nyannyan-v2-mkmlizer: ║ Version: 0.8.14 ║
arlineka-nyannyan-v2-mkmlizer: ║ Copyright 2023 MK ONE TECHNOLOGIES Inc. ║
arlineka-nyannyan-v2-mkmlizer: ║ https://mk1.ai ║
arlineka-nyannyan-v2-mkmlizer: ║ ║
arlineka-nyannyan-v2-mkmlizer: ║ The license key for the current software has been verified as ║
arlineka-nyannyan-v2-mkmlizer: ║ belonging to: ║
arlineka-nyannyan-v2-mkmlizer: ║ ║
arlineka-nyannyan-v2-mkmlizer: ║ Chai Research Corp. ║
arlineka-nyannyan-v2-mkmlizer: ║ Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f ║
arlineka-nyannyan-v2-mkmlizer: ║ Expiration: 2024-07-15 23:59:59 ║
arlineka-nyannyan-v2-mkmlizer: ║ ║
arlineka-nyannyan-v2-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
arlineka-nyannyan-v2-mkmlizer: /opt/conda/lib/python3.10/site-packages/huggingface_hub/utils/_deprecation.py:131: FutureWarning: 'list_files_info' (from 'huggingface_hub.hf_api') is deprecated and will be removed from version '0.23'. Use `list_repo_tree` and `get_paths_info` instead.
arlineka-nyannyan-v2-mkmlizer: warnings.warn(warning_message, FutureWarning)
arlineka-nyannyan-v2-mkmlizer: Downloaded to shared memory in 54.775s
arlineka-nyannyan-v2-mkmlizer: quantizing model to /dev/shm/model_cache
arlineka-nyannyan-v2-mkmlizer: Saving flywheel model at /dev/shm/model_cache
arlineka-nyannyan-v2-mkmlizer: Loading 0: 0%| | 0/291 [00:00<?, ?it/s] Loading 0: 1%| | 2/291 [00:04<11:08, 2.31s/it] Loading 0: 5%|▍ | 14/291 [00:04<01:09, 4.00it/s] Loading 0: 9%|▉ | 27/291 [00:04<00:28, 9.14it/s] Loading 0: 14%|█▍ | 41/291 [00:04<00:15, 16.18it/s] Loading 0: 19%|█▊ | 54/291 [00:05<00:09, 24.39it/s] Loading 0: 23%|██▎ | 66/291 [00:05<00:08, 27.11it/s] Loading 0: 26%|██▋ | 77/291 [00:05<00:06, 34.79it/s] Loading 0: 31%|███ | 90/291 [00:05<00:04, 46.26it/s] Loading 0: 35%|███▌ | 103/291 [00:05<00:03, 58.59it/s] Loading 0: 40%|███▉ | 115/291 [00:05<00:02, 67.66it/s] Loading 0: 45%|████▍ | 130/291 [00:05<00:01, 82.50it/s] Loading 0: 49%|████▉ | 142/291 [00:06<00:01, 87.87it/s] Loading 0: 54%|█████▍ | 157/291 [00:06<00:01, 101.00it/s] Loading 0: 58%|█████▊ | 170/291 [00:06<00:01, 67.10it/s] Loading 0: 63%|██████▎ | 184/291 [00:06<00:01, 79.43it/s] Loading 0: 67%|██████▋ | 195/291 [00:06<00:01, 83.36it/s] Loading 0: 73%|███████▎ | 211/291 [00:06<00:00, 98.66it/s] Loading 0: 77%|███████▋ | 223/291 [00:06<00:00, 100.52it/s] Loading 0: 82%|████████▏ | 238/291 [00:07<00:00, 111.08it/s] Loading 0: 86%|████████▋ | 251/291 [00:07<00:00, 111.58it/s] Loading 0: 91%|█████████ | 265/291 [00:07<00:00, 117.42it/s] Loading 0: 96%|█████████▌| 278/291 [00:07<00:00, 72.16it/s] Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
arlineka-nyannyan-v2-mkmlizer: quantized model in 23.224s
arlineka-nyannyan-v2-mkmlizer: Processed model arlineka/NyanNyan in 80.556s
arlineka-nyannyan-v2-mkmlizer: creating bucket guanaco-mkml-models
arlineka-nyannyan-v2-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
arlineka-nyannyan-v2-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/arlineka-nyannyan-v2
arlineka-nyannyan-v2-mkmlizer: DEBUG retryable error: RequestError: send request failed
arlineka-nyannyan-v2-mkmlizer: caused by: dial tcp 216.153.53.63:443: i/o timeout
arlineka-nyannyan-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/arlineka-nyannyan-v2/tokenizer_config.json
arlineka-nyannyan-v2-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/arlineka-nyannyan-v2/config.json
arlineka-nyannyan-v2-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/arlineka-nyannyan-v2/special_tokens_map.json
arlineka-nyannyan-v2-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/arlineka-nyannyan-v2/tokenizer.json
arlineka-nyannyan-v2-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/arlineka-nyannyan-v2/flywheel_model.0.safetensors
arlineka-nyannyan-v2-mkmlizer: loading reward model from ChaiML/reward_gpt2_medium_preference_24m_e2
arlineka-nyannyan-v2-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py:913: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
arlineka-nyannyan-v2-mkmlizer: warnings.warn(
arlineka-nyannyan-v2-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/tokenization_auto.py:757: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
arlineka-nyannyan-v2-mkmlizer: warnings.warn(
arlineka-nyannyan-v2-mkmlizer: /opt/conda/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py:468: FutureWarning: The `use_auth_token` argument is deprecated and will be removed in v5 of Transformers. Please use `token` instead.
arlineka-nyannyan-v2-mkmlizer: warnings.warn(
Retrying (%r) after connection broken by '%r': %s
arlineka-nyannyan-v2-mkmlizer: cp /tmp/reward_cache/reward.tensors s3://guanaco-reward-models/arlineka-nyannyan-v2_reward/reward.tensors
Job arlineka-nyannyan-v2-mkmlizer completed after 177.07s with status: succeeded
Stopping job with name arlineka-nyannyan-v2-mkmlizer
Pipeline stage MKMLizer completed in 180.83s
Running pipeline stage MKMLKubeTemplater
Pipeline stage MKMLKubeTemplater completed in 0.08s
Running pipeline stage ISVCDeployer
Creating inference service arlineka-nyannyan-v2
Waiting for inference service arlineka-nyannyan-v2 to be ready
Inference service arlineka-nyannyan-v2 ready after 50.34701085090637s
Pipeline stage ISVCDeployer completed in 57.41s
Running pipeline stage StressChecker
%s, retrying in %s seconds...
%s, retrying in %s seconds...
DeploymentChecksError('expected string or bytes-like object')
arlineka-nyannyan_v2 status is now failed due to DeploymentManager action
admin requested tearing down of arlineka-nyannyan_v2
Running pipeline stage ISVCDeleter
Checking if service arlineka-nyannyan-v2 is running
Tearing down inference service arlineka-nyannyan-v2
Toredown service arlineka-nyannyan-v2
Pipeline stage ISVCDeleter completed in 7.01s
Running pipeline stage MKMLModelDeleter
Cleaning model data from S3
Cleaning model data from model cache
Deleting key arlineka-nyannyan-v2/config.json from bucket guanaco-mkml-models
Deleting key arlineka-nyannyan-v2/flywheel_model.0.safetensors from bucket guanaco-mkml-models
Deleting key arlineka-nyannyan-v2/special_tokens_map.json from bucket guanaco-mkml-models
Deleting key arlineka-nyannyan-v2/tokenizer.json from bucket guanaco-mkml-models
Deleting key arlineka-nyannyan-v2/tokenizer_config.json from bucket guanaco-mkml-models
Cleaning model data from model cache
Deleting key arlineka-nyannyan-v2_reward/config.json from bucket guanaco-reward-models
Deleting key arlineka-nyannyan-v2_reward/merges.txt from bucket guanaco-reward-models
Deleting key arlineka-nyannyan-v2_reward/reward.tensors from bucket guanaco-reward-models
Deleting key arlineka-nyannyan-v2_reward/special_tokens_map.json from bucket guanaco-reward-models
Deleting key arlineka-nyannyan-v2_reward/tokenizer.json from bucket guanaco-reward-models
Deleting key arlineka-nyannyan-v2_reward/tokenizer_config.json from bucket guanaco-reward-models
Deleting key arlineka-nyannyan-v2_reward/vocab.json from bucket guanaco-reward-models
Pipeline stage MKMLModelDeleter completed in 2.38s
arlineka-nyannyan_v2 status is now torndown due to DeploymentManager action

Usage Metrics

Latency Metrics