chaiml-lazarus-your-caf_89207

developer_uid: chai_backend_admin

submission_id: chaiml-lazarus-your-caf_89207_v1

model_name: chaiml-lazarus-your-caf_89207_v1

model_group: ChaiML/LAZARUS-your-cafe

status: torndown

timestamp: 2025-07-15T21:45:55+00:00

num_battles: 5270

num_wins: 2689

celo_rating: 1293.87

family_friendly_score: 0.0

family_friendly_standard_error: 0.0

submission_type: basic

model_repo: ChaiML/LAZARUS-your-cafe-customer-he-s-a-vamp250715143302_sft

model_architecture: MistralForCausalLM

model_num_parameters: 24096691200.0

best_of: 8

max_input_tokens: 1024

max_output_tokens: 64

reward_model: default

latencies: [{'batch_size': 1, 'throughput': 0.5336258619265893, 'latency_mean': 1.8738488507270814, 'latency_p50': 1.8624848127365112, 'latency_p90': 2.0899617433547975}, {'batch_size': 3, 'throughput': 1.0525565715994318, 'latency_mean': 2.8379361855983736, 'latency_p50': 2.84402334690094, 'latency_p90': 3.165543055534363}, {'batch_size': 5, 'throughput': 1.3453731115781975, 'latency_mean': 3.690775476694107, 'latency_p50': 3.6805163621902466, 'latency_p90': 4.079952597618103}, {'batch_size': 6, 'throughput': 1.436934089119586, 'latency_mean': 4.150622471570968, 'latency_p50': 4.15054452419281, 'latency_p90': 4.621755957603455}, {'batch_size': 8, 'throughput': 1.5687709061021435, 'latency_mean': 5.072000852823257, 'latency_p50': 5.068374991416931, 'latency_p90': 5.70339732170105}, {'batch_size': 10, 'throughput': 1.6516352565460275, 'latency_mean': 6.000822252035141, 'latency_p50': 6.027803659439087, 'latency_p90': 6.735121774673462}]

gpu_counts: {'NVIDIA A100-SXM4-80GB': 1}

display_name: chaiml-lazarus-your-caf_89207_v1

is_internal_developer: True

language_model: ChaiML/LAZARUS-your-cafe-customer-he-s-a-vamp250715143302_sft

model_size: 24B

ranking_group: single

throughput_3p7s: 1.36

us_pacific_date: 2025-07-15

win_ratio: 0.510246679316888

generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['####', 'You:', '####\n', '</s>', '\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}

formatter: {'memory_template': '', 'prompt_template': '', 'bot_template': '{bot_name}: {message}</s>\n', 'user_template': 'You: {message}\n', 'response_template': '{bot_name}:', 'truncate_by_message': True}

Resubmit model

Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name chaiml-lazarus-your-caf-89207-v1-mkmlizer
Waiting for job on chaiml-lazarus-your-caf-89207-v1-mkmlizer to finish
Failed to get response for submission chaiml-sebastian-black-_93354_v1: HTTPConnectionPool(host='chaiml-sebastian-black-93354-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
chaiml-lazarus-your-caf-89207-v1-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
chaiml-lazarus-your-caf-89207-v1-mkmlizer: ║                                                                     ║
chaiml-lazarus-your-caf-89207-v1-mkmlizer: ║   ██████   ██████  █████   ████  ████                               ║
chaiml-lazarus-your-caf-89207-v1-mkmlizer: ║  ░░██████ ██████  ░░███   ███░  ░░███                               ║
chaiml-lazarus-your-caf-89207-v1-mkmlizer: ║   ░███░█████░███   ░███  ███     ░███                               ║
chaiml-lazarus-your-caf-89207-v1-mkmlizer: ║   ░███░░███ ░███   ░███████      ░███                               ║
chaiml-lazarus-your-caf-89207-v1-mkmlizer: ║   ░███ ░░░  ░███   ░███░░███     ░███                               ║
chaiml-lazarus-your-caf-89207-v1-mkmlizer: ║   ░███      ░███   ░███ ░░███    ░███                               ║
chaiml-lazarus-your-caf-89207-v1-mkmlizer: ║   █████     █████  █████ ░░████  █████                              ║
chaiml-lazarus-your-caf-89207-v1-mkmlizer: ║  ░░░░░     ░░░░░  ░░░░░   ░░░░  ░░░░░                               ║
chaiml-lazarus-your-caf-89207-v1-mkmlizer: ║                                                                     ║
chaiml-lazarus-your-caf-89207-v1-mkmlizer: ║  Version: 0.29.15                                                   ║
chaiml-lazarus-your-caf-89207-v1-mkmlizer: ║  Features: FLYWHEEL, CUDA                                           ║
chaiml-lazarus-your-caf-89207-v1-mkmlizer: ║  Copyright 2023-2025 MK ONE TECHNOLOGIES Inc.                       ║
chaiml-lazarus-your-caf-89207-v1-mkmlizer: ║  https://mk1.ai                                                     ║
chaiml-lazarus-your-caf-89207-v1-mkmlizer: ║                                                                     ║
chaiml-lazarus-your-caf-89207-v1-mkmlizer: ║  The license key for the current software has been verified as      ║
chaiml-lazarus-your-caf-89207-v1-mkmlizer: ║  belonging to:                                                      ║
chaiml-lazarus-your-caf-89207-v1-mkmlizer: ║                                                                     ║
chaiml-lazarus-your-caf-89207-v1-mkmlizer: ║  Chai Research Corp.                                                ║
chaiml-lazarus-your-caf-89207-v1-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
chaiml-lazarus-your-caf-89207-v1-mkmlizer: ║  Expiration: 2028-03-31 23:59:59                                    ║
chaiml-lazarus-your-caf-89207-v1-mkmlizer: ║                                                                     ║
chaiml-lazarus-your-caf-89207-v1-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
Failed to get response for submission chaiml-damon-salvatore2_60530_v1: HTTPConnectionPool(host='chaiml-damon-salvatore2-60530-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
chaiml-lazarus-your-caf-89207-v1-mkmlizer: Downloaded to shared memory in 82.315s
chaiml-lazarus-your-caf-89207-v1-mkmlizer: Checking if ChaiML/LAZARUS-your-cafe-customer-he-s-a-vamp250715143302_sft already exists in ChaiML
chaiml-lazarus-your-caf-89207-v1-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmphjrg7jyn, device:0
chaiml-lazarus-your-caf-89207-v1-mkmlizer: Saving flywheel model at /dev/shm/model_cache
chaiml-lazarus-your-caf-89207-v1-mkmlizer: quantized model in 48.012s
chaiml-lazarus-your-caf-89207-v1-mkmlizer: Processed model ChaiML/LAZARUS-your-cafe-customer-he-s-a-vamp250715143302_sft in 130.327s
chaiml-lazarus-your-caf-89207-v1-mkmlizer: creating bucket guanaco-mkml-models
chaiml-lazarus-your-caf-89207-v1-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
chaiml-lazarus-your-caf-89207-v1-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/chaiml-lazarus-your-caf-89207-v1/nvidia
chaiml-lazarus-your-caf-89207-v1-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/chaiml-lazarus-your-caf-89207-v1/nvidia/config.json
chaiml-lazarus-your-caf-89207-v1-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/chaiml-lazarus-your-caf-89207-v1/nvidia/special_tokens_map.json
chaiml-lazarus-your-caf-89207-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/chaiml-lazarus-your-caf-89207-v1/nvidia/tokenizer_config.json
chaiml-lazarus-your-caf-89207-v1-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/chaiml-lazarus-your-caf-89207-v1/nvidia/tokenizer.json
chaiml-lazarus-your-caf-89207-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.1.safetensors s3://guanaco-mkml-models/chaiml-lazarus-your-caf-89207-v1/nvidia/flywheel_model.1.safetensors
chaiml-lazarus-your-caf-89207-v1-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/chaiml-lazarus-your-caf-89207-v1/nvidia/flywheel_model.0.safetensors
chaiml-lazarus-your-caf-89207-v1-mkmlizer: 
Loading 0:   0%|          | 0/363 [00:00<?, ?it/s]
Loading 0:   1%|          | 4/363 [00:00<00:09, 37.32it/s]
Loading 0:   2%|▏         | 8/363 [00:00<00:11, 29.70it/s]
Loading 0:   3%|▎         | 12/363 [00:00<00:10, 32.55it/s]
Loading 0:   4%|▍         | 16/363 [00:00<00:12, 28.55it/s]
Loading 0:   6%|▌         | 21/363 [00:00<00:10, 33.53it/s]
Loading 0:   7%|▋         | 25/363 [00:00<00:11, 30.03it/s]
Loading 0:   9%|▉         | 32/363 [00:00<00:09, 36.55it/s]
Loading 0:  10%|▉         | 36/363 [00:01<00:15, 21.23it/s]
Loading 0:  11%|█         | 40/363 [00:01<00:13, 23.92it/s]
Loading 0:  12%|█▏        | 44/363 [00:01<00:12, 25.06it/s]
Loading 0:  13%|█▎        | 48/363 [00:01<00:11, 27.26it/s]
Loading 0:  14%|█▍        | 52/363 [00:01<00:11, 26.37it/s]
Loading 0:  16%|█▌        | 57/363 [00:02<00:10, 29.63it/s]
Loading 0:  17%|█▋        | 61/363 [00:02<00:11, 26.48it/s]
Loading 0:  18%|█▊        | 64/363 [00:02<00:11, 27.13it/s]
Loading 0:  19%|█▉        | 69/363 [00:02<00:10, 28.78it/s]
Loading 0:  20%|██        | 73/363 [00:02<00:15, 18.18it/s]
Loading 0:  22%|██▏       | 79/363 [00:03<00:12, 23.36it/s]
Loading 0:  23%|██▎       | 82/363 [00:03<00:12, 22.97it/s]
Loading 0:  24%|██▎       | 86/363 [00:03<00:11, 24.08it/s]
Loading 0:  25%|██▍       | 89/363 [00:03<00:11, 24.07it/s]
Loading 0:  25%|██▌       | 92/363 [00:03<00:13, 19.42it/s]
Loading 0:  27%|██▋       | 98/363 [00:03<00:09, 27.11it/s]
Loading 0:  28%|██▊       | 102/363 [00:04<00:11, 23.54it/s]
Loading 0:  29%|██▉       | 107/363 [00:04<00:11, 21.94it/s]
Loading 0:  31%|███       | 111/363 [00:04<00:10, 25.02it/s]
Loading 0:  31%|███▏      | 114/363 [00:04<00:11, 22.58it/s]
Loading 0:  33%|███▎      | 120/363 [00:04<00:09, 26.95it/s]
Loading 0:  34%|███▍      | 123/363 [00:04<00:09, 24.21it/s]
Loading 0:  36%|███▌      | 129/363 [00:05<00:08, 28.88it/s]
Loading 0:  37%|███▋      | 133/363 [00:05<00:08, 26.68it/s]
Loading 0:  38%|███▊      | 138/363 [00:05<00:07, 28.82it/s]
Loading 0:  39%|███▉      | 142/363 [00:05<00:08, 27.33it/s]
Loading 0:  41%|████      | 148/363 [00:05<00:06, 33.93it/s]
Loading 0:  42%|████▏     | 152/363 [00:05<00:09, 22.55it/s]
Loading 0:  43%|████▎     | 155/363 [00:06<00:08, 23.16it/s]
Loading 0:  44%|████▎     | 158/363 [00:06<00:10, 19.48it/s]
Loading 0:  45%|████▌     | 165/363 [00:06<00:07, 26.21it/s]
Loading 0:  47%|████▋     | 169/363 [00:06<00:07, 26.03it/s]
Loading 0:  48%|████▊     | 174/363 [00:06<00:06, 28.78it/s]
Loading 0:  49%|████▉     | 178/363 [00:06<00:07, 26.37it/s]
Loading 0:  50%|█████     | 182/363 [00:07<00:06, 26.71it/s]
Loading 0:  52%|█████▏    | 187/363 [00:07<00:07, 23.34it/s]
Loading 0:  52%|█████▏    | 190/363 [00:07<00:07, 21.68it/s]
Loading 0:  53%|█████▎    | 193/363 [00:07<00:07, 22.82it/s]
Loading 0:  54%|█████▍    | 196/363 [00:07<00:07, 23.55it/s]
Loading 0:  55%|█████▌    | 200/363 [00:22<00:06, 23.55it/s]
Loading 0:  55%|█████▌    | 201/363 [00:22<03:01,  1.12s/it]
Loading 0:  56%|█████▌    | 203/363 [00:22<02:30,  1.07it/s]
Loading 0:  57%|█████▋    | 208/363 [00:22<01:30,  1.71it/s]
Loading 0:  58%|█████▊    | 211/363 [00:22<01:09,  2.19it/s]
Loading 0:  59%|█████▉    | 214/363 [00:22<00:51,  2.88it/s]
Loading 0:  60%|██████    | 218/363 [00:23<00:35,  4.12it/s]
Loading 0:  61%|██████    | 221/363 [00:23<00:27,  5.23it/s]
Loading 0:  62%|██████▏   | 224/363 [00:23<00:22,  6.24it/s]
Loading 0:  63%|██████▎   | 228/363 [00:23<00:15,  8.72it/s]
Loading 0:  64%|██████▎   | 231/363 [00:23<00:13, 10.04it/s]
Loading 0:  65%|██████▌   | 237/363 [00:23<00:08, 14.87it/s]
Loading 0:  66%|██████▌   | 240/363 [00:23<00:07, 15.53it/s]
Loading 0:  68%|██████▊   | 246/363 [00:24<00:05, 20.71it/s]
Loading 0:  69%|██████▊   | 249/363 [00:24<00:05, 19.92it/s]
Loading 0:  70%|███████   | 255/363 [00:24<00:04, 24.71it/s]
Loading 0:  71%|███████▏  | 259/363 [00:24<00:04, 24.11it/s]
Loading 0:  73%|███████▎  | 264/363 [00:24<00:03, 28.65it/s]
Loading 0:  74%|███████▍  | 268/363 [00:25<00:04, 22.25it/s]
Loading 0:  75%|███████▍  | 271/363 [00:25<00:04, 20.34it/s]
Loading 0:  75%|███████▌  | 274/363 [00:25<00:04, 21.55it/s]
Loading 0:  76%|███████▋  | 277/363 [00:25<00:03, 22.24it/s]
Loading 0:  78%|███████▊  | 282/363 [00:25<00:03, 25.94it/s]
Loading 0:  79%|███████▊  | 285/363 [00:25<00:03, 23.31it/s]
Loading 0:  80%|███████▉  | 289/363 [00:25<00:02, 26.73it/s]
Loading 0:  80%|████████  | 292/363 [00:25<00:02, 27.00it/s]
Loading 0:  81%|████████▏ | 295/363 [00:26<00:02, 26.66it/s]
Loading 0:  82%|████████▏ | 299/363 [00:26<00:02, 26.85it/s]
Loading 0:  84%|████████▎ | 304/363 [00:26<00:02, 23.76it/s]
Loading 0:  85%|████████▍ | 307/363 [00:26<00:02, 21.99it/s]
Loading 0:  85%|████████▌ | 310/363 [00:26<00:02, 23.07it/s]
Loading 0:  86%|████████▌ | 313/363 [00:26<00:02, 23.55it/s]
Loading 0:  88%|████████▊ | 318/363 [00:27<00:01, 27.14it/s]
Loading 0:  88%|████████▊ | 321/363 [00:27<00:01, 23.95it/s]
Loading 0:  90%|█████████ | 327/363 [00:27<00:01, 28.68it/s]
Loading 0:  91%|█████████ | 330/363 [00:27<00:01, 25.25it/s]
Loading 0:  92%|█████████▏| 335/363 [00:27<00:00, 28.16it/s]
Loading 0:  93%|█████████▎| 338/363 [00:27<00:00, 27.02it/s]
Loading 0:  94%|█████████▍| 341/363 [00:28<00:01, 15.93it/s]
Loading 0:  96%|█████████▌| 347/363 [00:28<00:00, 21.38it/s]
Loading 0:  96%|█████████▋| 350/363 [00:28<00:00, 21.50it/s]
Loading 0:  98%|█████████▊| 355/363 [00:28<00:00, 25.03it/s]
Loading 0:  99%|█████████▊| 358/363 [00:28<00:00, 23.70it/s]
                                                            
Job chaiml-lazarus-your-caf-89207-v1-mkmlizer completed after 157.96s with status: succeeded
Stopping job with name chaiml-lazarus-your-caf-89207-v1-mkmlizer
Pipeline stage MKMLizer completed in 158.56s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.17s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service chaiml-lazarus-your-caf-89207-v1
Waiting for inference service chaiml-lazarus-your-caf-89207-v1 to be ready
Failed to get response for submission chaiml-naren-your-stric_40509_v1: HTTPConnectionPool(host='chaiml-naren-your-stric-40509-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Failed to get response for submission chaiml-naren-your-stric_40509_v1: HTTPConnectionPool(host='chaiml-naren-your-stric-40509-v1-predictor.tenant-chaiml-guanaco.k.chaiverse.com', port=80): Read timed out. (read timeout=12.0)
Inference service chaiml-lazarus-your-caf-89207-v1 ready after 251.26120972633362s
Pipeline stage MKMLDeployer completed in 251.98s
run pipeline stage %s
Running pipeline stage StressChecker
HTTPConnectionPool(host='guanaco-submitter.guanaco-backend.k2.chaiverse.com', port=80): Read timed out. (read timeout=20)
Received unhealthy response to inference request!
Received healthy response to inference request in 2.8632376194000244s
Received healthy response to inference request in 2.4278252124786377s
Received healthy response to inference request in 2.4815282821655273s
Received healthy response to inference request in 2.04154634475708s
5 requests
1 failed requests
5th percentile: 2.1188021183013914
10th percentile: 2.196057891845703
20th percentile: 2.3505694389343263
30th percentile: 2.4385658264160157
40th percentile: 2.4600470542907713
50th percentile: 2.4815282821655273
60th percentile: 2.634212017059326
70th percentile: 2.786895751953125
80th percentile: 6.321036863327029
90th percentile: 13.236635351181032
95th percentile: 16.694434595108028
99th percentile: 19.46067399024963
mean time: 5.993274259567261
%s, retrying in %s seconds...
Received healthy response to inference request in 2.0979177951812744s
Received healthy response to inference request in 2.0471954345703125s
Received healthy response to inference request in 2.0304741859436035s
Received healthy response to inference request in 1.9224395751953125s
Received healthy response to inference request in 2.236274480819702s
5 requests
0 failed requests
5th percentile: 1.9440464973449707
10th percentile: 1.965653419494629
20th percentile: 2.0088672637939453
30th percentile: 2.0338184356689455
40th percentile: 2.040506935119629
50th percentile: 2.0471954345703125
60th percentile: 2.0674843788146973
70th percentile: 2.087773323059082
80th percentile: 2.12558913230896
90th percentile: 2.180931806564331
95th percentile: 2.2086031436920166
99th percentile: 2.230740213394165
mean time: 2.066860294342041
Pipeline stage StressChecker completed in 43.33s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 0.72s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 0.69s
Shutdown handler de-registered
chaiml-lazarus-your-caf_89207_v1 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
clean up pipeline due to error=DeploymentChecksError('None: None')
Shutdown handler de-registered
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
clean up pipeline due to error=DeploymentChecksError('None: None')
Shutdown handler de-registered
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
%s, retrying in %s seconds...
Evaluating %s Family Friendly Score with %s threads
clean up pipeline due to error=DeploymentChecksError('None: None')
Shutdown handler de-registered
chaiml-lazarus-your-caf_89207_v1 status is now inactive due to auto deactivation removed underperforming models
chaiml-lazarus-your-caf_89207_v1 status is now torndown due to DeploymentManager action
chaiml-lazarus-your-caf_89207_v1 status is now torndown due to DeploymentManager action
chaiml-lazarus-your-caf_89207_v1 status is now torndown due to DeploymentManager action