developer_uid: cgato
submission_id: cgato-nemo-12b-humanize-_7413_v3
model_name: cgato-nemo-12b-humanize-_7413_v3
model_group: cgato/Nemo-12b-Humanize-
status: inactive
timestamp: 2024-12-12T06:01:49+00:00
num_battles: 21350
num_wins: 10300
celo_rating: 1250.89
family_friendly_score: 0.5602
family_friendly_standard_error: 0.007019629050028214
submission_type: basic
model_repo: cgato/Nemo-12b-Humanize-KTO-Experimental-Latest
model_architecture: MistralForCausalLM
model_num_parameters: 12772111360.0
best_of: 8
max_input_tokens: 1024
max_output_tokens: 64
latencies: [{'batch_size': 1, 'throughput': 0.6236390258064417, 'latency_mean': 1.6034348404407501, 'latency_p50': 1.6108484268188477, 'latency_p90': 1.7587440729141235}, {'batch_size': 3, 'throughput': 1.1609985599498909, 'latency_mean': 2.5807472920417784, 'latency_p50': 2.579269766807556, 'latency_p90': 2.8595030069351197}, {'batch_size': 5, 'throughput': 1.419586166641329, 'latency_mean': 3.5113778042793276, 'latency_p50': 3.5311776399612427, 'latency_p90': 3.8813069343566893}, {'batch_size': 6, 'throughput': 1.4856449108522873, 'latency_mean': 4.012174957990647, 'latency_p50': 4.023177742958069, 'latency_p90': 4.478644323348999}, {'batch_size': 8, 'throughput': 1.5391647882880999, 'latency_mean': 5.166860723495484, 'latency_p50': 5.154400110244751, 'latency_p90': 5.775869011878967}, {'batch_size': 10, 'throughput': 1.595407898344873, 'latency_mean': 6.2182609701156615, 'latency_p50': 6.180792808532715, 'latency_p90': 6.991973662376404}]
gpu_counts: {'NVIDIA RTX A5000': 1}
display_name: cgato-nemo-12b-humanize-_7413_v3
is_internal_developer: False
language_model: cgato/Nemo-12b-Humanize-KTO-Experimental-Latest
model_size: 13B
ranking_group: single
throughput_3p7s: 1.45
us_pacific_date: 2024-12-11
win_ratio: 0.48243559718969553
generation_params: {'temperature': 1.0, 'top_p': 1.0, 'min_p': 0.0, 'top_k': 40, 'presence_penalty': 0.0, 'frequency_penalty': 0.0, 'stopping_words': ['\n'], 'max_input_tokens': 1024, 'best_of': 8, 'max_output_tokens': 64}
formatter: {'memory_template': '<|im_start|>system\n{memory}<|im_end|>\n', 'prompt_template': '<|im_start|>user\n{prompt}<|im_end|>\n', 'bot_template': '<|im_start|>assistant\n{bot_name}: {message}<|im_end|>\n', 'user_template': '<|im_start|>user\n{user_name}: {message}<|im_end|>\n', 'response_template': '<|im_start|>assistant\n{bot_name}:', 'truncate_by_message': True}
Resubmit model
Shutdown handler not registered because Python interpreter is not running in the main thread
run pipeline %s
run pipeline stage %s
Running pipeline stage MKMLizer
Starting job with name cgato-nemo-12b-humanize-7413-v3-mkmlizer
Waiting for job on cgato-nemo-12b-humanize-7413-v3-mkmlizer to finish
cgato-nemo-12b-humanize-7413-v3-mkmlizer: ╔═════════════════════════════════════════════════════════════════════╗
cgato-nemo-12b-humanize-7413-v3-mkmlizer: ║     _____            __           __                                ║
cgato-nemo-12b-humanize-7413-v3-mkmlizer: ║    / _/ /_ ___    __/ /  ___ ___ / /                                ║
cgato-nemo-12b-humanize-7413-v3-mkmlizer: ║   / _/ / // / |/|/ / _ \/ -_) -_) /                                 ║
cgato-nemo-12b-humanize-7413-v3-mkmlizer: ║  /_//_/\_, /|__,__/_//_/\__/\__/_/                                  ║
cgato-nemo-12b-humanize-7413-v3-mkmlizer: ║       /___/                                                         ║
cgato-nemo-12b-humanize-7413-v3-mkmlizer: ║                                                                     ║
cgato-nemo-12b-humanize-7413-v3-mkmlizer: ║  Version: 0.11.12                                                   ║
cgato-nemo-12b-humanize-7413-v3-mkmlizer: ║  Copyright 2023 MK ONE TECHNOLOGIES Inc.                            ║
cgato-nemo-12b-humanize-7413-v3-mkmlizer: ║  https://mk1.ai                                                     ║
cgato-nemo-12b-humanize-7413-v3-mkmlizer: ║                                                                     ║
cgato-nemo-12b-humanize-7413-v3-mkmlizer: ║  The license key for the current software has been verified as      ║
cgato-nemo-12b-humanize-7413-v3-mkmlizer: ║  belonging to:                                                      ║
cgato-nemo-12b-humanize-7413-v3-mkmlizer: ║                                                                     ║
cgato-nemo-12b-humanize-7413-v3-mkmlizer: ║  Chai Research Corp.                                                ║
cgato-nemo-12b-humanize-7413-v3-mkmlizer: ║  Account ID: 7997a29f-0ceb-4cc7-9adf-840c57b4ae6f                   ║
cgato-nemo-12b-humanize-7413-v3-mkmlizer: ║  Expiration: 2025-01-15 23:59:59                                    ║
cgato-nemo-12b-humanize-7413-v3-mkmlizer: ║                                                                     ║
cgato-nemo-12b-humanize-7413-v3-mkmlizer: ╚═════════════════════════════════════════════════════════════════════╝
cgato-nemo-12b-humanize-7413-v3-mkmlizer: Downloaded to shared memory in 47.543s
cgato-nemo-12b-humanize-7413-v3-mkmlizer: quantizing model to /dev/shm/model_cache, profile:s0, folder:/tmp/tmp1k16865c, device:0
cgato-nemo-12b-humanize-7413-v3-mkmlizer: Saving flywheel model at /dev/shm/model_cache
cgato-nemo-12b-humanize-7413-v3-mkmlizer: quantized model in 37.109s
cgato-nemo-12b-humanize-7413-v3-mkmlizer: Processed model cgato/Nemo-12b-Humanize-KTO-Experimental-Latest in 84.652s
cgato-nemo-12b-humanize-7413-v3-mkmlizer: creating bucket guanaco-mkml-models
cgato-nemo-12b-humanize-7413-v3-mkmlizer: Bucket 's3://guanaco-mkml-models/' created
cgato-nemo-12b-humanize-7413-v3-mkmlizer: uploading /dev/shm/model_cache to s3://guanaco-mkml-models/cgato-nemo-12b-humanize-7413-v3
cgato-nemo-12b-humanize-7413-v3-mkmlizer: cp /dev/shm/model_cache/special_tokens_map.json s3://guanaco-mkml-models/cgato-nemo-12b-humanize-7413-v3/special_tokens_map.json
cgato-nemo-12b-humanize-7413-v3-mkmlizer: cp /dev/shm/model_cache/config.json s3://guanaco-mkml-models/cgato-nemo-12b-humanize-7413-v3/config.json
cgato-nemo-12b-humanize-7413-v3-mkmlizer: cp /dev/shm/model_cache/tokenizer_config.json s3://guanaco-mkml-models/cgato-nemo-12b-humanize-7413-v3/tokenizer_config.json
cgato-nemo-12b-humanize-7413-v3-mkmlizer: cp /dev/shm/model_cache/tokenizer.json s3://guanaco-mkml-models/cgato-nemo-12b-humanize-7413-v3/tokenizer.json
cgato-nemo-12b-humanize-7413-v3-mkmlizer: cp /dev/shm/model_cache/flywheel_model.0.safetensors s3://guanaco-mkml-models/cgato-nemo-12b-humanize-7413-v3/flywheel_model.0.safetensors
Job cgato-nemo-12b-humanize-7413-v3-mkmlizer completed after 114.49s with status: succeeded
Stopping job with name cgato-nemo-12b-humanize-7413-v3-mkmlizer
Pipeline stage MKMLizer completed in 115.03s
run pipeline stage %s
Running pipeline stage MKMLTemplater
Pipeline stage MKMLTemplater completed in 0.16s
run pipeline stage %s
Running pipeline stage MKMLDeployer
Creating inference service cgato-nemo-12b-humanize-7413-v3
Waiting for inference service cgato-nemo-12b-humanize-7413-v3 to be ready
Inference service cgato-nemo-12b-humanize-7413-v3 ready after 180.65508818626404s
Pipeline stage MKMLDeployer completed in 181.21s
run pipeline stage %s
Running pipeline stage StressChecker
Received healthy response to inference request in 2.0332319736480713s
Received healthy response to inference request in 1.9812681674957275s
Received healthy response to inference request in 1.4569361209869385s
Received healthy response to inference request in 1.580282211303711s
Received healthy response to inference request in 1.465677261352539s
5 requests
0 failed requests
5th percentile: 1.4586843490600585
10th percentile: 1.4604325771331788
20th percentile: 1.463929033279419
30th percentile: 1.4885982513427733
40th percentile: 1.5344402313232421
50th percentile: 1.580282211303711
60th percentile: 1.7406765937805175
70th percentile: 1.9010709762573241
80th percentile: 1.9916609287261964
90th percentile: 2.0124464511871336
95th percentile: 2.0228392124176025
99th percentile: 2.0311534214019775
mean time: 1.7034791469573975
Pipeline stage StressChecker completed in 9.82s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyTriggerPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage OfflineFamilyFriendlyTriggerPipeline completed in 2.42s
run pipeline stage %s
Running pipeline stage TriggerMKMLProfilingPipeline
run_pipeline:run_in_cloud %s
starting trigger_guanaco_pipeline args=%s
triggered trigger_guanaco_pipeline args=%s
Pipeline stage TriggerMKMLProfilingPipeline completed in 2.28s
Shutdown handler de-registered
cgato-nemo-12b-humanize-_7413_v3 status is now deployed due to DeploymentManager action
Shutdown handler registered
run pipeline %s
run pipeline stage %s
Running pipeline stage OfflineFamilyFriendlyScorer
Evaluating %s Family Friendly Score with %s threads
Pipeline stage OfflineFamilyFriendlyScorer completed in 2811.56s
Shutdown handler de-registered
cgato-nemo-12b-humanize-_7413_v3 status is now inactive due to auto deactivation removed underperforming models