We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent 463f63a commit 950bcccCopy full SHA for 950bccc
demos/common/export_models/README.md
@@ -31,6 +31,7 @@ usage: export_model.py text_generation [-h]
31
[--config_file_path CONFIG_FILE_PATH]
32
[--overwrite_models]
33
[--target_device TARGET_DEVICE]
34
+ [--pipeline_type PIPELINE_TYPE]
35
[--kv_cache_precision {u8}]
36
[--enable_prefix_caching]
37
[--disable_dynamic_split_fuse]
@@ -58,6 +59,9 @@ options:
58
59
repository
60
--target_device TARGET_DEVICE
61
CPU or GPU, default is CPU
62
+ --pipeline_type PIPELINE_TYPE
63
+ Type of the pipeline to be used. Can be either
64
+ CONTINUOUS_BATCHING or VISUAL_LANGUAGE_MODEL.
65
--kv_cache_precision {u8}
66
u8 or empty (model default). Reduced kv cache
67
precision to u8 lowers the cache size consumption.
0 commit comments