support for llama 3

[avianion]

will this project plan to support llama 3 70b or 8b?

[npuichigo]

llama3 should already be supported with template https://github.com/npuichigo/openai_trtllm/blob/main/templates/history_template_llama3.liquid. To get the model, please refer to https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/llama#llama-v3-updates

[avianion]

ok great. @npuichigo but what is the model name? it keeps saying to me model not found and i have tried many model names. with llama 3 70b

[npuichigo]

it's ensemble if the structure looks like https://github.com/triton-inference-server/tensorrtllm_backend/tree/v0.9.0/all_models/inflight_batcher_llm

[avianion]

Should skip_special_tokens be True or False? and same with add_special_tokens in the preprocessing config.pbtxt?