Abstract:

In the field of intelligent connected vehicles， the recognition accuracy of in-car systems for non-command voice input in complex environment （the proportion of correct voice input recognition by the system） is of great significance. To address this challenge， in this paper a multimodal rejection model is proposed. The model is based on the open-source ChatGLM2-6B large language model and has undergone exclusive rejection dataset construction and model fine-tuning for the in-vehicle interaction scenario. The rejection dataset is collected from real driving scenarios， integrating voice information with the driver's facial orientation， gestures， and emotion， and other non-verbal signals to provide richer interaction information， effectively overcoming the limitation of pure language recognition mechanisms in complex environment. Through experiments， it is found that the multimodal rejection model shows higher recognition accuracy （ACC） and lower false rejection rate （FRR） on the test set compared to the pure language rejection model.

Key words: intelligent connected vehicles, in-vehicle speech interaction, rejection, large model