npu驱动升级到0.9.8后推理变慢

[Jackie-Tang1]

orangepi5 rk3588 npu升级到0.9.8后推理速度变得很慢，是就是这样吗？

[Jackie-Tang1]

推理结果倒是没有什么问题，就是推理速度极其慢，同样Qwen 1.8B的模型在0.9.6下跑的飞起来

[ZIFENG278]

如何编译模型他才正常推理，我的同qwen-1.8 效果有点奇怪

rock@rock-5b-plus:~/ssd/rkllm$ ./llm_demo ./qwen.rkllm 1024 1024
rkllm init start
I rkllm: rkllm-runtime version: 1.1.4, rknpu driver version: 0.9.8, platform: RK3588

rkllm init success

**********************可输入以下问题对应序号获取回答/或自定义输入********************

[0] 把下面的现代文翻译成文言文: 到了春风和煦，阳光明媚的时候，湖面平静，没有惊涛骇浪，天色湖光相连，一片碧绿，广阔无际；沙洲上的鸥鸟，时而飞翔，时而停歇，美丽的鱼游来游去，岸上与小洲上的花草，青翠欲滴。
[1] 以咏梅为题目，帮我写一首古诗，要求包含梅花、白雪等元素。
[2] 上联: 江边惯看千帆过
[3] 把这句话翻译成中文: Knowledge can be acquired from many sources. These include books, teachers and practical experience, and each has its own advantages. The knowledge we gain from books and formal education enables us to learn about things that we have no opportunity to experience in daily life. We can also develop our analytical skills and learn how to view and interpret the world around us in different ways. Furthermore, we can learn from the past by reading books. In this way, we won't repeat the mistakes of others and can build on their achievements.
[4] 把这句话翻译成英文: RK3588是新一代高端处理器，具有高算力、低功耗、超强多媒体、丰富数据接口等特点

*************************************************************************


user: 你是谁
robot: 慢慢的是谁



user: 3
把这句话翻译成中文: Knowledge can be acquired from many sources. These include books, teachers and practical experience, and each has its own advantages. The knowledge we gain from books and formal education enables us to learn about things that we have no opportunity to experience in daily life. We can also develop our analytical skills and learn how to view and interpret the world around us in different ways. Furthermore, we can learn from the past by reading books. In this way, we won't repeat the mistakes of others and can build on their achievements.
robot: 

user: 

[Jackie-Tang1]

你这个试着将模型输入改成text = PROMPT_TEXT_PREFIX + input_str + PROMPT_TEXT_POSTFIX;增加prompt试一下，应该就好了