New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Question] 请问7B没有用上FlashAttention吗？ #130

Open

5 tasks done

nezhazheng opened this issue Sep 7, 2023 · 1 comment

Open

5 tasks done

[Question] 请问7B没有用上FlashAttention吗？ #130

nezhazheng opened this issue Sep 7, 2023 · 1 comment

Labels

question

nezhazheng commented Sep 7, 2023

Required prerequisites

I have read the documentation https://github.com/baichuan-inc/baichuan-7B/blob/HEAD/README.md.
I have searched the Issue Tracker and Discussions that this hasn't already been reported. (+1 or comment there if it has.)
Consider asking first in a Discussion.

Questions

请问7B没有用上FlashAttention吗？看了下7B代码，没发现这块的逻辑。

Checklist

I have provided all relevant and necessary information above.
I have chosen a suitable title for this issue.

nezhazheng added the question label

mmmans commented Sep 7, 2023

No. We user xformers for training, and naive impl for inference.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment