How do I manage external libraries from bower in grunt?

Currently I have bower putting all my lib files into: src/libs

I have grunt reading all files in src folder and minifying.

this seems like a lot considering the libraries are already minified, should I just point bower to my dist folder?

Will it still work if I keep them in src?