"files" in package.json and .npmignore do not work with "npm install" from GitHub repo

I'm trying to install a private Node package directly from its GitHub repo. Even though the package.json has the "files" property set with only 1 folder to be copied, the entire GitHub repo is copied into node_modules. I also tried .npmignore, but same issue, I keep getting the entire repo.

Any idea who I can get this working?