next() call in middleware of mounted apps in ExpressJS

I have an Express application:

var site = new express();

Then I make the admin part a sub app:

var admin = new express();
admin.get("/users", function(req, res) {...});

Finally I mount admin app onto the main site with a permission checking middleware:

var isAdmin = function(req, res, next) {
    if (!req.user.admin) {
        return res.send("403");
    }
    next();
};

site.use("/admin", isAdmin, admin);

Problem is: when I open /admin/users page, the next() call in permission checking leads to a 404 in the end. I feel it calls the next callback function in site instead of going into admin.

If I remove the permission checking middleware, write just like this:

site.use("/admin", admin);

Then /admin/users can be accessed finely. Without any ACL of course...

Question is How can I make the code goes into admin sub app in the middleware chain?