angularjs isolated scope expression not working

Long story short my isolated scope is working for "=" binding not for "&" expression

here is my code so far

In controller

$scope.goToPage = function (pageNo) {              

        console.log(pageNo);
    };

In directive

scope: {
            'pagination': '=paginate',
            'paginateTo' : '&clickAction'
        },

The directive

<pagination paginate="pagination" click-action="goToPage(pageNo)"></pagination>

The directive View

<a class="button button-small" ng-click="paginateTo({ pageNo : 2 })"> Click ME</a>

Nothing happens. If i remove the isolated scope and change the code. It works fine. the binding is working pretty well so i didn't paste it here.

Any ideas what i am doing wrong ???