Using ngView directive without hash in angularjs

AngularJS Part:

myApp.config(['$routeProvider','$locationProvider',function($routeProvider,$locationProvider) {
    $routeProvider.
    when('/main', {templateUrl: 'sub/main.tpl', controller: MainCtrl}).
    when('/users', {templateUrl: 'sub/users.tpl', controller: UserCtrl}).
    otherwise({redirectTo: '/main'});
}]);

and my index.html

<a href="#/main">Main</a> | <a href="#/users">Display Users</a>
<div ng-view></div>

It works great but I am having problem with hashes.

When I try to use an anchor i.e:

<a href="#test">test</a>

when I click on this anchor it is calling the main page.

Is it possible to get rid of the hashes when we use ng-view?