Input field not editable or somehow readonly when using in directive with ion-toggle

Codepen with the problem

this is my directive template:

<ion-toggle class="cw-alerttimebutton" ng-model="targetobject.isEnabled"
            ng-checked="targetobject.isEnabled">
    <input type="text" ng-model="targetobject.time" value="{{targetobject.time}}"> {{ text }}
</ion-toggle>

this is my app setup:

app.directive('cwAlertTimeButton', function() {
    return {
        restrict: 'AE',
        replace: false, //toggle already replaces
        templateUrl: 'templates/directives/cwAlertTimeButton.html',
        link: function(scope, elem, attrs) {
            scope.targetobject = scope.$eval(attrs.targetobject);
            scope.text = attrs.text;
        }
    };
});

The questions:

<input type="text" ng-model="targetobject.time" value="1980-01-01T11:45:00.000Z" class="ng-pristine ng-untouched ng-valid">

is write protected nor accessible in this place. ion-toggle seems to use a class item-toggle that causes problems.

If I move the input outside of the containing element with class item-toggle it works as expected. As well as when I remove the class item-toggle from its parent.

How can I get it to work inside this construct?

The idea is to get an iphone like alarm. E.g. left the time, you click in it, you change it, right you have a toggle to enable disable it.

I use input type=text only to see if it works at all, but it doesn't.

enter image description here