Fabric JS not functioning in Angular JS

I am trying to do canvas manipulation with Fabric.js in an Angular.js application, and I can't even get the very basics working. I am just trying to set the background image, but at this point, I can't get that or even basic objects to show up on the canvas. I would really like that background image to be resized to whatever the zoom is set to as in this question, but that answer also results in a blank canvas for me, and I figure I can get to that later if I can just get the background on there in the first place.

Here is the directive:

app.directive('canvasDirective', function() {
    return {
        restrict: 'A',
        require: '^ngModel',
        link : function ($scope, elem, attrs) {
            var setBackground = function (src) {
                $scope.origImage.src = src;

                //This gives me a blank canvas:
                //$scope.canvas.setBackgroundImage(src, $scope.canvas.renderAll.bind($scope.canvas));

                //This also gives me a blank canvas:
                $scope.origImage.onload = function () {
                    var zoomCanvas = document.createElement("canvas");
                    var zoomCtx = zoomCanvas.getContext("2d");

                    zoomCanvas.width = $scope.origImage.width*$scope.zoom;
                    zoomCanvas.height = $scope.origImage.height*$scope.zoom;
                    // draw a scaled version of fullBk to zoomCanvas
                    zoomCtx.drawImage($scope.origImage,
                        0,0,$scope.origImage.width,$scope.origImage.height,
                        0,0,zoomCanvas.width,zoomCanvas.height);
                    $scope.canvas.setBackgroundImage( zoomCanvas.toDataURL(), function(){
                        $scope.canvas.backgroundImageStretch=true;
                        $scope.canvas.renderAll();
                    });
                };

                //This rectangle does not appear on the canvas
                var rect = new fabric.Rect({
                    left: 100,
                    top: 100,
                    fill: 'red',
                    width: 200,
                    height: 200
                });

                $scope.canvas.add(rect);
                $scope.canvas.renderAll();
            };
            setBackground($scope.pageSrc);

            $scope.$watch(attrs.ngModel, function (newVal, oldVal) {
                if (oldVal !== newVal) {
                    setBackground(newVal);
                }
            }, true);
        }
    };
});

Here is the partial:

<div>
    <canvas canvas-directive id="editPageArea"
            width="{{zoom*origImage.width}}" height="{{zoom*origImage.height}}"
            ng-model="pageSrc"
            style="border:1px solid red">
    </canvas>

</div>

If I pass an image source in to my setBackground function, then due to the red border, I can see that the width and height of the canvas are being set correctly. The origImage image, if I display it, is showing the correct image, so I know that the src url is correct. If I use a debugger, I can verify that all the lines of code are being called, right down to the inside of canvas.setBackgroundImage. However, the canvas remains blank. There is also no red rectangle being drawn inside the canvas. What am I doing wrong?