Node.js & the instagram API, 'Could not decode a text frame as UTF-8'

Question

Node.js & the instagram API, 'Could not decode a text frame as UTF-8'

I am doing a simple instagram API call with the http node.js module. Just a simple request like this:

getJSON : function(options, on_result, on_error) {

    var req = http.request(options, function(res) {

        var output = '';
        res.setEncoding('utf8');

        res.on('data', function (chunk) {
            console.log(chunk);
            return output += chunk;
        });

        return res.on('end', function() {
            var obj = JSON.parse(output);
            on_result(res.statusCode, obj);
        });
    });

    req.on('error', function(err) {
        on_error(err);
    });

    return req.end();
}

The on_result function just stringifies it with JSON.stringify, and sends it across a web socket with this function:

var connection = request.accept(null, request.origin);
connection.sendUTF(JSON.stringify(obj));

When I start the node server and point my browser to my url, I get this from chrome and firefox:

Could not decode a text frame as UTF-8.

I had absolutely no problems running this yesterday.

In the http 'on data' function, I am logging out the data to the console, and what I see onscreen looks ok except certain escape characters like these:

"text":"\ud83d\ude1d\ud83d\ude02\ud83d\udc4f\ud83d\udc8b"
"full_name":"\ud83c\uddee\ud83c\uddf9\u21e8\u261eSara\u261c\u21e6\ud83c\uddee\ud83c\uddf9"

Then, when it gets JSON stringified, these escape sequences turn into this:

"text":"<d83d><de1d><d83d><de02><d83d><dc4f><d83d><dc8b>"
"full_name":"<d83c><ddee><d83c><ddf9>â¨âSaraââ¦<d83c><ddee><d83c><ddf9>"

Which explains why the browsers complain about UTF-8.

Am I doing something wrong?

javascript
node.js
instagram

Answer 1

you can try to escape \u as follows:

    return res.on('end', function() {
        var escaped_output = output.replace(/\\u/g, "\\\\u");  //<--
        var obj = JSON.parse(escape_output);
    });

Also, why do you convert 'output' to object and then back to string and not pass it to the websocket as received at the response.

Answer 2

I've determined that the data is coming from instagram like this and there is not much that can be done, except to try escaping or removing the non utf-8 characters. Since only certain fields are like this and my application has no need for those fields, I can get around it.