Node.js mikeal/request module - Garbled non-utf8 website (Shift_JIS)

Question

Node.js mikeal/request module - Garbled non-utf8 website (Shift_JIS)

I am trying to access a non utf-8 website using request module. Response is garbled for this request.

var request = require('request');
request('http://www.alc.co.jp/', function (error, response, body) {
  if (!error && response.statusCode == 200) {
    console.log(body) // Print the web page.
  }
});

Even after setting the encoding option to Shift_JIS I am seeing garbled Japanese text.

javascript
node.js
encoding
request
shift-jis

Answer 1

You need to do the conversion yourself. The example code below uses node-iconv.

    var Iconv = require('iconv').Iconv;
    var request = require('request');
    request({
      uri: 'http://www.jalan.net/',
      encoding: null,
    }, function (error, response, body) {
      if (!error && response.statusCode == 200) {
        body = new Iconv('shift_jis', 'utf-8').convert(body).toString();
        console.log(body); // Print the web page.
      }
    });

The encoding: null parameter asks request not to convert the Buffer (a byte array) into String yet.
We pass this buffer to Iconv for converting into another Buffer of UTF-8 encoding.
Now this Buffer is good for being converted into a String.

(BTW, http://www.alc.co.jp has switched to UTF-8, so I substituted with another site.)