Getting webpage data to a JavaScript that can only run out of the webpage

I'm making a chrome extension that injects JavaScript into a certain forum website. The JavaScript uses jQuery to add a button to the webpage whenever you're viewing a profile on the forum. There's a link already on every profile page that can show all of that profile's posts on the forum. The button added by jQuery (by the chrome extension) is supposed to use the 'show last posts' link already on each profile page to scrape through all of a profile's last posts and find posts which are actually new topics and not just replies to other topics on the forum. The chrome extension's purpose is to display all of a profile's last topics.

The problem is that the Node.js code that I use to scrape the webpages can't run in the context of a webpage. I think it doesn't work because it uses node modules 'request' and 'cheerio,' which don't have browserify support.

Right now I can only run the Node.js scraper by manually inserting into its code the link to the 'show last posts' page that I want to be scraped and then executing it in my local command prompt. There's two questions to all of this:

  1. How do I make the Node.js scraper execute to run out of the webpage's context when someone clicks the button that's added by the chrome extension?

  2. How do I send the 'show last posts' link from the chrome extension's content script (jQuery button) to the scraper? (since the scraper can't run in the browser to grab the link itself)

Please provide a number before your answers so everyone can tell which questions you're answering. Thanks

  1. To start with, you'll need to run your Node scraper on a server that's accessible by the Extension.

    You can host the server on a service like Heroku after which you'll need to add the server url to the Manifest file's permissions

    "permissions": [ "tabs",
    "*://link-to-your-herokuapp.herokuapp.com/*"
 ],

  2. To send data to the server, you can use a URL say /getLastPost and add request parameters as /getLastPost?parameter=value&parameter2=value2. In your case, you'll need to send the link (if i'm not mistaken) for which you can do something like getLastPost?link=someLinkHere. Then in your node.js scraper add a URL controller as follows: 

    var url = require('url'); 
app.get('/getLastPost', function(req, res){
    var url_parts = url.parse(req.url, true);
    var link = url_parts.query; //this would give you your link
    functionThatPerformsScraping(link); //send the link as a parameter to the function where you require the link
});

I hope this is good enough to get you started.

EDIT:

Look up Ajax here: http://api.jquery.com/jquery.ajax/. So this is what will get done on button click

$('.some-btn-class').on('click', function(
    $.ajax({
        url:'link-to-herokuapp.herokuapp.com/getLastPost?link='+val,
    }).done(function(){
        //whatever needs to be performed once ajax is done
    });
)});

It seems like you are new to Jquery and Node.js. Go through the different documentations.