I am trying to scrape https://www.freelance.nl/opdrachten/zoeken for data using Request and Cheerio but I am running into issues posting search terms.
I cannot see where the search string and selected category are sent during the post when using the site and how I can use them in Request to automate searches from my node application.
Basically I want to be able to send different search terms using Request then I can scrape the returned html for the data I need.
So far I have this:
request.post('https://www.freelance.nl/opdrachten/zoeken', { form: { key: 'value' } },
function (error, response, body) {
if (!error && response.statusCode == 200) {
console.log(body)
}
}
);
But as I cannot see where the form data is stored in dev tools, I cannot send the correct values in the 'form' object. I'm pretty sure it's in request payload, but how do I get to that from my node application?
Is there an easier way to do this? Am I completely wasting my time?
I've slighly modified your code:
payload = {'projectFilterForm[keywords]':'javascript','projectFilterForm[category][]': '1'}
request.post('https://www.freelance.nl/opdrachten/zoeken', { data:payload },
function (error, response, body) {
if (!error && response.statusCode == 200) {
console.log(body)
}
}
);
I know you can do ternary expressions in Javascript for an if - else statement, but how about an else- else if- else statement? I thought that surely this would be supported but I haven't been able to ...
I know you can do ternary expressions in Javascript for an if - else statement, but how about an else- else if- else statement? I thought that surely this would be supported but I haven't been able to ...
Being fairly new to jquery and javascript I can't seem to understand how the .text() method really works. I read through the jQuery documentation but still can't figure it out. for (var i=0; i < ...
Being fairly new to jquery and javascript I can't seem to understand how the .text() method really works. I read through the jQuery documentation but still can't figure it out. for (var i=0; i < ...
When adding two or more functions in my jquery script file the function/s stops working. It's working perfectly fine when I've only written one function. But when I'm adding a second one both of ...
When adding two or more functions in my jquery script file the function/s stops working. It's working perfectly fine when I've only written one function. But when I'm adding a second one both of ...
Calling this async Web Method, I get a 500 error. Are there other ways to call a web method async? Stacktrace: Server Error in '/' Application. Unknown web method SendMessage. Parameter name: ...
Calling this async Web Method, I get a 500 error. Are there other ways to call a web method async? Stacktrace: Server Error in '/' Application. Unknown web method SendMessage. Parameter name: ...