JavaScript Stream query results with the native mongoDB driver for node streaming query results,mysql stream query results

I have 100,000 records in a mongoDB collection and trying to retrieve them in a node.js application using the native driver.

I follow the example in MongoDB doc for CursorStream but get the error:

RangeError: Maximum call stack size exceeded

Before this error I get many of these:

(node) warning: Recursive process.nextTick detected. This will break in the next version of node. Please use setImmediate for recursive deferral.

here is my code:

var query = {...};
var fields = {...};
var options = {
    // "limit": 10
    //"skip": 10,
    //"sort": title
}

var stream = myCollection.find(query, fields, options).stream();
//  stream.pause();
var results = [];
stream.on('data', function (item){
    results.push(item);
    stream.pause();
    // Restart the stream after 1 miliscecond
    setTimeout(function() {
        stream.resume();
    }, 1);
});

stream.on('close'.....

The error occurs also when I don't define a listener for the 'data' event. but it doesn't occur if a pause the stream right after its creation.

The mongod version is v2.4.1 The node driver version is 1.2.x

Any help/hint would be appreciated.

Answer:1

It looks like the problem is solved by setting a batch size in the Cursor Stream:

var stream = myCollection.find(query, fields, options).batchSize(10000).stream();
Answer:2

I've spent a long time trying to get this working. I have a section called "RightExtra" and a div inside it called "RightExtraContent". I'm trying to make it so that these two divs can move freely ...

I've spent a long time trying to get this working. I have a section called "RightExtra" and a div inside it called "RightExtraContent". I'm trying to make it so that these two divs can move freely ...

  1. move window with keyboard
  2. move window with arrow keys
  3. move window with keyboard mac
  4. move window with three fingers mac
  5. move window with keys
  6. move window with 3 fingers mac
  7. move window with keyboard ubuntu
  8. move window with trackpad mac
  9. move window with keyboard win 10
  10. move window with keyboard shortcut
  11. move window with arrows
  12. move window with mouse
  13. move window with keyboard linux
  14. move window with shortcut
  15. move window with powershell
  16. move window with keyboard macos
  17. move window with hotkey
  18. move window with keyboard off screen
  19. move window with wmctrl
  20. move window with monitor

OBJECTIVE Have Angular access a Service, retreive information, and display Service Information in two different controllers (later these controllers will manipulate this data, independently). LIVE ...

OBJECTIVE Have Angular access a Service, retreive information, and display Service Information in two different controllers (later these controllers will manipulate this data, independently). LIVE ...

Environment: Visual Studio 2008 SP1, ASP.NET and JavaScript I'm trying to do a quick document.write test but as soon as I add script tags Visual Studio editor doesn't like it. Specifically, the ...

Environment: Visual Studio 2008 SP1, ASP.NET and JavaScript I'm trying to do a quick document.write test but as soon as I add script tags Visual Studio editor doesn't like it. Specifically, the ...

Weird, line one works fine but line 3 give me TypeError: dojo.byId(...).attr is not a function. There is hidden fields that hold all student pair that as <input type="hidden" id="...

Weird, line one works fine but line 3 give me TypeError: dojo.byId(...).attr is not a function. There is hidden fields that hold all student pair that as <input type="hidden" id="...