This relates to MongoDB and the Node.js Mongoose library.
I am dealing with a situation where there is a fair amount of data from one schema that needs to be replicated in another. I either need to load that data separately or use references as the data needs to be up to date (aka it changes frequently). These frequent changes make embeded documents impractical.
Is this a bad idea if I am going to end up with 5-10 references within an object (not always, that's probably a mid to high number)?
If that is a bad idea is it better to separately query for each of the needed sub-documents with another query or set of queries?
To add to the mix, I will be putting these behind a memory cache so it's not going to be reloaded constantly. It will only be touched when the cache expires or if the document changes.
Your thoughts and experiences?
Based on some requests here's an example and some more information:
var postSchema = new Schema({
postType: String,
created: {type: Date, default: Date.now},
event: {type: Schema.Types.ObjectId, ref:'Event'},
...
});
In this case the event would be loaded almost every time. In most other cases, it would only be loaded conditionally.
To answer the questions of scale, it's going to be pretty heavy usage. Probably closer to millions than hundreds.
Tell me this, would it be better to cache the event object in redis (our caching target), and call it post load from there and if it's not in cache do a separate query? I'm thinking that might be a better course than references. Agree? Disagree?