I'm new in node js and MongoDB. I'm working on MongoDB search and pagination it is working good, but I have an issue is with time. it is tacked too much time in execution of county record and search record.
I use the small word for search then it works fast, if I use "long string" or "no record in database" then it's tack too much time 50 to 186.30 seconds. (it too too much time, I'm expected it 1 to 2 seconds)
I have more than 10,00,000 data on my record.
If I'm not including a count of the search word. it is tack 0.20 to 1.5 seconds, but when I'm counting recode with search word it is tack 25.0 to 35.0 seconds.
I have no idea how to decrease this time for count recodes with the search word(query optimization).
I have maximum try to query optimization.
I have also tried with
{
$count: "passing_scores"
}
but no change on time. I'm stuck on it. I have to decrease the time of count with the search word.
SQL Query for example
SELECT * FROM `post`
Left JOIN catagory ON post.catid=catagory.id
WHERE post_name LIKE '%a%' OR post_data LIKE '%a%' OR tags LIKE '%a%' OR post_url LIKE '%a%'
NODE and MongoDB
PostObj.count({},function(err,totalCount) {
if(err) {
response = {"error" : true,"message" : "Error fetching data"}
}
PostObj.aggregate([
{ $lookup:
{
from: 'catagories',
localField: 'catagory.catagory_id',
foreignField: '_id',
as: 'catagories_data'
}
},
{
$match:
{
$or: [
{"catagories_data.catagory_name": { $regex: new RegExp(search_data)}},
{"postname": { $regex: new RegExp(search_data) }},
{"posturl": { $regex: new RegExp(search_data) }},
{"postdata": { $regex: new RegExp(search_data) }},
{"tags": { $regex: new RegExp(search_data) }}
]
}
},
{ $limit : search_limit },
{ $skip : search_skip },
{ $group : { _id : "$_id", postname: { $push: "$postname" } , posturl: { $push: "$posturl" } } }
]).exec(function (err, data){
//end insert log data
if(err) {
response = {"error" : true,"message" :err};
}
if(search_data != "")
{
// count record using search word
PostObj.aggregate([
{ $lookup:
{
from: 'catagories',
localField: 'catagory.catagory_id',
foreignField: '_id',
as: 'catagories_data'
}
},
{
$match:
{
$or: [
{"catagories_data.catagory_name": { $regex: new RegExp(search_data)}},
{"postname": { $regex: new RegExp(search_data) }},
{"posturl": { $regex: new RegExp(search_data) }},
{"postdata": { $regex: new RegExp(search_data) }},
{"tags": { $regex: new RegExp(search_data) }}
]
}
},
{ $group: { _id: null, myCount: { $sum: 1 } } },
{ $project: { _id: 0 } }
]).exec(function (err, Countdata){
res.json({
sEcho : req.body.draw,
iTotalRecords: Countdata.myCount,
iTotalDispla,yRecords: Countdata.myCount,
aaData: data
});
}
res.json({
sEcho : req.body.draw,
iTotalRecords: totalPages,
iTotalDisplayRecords: totalPages,
aaData: data
});
});
});
Also, I have to try this way but it is tack 35.0 to 49.0 seconds more than 1st code.
PostObj.aggregate([
{ $lookup:
{
from: 'catagories',
localField: 'catagory.catagory_id',
foreignField: '_id',
as: 'catagories_data'
}
},
{
$match:
{
$or: [
{"catagories_data.catagory_name": { $regex: new RegExp(search_data)}},
{"postname": { $regex: new RegExp(search_data) }},
{"posturl": { $regex: new RegExp(search_data) }},
{"postdata": { $regex: new RegExp(search_data) }},
{"tags": { $regex: new RegExp(search_data) }}
]
}
},
{ '$facet' : {
metadata: [ { $count: "total" }, { $addFields: { page: NumberInt(3) } } ],
data: [ { $skip: 20 }, { $limit: 10 } ] // add projection here wish you re-shape the docs
} }
] )
If I do not use search word it is work good. I have an issue with when searching any word(count of records of that work without skip and limit)
Any solution for it?
from MongoDB search and pagination Aggregation Performance issue
ReplyDeleteVery Nice blog thanks for sharing valuable information with us.
Full Stack online Training
Full Stack Training
Full Stack Developer Online Training