Cooking up the Treats => General Development => Topic started by: newt on February 17, 2010, 10:19:38 am

Title: [BROKEN AGAIN] Reasonable Forum Search Option
Post by: newt on February 17, 2010, 10:19:38 am
I'm not sure where we stand in terms of a reasonable and reliable search option for our forum, but what's currently in place does not work - or, at least, it does not work in Opera.  It's been long enough that we've lived without a decent search tool and I think it's time to get it fixed.  The single best search tool would be to have the integrated SMF search tool working.

I've tried the upper-right (powered by KnowledgeDex) search tool and it is working very well, though it _mostly_ worked for a while, it does not work any more (at least not in Opera).  The wheels just keep on spinning and no results are displayed.  When it DID work it would provide mostly useful results but after visiting a result and going back would cause a "skip" in the process. IIRC, it was as though you had hit the back button twice, however there was no way to move forward without ending up at the forum result again. Quirky to say the least.

I've tried the SMF navigation bar search tool (Home Help Search Profile etc) and the results it brings back are poor, at best. Often times my search terms are not even found in the "most relevant" results that are shown - it takes digging through the results to find something useful.  On the bright side, at least this tool gives results - knowledgedex does not.  When a forum's search tool is working (phpBB or smf), it is very easy to work with. I think this would be choice #1 for me in terms of a search tool.

I've tried the google method (search terms This is by far the most reliable search results at the present time, but a very non-standard way of search a forum.  Not many folks actually think that in order to search a forum they're currently visiting they'll need to leave the site, visit google, and enter an "advanced" search string like the aforementioned string.  On the bright side, it gives reliably reasonable and relevant results.  Regardless of actual forum search features working or not, this is a viable _backup_ option. It should not be the standard since many folks are not familiar with the "" method of searching a website.

If help is needed to get the SMF search tool fixed then I'll certainly lend a hand - I've worked with phpBB forum several times in the past and am familiar with the web technologies involved (php, mysql, javascript, etc). I would guess that a simple method of backup_forum>install_new_forum>import_backup that it should 1) fix the search feature, and 2) bring us back to current.  Of course, running this process in a test environment first would be necessary.
Title: Re: Reasonable Forum Search Option
Post by: bigpaws on February 17, 2010, 12:30:26 pm
Hi Newt,

I am not aware of problems with Opera, although I have never tried it.

The search is a combined search for several sources not just the forum, which is why
the returns take time. I have tried SMF search myself with poor results which is why
our hosting service has another solution in place.

I will forward this to those that take care of the search function with Opera.

If there is no response to this post I will be glad to post the response myself.

Title: Re: Reasonable Forum Search Option
Post by: newt on February 17, 2010, 02:00:11 pm

Thanks for your reply, and you are right (as usual). If I just wait a bit longer I will see the results in the KnowledgeDex search option.  It takes longer than I'd expect a 'search' function to take but as long as I'm aware of the time issue and the results are relevant, I won't complain.  I did a test search for 'dosemu' and it took about 25 seconds to return the forum results, but they actually showed up and were very relevant.  I should learn to be more patient; I've just become accustomed to quick searches.  Increasing the returned result time for KnowledgeDex would be a bonus, if it's possible.

I'll need to test the strange "double-back" issue I recall from a while back. It's been so long since I've used the knowledgedex search that perhaps opera 10.10 has the issue resolved.  I'll certainly let you know if the problem has been fixed or persists.

Thanks again!

The strange 'back' behavior still persists. Here's what I did, and what happened. In knowledgedex I searched for 'dosemu', then searched for 'xorg.conf', and then searched for 'nvidia'. I click on one of the nvidia forum results. Then clicked the opera back button (mouse gesture), and it took me back to knowledgedex but was re-searching for 'dosemu'. Does that make sense? Basically, after multiple searches, clicking on a result, and clicking back it took me to my initial search rather than the most recent.
Title: Re: Reasonable Forum Search Option
Post by: bigpaws on February 17, 2010, 05:01:16 pm
Hi Newt,

I did send an email about the problem. The searches appear to be slower than normal
which may just need to be reindexing. I will send another email if it has not changed
in the next day.

Title: Re: Reasonable Forum Search Option
Post by: nightflier on February 17, 2010, 05:01:58 pm
On my end, I get identical results from the KnowleDex search in FF 3.6 and Opera 10.10.
Title: Re: Reasonable Forum Search Option
Post by: Deach on February 17, 2010, 09:39:42 pm
I too got the same results in FF and Opera 10.10. I agree with "taking a little bit longer than before" statement.  This happened on both browsers though. 
Title: Re: Reasonable Forum Search Option
Post by: Leo Scott on February 18, 2010, 07:58:04 am

We are currently having speed issues with KnowledgeDex on all browsers. We are in the process of resolving this issue as it affects all of our sites that use our enterprise search solution. Also, the back-button issue is caused by having to completely reload the search page. Since the search page is pre-loaded with the search term entered in the search box on the forum, it will reload with that term. Subsequent searches are performed without reloading the search page by using AJAX(or rather, AJAJ). The reason that this was done this way is that we needed a way to asynchronously show results from all knowledge stores (such as forum and wiki). Also.. we have to use AJAX to load search results because they can vary in the amount of time it takes to generate the results... It is not just a result of a search engine; each search result has to be permission-checked in real-time as the search engine itself may provide pages that you do not have permissions to view (hence, enterprise search).

In short, this is not a problem in just Opera. The first problem is a delay in the search engine response which we are actively working on. The second problem is a bug where we do not update the address bar after a new search time is entered. I can not give a timeline on fixing this second issue at the moment.I suggest in the mean time that you open search results in new tabs. As soon as we resolve the bug (it exists for all of our clients), Vector's installation will be updated at the same time as our commercial clients. The first issue should be fixed today or tomorrow (#1 on my list today).

-Steve @ SQI
Title: Re: Reasonable Forum Search Option
Post by: newt on February 18, 2010, 08:28:00 am
That's great news Steve, and thanks for the detailed accounting of the problem.  If the speed issue were taken care of then researching wouldn't be a big deal since the searches would be quick; though it would still be good to fix that if at all possible.  I remember when KD was first implemented it worked quickly - i.e. typical search speed.  If it were able to get back to that speed or close then successive searches aren't nearly as painful, but waiting ~25-30 seconds per search seems a bit long.

Thanks again for your help with all of this!
Title: Re: Reasonable Forum Search Option
Post by: Leo Scott on February 18, 2010, 11:11:04 am
KnowledgeDex is now quick again! Newt, the issue was actually a problem with our upstream ISP's reverse DNS entries. And since I could not turn off rDNS on our search engine (long story), I was able to put a local DNS server to send empty rDNS responses.

In short... it was quick before and now it is quick again. Sorry for the ..ahem.. delay ;)

-Steve @ SQI
Title: Re: [SOLVED] Reasonable Forum Search Option
Post by: newt on February 18, 2010, 03:38:20 pm
Steve, I tried the fixed KnowledgeDex and it's working wonderfully - quick - like it used to.  Thanks for finding the issue and fixing it!!

The pre-loaded search term issue of clicking 'back' and seeing my initial search term still persists, but like I said, as long as the results are quick to show then I have no problems just re-searching for a follow-up term.  On the other hand, if there is potential to devise a robust solution to the pre-loaded/follow-up search term issue then that would make just that much smoother.

Thanks for everything!!
Title: Re: [SOLVED] Reasonable Forum Search Option
Post by: newt on February 18, 2010, 03:46:09 pm
I'm just observing the pre-loaded issue you mention above and notice that the knowledgedex search page is initially passed a 'query' variable via address (GET), however, subsequent searches do not modify or update the 'query' variable and it just remains the same.  I would think that if a search directly from the knowledgedex could update the 'query' variable then when a user clicks back they would be presented with their previous search rather than initial search.  Either that, or somehow modify the forum search field to pass the 'query' variable via POST rather than GET.  Obviously I haven't thought it all through but these are just my initial thoughts and observations.

For instance, try this:
-Enter a search term in the forum KnowledgeDex field (upper-right)
-Alter the search term in the addresses 'query' variable (https: //
-Again, alter the search term in the address
-Click on a forum result
-Now click 'back' on your browser (it should take you back to your most recent search term rather than the initial search term)
Title: Re: [BROKEN AGAIN] Reasonable Forum Search Option
Post by: newt on November 20, 2010, 07:48:40 pm
Seems the search is down for me again. A simple search of 'xorg' reveals no results in any category. Not sure if anyone is aware of this yet.

Plus, my previous post (from Feb '10) addresses an 'annoyance' issue I (personally) have with the search tool and a reasonable solution.
Title: Re: [BROKEN AGAIN] Reasonable Forum Search Option
Post by: bigpaws on November 20, 2010, 07:56:47 pm
Interesting that I used the search yesterday without any problems.

There is more to the search than just the forum. The search
involves all the above you mention plus Lucene to combine
the search from the knowledge base, forum and support
ticket system.

I have sent an email to gentleman that takes care of this.