HP TRIM 7.2 and IDOL Document Content Index (2589 Views)
Reply
Honored Contributor
EWillsey
Posts: 1,923
Registered: ‎04-20-2010
Message 1 of 14 (2,589 Views)

HP TRIM 7.2 and IDOL Document Content Index

Hey trimmers,

 

Would anyone know the best place to find documentation related to how IDOL works within the TRIM environment?  I see there are three additional windows services devoted to it.  So far my testing has gone well, but I'm not used to it and am looking for some of the configuration documentation.  Specfically around: stemming, hyphenChars, augmentseparators, indexnumber, stoplist, and the rest of the configuration file.

 

I also noticed that when I register a document and it's processed by the event server, it can take up to 2 minutes to become available in the document content search.  If I manually reindex through the TES then it's instantly available for a search.  Anyone know why?

 

Also how do you disable the log from showing every DCI query? 

 

Thanks!

Erik

Respected Contributor
Philip Harney
Posts: 287
Registered: ‎08-02-2009
Message 2 of 14 (2,563 Views)

Re: HP TRIM 7.2 and IDOL Document Content Index

Hello Eric,

 

There's always been a delay before processing events with the event processor, I expect its still the same in TRIM 7 even with the new design of the indexer. 

 

Think the registry key is

 

HKLM\SOFTWARE\Hewlett-Packard\HP TRIM\WorkgroupServers\DatabaseEventPollIntervalM


I've done a fair bit of looking online for information on the IDOL engine but couldnt find much either, and the documentation provided by HP has been minimal.

 

If I find anything I will post it

 

Phil

Trusted Contributor
Rich_Kid
Posts: 239
Registered: ‎03-31-2010
Message 3 of 14 (2,559 Views)

Re: HP TRIM 7.2 and IDOL Document Content Index

I'm especially interested in this as well.

 

I would like to know what type of IDOL HP have bundled with Trim. With ISYS we got a cut down version of the product that did DCI and thats it. If you wanted the advanced features ISYS offered that was a seperate purchase.

 

Is this the case with IDOL, we get a basic DCI or are we getting additional features. If we don't get the full feature set is there a posibility to upgrade it with the Trim license rather than a seperate purchase.

 

Cheers

 

Esteemed Contributor
Richard Kelly_7
Posts: 205
Registered: ‎02-15-2007
Message 4 of 14 (2,552 Views)

Re: HP TRIM 7.2 and IDOL Document Content Index

The delay for content indexing (actually all events) is now configurable via Enterprise Studio - 120 seconds is the default value.

 

The IDOL shipped with TRIM is cut down - but mostly from the point of view of non-content indexing functionality and only "cut-down" by not having some of the 500+ features enabled in the license.  You can replace it with a standard IDOL implementation, but it is not a simple product to use - the basic IDOL Administrator course is a full-on 4 day session. I suspect this is why the IDOL documentation does not ship with TRIM - just too much functionality for people to get wrong!

Note: Any posts I make on this forum are my own personal opinion and do not constitute a formal commitment on behalf of HP.
Honored Contributor
EWillsey
Posts: 1,923
Registered: ‎04-20-2010
Message 5 of 14 (2,538 Views)

Re: HP TRIM 7.2 and IDOL Document Content Index

Thanks. What I really meant was why is there a delay from when TRIM says it's indexed to when it shows up in the search results. However I found in the post-release notes (KM1317133) this is referenced as a known issue:

Symptoms:
It is possible for TES to report that IDOL has finished processing something while IDOL is still working.
This can occur when creating a new IDOL instance (TES will report the operation is finished before IDOL has created the new instance) or during a Reindex (TES will report the reindex is complete, while IDOL has yet to process all of the documents).
Cause
This issue is due to the way TRIM Enterprise Studio communicates with the IDOL server, and the way the IDOL server processes incoming requests.
Resolution
It is difficult for the administrator to directly check the IDOL server to see if operations have actually completed. It is suggested that an administrator waits for a short period after creating a new IDOL instance before performing an reindex. Similarly, it may take some time after a reindex has “completed” for all of the records to appear in a DCI search.





I'm still interested in the other settings asked about. Can't easily find that documented anywhere on the web.
Frequent Advisor
alexw_1
Posts: 55
Registered: ‎05-05-2010
Message 6 of 14 (2,515 Views)

Re: HP TRIM 7.2 and IDOL Document Content Index

Additionally, it would appear that the DCI maintenance tools within TES now do nothing (yet are still present).

It would appear that IDOL is a black box for the DCI, and if there are problems, all an administrator can do it perform a full rebuild of the index?
Honored Contributor
EWillsey
Posts: 1,923
Registered: ‎04-20-2010
Message 7 of 14 (2,513 Views)

Re: HP TRIM 7.2 and IDOL Document Content Index

The post-release notes mention that those features (check and optimize) don't function, but also doesn't indicate if they intend to be fixed.   I'd assume they do.  The reindex function does work however.

Valued Contributor
raynebc
Posts: 521
Registered: ‎05-06-2009
Message 8 of 14 (2,503 Views)

Re: HP TRIM 7.2 and IDOL Document Content Index

I'm having trouble finding the post-release notes on HP's support portal.  Could you lead me in the right direction?

Honored Contributor
EWillsey
Posts: 1,923
Registered: ‎04-20-2010
Message 9 of 14 (2,501 Views)

Re: HP TRIM 7.2 and IDOL Document Content Index

Honored Contributor
Grundy
Posts: 2,849
Registered: ‎02-16-2009
Message 10 of 14 (2,448 Views)

Re: HP TRIM 7.2 and IDOL Document Content Index

I put up that tech note.

I accidentally made it customer visible when it was supposed to be internal only (according to the higher-ups), but glad I made it public. Looks like it's answered a few questions already.  :)

 

If you guys need any help and have any specific questions, log a case.

Paul (Burkle) and myself have had enough IDOL training and are familiar enough with the implementation to answer most basic questions.



::::::::::::::::::::::
NOT A HP EMPLOYEE
::::::::::::::::::::::

Kapish.com.au
Occasional Contributor
estryzek
Posts: 2
Registered: ‎08-20-2012
Message 11 of 14 (2,186 Views)

Re: HP TRIM 7.2 and IDOL Document Content Index

With IDOL Service. AS with Space limitations by design in VM environment, How to relocate the index to new ISCSI LUN without complete uninstall and reinstall.

With a VM Guest Server we store user data separate from server images. This would also include DCI DATA. There should be a way to move a DCI to a Shared Volume or at minim a different VOL then primary install location of services applications.

In TES we only have the option to create new IDOL index and no choice to the location of it.

So I now ask how to move a Location of the IDOL Storage/Index Locations ?

Honored Contributor
Grundy
Posts: 2,849
Registered: ‎02-16-2009
Message 12 of 14 (2,173 Views)

Re: HP TRIM 7.2 and IDOL Document Content Index

This is all configurable in the IDOL config files.

 

See attached tech note.



::::::::::::::::::::::
NOT A HP EMPLOYEE
::::::::::::::::::::::

Kapish.com.au
Occasional Advisor
Jai Shankar
Posts: 6
Registered: ‎11-09-2008
Message 13 of 14 (1,829 Views)

Re: HP TRIM 7.2 and IDOL Document Content Index

Hi,

 

We recently upgraded to TRIM 7.2.1 and on build 3519.  We don't have a huge database and stats show that there were about 7600 records added since the start of the year.

 

However the TRIM server has regular very high volume disk read/writes.

 

Looking at the TRIM server CPU, Memory, and Disk statistics – the server seems to go into a high load state every 10-15mins, resulting in very high disk read/writes which causes the lag. At about the same time there is a 34GB lag in the storage replication.  Normal lag is a few hundred MB. 

 

We were doing a DR Exercise and there was a 34GB lag between Production and DR storage replication but there were  only 77 documents added to TRIM on that day.  Has anyone have any thoughts on this?

 

Honored Contributor
Sander Hoogwerf
Posts: 378
Registered: ‎11-05-2008
Message 14 of 14 (1,824 Views)

Re: HP TRIM 7.2 and IDOL Document Content Index

Define "huge", for one 1GB may seem huge, for others 1PB may still be considered small.

 

Use Windows Resource Monitor to pinpoint which file operations take place. From there you can identify which application is.

 

If IDOL is active on that server, then it may be a likely candidate for the high amount of disk access. IDOL writes a new index file on each commit in the dynterm folder which each have a maximum size of ~12GB and then switches over.

Still a long way from your 34GB, but getting closer than any of the other TRIM components.

Since new index is built up in parallel, your DR solution may consider this a new file an try to sync the entire file size instead of just the delta between the old and new index.

You can check the IDOL logfiles to have better understanding of what it's doing.

(Any opinions expressed in this forum are my own personal opinion and should not be construed as an official statement by Hewlett Packard.)

Analytics & Data Management
Application & Business Services
HP Enterprise Services
The opinions expressed above are the personal opinions of the authors, not of HP. By using this site, you accept the Terms of Use and Rules of Participation.