XPsp2, D6sp2, DBISAM 4.22b5

I'm rewriting an app and decided for the email part that rather than decode emails on arrival and store the bits in separate fields (_Headers, _Message, _InLinPics, _Attachments) I'd just receive them and store in encoded format (splitting out subject etc for speed of display in a grid) and decode as and when necessary.

The problem I'm encountering is with full text indexing on the message. Decode the message, strip to plain text, massage to leaving just the words I want.

If I create an extra field (_FastSearch), set up full text indexing and then loop through the table decoding the messages and stuffing the things I want indexing into _FastSearch its fine. The average time taken per record is

Decode and stuff into _FastSearch - 11ms
Decode and stuff into _FastSearch with fti - 35ms

As you can see fti gives a fair performance hit - but still very acceptable. However, if I try and use the same code in OnTextIndexFilter and use sql to add the encoded field to the full text index I don't know how long things take because I decided to stop it after 12 hours.

I don't want to store the data multiple times if I can avoid it - any suggestions.


In ELN there are about 25k records in total, 1.5k records where there is no data in _Encoded (the field holding the message data), about 15k that are plain text, less than 1k are rtf and the remainder are mime.

Roy Lambert

I feel like a complete prat - I had a test in (FileName <> '_encoded' ) should have been a capital E


Poor old fti system was trying to index the mime encoded stuff including upto 15Mb of attachments so its forgiven for taking so long


Roy Lambert