Item5647: Make the indexing process more robust
Current State: Being Worked On
Target Release: n/a
A crash in indexing one file or one topic should not lead to a stop of the whole indexing process.
- 21 May 2008
Re-naming the plugin, and needs to check if this still applies.
did a lot of work to improve robustness, but I'm not sure about a crash of a single file. He tried to remove crashes all together, but it needs testing...
- 25 Sep 2009
i can confirm this can still happen. strangely, the
was able to stringify the spreadsheet (about 18MB) in a couple of minutes, but
didn't manage to process the same file, even overnight.
- 27 Feb 2010
Most probably your xls file is password protected. There are no third-party libraries to even detect that an xls is password protected. Instead the reader will consume it as is and generate large internal data structures while reading some arbitrary bits.
Been addressed in StringifierContrib
, sort of, by creating a separate cmdline perl to real the xls. If it crashes, it does not take
the rest of the indexer with it.
- 27 Feb 2010
well, i don't think it was password-protected as
was able to parse and produce results for the same file which was hanging the indexer. so, that's very disconcerting. also, it's not about the indexer crashing, but instead hanging. so, the wrapper will need some sort of watchdog capability. i'm sure StringifierContrib
will provide some inspiration, though the long term goal is to refactor KinoSearchContrib to use StringifierContrib
- 11 Mar 2010