Review main Merged

74742[BUGFIX] Prevent exception caused by hash collisions in indexed_search

Florian Schöppe CR +2 V +2 Normal +152 −2 updated 07/05/23, 17:18

Commit message

The indexed_search extension uses a truncated md5 hash for building a
word index. The truncation greatly increases the probability for
collisions. These collisions are the root cause for the problems that
are described in the related issues.

The best solution would be to either use full-length hashes or no
hashes at all. As this is not possible without changing the database
schema and rebuilding the whole word index, this patch prevents the
exceptions caused by the hash collisions. This is done by removing all
words with colliding hashes before the word index is updated.

Note that this patch just makes it possible to index page contents that
contain hash collisions (words with colliding hashes) again. The
affected words are obviously not added to the word index and though
won't yield results when searched for.

Review

  • Code review CR +2

    Imported before Forger kept the single votes, the next import adds them.

  • Verified V +2

    Imported before Forger kept the single votes, the next import adds them.

Issues 4

Files 5