Uploaded image for project: 'Search Module'
  1. Search Module
  2. SEARCH-27

Deal with really large indexes

    XMLWordPrintable

    Details

    • Complexity:
      Medium

      Description

      This is something that came up in the Lucene users list, and might be problem on massively scaled (600 million rows +) data. Which is that many operating systems have a limit on the directory size limit or will run out of memory on a Lucene search. We should investigate how the index will scale and figure out when/if we need to start sharding indexes. The result of this investigation should 1. be an upper bound on a single index. 2. the system resources that are required to do searches on very large applications.

        Attachments

          Activity

            People

            Assignee:
            Unassigned
            Reporter:
            czakian Christopher Zakian
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Dates

              Created:
              Updated: