Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Challenge in implementing suggester over huge chunks of text #1

Open
innovationchef opened this issue Aug 29, 2018 · 0 comments
Open

Comments

@innovationchef
Copy link
Member

innovationchef commented Aug 29, 2018

I added the following XML to the solrconfig.xml file for implementing suggester module.

 <searchComponent name="suggest" class="solr.SuggestComponent">
      <lst name="suggester">
        <str name="name">mySuggester</str>
        <str name="lookupImpl">FuzzyLookupFactory</str>
        <str name="dictionaryImpl">DocumentDictionaryFactory</str>
        <str name="field">BioChemEntity.description</str>
        <str name="suggestAnalyzerFieldType">string</str>
      </lst>
    </searchComponent>
    <requestHandler name="/suggest" class="solr.SearchHandler"
                    startup="lazy" >
      <lst name="defaults">
        <str name="suggest">true</str>
        <str name="suggest.count">10</str>
      </lst>
      <arr name="components">
        <str>suggest</str>
      </arr>
    </requestHandler>

This would capture the BioChemEntity.description entries and populate the suggester dictionary that is used internally by Solr to provide suggestions.

The suggestions were captured by the following -
http://localhost:8983/solr/solr_core_name/suggest?suggest=true&suggest.build=true&suggest.dictionary=mySuggester&suggest.q=can
to get suggestions for the word starting with 'can'. However, it throws an error in building up the dictionary. When the above link is posted on the browser, I receive an error (attaching the error file). The error means that Solr was not able to build its suggester dictionary for 106713 docs(that I indexed in one go) while using the description of samples.

Note: Description in biochementities contains huge chunks of text.
suggester_module_error.txt

innovationchef added a commit that referenced this issue Aug 29, 2018
…n our project (Note: the script works fine in general for Suggester implementation, however in our case, there are other issues as explained in issue #1 and #2)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant