au.org.ala.checklist.lucene
Class CBIndexSearch

java.lang.Object
  au.org.ala.checklist.lucene.CBIndexSearch

public class CBIndexSearch
extends java.lang.Object

The API used to perform a search on the CB Lucene Index. It follows the following algorithm when trying to find a match: 1. Search for a direct match for supplied name on the name field (with the optional rank provided). 2. Search for a match on the alternative name field (with optional rank) 3. Generate a searchable canonical name for the supplied name. Search for a match on the searchable canonical field using the generated name 4. Clean up the supplied name using the ECAT name parser. Repeat steps 1 to 3 on the clean name until a match is found 5. No match is found When a match is found the existence of homonyms are checked. Where a homonym exists, if the kingdom of the result does not match the supplied kingdom a HomonymException is thrown.

Author:

Natasha

Field Summary

static java.util.regex.Pattern	affPattern
static java.util.regex.Pattern	cfPattern
protected org.apache.commons.logging.Log	log
protected TaxonNameSoundEx	tnse
static java.util.regex.Pattern	virusStopPattern
static java.util.regex.Pattern	voucherRemovePattern

Constructor Summary

CBIndexSearch()

CBIndexSearch(java.lang.String indexDirectory)
Creates a new name searcher.

Method Summary

void	dumpSpecies() Dumps a list of the species LSID's that are contained in the index.
org.apache.lucene.search.TopDocs	getIRMNGGenus(LinnaeanRankClassification cl, RankType rank) Multiple genus indicate that an unresolved homonym exists for the supplied search details.
java.lang.String	getPrimaryLsid(java.lang.String lsid) Returns the primary LSID for the supplied lsid.
void	reopenReaders()
RankType	resolveIRMNGHomonym(LinnaeanRankClassification cl, RankType rank) Attempt to resolve the homonym using the IRMNG index.
java.lang.String	searchForAcceptedLsidDefaultHandling(LinnaeanRankClassification cl, boolean fuzzy) Returns the accepted LSID for the supplied classification.
java.lang.String	searchForAcceptedLsidDefaultHandling(LinnaeanRankClassification cl, boolean fuzzy, boolean ignoreHomonyms)
NameSearchResult	searchForAcceptedRecordDefaultHandling(LinnaeanRankClassification cl, boolean fuzzy) Returns the accepted result for the supplied classification.
NameSearchResult	searchForAcceptedRecordDefaultHandling(LinnaeanRankClassification cl, boolean fuzzy, boolean ignoreHomonym)
NameSearchResult	searchForCommonName(java.lang.String name) Performs a search on the supplied common name returning a NameSearchResult.
java.lang.String	searchForLSID(LinnaeanRankClassification cl, boolean recursiveMatching) Search for an LSID with the supplied classification without a fuzzy match.
java.lang.String	searchForLSID(java.lang.String name) Searches for the name without using fuzzy name matching...
java.lang.String	searchForLSID(java.lang.String name, boolean fuzzy) Searches the index for the supplied name with or without fuzzy name matching.
java.lang.String	searchForLSID(java.lang.String name, boolean fuzzy, boolean ignoreHomonyms)
java.lang.String	searchForLSID(java.lang.String name, LinnaeanRankClassification cl, RankType rank) Search for an LSID based on suppled name, classification and rank without a fuzzy match...
java.lang.String	searchForLSID(java.lang.String name, LinnaeanRankClassification cl, RankType rank, boolean fuzzy, boolean ignoreHomonym) Search for an LSID based on the supplied name, classification and rank with or without fuzzy name matching.
java.lang.String	searchForLSID(java.lang.String name, RankType rank) Searches for an LSID of the supplied name and rank without a fuzzy match...
java.lang.String	searchForLSID(java.lang.String name, RankType rank, boolean fuzzy) Searches the index for the supplied name of the specified rank with or without fuzzy name matching.
java.lang.String	searchForLSID(java.lang.String name, RankType rank, boolean fuzzy, boolean ignoreHomonyms) Searches for the supplied name of the specified rank with or without fuzzy name matching.
java.lang.String	searchForLSID(java.lang.String name, java.lang.String kingdom, java.lang.String scientificName, RankType rank) Deprecated. Use #searchForLSID(java.lang.String, au.org.ala.data.model.LinnaeanRankClassification, au.org.ala.data.util.RankType, boolean) instead. It is more extensible to supply a classification object then a list of higher classification
java.lang.String	searchForLsidById(java.lang.String id) Gets the LSID for the record that has the supplied checklist bank id.
java.lang.String	searchForLSIDCommonName(java.lang.String commonName) Performs a search on the common name index for the supplied name.
NameSearchResult	searchForRecord(LinnaeanRankClassification cl, boolean recursiveMatching)
NameSearchResult	searchForRecord(LinnaeanRankClassification cl, boolean recursiveMatching, boolean fuzzy)
NameSearchResult	searchForRecord(LinnaeanRankClassification cl, boolean recursiveMatching, boolean addGuids, boolean fuzzy) Search for an LSID with the supplied classification without a fuzzy match.
NameSearchResult	searchForRecord(java.lang.String name, LinnaeanRankClassification cl, RankType rank) Searches for a record based on the supplied name, classification and rank without fuzzy name matching
NameSearchResult	searchForRecord(java.lang.String name, LinnaeanRankClassification cl, RankType rank, boolean fuzzy)
NameSearchResult	searchForRecord(java.lang.String name, LinnaeanRankClassification cl, RankType rank, boolean fuzzy, boolean ignoreHomonyms) Searches for a record based on the supplied name, rank and classification with or without fuzzy name matching.
NameSearchResult	searchForRecord(java.lang.String name, RankType rank) Searches index for the supplied name and rank without a fuzzy match.
NameSearchResult	searchForRecord(java.lang.String name, RankType rank, boolean fuzzy) Searches the index for the supplied name of the specified rank.
NameSearchResult	searchForRecord(java.lang.String name, java.lang.String kingdom, java.lang.String genus, RankType rank) Deprecated. Use searchForRecord(java.lang.String, au.org.ala.data.model.LinnaeanRankClassification, au.org.ala.data.util.RankType, boolean) instead. It is more extensible to supply a classification object then a list of higher classification
NameSearchResult	searchForRecordByID(java.lang.String id) Returns the records that has the supplied checklist bank id
NameSearchResult	searchForRecordByLsid(java.lang.String lsid)
MetricsResultDTO	searchForRecordMetrics(LinnaeanRankClassification cl, boolean recursiveMatching)
MetricsResultDTO	searchForRecordMetrics(LinnaeanRankClassification cl, boolean recursiveMatching, boolean fuzzy)
MetricsResultDTO	searchForRecordMetrics(LinnaeanRankClassification cl, boolean recursiveMatching, boolean addGuids, boolean fuzzy) Search for a specific name returning extra metrics that can be reported as name match quality...
MetricsResultDTO	searchForRecordMetrics(LinnaeanRankClassification cl, boolean recursiveMatching, boolean addGuids, boolean fuzzy, boolean ignoreHomonym)
java.util.List<NameSearchResult>	searchForRecords(java.lang.String name, RankType rank, boolean fuzzy) Searches for records with the specified name and rank with or without fuzzy name matching
java.util.List<NameSearchResult>	searchForRecords(java.lang.String name, RankType rank, LinnaeanRankClassification cl, int max) Searches for a list of results for the supplied name, classification and rank without fuzzy match
java.util.List<NameSearchResult>	searchForRecords(java.lang.String name, RankType rank, LinnaeanRankClassification cl, int max, boolean fuzzy) Searches for the records that satisfy the given conditions using the algorithm outlined in the class description.
java.util.List<NameSearchResult>	searchForRecords(java.lang.String name, RankType rank, LinnaeanRankClassification cl, int max, boolean fuzzy, boolean ignoreHomonyms)
void	updateClassificationWithGUID(LinnaeanRankClassification cl) Updates the supplied classification so that the supplied ID's are substituted with GUIDs.
NameSearchResult	validateHomonymByAuthor(java.util.List<NameSearchResult> result, java.lang.String name, LinnaeanRankClassification cl)
NameSearchResult	validateHomonyms(java.util.List<NameSearchResult> results, java.lang.String name, LinnaeanRankClassification cl) Takes a result set that contains a homonym and then either throws a HomonymException or returns the first result that matches the supplied taxa.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

log

protected org.apache.commons.logging.Log log

tnse

protected TaxonNameSoundEx tnse

virusStopPattern

public static final java.util.regex.Pattern virusStopPattern

voucherRemovePattern

public static final java.util.regex.Pattern voucherRemovePattern

affPattern

public static final java.util.regex.Pattern affPattern

cfPattern

public static final java.util.regex.Pattern cfPattern

Constructor Detail

CBIndexSearch

public CBIndexSearch()

CBIndexSearch

public CBIndexSearch(java.lang.String indexDirectory)
              throws org.apache.lucene.index.CorruptIndexException,
                     java.io.IOException

Creates a new name searcher. Using the indexDirectory as the source directory

Parameters:

indexDirectory - The directory that contains the CB and IRMNG index.

Throws:

org.apache.lucene.index.CorruptIndexException

java.io.IOException

Method Detail

reopenReaders

public void reopenReaders()

dumpSpecies

public void dumpSpecies()

Dumps a list of the species LSID's that are contained in the index.

searchForLSID

public java.lang.String searchForLSID(java.lang.String name,
                                      boolean fuzzy)
                               throws SearchResultException

Searches the index for the supplied name with or without fuzzy name matching. Returns null when there is no result or the LSID for the first result. Where no LSID exist for the record the CB ID is returned instead

Parameters:

name -

fuzzy - look for a fuzzy match

Returns:

Throws:

HomonymException - when an unresolved homonym is detected

SearchResultException

searchForLSID

public java.lang.String searchForLSID(java.lang.String name,
                                      boolean fuzzy,
                                      boolean ignoreHomonyms)
                               throws SearchResultException

Throws:

SearchResultException

searchForLSID

public java.lang.String searchForLSID(java.lang.String name)
                               throws SearchResultException

Searches for the name without using fuzzy name matching...

Parameters:

name - scientific name for a taxon

Throws:

SearchResultException

See Also:

searchForLSID(java.lang.String, boolean)

searchForLSID

public java.lang.String searchForLSID(java.lang.String name,
                                      RankType rank,
                                      boolean fuzzy)
                               throws SearchResultException

Searches the index for the supplied name of the specified rank with or without fuzzy name matching. Returns null when there is no result or the LSID for the first result. Where no LSID exist for the record the CB ID is returned instead When the result is a synonym the "accepted" taxons's LSID is returned.

Parameters:

name -

rank -

fuzzy - look for a fuzzy match

Returns:

Throws:

HomonymException - when an unresolved homonym is detected

SearchResultException

searchForLSID

public java.lang.String searchForLSID(java.lang.String name,
                                      RankType rank,
                                      boolean fuzzy,
                                      boolean ignoreHomonyms)
                               throws SearchResultException

Searches for the supplied name of the specified rank with or without fuzzy name matching. When ignoreHomonyms is true A homonym exception will only be thrown if a homonym is detected where the ALA has both names.

Parameters:

name -

rank -

fuzzy -

ignoreHomonyms -

Returns:

Throws:

SearchResultException

searchForLSID

public java.lang.String searchForLSID(java.lang.String name,
                                      RankType rank)
                               throws SearchResultException

Searches for an LSID of the supplied name and rank without a fuzzy match...

Parameters:

name -

rank -

Returns:

Throws:

SearchResultException

See Also:

searchForLSID(java.lang.String, au.org.ala.data.util.RankType, boolean)

searchForLSID

@Deprecated
public java.lang.String searchForLSID(java.lang.String name,
                                                 java.lang.String kingdom,
                                                 java.lang.String scientificName,
                                                 RankType rank)
                               throws SearchResultException

Deprecated. Use #searchForLSID(java.lang.String, au.org.ala.data.model.LinnaeanRankClassification, au.org.ala.data.util.RankType, boolean) instead. It is more extensible to supply a classification object then a list of higher classification

Searches for the LSID of the supplied name and rank. Using the kingdom to resolve homonym issues.

Parameters:

name -

kingdom -

genus -

rank -

Returns:

Throws:

SearchResultException

searchForLSID

public java.lang.String searchForLSID(java.lang.String name,
                                      LinnaeanRankClassification cl,
                                      RankType rank,
                                      boolean fuzzy,
                                      boolean ignoreHomonym)
                               throws SearchResultException

Search for an LSID based on the supplied name, classification and rank with or without fuzzy name matching. When a classification is supplied it is used for 2 purposes:

1. To try and resolve potential homonyms
2. To provide "optional" components to the search. Thus an incorrect higher classification will not prevent matches from occurring.

If it is not provided and a homonym is detected in the result a HomonymException is thrown.

Parameters:

name -

cl - The high taxa that form the classification for the search item

rank -

fuzzy - look for a fuzzy match

Returns:

Throws:

HomonymException - When an unresolved homonym is detected

SearchResultException

searchForLSID

public java.lang.String searchForLSID(LinnaeanRankClassification cl,
                                      boolean recursiveMatching)
                               throws SearchResultException

Search for an LSID with the supplied classification without a fuzzy match. Supplying to classification in this way allows the API to try and ascertain the rank and the correct scientific name to use.

Parameters:

cl - the classification to work with

Returns:

An LSID for the taxon or null if nothing matched or homonym issues detected

Throws:

SearchResultException

updateClassificationWithGUID

public void updateClassificationWithGUID(LinnaeanRankClassification cl)

Updates the supplied classification so that the supplied ID's are substituted with GUIDs.

Parameters:

cl -

searchForRecord

public NameSearchResult searchForRecord(LinnaeanRankClassification cl,
                                        boolean recursiveMatching)
                                 throws SearchResultException

Throws:

SearchResultException

searchForRecordMetrics

public MetricsResultDTO searchForRecordMetrics(LinnaeanRankClassification cl,
                                               boolean recursiveMatching)
                                        throws SearchResultException

Throws:

SearchResultException

searchForRecordMetrics

public MetricsResultDTO searchForRecordMetrics(LinnaeanRankClassification cl,
                                               boolean recursiveMatching,
                                               boolean fuzzy)
                                        throws SearchResultException

Throws:

SearchResultException

searchForRecord

public NameSearchResult searchForRecord(LinnaeanRankClassification cl,
                                        boolean recursiveMatching,
                                        boolean fuzzy)
                                 throws SearchResultException

Throws:

SearchResultException

searchForRecord

public NameSearchResult searchForRecord(LinnaeanRankClassification cl,
                                        boolean recursiveMatching,
                                        boolean addGuids,
                                        boolean fuzzy)
                                 throws SearchResultException

Search for an LSID with the supplied classification without a fuzzy match. Supplying to classification in this way allows the API to try and ascertain the rank and the correct scientific name to use.

Parameters:

cl - the classification to work with

recursiveMatching - whether to try matching to a higher taxon when leaf taxa matching fails

Returns:

An LSID for the taxon or null if nothing matched or homonym issues detected

Throws:

SearchResultException

searchForRecordMetrics

public MetricsResultDTO searchForRecordMetrics(LinnaeanRankClassification cl,
                                               boolean recursiveMatching,
                                               boolean addGuids,
                                               boolean fuzzy)

Search for a specific name returning extra metrics that can be reported as name match quality...

Parameters:

cl -

recursiveMatching -

addGuids -

fuzzy -

Returns:

searchForRecordMetrics

public MetricsResultDTO searchForRecordMetrics(LinnaeanRankClassification cl,
                                               boolean recursiveMatching,
                                               boolean addGuids,
                                               boolean fuzzy,
                                               boolean ignoreHomonym)

searchForLSID

public java.lang.String searchForLSID(java.lang.String name,
                                      LinnaeanRankClassification cl,
                                      RankType rank)
                               throws SearchResultException

Search for an LSID based on suppled name, classification and rank without a fuzzy match...

Parameters:

name -

cl -

rank -

Returns:

Throws:

SearchResultException

searchForRecord

public NameSearchResult searchForRecord(java.lang.String name,
                                        RankType rank,
                                        boolean fuzzy)
                                 throws SearchResultException

Searches the index for the supplied name of the specified rank. Returns null when there is no result or the result object for the first result.

Parameters:

name -

rank -

fuzzy - look for a fuzzy match

Returns:

Throws:

SearchResultException

searchForRecord

public NameSearchResult searchForRecord(java.lang.String name,
                                        RankType rank)
                                 throws SearchResultException

Searches index for the supplied name and rank without a fuzzy match.

Parameters:

name -

rank -

Returns:

Throws:

SearchResultException

searchForAcceptedLsidDefaultHandling

public java.lang.String searchForAcceptedLsidDefaultHandling(LinnaeanRankClassification cl,
                                                             boolean fuzzy)

Returns the accepted LSID for the supplied classification. If a synonym is matched the accepted LSID is retrieved and returned. It uses the default error handling. For example matches to excluded concepts are permitted. Do not use this method if you would like control over how error conditions are handled.

Parameters:

cl -

fuzzy -

Returns:

searchForAcceptedLsidDefaultHandling

public java.lang.String searchForAcceptedLsidDefaultHandling(LinnaeanRankClassification cl,
                                                             boolean fuzzy,
                                                             boolean ignoreHomonyms)

searchForAcceptedRecordDefaultHandling

public NameSearchResult searchForAcceptedRecordDefaultHandling(LinnaeanRankClassification cl,
                                                               boolean fuzzy)

Returns the accepted result for the supplied classification. If a synonym is matched the accepted result is retrieved and returned. It uses the default error handling. For example matches to excluded concepts are permitted. Do not use this method if you would like control over how error conditions are handled.

Parameters:

cl -

fuzzy -

Returns:

searchForAcceptedRecordDefaultHandling

public NameSearchResult searchForAcceptedRecordDefaultHandling(LinnaeanRankClassification cl,
                                                               boolean fuzzy,
                                                               boolean ignoreHomonym)

searchForRecord

@Deprecated
public NameSearchResult searchForRecord(java.lang.String name,
                                                   java.lang.String kingdom,
                                                   java.lang.String genus,
                                                   RankType rank)
                                 throws SearchResultException

Deprecated. Use searchForRecord(java.lang.String, au.org.ala.data.model.LinnaeanRankClassification, au.org.ala.data.util.RankType, boolean) instead. It is more extensible to supply a classification object then a list of higher classification

Searches for a record based on the supplied name and rank. It uses the kingdom and genus to resolve homonyms.

Parameters:

name -

kingdom -

genus -

rank -

Returns:

Throws:

SearchResultException

searchForRecord

public NameSearchResult searchForRecord(java.lang.String name,
                                        LinnaeanRankClassification cl,
                                        RankType rank,
                                        boolean fuzzy)
                                 throws SearchResultException

Throws:

SearchResultException

searchForRecord

public NameSearchResult searchForRecord(java.lang.String name,
                                        LinnaeanRankClassification cl,
                                        RankType rank,
                                        boolean fuzzy,
                                        boolean ignoreHomonyms)
                                 throws SearchResultException

Searches for a record based on the supplied name, rank and classification with or without fuzzy name matching.

Parameters:

name -

cl -

rank -

fuzzy -

Returns:

Throws:

SearchResultException

searchForRecord

public NameSearchResult searchForRecord(java.lang.String name,
                                        LinnaeanRankClassification cl,
                                        RankType rank)
                                 throws SearchResultException

Searches for a record based on the supplied name, classification and rank without fuzzy name matching

Parameters:

name -

cl -

rank -

Returns:

Throws:

SearchResultException

searchForRecordByID

public NameSearchResult searchForRecordByID(java.lang.String id)

Returns the records that has the supplied checklist bank id

Parameters:

id -

Returns:

searchForLsidById

public java.lang.String searchForLsidById(java.lang.String id)

Gets the LSID for the record that has the supplied checklist bank id.

Parameters:

id -

Returns:

searchForRecords

public java.util.List<NameSearchResult> searchForRecords(java.lang.String name,
                                                         RankType rank,
                                                         boolean fuzzy)
                                                  throws SearchResultException

Searches for records with the specified name and rank with or without fuzzy name matching

Parameters:

name -

rank -

fuzzy - search for a fuzzy match

Returns:

Throws:

SearchResultException

searchForRecords

public java.util.List<NameSearchResult> searchForRecords(java.lang.String name,
                                                         RankType rank,
                                                         LinnaeanRankClassification cl,
                                                         int max)
                                                  throws SearchResultException

Searches for a list of results for the supplied name, classification and rank without fuzzy match

Parameters:

name -

rank -

cl -

max -

Returns:

Throws:

SearchResultException

searchForRecords

public java.util.List<NameSearchResult> searchForRecords(java.lang.String name,
                                                         RankType rank,
                                                         LinnaeanRankClassification cl,
                                                         int max,
                                                         boolean fuzzy)
                                                  throws SearchResultException

Searches for the records that satisfy the given conditions using the algorithm outlined in the class description.

Parameters:

name -

rank -

kingdom -

genus -

max - The maximum number of results to return

fuzzy - search for a fuzzy match

Returns:

Throws:

SearchResultException

searchForRecords

public java.util.List<NameSearchResult> searchForRecords(java.lang.String name,
                                                         RankType rank,
                                                         LinnaeanRankClassification cl,
                                                         int max,
                                                         boolean fuzzy,
                                                         boolean ignoreHomonyms)
                                                  throws SearchResultException

Throws:

SearchResultException

validateHomonymByAuthor

public NameSearchResult validateHomonymByAuthor(java.util.List<NameSearchResult> result,
                                                java.lang.String name,
                                                LinnaeanRankClassification cl)
                                         throws HomonymException

Throws:

HomonymException

validateHomonyms

public NameSearchResult validateHomonyms(java.util.List<NameSearchResult> results,
                                         java.lang.String name,
                                         LinnaeanRankClassification cl)
                                  throws HomonymException

Takes a result set that contains a homonym and then either throws a HomonymException or returns the first result that matches the supplied taxa. AS OF 22/07/2010: Homonyms are ONLY being tested if the result was a genus. According to Tony it is very rare for a species to be a homonym with another species that belongs to a homonym of the same genus. Eventually we should get a list of the known cases to test against. This should provide overall better name matching. 2011-01-14: The homonym validation has been modified to include species level homonyms. The indexing of the irmng species is different to the genus. IRMNG has a more complete genus coverage than species. Thus only the species that are homonyms are included in the index.

Parameters:

results -

k -

Returns:

Throws:

HomonymException

getIRMNGGenus

public org.apache.lucene.search.TopDocs getIRMNGGenus(LinnaeanRankClassification cl,
                                                      RankType rank)

Multiple genus indicate that an unresolved homonym exists for the supplied search details.

Parameters:

cl - The classification to test

rank - The rank level of the homonym being tested either RankType.GENUS or RankType.SPECIES

resolveIRMNGHomonym

public RankType resolveIRMNGHomonym(LinnaeanRankClassification cl,
                                    RankType rank)
                             throws HomonymException

Attempt to resolve the homonym using the IRMNG index. The ability to resolve the homonym is dependent on the quality and quantity of the higher taxa provided in the search via cl.

Parameters:

cl - The classification used to determine the rank at which the homonym is resolvable

Returns:

Throws:

HomonymException

searchForLSIDCommonName

public java.lang.String searchForLSIDCommonName(java.lang.String commonName)

Performs a search on the common name index for the supplied name.

Parameters:

commonName -

Returns:

searchForCommonName

public NameSearchResult searchForCommonName(java.lang.String name)

Performs a search on the supplied common name returning a NameSearchResult. Useful if you required CB ID's etc.

Parameters:

name -

Returns:

getPrimaryLsid

public java.lang.String getPrimaryLsid(java.lang.String lsid)

Returns the primary LSID for the supplied lsid. This is useful in the situation where multiple LSIDs are associated with a scientific name and there is a reference to the non-primary LSID.

Parameters:

lsid -

Returns:

searchForRecordByLsid

public NameSearchResult searchForRecordByLsid(java.lang.String lsid)