org.apache.tika.sax

Class StandardsText

java.lang.Object

org.apache.tika.sax.StandardsText

public class StandardsText
extends java.lang.Object

StandardText relies on regular expressions to extract standard references from text.

This class helps to find the standard references from text by performing the following steps:

1. searches for headers;
2. searches for patterns that are supposed to be standard references (basically, every string mostly composed of uppercase letters followed by an alphanumeric characters);
3. each potential standard reference starts with score equal to 0.25;
4. increases by 0.25 the score of references which include the name of a known standard organization (StandardOrganizations);
5. increases by 0.25 the score of references which include the word Publication or Standard;
6. increases by 0.25 the score of references which have been found within "Applicable Documents" and equivalent sections;
7. returns the standard references along with scores.

Field Summary

Fields

Modifier and Type	Field and Description
private static java.lang.String	REGEX_APPLICABLE_DOCUMENTS
private static java.lang.String	REGEX_FALLBACK
private static java.lang.String	REGEX_HEADER
private static java.lang.String	REGEX_IDENTIFIER
private static java.lang.String	REGEX_ORGANIZATION
private static java.lang.String	REGEX_STANDARD
private static java.lang.String	REGEX_STANDARD_TYPE

Constructor Summary

Constructors

Constructor and Description
StandardsText()

Method Summary

Modifier and Type	Method and Description
static java.util.ArrayList<StandardReference>	extractStandardReferences(java.lang.String text, double threshold) Extracts the standard references found within the given text.
private static java.util.Map<java.lang.Integer,java.lang.String>	findHeaders(java.lang.String text) This method helps to find the headers within the given text.
private static java.util.ArrayList<StandardReference>	findStandards(java.lang.String text, java.util.Map<java.lang.Integer,java.lang.String> headers, double threshold) This method helps to find the standard references within the given text.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

REGEX_HEADER

private static final java.lang.String REGEX_HEADER

See Also:

Constant Field Values

REGEX_APPLICABLE_DOCUMENTS

private static final java.lang.String REGEX_APPLICABLE_DOCUMENTS

See Also:

Constant Field Values

REGEX_IDENTIFIER

private static final java.lang.String REGEX_IDENTIFIER

See Also:

Constant Field Values

REGEX_ORGANIZATION

private static final java.lang.String REGEX_ORGANIZATION

REGEX_STANDARD_TYPE

private static final java.lang.String REGEX_STANDARD_TYPE

See Also:

Constant Field Values

REGEX_FALLBACK

private static final java.lang.String REGEX_FALLBACK

See Also:

Constant Field Values

REGEX_STANDARD

private static final java.lang.String REGEX_STANDARD

Constructor Detail

StandardsText

public StandardsText()

Method Detail

extractStandardReferences

public static java.util.ArrayList<StandardReference> extractStandardReferences(java.lang.String text,
                                                                               double threshold)

Extracts the standard references found within the given text.

Parameters:

text - the text from which the standard references are extracted.

threshold - the lower bound limit to be used in order to select only the standard references with score greater than or equal to the threshold. For instance, using a threshold of 0.75 means that only the patterns with score greater than or equal to 0.75 will be returned.

Returns:

the list of standard references extracted from the given text.

findHeaders

private static java.util.Map<java.lang.Integer,java.lang.String> findHeaders(java.lang.String text)

This method helps to find the headers within the given text.

Parameters:

text - the text from which the headers are extracted.

Returns:

the list of headers found within the given text.

findStandards

private static java.util.ArrayList<StandardReference> findStandards(java.lang.String text,
                                                                    java.util.Map<java.lang.Integer,java.lang.String> headers,
                                                                    double threshold)

This method helps to find the standard references within the given text.

Parameters:

text - the text from which the standards references are extracted.

headers - the list of headers found within the given text.

threshold - the lower bound limit to be used in order to select only the standard references with score greater than or equal to the threshold.

Returns:

the list of standard references extracted from the given text.