org.apache.tika.parser.external

Class ExternalParser

java.lang.Object

org.apache.tika.parser.AbstractParser

org.apache.tika.parser.external.ExternalParser

All Implemented Interfaces:

java.io.Serializable, Parser

public class ExternalParser
extends AbstractParser

Parser that uses an external program (like catdoc or pdf2txt) to extract text content and metadata from a given document.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	ExternalParser.LineConsumer Consumer contract

Field Summary

Fields

Modifier and Type	Field and Description
private java.lang.String[]	command The external command to invoke.
private ExternalParser.LineConsumer	ignoredLineConsumer A consumer for ignored Lines
static java.lang.String	INPUT_FILE_TOKEN The token, which if present in the Command string, will be replaced with the input filename.
private java.util.Map<java.util.regex.Pattern,java.lang.String>	metadataPatterns Regular Expressions to run over STDOUT to extract Metadata.
static java.lang.String	OUTPUT_FILE_TOKEN The token, which if present in the Command string, will be replaced with the output filename.
private static long	serialVersionUID
private java.util.Set<MediaType>	supportedTypes Media types supported by the external program.

Constructor Summary

Constructors

Constructor and Description
ExternalParser()

Method Summary

Modifier and Type	Method and Description
static boolean	check(java.lang.String[] checkCmd, int... errorValue)
static boolean	check(java.lang.String checkCmd, int... errorValue) Checks to see if the command can be run.
private void	extractMetadata(java.io.InputStream stream, Metadata metadata)
private void	extractOutput(java.io.InputStream stream, XHTMLContentHandler xhtml) Starts a thread that extracts the contents of the standard output stream of the given process to the given XHTML content handler.
java.lang.String[]	getCommand()
ExternalParser.LineConsumer	getIgnoredLineConsumer() Gets lines consumer
java.util.Map<java.util.regex.Pattern,java.lang.String>	getMetadataExtractionPatterns()
java.util.Set<MediaType>	getSupportedTypes()
java.util.Set<MediaType>	getSupportedTypes(ParseContext context) Returns the set of media types supported by this parser when used with the given parse context.
private static void	ignoreStream(java.io.InputStream stream) Starts a thread that reads and discards the contents of the standard stream of the given process.
private static java.lang.Thread	ignoreStream(java.io.InputStream stream, boolean waitForDeath) Starts a thread that reads and discards the contents of the standard stream of the given process.
void	parse(java.io.InputStream stream, org.xml.sax.ContentHandler handler, Metadata metadata, ParseContext context) Executes the configured external command and passes the given document stream as a simple XHTML document to the given SAX content handler.
private void	parse(TikaInputStream stream, XHTMLContentHandler xhtml, Metadata metadata, TemporaryResources tmp)
private void	sendInput(java.lang.Process process, java.io.InputStream stream) Starts a thread that sends the contents of the given input stream to the standard input stream of the given process.
void	setCommand(java.lang.String... command) Sets the command to be run.
void	setIgnoredLineConsumer(ExternalParser.LineConsumer ignoredLineConsumer) Set a consumer for the lines ignored by the parse functions
void	setMetadataExtractionPatterns(java.util.Map<java.util.regex.Pattern,java.lang.String> patterns) Sets the map of regular expression patterns and Metadata keys.
void	setSupportedTypes(java.util.Set<MediaType> supportedTypes)

Methods inherited from class org.apache.tika.parser.AbstractParser

parse

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

serialVersionUID

private static final long serialVersionUID

See Also:

Constant Field Values

INPUT_FILE_TOKEN

public static final java.lang.String INPUT_FILE_TOKEN

The token, which if present in the Command string, will be replaced with the input filename. Alternately, the input data can be streamed over STDIN.

See Also:

Constant Field Values

OUTPUT_FILE_TOKEN

public static final java.lang.String OUTPUT_FILE_TOKEN

The token, which if present in the Command string, will be replaced with the output filename. Alternately, the output data can be collected on STDOUT.

See Also:

Constant Field Values

supportedTypes

private java.util.Set<MediaType> supportedTypes

Media types supported by the external program.

metadataPatterns

private java.util.Map<java.util.regex.Pattern,java.lang.String> metadataPatterns

Regular Expressions to run over STDOUT to extract Metadata.

command

private java.lang.String[] command

The external command to invoke.

See Also:

Runtime.exec(String[])

ignoredLineConsumer

private ExternalParser.LineConsumer ignoredLineConsumer

A consumer for ignored Lines

Constructor Detail

ExternalParser

public ExternalParser()

Method Detail

getSupportedTypes

public java.util.Set<MediaType> getSupportedTypes(ParseContext context)

Description copied from interface: Parser

Returns the set of media types supported by this parser when used with the given parse context.

Parameters:

context - parse context

Returns:

immutable set of media types

getSupportedTypes

public java.util.Set<MediaType> getSupportedTypes()

setSupportedTypes

public void setSupportedTypes(java.util.Set<MediaType> supportedTypes)

getCommand

public java.lang.String[] getCommand()

setCommand

public void setCommand(java.lang.String... command)

Sets the command to be run. This can include either of INPUT_FILE_TOKEN or OUTPUT_FILE_TOKEN if the command needs filenames.

See Also:

Runtime.exec(String[])

getIgnoredLineConsumer

public ExternalParser.LineConsumer getIgnoredLineConsumer()

Gets lines consumer

Returns:

consumer instance

setIgnoredLineConsumer

public void setIgnoredLineConsumer(ExternalParser.LineConsumer ignoredLineConsumer)

Set a consumer for the lines ignored by the parse functions

Parameters:

ignoredLineConsumer - consumer instance

getMetadataExtractionPatterns

public java.util.Map<java.util.regex.Pattern,java.lang.String> getMetadataExtractionPatterns()

setMetadataExtractionPatterns

public void setMetadataExtractionPatterns(java.util.Map<java.util.regex.Pattern,java.lang.String> patterns)

Sets the map of regular expression patterns and Metadata keys. Any matching patterns will have the matching metadata entries set. Set this to null to disable Metadata extraction.

parse

public void parse(java.io.InputStream stream,
                  org.xml.sax.ContentHandler handler,
                  Metadata metadata,
                  ParseContext context)
           throws java.io.IOException,
                  org.xml.sax.SAXException,
                  TikaException

Executes the configured external command and passes the given document stream as a simple XHTML document to the given SAX content handler. Metadata is only extracted if setMetadataExtractionPatterns(Map) has been called to set patterns.

Parameters:

stream - the document stream (input)

handler - handler for the XHTML SAX events (output)

metadata - document metadata (input and output)

context - parse context

Throws:

java.io.IOException - if the document stream could not be read

org.xml.sax.SAXException - if the SAX events could not be processed

TikaException - if the document could not be parsed

parse

private void parse(TikaInputStream stream,
                   XHTMLContentHandler xhtml,
                   Metadata metadata,
                   TemporaryResources tmp)
            throws java.io.IOException,
                   org.xml.sax.SAXException,
                   TikaException

Throws:

java.io.IOException

org.xml.sax.SAXException

TikaException

extractOutput

private void extractOutput(java.io.InputStream stream,
                           XHTMLContentHandler xhtml)
                    throws org.xml.sax.SAXException,
                           java.io.IOException

Starts a thread that extracts the contents of the standard output stream of the given process to the given XHTML content handler. The standard output stream is closed once fully processed.

Parameters:

process - process

xhtml - XHTML content handler

Throws:

org.xml.sax.SAXException - if the XHTML SAX events could not be handled

java.io.IOException - if an input error occurred

sendInput

private void sendInput(java.lang.Process process,
                       java.io.InputStream stream)

Starts a thread that sends the contents of the given input stream to the standard input stream of the given process. Potential exceptions are ignored, and the standard input stream is closed once fully processed. Note that the given input stream is not closed by this method.

Parameters:

process - process

stream - input stream

ignoreStream

private static void ignoreStream(java.io.InputStream stream)

Starts a thread that reads and discards the contents of the standard stream of the given process. Potential exceptions are ignored, and the stream is closed once fully processed. Note: calling this starts a new thread and blocks the current(caller) thread until the new thread dies

Parameters:

stream - stream to be ignored

ignoreStream

private static java.lang.Thread ignoreStream(java.io.InputStream stream,
                                             boolean waitForDeath)

Starts a thread that reads and discards the contents of the standard stream of the given process. Potential exceptions are ignored, and the stream is closed once fully processed.

Parameters:

stream - stream to sent to black hole (a k a null)

waitForDeath - when true the caller thread will be blocked till the death of new thread.

Returns:

The thread that is created and started

extractMetadata

private void extractMetadata(java.io.InputStream stream,
                             Metadata metadata)

check

public static boolean check(java.lang.String checkCmd,
                            int... errorValue)

Checks to see if the command can be run. Typically used with something like "myapp --version" to check to see if "myapp" is installed and on the path.

Parameters:

checkCmd - The check command to run

errorValue - What is considered an error value?

check

public static boolean check(java.lang.String[] checkCmd,
                            int... errorValue)