org.apache.lucene.analysis.cn.smart

Class WordSegmenter

java.lang.Object

org.apache.lucene.analysis.cn.smart.WordSegmenter

class WordSegmenter
extends java.lang.Object

Segment a sentence of Chinese text into words.

Field Summary

Fields

Modifier and Type	Field and Description
private HHMMSegmenter	hhmmSegmenter
private SegTokenFilter	tokenFilter

Constructor Summary

Constructors

Constructor and Description
WordSegmenter()

Method Summary

Modifier and Type	Method and Description
SegToken	convertSegToken(SegToken st, java.lang.String sentence, int sentenceStartOffset) Process a SegToken so that it is ready for indexing.
java.util.List<SegToken>	segmentSentence(java.lang.String sentence, int startOffset) Segment a sentence into words with HHMMSegmenter

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

hhmmSegmenter

private HHMMSegmenter hhmmSegmenter

tokenFilter

private SegTokenFilter tokenFilter

Constructor Detail

WordSegmenter

WordSegmenter()

Method Detail

segmentSentence

public java.util.List<SegToken> segmentSentence(java.lang.String sentence,
                                                int startOffset)

Segment a sentence into words with HHMMSegmenter

Parameters:

sentence - input sentence

startOffset - start offset of sentence

Returns:

List of SegToken

convertSegToken

public SegToken convertSegToken(SegToken st,
                                java.lang.String sentence,
                                int sentenceStartOffset)

Process a SegToken so that it is ready for indexing. This method calculates offsets and normalizes the token with SegTokenFilter.

Parameters:

st - input SegToken

sentence - associated Sentence

sentenceStartOffset - offset into sentence

Returns:

Lucene SegToken