Package org.apache.lucene.demo.knn

Class KnnVectorDictFilter

java.lang.Object
org.apache.lucene.util.AttributeSource
org.apache.lucene.analysis.TokenStream
org.apache.lucene.analysis.TokenFilter
org.apache.lucene.demo.knn.KnnVectorDictFilter
All Implemented Interfaces:
Closeable, AutoCloseable, org.apache.lucene.util.Unwrappable<org.apache.lucene.analysis.TokenStream>
public final class KnnVectorDictFilter extends org.apache.lucene.analysis.TokenFilter
Looks up each tokens in a dictionary, and sums the token vectors. Unrecognized tokens are ignored. The resulting vector is normalized to unit length.

  • Nested Class Summary

    Nested classes/interfaces inherited from class org.apache.lucene.util.AttributeSource

    org.apache.lucene.util.AttributeSource.State

  • Field Summary

    Fields inherited from class org.apache.lucene.analysis.TokenFilter

    input

    Fields inherited from class org.apache.lucene.analysis.TokenStream

    DEFAULT_TOKEN_ATTRIBUTE_FACTORY

  • Constructor Summary

    Constructors
    Constructor
    Description
    KnnVectorDictFilter(org.apache.lucene.analysis.TokenStream input, KnnVectorDict dict)
    sole constructor

  • Method Summary

    Modifier and Type
    Method
    Description
    void
    end()
     
    float[]
    getResult()
    Get the vector computed from the input
    boolean
    incrementToken()
     
    void
    reset()
     

    Methods inherited from class org.apache.lucene.analysis.TokenFilter

    close, unwrap

    Methods inherited from class org.apache.lucene.util.AttributeSource

    addAttribute, addAttributeImpl, captureState, clearAttributes, cloneAttributes, copyTo, endAttributes, equals, getAttribute, getAttributeClassesIterator, getAttributeFactory, getAttributeImplsIterator, hasAttribute, hasAttributes, hashCode, reflectAsString, reflectWith, removeAllAttributes, restoreState, toString

    Methods inherited from class java.lang.Object

    clone, finalize, getClass, notify, notifyAll, wait, wait, wait

  • Constructor Details

    • KnnVectorDictFilter

      public KnnVectorDictFilter(org.apache.lucene.analysis.TokenStream input, KnnVectorDict dict)
      sole constructor
      Parameters:
      input - the input token stream to filter.
      dict - a token to vector dictionary, used to look up the token vectors.

  • Method Details

    • incrementToken

      public boolean incrementToken() throws IOException
      Specified by:
      incrementToken in class org.apache.lucene.analysis.TokenStream
      Throws:
      IOException

    • reset

      public void reset() throws IOException
      Overrides:
      reset in class org.apache.lucene.analysis.TokenFilter
      Throws:
      IOException

    • end

      public void end() throws IOException
      Overrides:
      end in class org.apache.lucene.analysis.TokenFilter
      Throws:
      IOException

    • getResult

      public float[] getResult()
      Get the vector computed from the input
      Returns:
      the resultant sum of the vectors of each term.