Class TypeTokenFilter

  • All Implemented Interfaces:
    java.io.Closeable, java.lang.AutoCloseable

    public final class TypeTokenFilter
    extends FilteringTokenFilter
    Removes tokens whose types appear in a set of blocked types from a token stream.
    • Field Detail

      • stopTypes

        private final java.util.Set<java.lang.String> stopTypes
      • useWhiteList

        private final boolean useWhiteList
    • Constructor Detail

      • TypeTokenFilter

        public TypeTokenFilter​(TokenStream input,
                               java.util.Set<java.lang.String> stopTypes,
                               boolean useWhiteList)
        Create a new TypeTokenFilter.
        Parameters:
        input - the TokenStream to consume
        stopTypes - the types to filter
        useWhiteList - if true, then tokens whose type is in stopTypes will be kept, otherwise they will be filtered out
      • TypeTokenFilter

        public TypeTokenFilter​(TokenStream input,
                               java.util.Set<java.lang.String> stopTypes)
        Create a new TypeTokenFilter that filters tokens out (useWhiteList=false).
    • Method Detail

      • accept

        protected boolean accept()
        By default accept the token if its type is not a stop type. When the useWhiteList parameter is set to true then accept the token if its type is contained in the stopTypes
        Specified by:
        accept in class FilteringTokenFilter