Ignore:
Timestamp:
Sep 11, 2009, 12:35:21 PM (12 years ago)
Author:
Matthäus Wander
Message:

Tokenizer:

  • set tokenizer constructors to private
  • added public static tokenize(...) methods instead
  • added tokenize(FileInfo) implementation based on yield return statement
  • added KeyValueReader which reads text files with "key = value" format
File:
1 edited

Legend:

Unmodified
Added
Removed
  • trunk/CrypPlugins/WordPatterns/WordPatterns.cs

    r438 r510  
    132132                dictPatterns = new Dictionary<Pattern, IList<string>>();
    133133                int wordCount = 0;
    134                 WordTokenEnum enumerator = new WordTokenizer(inputDict).GetSpecialEnumerator();
     134                WordEnumerator enumerator = WordTokenizer.tokenize(inputDict).GetCustomEnumerator();
    135135                while(enumerator.MoveNext() && !stop) // has next
    136136                {
Note: See TracChangeset for help on using the changeset viewer.