YuStemmer is a natural language stemming library. Its purpose is to reduce an inflected word to a common stem or root form. The English stemmer, for example, returns "write" for "write", "writes", "writing", and "writings". Stemmers are available for these languages:

• Danish, Dutch, English, Finnish, French, German, German2 (different Umlaut handling), Hungarian, Italian, Norwegian, Porter (English), Portuguese, Romanian, Russian, Spanish, Swedish, Turkish.

Applications for stemmers are usually query and search systems. Stemming enables them to return related results with similar meaning but slightly different spelling. YuStemmer was initially developed for the DISQLite3 Full Text Search (FTS) engine, but fits other purposes we well.

YuStemmer is fully algorithmic. No extensive lookup dictionaries are needed. This results in small memory footprint and leads to excellent performance.

YuStemmer is organized into different classes, each of them optimized for a particular string type and text encoding:

• TYuStemmer class:
  • ANSI text, 8-bit, usually in ISO-8859-1, unless otherwise noted.
  • Some stemmers might not be available for this class.
• TYuStemmer_8 class:
  • UTF-8 text, 8-bit.
  • All stemmers are available for this class.
• TYuStemmer_8 class:
  • UTF-16 text, 16-bit. This corresponds to Delphi's WideString / UnicodeString.
  • All stemmers are available for this class.

Make sure to choose the stemmer class matching your string type and character set. Otherwise you will suffer a performance penalty caused by avoidable string conversions. In Delphi, such conversions usually happen implicitly and go unnoticed by most developers. Therefore, pay close attention here to make the most of YuStemmer!

The Stem() method does the work for all of the above classes. It expects a single word and returns its stem. If there is no stem, the original word is returned unchanged.

function StemFrench(const AWord: AnsiString): AnsiString;
var
  Stemmer: TYuStemmer;
begin
  Stemmer := TYuStemmer_French.Create;
  Result := Stemmer.Stem(AWord);
  Stemmer.Free;
end;

To improve performance when stemming a great number of words, it is safe to reuse the same instance of a stemmer class multiple times.

procedure StemItalian(const AWords: TStringList);
var
  i: Integer;
  { TStringList is UTF-16 in Unicode Delphis. }
  Stemmer: {$IFDEF Unicode}TYuStemmer_16{$ELSE}TYuStemmer{$ENDIF};
begin
  {$IFDEF Unicode}
  Stemmer := TYuStemmer_Italian_16.Create;
  {$ELSE Unicode}
  Stemmer := TYuStemmer_Italian.Create;
  {$ENDIF Unicode}
  for i := 0 to AWords.Count - 1 do
    AWords[i] := Stemmer.Stem(AWords[i]);
  Stemmer.Free;
end;