Helping task for natural language processing purposes. We used lemmatization in several projects, first time it was required because of significantly larger words forms quantity in Russian, standard svm and nb classifiers failed, so we needed to reduce dimensionality, lemmatization and stemming helped here. We developed several algorithms and data structures for an optimized memory/cpu lemmatization (and partially stemming).
Special model format to fit mobile device memory requirements
Special model format for improving speed for web-service
Fast search of lemma O(log N); even O(1) (counting average word length as a constant)
Retrieving an original word form (lemma) in two steps: first step is to request the model, if it fails to find a lemma, use Porter Stemmer.