Skip to content

LLM

tokenizer

  • split text into tokens
  • depends on similarity and frequency of tokens

Evaluation

- Sensitivity to prompting/inconsistencies