(Information Science Expert)
- Using the decoder of the Transformer model.
- Developed by Open AI, the Language Model (v3 of GPT) is extremely large.
- It utilizes the Common Crawl corpus, books, and Wikipedia, among others.
- With such a massive corpus, it can prevent the repetition of the same sentences, allowing for training without memorizing them.
- The Language Model can be applied to various other tasks (how?).
- Types of tasks include:
- News article generation
- Sentence completion
- Machine translation (!?) (it can achieve high-quality translation using only a small portion (7%) of non-English information in the corpus)
- Weakness: Agreement recognition (determining if A can be inferred from B), as it lacks training in comparing two sentences.
- #naturallanguageprocessing