Skip to main content

๐Ÿ”ฅ Released OpenThaiGPT 7b <1.0.0-beta> (16/08/23)

๐Ÿ‡น๐Ÿ‡ญ OpenThaiGPT 1.0.0-beta (16 August 2023)

๐Ÿ‡น๐Ÿ‡ญ OpenThaiGPT Version 1.0.0-beta is a Thai language 7B-parameter LLaMA v2 Chat model finetuned to follow Thai translated instructions and extend more than 24,554 most popular Thai words vocabularies into LLM's dictionary for turbo speed.

Web Demoโ€‹

OpenThaiGPT 1.0.0-beta Web Demodemo-beta.openthaigpt.aieat.or.th

Colab Demoโ€‹

OpenThaiGPT 7b 1.0.0-beta โ€” Open in Google Colabcolab.research.google.com

Change Logsโ€‹

๐Ÿ‡น๐Ÿ‡ญ Version 1.0.0-beta (Llama v2 + 24,554 Thai word extension)โ€‹

Release date: 16 August 2023

๐Ÿ‡น๐Ÿ‡ญ OpenThaiGPT Version 1.0.0-beta is a Thai language 7B-parameter LLaMA v2 Chat model finetuned to follow Thai translated instructions and extend 24,554 Thai words vocabularies for turbo speed.

Licenseโ€‹

  • Source Code: License Apache Software License 2.0.
  • Weight: Research and Commercial uses.

Code and Weightโ€‹

Authorsโ€‹

๐Ÿ‡น๐Ÿ‡ญ Version 1.0.0-alpha (Facebook LLama V2 Model)โ€‹

Release date: 3 August 2023

๐Ÿ‡น๐Ÿ‡ญ OpenThaiGPT Version 1.0.0-alpha is the first Thai implementation of a 7B-parameter LLaMA v2 Chat model finetuned to follow Thai translated instructions and makes use of the Huggingface LLaMA implementation.

Changesโ€‹

  1. Using Facebook LLama v2 model 7b chat as a base model which is pretrained on over 2 trillion token.
  2. Context Length is upgraded from 2048 token to 4096 token
  3. Allow research and commercial use.

Licenseโ€‹

  • Source Code: License Apache Software License 2.0.
  • Weight: Research and commercial uses.

Code and Weightโ€‹

Authorsโ€‹

Version 0.1.0-beta (Facebook LLama Model)โ€‹

Release date: 16 May 2023

OpenThaiGPT Version 0.1.0-beta is a 7B-parameter LLaMA model finetuned to follow Thai translated instructions below and makes use of the Huggingface LLaMA implementation.

Statisticsโ€‹

  • Number of parameters: 7B
  • Dimension: 4096
  • Context Length: 2048
  • n heads: 32
  • n layers: 32
  • n tokens: 1T

Licenseโ€‹

  • Source Code: License Apache Software License 2.0.
  • Weight: For research use only (due to the Facebook LLama's Weight LICENSE).

Note that: A commercial use license for OpenThaiGPT 0.1.0 weight will be released later soon!

Code and Weightโ€‹

Authorsโ€‹

Kobkrit Viriyayudhakorn (kobkrit@aieat.or.th), Sumeth Yuenyong (sumeth.yue@mahidol.edu) and Thaweewat Rugsujarit (thaweewr@scg.com).

Trained Datasetsโ€‹

Dataset NameInstruction PairsDescriptions
Thaweewat/alpaca-finance-43k-th43,000Alpaca Finance Instruction translated into Thai by Thaweewat Ruksujarit.
kobkrit/rd-taxqa600RD's Tax QA Chatbot Training set by เธ—เธฃเธ‡เธงเธธเธ’เธด เธšเธธเธฃเธ‡เธ„เนŒ
datasets/iapp_wiki_qa_squad4,000iApp Technology's Extractive QA Dataset in Thai language
Thaweewat/databricks-dolly-15k-th15,000Databrick's Dolly Instruction translated into Thai by Thaweewat Ruksujarit.
Thaweewat/instruction-wild-52k-th52,000Instruction Wild's translated into Thai by Thaweewat Ruksujarit.
Thaweewat/alpaca-cleaned-52k-th51,000Standford Alpaca's translated into Thai by Thaweewat Ruksujarit.
Thaweewat/gpteacher-20k-th20,000GPT Teacher's Instruction translated into Thai by Thaweewat Ruksujarit.
Thaweewat/onet-m6-social600ONET m6 Social Exam
datasets/Thaweewat/hc3-24k-th24,000Hello Simple AI Summary Dataset translated into Thai by Thaweewat Ruksujarit.
OpenThaiGPT Self Instruct (Google Sheets)5,000

Thai SelfInstruct Dataset
(Automatic Generated) by OpenThaiGPT

Version 0.1.0-alpha (ByT5-XL Model)โ€‹

Release date: 24 April 2023

OpenThaiGPT version 0.1.0-alpha

Thai First 3 billion params models

  • First Thai Byte-Level Text-to-Text Transfer Transformer
  • Support Instruction following
    • Translation to Thai
    • Explanation
    • Paraphrase
  • Zero-shot and Few-shot Learning
  • Pretraining Model: ByT5-XL (3.74 billion params)
  • InstructDataset: 50,000 Thai SelfInstruct
  • RLHF: None
  • Developer: Sumeth Yuenyong, Kobkrit Viriyayudhakorn (kobkrit@iapp.co.th)

PoC Version 0.0.4 (The Fourth PoC Version)โ€‹

Release date: 12 March 2023

OpenThaiGPT version 0.0.4

The Fourth PoC Model

  • Answers questions in greater detail and, for the most part, responds better than 0.0.3
  • Pretraining Model: GPT-2 Thai-base
  • InstructDataset: 300,000 Pantip + 5,000 Wiki QA => 12,920 Thai InstructGPT
  • RLHF: None
  • Developer: Kobkrit Viriyayudhakorn (kobkrit@iapp.co.th)

PoC Version 0.0.3 (The Third PoC Version)โ€‹

Release date: 28 February 2023

OpenThaiGPT version 0.0.3

The Third PoC Model

  • Pretraining Model: GPT-2 Thai-base
  • InstructDataset: 300,000 Pantip + 5,000 Wiki QA => 7,000 Thai InstructGPT
  • RLHF: None
  • Developer: Kobkrit Viriyayudhakorn (kobkrit@iapp.co.th)

PoC Version 0.0.2 (The Second PoC Version)โ€‹

Release date: 27 February 2023

OpenThaiGPT version 0.0.2

The Second PoC Model

  • Pretraining Model: GPT-2 Thai-base
  • InstructDataset: 7,000 Thai InstructGPT
  • RLHF: None
  • Developer: Kobkrit Viriyayudhakorn (kobkrit@iapp.co.th)

PoC Version 0.0.1 (Very First PoC Version)โ€‹

Release date: 20 February 2023

The Very First PoC Model

  • Pretraining Model: GPT-2 Thai-base
  • InstructDataset: 298,678 QA Pairs getting from 70,000 Pantip katoos + Wikipedia QA by iApp
  • RLHF: None
  • Developer: Kobkrit Viriyayudhakorn (kobkrit@iapp.co.th)