Skip to main content

🔥 Released Models Version <0.1.0-beta> (16/05/23)

Version 0.1.0-beta (16 May 2023)​

Demo​

ทดลอง OpenThaiGPT 0.1.0-beta — Google Colabcolab.research.google.com

Change Logs​

Version 0.1.0-beta (Facebook LLama Model)​

Release date: 16 May 2023

OpenThaiGPT Version 0.1.0-beta is a 7B-parameter LLaMA model finetuned to follow Thai translated instructions below and makes use of the Huggingface LLaMA implementation.

Statistics​

  • Number of parameters: 7B
  • Dimension: 4096
  • Max Length Token: 2048
  • n heads: 32
  • n layers: 32
  • n tokens: 1T

License​

  • Source Code: License Apache Software License 2.0.
  • Weight: For research use only (due to the Facebook LLama's Weight LICENSE).

Note that: A commercial use license for OpenThaiGPT 0.1.0 weight will be released later soon!

Code and Weight​

Authors​

Kobkrit Viriyayudhakorn (kobkrit@aieat.or.th), Sumeth Yuenyong (sumeth.yue@mahidol.edu) and Thaweewat Rugsujarit (thaweewr@scg.com).

Trained Datasets​

Dataset NameInstruction PairsDescriptions
Thaweewat/alpaca-finance-43k-th43,000Alpaca Finance Instruction translated into Thai by Thaweewat Ruksujarit.
kobkrit/rd-taxqa600RD's Tax QA Chatbot Training set by ทรงวุฒิ บุรงค์
datasets/iapp_wiki_qa_squad4,000iApp Technology's Extractive QA Dataset in Thai language
Thaweewat/databricks-dolly-15k-th15,000Databrick's Dolly Instruction translated into Thai by Thaweewat Ruksujarit.
Thaweewat/instruction-wild-52k-th52,000Instruction Wild's translated into Thai by Thaweewat Ruksujarit.
Thaweewat/alpaca-cleaned-52k-th51,000Standford Alpaca's translated into Thai by Thaweewat Ruksujarit.
Thaweewat/gpteacher-20k-th20,000GPT Teacher's Instruction translated into Thai by Thaweewat Ruksujarit.
Thaweewat/onet-m6-social600ONET m6 Social Exam
datasets/Thaweewat/hc3-24k-th24,000Hello Simple AI Summary Dataset translated into Thai by Thaweewat Ruksujarit.
OpenThaiGPT Self Instruct (https://docs.google.com/spreadsheets/d/1BSHkpRyD5RH90E85tLWe4UzpgfDHZafE2rKxLincyWI/edit?usp=sharing)5,000

Thai SelfInstruct Dataset
(Automatic Generated) by OpenThaiGPT

Version 0.1.0-alpha (ByT5-XL Model)​

Release date: 24 April 2023

OpenThaiGPT version 0.1.0-alpha

Thai First 3 billion params models

  • First Thai Byte-Level Text-to-Text Transfer Transformer
  • Support Instruction following
    • Translation to Thai
    • Explanation
    • Paraphrase
  • Zero-shot and Few-shot Learning
  • Pretraining Model: ByT5-XL (3.74 billion params)
  • InstructDataset: 50,000 Thai SelfInstruct
  • RLHF: None
  • Developer: Sumeth Yuenyong, Kobkrit Viriyayudhakorn (kobkrit@iapp.co.th)

PoC Version 0.0.4 (The Fourth PoC Version)​

Release date: 12 March 2023

OpenThaiGPT version 0.0.4

The Fourth PoC Model

  • ตอบคำถามได้ลงรายละเอียดมากขึ้น และตอบคำถามได้ดีขึ้นกว่า 0.0.3 เป็นส่วนมาก
  • Pretraining Model: GPT-2 Thai-base
  • InstructDataset: 300,000 Pantip + 5,000 Wiki QA => 12,920 Thai InstructGPT
  • RLHF: None
  • Developer: Kobkrit Viriyayudhakorn (kobkrit@iapp.co.th)

PoC Version 0.0.3 (The Third PoC Version)​

Release date: 28 February 2023

OpenThaiGPT version 0.0.3

The Third PoC Model

  • Pretraining Model: GPT-2 Thai-base
  • InstructDataset: 300,000 Pantip + 5,000 Wiki QA => 7,000 Thai InstructGPT
  • RLHF: None
  • Developer: Kobkrit Viriyayudhakorn (kobkrit@iapp.co.th)

PoC Version 0.0.2 (The Second PoC Version)​

Release date: 27 February 2023

OpenThaiGPT version 0.0.2

The Second PoC Model

  • Pretraining Model: GPT-2 Thai-base
  • InstructDataset: 7,000 Thai InstructGPT
  • RLHF: None
  • Developer: Kobkrit Viriyayudhakorn (kobkrit@iapp.co.th)

PoC Version 0.0.1 (Very First PoC Version)​

Release date: 20 February 2023

The Very First PoC Model

  • Pretraining Model: GPT-2 Thai-base
  • InstructDataset: 298,678 QA Pairs getting from 70,000 Pantip katoos + Wikipedia QA by iApp
  • RLHF: None
  • Developer: Kobkrit Viriyayudhakorn (kobkrit@iapp.co.th)