Building-a-GPT-Inspired-Transformer-from-Scratch

Crafting the Attention! This repository contains an implementation of a decoder Transformer model inspired by the original GPT architecture, designed to capture the writing style and speech patterns of characters from Classroom of the Elite just for educational purposes. The model is trained to generate text that mimics the linguistic style of the light novel, producing new sentences in the same tone as the source material. It still carries some fragmented, abstract tone , so it can be enhanced more and better . 📜 Paper Implementation This repository is inspired by the "Attention Is All You Need" paper by Vaswani et al. (2017), which introduced the Transformer architecture , we closely follow its core principles while making a slight modification based on the OpenAI paper "Language Models are Unsupervised Multitask Learners". Our model network consists of 10.82 million parameters, optimized for text generation tasks. The loss function used is Average Negative Log-Likelihood (ANLL), making it suitable for evaluating language modeling quality. 📂 Dataset