In discussions about why ChatGPT has captured our fascination, two common themes emerge: 1. Scale: Increasing data and computational resources. 2. User Experience (UX): Transitioning from prompt-based interactions to more natural chat interfaces. However, there's an aspect often overlooked – the remarkable technical innovation behind the success of models like ChatGPT.