Some notes for transformer architecture

This note is designed for my personal use; it’s not a comprehensive SQL cheat sheet, as I only record the functions/codes that I think are worthy

My notes refer to the study manual BASIC RATEMAKING Fifth Edition, May 2016.

Tokenization in natural language processing (NLP)involves breaking down text into smaller units called tokens. These tokens can be words, characters, or subwords. This process is for preparing text for further processing.

Principal Components Overview

Neural Network Overview Neural Network is inspired by biological phenomena in the human brain - how information is transferred as signals in the human brain. There are many types of Neural Networks that exist. The basic neural network architecture can be classified as Feed-forward Neural Network, CNN, RNN, and etc.

Decision Tree Overview Decision trees are a type of supervised machine learning algorithm used for classification and regression tasks with a tree-like model. The basic structure of a tree consists of the root node, decision node, and leaf node. The general procedure is that it starts at the root, splits based on the best attribute, and repeats until meeting stopping criteria (e.g., all instances in a leaf node belong to the same class).

Generalized Linear Regression

Multicollinearity

Linear Regression Overview Linear regression is a statistical method used to model the linear relationship between a dependent variable y and one or more independent variables \(x_{1}, x_{2}, \ldots, x_{p}\). It is simple, yet it remains one of the most useful analytical tools. While it may not always yield the highest accuracy, the model simplicity and interpibility are very good.