[Probability] 2. Random Process, Random Variable, Functional analysis, Kernel function

이 글은 최성준 교수님의 Bayesian Deep Learning 강좌를 요약한 글로, 필자의 이해를 위해 작성된 글입니다.

🦉 전체 흐름

Random Process를 이해하기에 앞서 Random Variable을 이해하고, RV는 sigma-field에서 정의되는 function이므로 이 일련의 과정을 이해하고자 한다.

🐻 Set, sigma-field, Measure --> Probability

• Set
• Set function: a function assigning a number of a set
  • Measure is a set function
    • sigma-field: a collection of subsets of U such that axioms(Sigma-field is designed to define a measure)
    • Probability is a (normalized) measure such that \mu(U)=1 / a set function P:A-->[0,1] where A is a sigma-field

🐻 Sample point, Sample space --> Probability

• Sample point: a point representing an outcome
• Sample space: the set of all the sample points
  • Probability allocation function: 
  • Conditional probability P(A|B)
    • Bayes' rule (Posterior probability, Prior probability)

🐻 Random Variable

• Random variable: a real-valued function defined on sample space that is measureable w.r.t the probability space and the Borel measureable space exists 주의점: Distribution이 정해져있지 않은 상태이다.$$X:\mathrm{\Omega }⟶\mathbb{R}suchthat\mathrm{\forall }B\in \mathbb{B},X^{-1}(B)\in \mathbb{A}$$
  • $$P(X\in B)forB\in \mathbb{B}=P(X^{-1}(B))=P(\{w:X(w)\in B\})$$
  • Sample space X에 관해 값을 하나 random 으로 뽑았을 때의 값: Random variable, P는 그것의 면적(measure)값
• Features of Random Variable
  • Probability mass function: X distribution에서 x가 나올 probability(measure)를 의미한다., 주로 discrete distribution X에 사용된다. (X는 distribution, x는 X distribution에서 random으로 뽑혀진 값 하나를 의미한다.) $$p_X(x)=P(X=x)$$
  • Probability density function: continuous distribution X에 사용된다. $$f_X(x)=\underset{\mathrm{\Delta }x\to 0}{lim}\frac{P(x<X\le x+\mathrm{\Delta }x}{\mathrm{\Delta }x},P(X\in B)={\int }_{x\in B}{f}_X(x)dx$$
  • Expectation 
  • Conditional Expectation: X average over the event whtere Y=y (a function of Y), Y가 잘게 분리될수록 different sigma-field를 나타낸다. $$E(X|Y)$$
  • Moment(Mean, Variance, Skewness, kurtosis etc.)
  • Joint Moment(Correlation, Covariance, Correlation coefficient etc.)

🐻 Random Process

• Random process: 무한 차원의 Random variable을 이야기 $$X_t(w),t\in I$$
  • $$X_t:I⟶$$ indexed family of infinite number of random variables
  • $$X_t:\mathrm{\Omega }⟶$$ 

🐻 Functional Analysis