Stochastic Processes

A stochastic process is a mathematical object defined as a collection of random variables defined on a common probability space $(\Omega, \mathcal{F}, \mathbb{P})$, indexed by a totally ordered set $T$ (usually representing time). Formally, a stochastic process is parameterized as $X = \{X_t : t \in T\}$, where for each $t \in T$, $X_t$ is an $\mathcal{F}$-measurable function mapping $\Omega \to S$ for measurable state space $(S, \mathcal{S})$.

When $T = \mathbb{N}$ or $\mathbb{Z}$, the process is cast as a discrete-time stochastic process. If $T = [0, \infty)$ or $T \subset \mathbb{R}$, it represents a continuous-time stochastic process. The state space $S$ determines whether the process is discrete-state (e.g., integer values) or continuous-state (e.g., real-valued).

Filtrations and Information

To rigorously describe the evolution of a stochastic process, it is essential to capture the accumulation of information over time. This is formalized by a filtration $\mathbb{F} = \{\mathcal{F}_t\}_{t \in T}$, which is an increasing family of sub-$\sigma$-algebras of $\mathcal{F}$. That is, $\mathcal{F}_s \subseteq \mathcal{F}_t \subseteq \mathcal{F}$ for all $s \leq t$.

The intuitive interpretation of $\mathcal{F}_t$ is the “history” or the “available information” up to time $t$. A stochastic process $\{X_t\}_{t \in T}$ is said to be adapted to the filtration $\mathbb{F}$ if, for every $t \in T$, the random variable $X_t$ is $\mathcal{F}_t$-measurable. This implies that if one observes the state of the universe up to time $t$, the value of $X_t$ is completely known.

Martingales

Martingales constitute one of the most fundamental classes of stochastic processes, generalizing the concept of a “fair game” where knowledge of past events never helps predict expected future winnings.

Let $(\Omega, \mathcal{F}, \{\mathcal{F}_t\}, \mathbb{P})$ be a filtered probability space. A real-valued stochastic process $\{M_t\}_{t \in T}$ is a martingale with respect to the filtration $\{\mathcal{F}_t\}$ and probability measure $\mathbb{P}$ if it satisfies the following three conditions:

1. Adaptedness: $M_t$ is $\mathcal{F}_t$-measurable for all $t$.
2. Integrability: $\mathbb{E}[|M_t|] < \infty$ for all $t$ (i.e., $M_t \in L^1(\mathbb{P})$).
3. Martingale Property: For all $s \leq t$, the conditional expectation satisfies: $\mathbb{E}[M_t \mid \mathcal{F}_s] = M_s \quad \text{almost surely (a.s.)}$

If the equality in the third condition is replaced with $\leq$ (or $\geq$), the process is termed a supermartingale (or submartingale). In a supermartingale, the expected future value is less than or equal to the current value (a losing game), whereas in a submartingale, it is greater than or equal to the current value (a winning game).

Discrete-Time Martingales

Consider a simple symmetric random walk $S_n = \sum_{i=1}^n X_i$, where the increments $X_i$ are independent, identically distributed (i.i.d.) random variables with $\mathbb{P}(X_i = 1) = 1/2$ and $\mathbb{P}(X_i = -1) = 1/2$. Let $\mathcal{F}_n = \sigma(X_1, \dots, X_n)$ be the natural filtration. Check that $S_n$ is a martingale:

$\mathbb{E}[S_{n+1} \mid \mathcal{F}_n] = \mathbb{E}[S_n + X_{n+1} \mid \mathcal{F}_n] = \mathbb{E}[S_n \mid \mathcal{F}_n] + \mathbb{E}[X_{n+1} \mid \mathcal{F}_n]$

Since $S_n$ is $\mathcal{F}_n$-measurable, $\mathbb{E}[S_n \mid \mathcal{F}_n] = S_n$. Since $X_{n+1}$ is independent of $\mathcal{F}_n$, $\mathbb{E}[X_{n+1} \mid \mathcal{F}_n] = \mathbb{E}[X_{n+1}] = 0$. Thus, $\mathbb{E}[S_{n+1} \mid \mathcal{F}_n] = S_n$, proving $S_n$ is a discrete-time martingale.

Stopping Times

In many practical and theoretical contexts, we are interested in evaluating models at random times (e.g., the time a stock hits a certain price or the time a gambler goes bankrupt). This gives rise to the concept of a stopping time.

A random variable $\tau: \Omega \to T \cup \{\infty\}$ is a stopping time (or Markov time) with respect to a filtration $\{\mathcal{F}_t\}$ if, for every $t \in T$, the event $\{\tau \leq t\} \in \mathcal{F}_t$. Intuitively, at any given time $t$, one can determine whether the stopping time has occurred strictly based on the information available up to time $t$. A stopping time cannot look into the future.

For a stochastic process $\{X_t\}$, the first hitting time of a Borel set $B \in \mathcal{B}(\mathbb{R})$ is defined as: $\tau_B = \inf \{ t \geq 0 : X_t \in B \}$ When the process has right-continuous paths and $B$ is a closed set, $\tau_B$ is guaranteed to be a stopping time.

Optional Stopping Theorem

Does evaluating a martingale at a stopping time $\tau$ preserve its expected value? In general, it might not. However, Doob’s Optional Stopping Theorem establishes the conditions under which the expected value at the stopping time equals the initial expected value, i.e., $\mathbb{E}[M_\tau] = \mathbb{E}[M_0]$.

Let $(M_n)_{n \geq 0}$ be a discrete-time martingale and $\tau$ be a stopping time with respect to the filtration $(\mathcal{F}_n)$. Then $\mathbb{E}[M_\tau] = \mathbb{E}[M_0]$ holds if any of the following conditions is satisfied:

1. The stopping time is bounded almost surely: $\mathbb{P}(\tau \leq N) = 1$ for some deterministic integer $N$.
2. The stopping time has a finite expectation $\mathbb{E}[\tau] < \infty$, and the increments are conditionally bounded: there exists $c > 0$ such that $\mathbb{E}[|M_{n+1} - M_n| \mid \mathcal{F}_n] \leq c$ a.s. on $\{\tau > n\}$.
3. There exists a constant $C$ such that $|M_{n \wedge \tau}| \leq C$ almost surely for all $n$.

This theorem highlights the impossibility of formulating a systemic winning strategy in a fair game under bounded resource constraints (the origin of the impossibility of the classical “Martingale betting strategy”).

Brownian Motion (Wiener Process)

The Wiener process (or standard Brownian motion) is the fundamental continuous-time analog of the random walk. It drives modern financial theory, statistical mechanics, and continuous-state probability.

A standard one-dimensional Wiener process $W = \{W_t\}_{t \ge 0}$ is a stochastic process characterized by the following properties:

1. $W_0 = 0$ almost surely.
2. $W$ has independent increments: For any $0 \leq t_1 < t_2 < \dots < t_k$, the random variables $W_{t_1}, W_{t_2} - W_{t_1}, \dots, W_{t_k} - W_{t_{k-1}}$ are independent.
3. $W$ has stationary normally distributed increments: For any $0 \leq s < t$, the increment $W_t - W_s$ follows a normal distribution: $W_t - W_s \sim \mathcal{N}(0, t - s)$
4. The paths $t \mapsto W_t$ are almost surely continuous.

Despite being continuous everywhere, the path of a Brownian motion is differentiable nowhere. Its quadratic variation over the interval $[0,t]$ is exactly $t$. That is, $\lim_{||\Pi|| \to 0} \sum_{i=0}^{n-1} (W_{t_{i+1}} - W_{t_i})^2 = t$. This strict non-zero quadratic variation is the very reason why ordinary calculus (Newton-Leibniz) fails for stochastic processes and necessitate a distinct calculus.

Itô’s Lemma

Because Brownian motion has non-zero quadratic variation, the standard chain rule of differential calculus does not hold. Instead, we use Itô’s Calculus, anchored by Itô’s Lemma.

Let $X_t$ be an Itô drift-diffusion process satisfying the stochastic differential equation: $dX_t = \mu_t dt + \sigma_t dW_t$ where $W_t$ is a standard Wiener process, and $\mu_t, \sigma_t$ are adapted processes. Let $f(t,x)$ be a scalar function that is twice continuously differentiable in $x$ and once in $t$ (i.e., $f \in C^{1,2}([0, \infty) \times \mathbb{R})$).

By Itô’s Lemma, the process $Y_t = f(t, X_t)$ is also an Itô process whose differential is given by: $df(t, X_t) = \left( \frac{\partial f}{\partial t} + \mu_t \frac{\partial f}{\partial x} + \frac{1}{2} \sigma_t^2 \frac{\partial^2 f}{\partial x^2} \right) dt + \sigma_t \frac{\partial f}{\partial x} dW_t$

The profound emergence of the term $\frac{1}{2} \sigma_t^2 \frac{\partial^2 f}{\partial x^2} dt$ reflects the quadratic variation of $X_t$, often formalized by the heuristic multiplication rules: $dt \cdot dt = 0, \quad dt \cdot dW_t = 0, \quad (dW_t)^2 = dt$

Stochastic Differential Equations (SDEs)

A Stochastic Differential Equation relates the continuous-time dynamics of a stochastic process to a deterministic drift part and a stochastic diffusion part. The general form is: $dX_t = b(t, X_t) dt + \sigma(t, X_t) dW_t$ This equation is simply a symbolic shorthand for the integral equation: $X_t = X_0 + \int_0^t b(s, X_s) ds + \int_0^t \sigma(s, X_s) dW_s$ where the first integral is a standard Lebesgue/Riemann integral and the second is an Itô stochastic integral.

Existence and Uniqueness

Much like Picard–Lindelöf for deterministic ODEs, there are conditions for the strong existence and uniqueness of solutions to SDEs. Under Lipschitz continuity and linear growth bounding conditions:

1. Lipschitz Condition: $|b(t, x) - b(t, y)| + |\sigma(t, x) - \sigma(t, y)| \leq K|x - y|$
2. Linear Growth: $|b(t, x)|^2 + |\sigma(t, x)|^2 \leq C(1 + |x|^2)$

for some constants $K, C > 0$ and all $t, x, y$, there exists a unique strong solution $X_t$ to the SDE.

The analysis, simulation, and integration of SDEs form the bedrock of continuously evolving systems subject to noise across physics, mathematical biology, and finance.