MullOverThing

Useful tips for everyday

# What do you need to know about Gaussian processes?

## What do you need to know about Gaussian processes?

For a Gaussian process, we need to define mean and covariance functions, specified by hyperparameters φ. y x A Gaussian process represents a powerful way to perform Bayesian inference about functions. A Gaussian process produces a mean estimate.\

### How is a Gaussian process used in Bayesian inference?

A Gaussian process can be used as a prior probability distribution over functions in Bayesian inference. Given any set of N points in the desired domain of your functions, take a multivariate Gaussian whose covariance matrix parameter is the Gram matrix of your N points with some desired kernel, and sample from that Gaussian.

#### Which is the generalisation of a Gaussian distribution?

A Gaussian process is the generalisation of a multivariate Gaussian distribution to a potentially infinite number of variables.\ y x x 3 1 3 2 3 3 2 1 2 2 2 3 1 1 1 2 1 3 31 32 33 21 22 23 11 21 13 3 2 1 3 2 1

How are Gaussian processes defined by second order statistics?

A key fact of Gaussian processes is that they can be completely defined by their second-order statistics. Thus, if a Gaussian process is assumed to have mean zero, defining the covariance function completely defines the process’ behaviour.

When to use a weighted noise kernel in Gaussian regression?

In this article, we introduce a weighted noise kernel for Gaussian processes allowing to account for varying noise when the ratio between noise variances for different points is known, such as in the case when an observation is the sample mean of multiple samples, and the number of samples varies between observations.

## Is it possible to approximate symmetric non Gaussian noise?

Approximating symmetric non-Gaussian noise with Gaussian noise will work sufficiently well in most cases (see the Central limit theorem ), however observations with strongly varying noise will result in either overestimating the noise variance or overfitting the data.

### How are hyperparameters tuned in Gaussian process regression?

The hyperparameters are inferred (‘tuned’), e.g. by maximizing the likelihood on the training set. To deal with noisy observations, a small constant is customarily added to the diagonal of the covariance matrix : The constant is interpreted as the variance of observation noise, normally distributed with zero mean.