As my first post I'll be attempting to make almost the whole inference of the Poisson regression model, which is also a course work for my class of Pattern recognition, so don't try this at home.

In this post we will cover:- What is the Poisson Regression?
- The Poisson Distribution as part of the Exponential Family
- Maximum Log-Likelihood method
- Derivating the \(loss\) function
- The matricial representation
- A simple implementation
- Awesome references

We already know about the Linear Regression, which helps us answer questions like "How much will a house with these characteristics cost?". Or the Logistic Regression, which is used to model the probability of a certain class or event existing such as pass/fail, win/lose, alive/dead or healthy/sick.

But, what happens when the questions are "How many costumers will come today?", "How many people are in line at the grocery store?", "How **many**...?",
and one way to answer these questions is using the Poisson Regression Model.

Poisson Regression is used to model count data. For this, we assume the response variable \(Y\) has a Poisson Distribution, and assumes the logarithm of its expected value can be modeled by a linear combinations of unknown parameters.

First, if we have a random variable \(X\) such that \[X\sim ExpFam(\theta)\] we know we can express its probability function as the following \begin{equation} P_X(X=x\ | \ \theta) = h(x) e^{\eta(\theta)T(x) - A(\theta)} \ \ \ \ (1) \end{equation} Okay, now let's do this, assume we have a random variable \(Y\) that is distributed as a Poisson with a parameter \(\lambda\) \[Y\sim Poisson(\lambda)\] with its probability function being \[P_Y(Y=y \ | \ \lambda) = \frac{e^{-\lambda}\lambda^{y}}{y!}\] We know that \[ P_y(Y=y \ | \ \lambda) = \frac{e^{-\lambda}\lambda^{y}}{y!} = \\ \frac{1}{y!} e^{-\lambda}\lambda^{y} = \\ \frac{1}{y!}e^{\log(e^{-\lambda}\lambda^{y})} = \] \[ \frac{1}{y!}e^{\log(e^{-\lambda}) + \log(\lambda^{y})} = \\ \frac{1}{y!} e^{-\lambda + y \log(\lambda)} = \\ \frac{1}{y!} e^{y \log(\lambda) - \lambda} \] and thus \begin{equation} P_y(Y=y \ | \ \lambda) = \frac{1}{y!} e^{y \log(\lambda) - \lambda} \ \ \ \ (2) \end{equation} but wait a minute, the equation \((2)\) has the same form as the equation \((1)\) ! \[\theta = \lambda\] \[h(y) = \frac{1}{y!}\] \[\eta(\lambda) = \log(\lambda)\] \[T(y) = y\] \[A(\lambda) = \lambda\] and thus \begin{equation} P_Y(Y=y \ | \ \lambda) = h(y) e^{\eta(\lambda)T(y) - A(\lambda)} \ \ \ \ (3) \end{equation} Finally we can say \[Y \sim ExpFam(\lambda)\]

Alright! Now it's time to code these results, you can check out the Jupyter Notebook too see the full setup and implementation, but here I'll leave the important parts

We know that we will need the \(loss\) function, so let's start with it

Done! Let's remember we also will need to calculate the gradient with respect \(w\) and also respect with \(b\), down bellow I'll show you the implementation following with the gradient_descent algorithm

We used these functions with a generated dataset and we calculate the following results (with the \(loss\) function decreasing as we can see)

And that is pretty much it, thank you for coming to my TED Talk

- Akbayrak, S. (2018, February 12). Generalized Linear Models. Retrieved October 17, 2020
- Exponential family. (2020, September 16). Retrieved September 27, 2020
- 4. Poisson Regression, Gradient Descent¶. (2020, July 22). Retrieved October 17, 2020
- Poisson regression. (2020, August 17). Retrieved October 30, 2020