Estimate exponential cutoff in power law distribution

Question

As I already did the analysis of the social network, I came across the problem of selecting the probability distribution according to the degree of the network.

So, I have a probability distribution P(X >= x) , which from visual control follows a power law with exponential cutoff, and not a pure power law (straight line).

So, given that the equation of distribution of power laws with exponential cutoff:

f (x) = x ** alpha * exp (beta * x)

How can I evaluate alpha and beta parameters using Python?

I know that the scipy.stats.powerlaw package exists, and they have a .fit() function, but this does not seem to complete the task, since it only returns the location and scale of the graph, which, apparently, is only useful for normal distribution? There are also not enough textbooks in this package.

PS I am well aware of the implementation of CLauset et al. , But they seem to provide no way to evaluate the parameters of alternative distributions.

Answer 1

The scipy.stats.powerlaw.fit function may still work for your purposes. This is a bit confusing how distributions in scipy.stats work (the documentation for each relates to the optional loc and scale parameters, although not all of them use these parameters, and each uses them differently). If you look at the docs:

http://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.powerlaw.html

there is also a second optional parameter “a”, which is “form parameters”. In the case of powerlaw, this contains a single parameter. Do not worry about "loc" and "scale".

Edit: Sorry, forgot that you also need the beta parameter. It is best to define the powerlaw function that you want yourself and then use the scipy generic algorithm to examine the parameters. For example: http://www.scipy.org/Cookbook/FittingData#head-5eba0779a34c07f5a596bbcf99dbc7886eac18e5

Answer 2

Here is a tool for estimating the scale factor and the exponential speed of a power law with exponential shutdown by maximizing the likelihood in R:

 # Input: Data vector, lower threshold # Output: List, giving type ("powerexp"), scaling exponent, exponential rate, lower threshold, log-likelihood powerexp.fit <- function(data,threshold=1,method="constrOptim",initial_rate=-1) { x <- data[data>=threshold] negloglike <- function(theta) { -powerexp.loglike(x,threshold,exponent=theta[1],rate=theta[2]) } # Fit a pure power-law distribution pure_powerlaw <- pareto.fit(data,threshold) # Use this as a first guess at the exponent initial_exponent <- pure_powerlaw$exponent if (initial_rate < 0) { initial_rate <- exp.fit(data,threshold)$rate } minute_rate <- 1e-6 theta_0 <- as.vector(c(initial_exponent,initial_rate)) theta_1 <- as.vector(c(initial_exponent,minute_rate)) switch(method, constrOptim = { # Impose the constraint that rate >= 0 # and that exponent >= -1 ui <- rbind(c(1,0),c(0,1)) ci <- c(-1,0) # Can't start with values on the boundary of the feasible set so add # tiny amounts just in case if (theta_0[1] == -1) {theta_0[1] <- theta_0[1] + minute_rate} if (theta_0[2] == 0) {theta_0[2] <- theta_0[2] + minute_rate} est <- constrOptim(theta=theta_0,f=negloglike,grad=NULL,ui=ui,ci=ci) alpha <- est$par[1] lambda <- est$par[2] loglike <- -est$value}, optim = { est <- optim(par=theta_0,fn=negloglike) alpha <- est$par[1] lambda <- est$par[2] loglike <- -est$value}, nlm = { est.0 <- nlm(f=negloglike,p=theta_0) est.1 <- nlm(f=negloglike,p=theta_1) est <- est.0 if (-est.1$minimum > -est.0$minimum) { est <- est.1;cat("NLM had to switch\n") } alpha <- est$estimate[1] lambda <- est$estimate[2] loglike <- -est$minimum}, {cat("Unknown method",method,"\n"); alpha<-NA; lambda<-NA; loglike<-NA} ) fit <- list(type="powerexp", exponent=alpha, rate=lambda, xmin=threshold, loglike=loglike, samples.over.threshold=length(x)) return(fit) } 

For more information contact https://github.com/jeffalstott/powerlaw/

Answer 3

The Powerlaw library can be directly used to evaluate parameters as follows:

• Install all pythons dependencies:

   pip install powerlaw mpmath scipy 

• Run the powerlaw package in python environment:

   import powerlaw data = [5, 4, ... ] results = powerlaw.Fit(data) 

• get parameters from the results

   results.truncated_power_law.parameter1 # power law parameter (alpha) results.truncated_power_law.parameter2 # exponential cut-off parameter (beta)