This website is still in a beta testing phase. Data is not exhaustive. Updates are occurring frequently.

At the top of the IATID chart the 'p-value' is shown for the current dataset. For a technical explanation of what the p-value is, see here.

Benford's law only gives an approximation of the expected frequency of digits. In any given sample it is highly unlikely that the distribution will match the expected frequency exactly. Moreover, the smaller the size of the subset, the more noticeable small variations will be. The p-value is calculated and shown here to give some sense of the significance of the data matching (or not matching) the predictions of Benford's law, given the size of the underlying dataset.

In very simple terms: the higher the p-value, the higher the chance that the data shown follows the distribution suggested by Benford's law. However, there are important facts to be aware of when interpreting a p-value. Look at the graphs below. On the left you can see all IATI data at the time of writing and on the right is the distribution for 97 transactions from Kyrgyzstan.

All IATI data compared to Benford's law IATI data for Kyrgyzstan compared to Benford's law

Over more than 250,000 transaction values, the expected and actual lines on this graph are so close that in some places you can't even separate them with the naked eye. But the p-value for this data is miniscule (less than 0.000000001). If we go by the rule that the higher the p-value, the closer the data fits the expected distribution, then we must decide that the expected and actual data is totally different. Which contradicts what we can interpret intuitively from the graph.

For the 97 transactions in Kyrgyzstan, we get a p-value of higher than 0.9. Even though the dataset is a fraction of the size and, as you can see from the graph, has some noticeable variations (relatively speaking), the data is mathematically a much better fit.

The lesson here is that there is a difference between statistical significance and practical significance. For a good explanation of this counter-intuitive result, see here.

The p-value is provided as a matter of interest but remember:

"The difference between two treatments or between two areas or two groups of people, will show up as 'significantly different' if the experiment be conducted through a sufficient number of trials, even thought the difference be so small that it is of no scientific or economic consequence." Deming, W.E. 1975. On probability as a basis for action. American Statistician 29: 146-152.
"In any particular application, graphical or other informal analysis may show that consistency or inconsistency with H0 is so clear cut that explicit calculation of p is unnecessary." Cox, D.R. 1977. The role of significance tests. Scandanavian Journal of Statistics 4: 49-70.