Dark diversity estimation based on a single matrix of binary observations

Abstract

Ecological theory and nature conservation have traditionally relied solely on observed local diversity. However, the biodiversity of a site also includes the absent species that are present in the surrounding region and can potentially inhabit the site’s particular ecological conditions. These unobserved species constitute the “dark diversity” of the site. Dark diversity is by definition unobservable and can only be estimated - in binary fashion or as a degree of certainty about species membership. This thesis compares the effectiveness of several implementations of non-negative matrix factorization (NMF) and basic autoencoders (a type of artificial neural network) to generate probabilistic values about a species’ membership in a specific site based on a single matrix of binary observations. We find that it is possible to generate a suitability matrix that is highly correlated with the underlying suitability values with both methods and that using autoencoders specifically for dark diversity predictions has a lot of potential for even more future improvements.