# ðŸ”¬ dnn-inferenceÂ¶

**dnn-inference** is a Python module for hypothesis testing based on blackbox models, including **deep neural networks**.

GitHub repo: https://github.com/statmlben/dnn-inference

Documentation: https://dnn-inference.readthedocs.io

Open Source: MIT license

Paper: arXiv:2103.04985

## ðŸŽ¯ What We Can DoÂ¶

**dnn-inference** is able to provide an asymptotically valid p-value to examine if \(\mathcal{S}\) is discriminative features to predict \(Y\).
Specifically, the proposed testing is:

where \(\mathcal{S}\) is a collection of hypothesized features, \(R\) and \(R_{\mathcal{S}}\) are risk functions with/without the hypothesized features \(\mathbf{X}_{\mathcal{S}}\), and \(f^*\) and \(g^*\) are population minimizers on \(R\) and \(R_{\mathcal{S}}\) respectively. The proposed test just considers the difference between the best predictive scores with/without hypothesized features. Please check more details in our paper arXiv:2103.04985.

When log-likelihood is used as a loss function, then the test is equivalent to a conditional independence test: \(Y \perp X_{\mathcal{S}} | X_{\mathcal{S}^c}\).

Only a small number of fitting on neural networks is required, and the number can be as small as 1.

Asymptotically Type I error control and power consistency.

## ReferenceÂ¶

**If you use this code please star the repository and cite the following paper:**

```
@misc{dai2021significance,
title={Significance tests of feature relevance for a blackbox learner},
author={Ben Dai and Xiaotong Shen and Wei Pan},
year={2021},
eprint={2103.04985},
archivePrefix={arXiv},
primaryClass={stat.ML}
}
```