{
"cells": [
{
"cell_type": "markdown",
"id": "bef88828",
"metadata": {},
"source": [
"# Propensity Score Matcher\n",
"\n",
"In this notebook, we show the basic usage of the PropensityScoreMatcher. Unlike the GeneticMatcher and ConstraintSatisfactionMatcher, the PropensityScoreMatcher does not directly optimize a particular balance score. Instead, the PropensityScoreMatcher uses the given objective as a measure of \"correctness\" of the propensity score model. The matcher tries a (possibly large) number of potential models and returns the model with the best score according to the given metric. In doing this, we are essentially automating an often manual process of hyperparameter optimization that accompanies propensity score matching.\n",
"\n",
"We show that the hyperparameter search still leaves unoptimized balance by apply a ConstraintSatisfactionMatcher\n",
"to the resulting population from the PropensityScoreMatcher. The residual unoptimzed balance highlights a major\n",
"limitation of propensity score matching in general."
]
},
{
"cell_type": "code",
"execution_count": 1,
"id": "347d7457",
"metadata": {},
"outputs": [],
"source": [
"import logging \n",
"logging.basicConfig(\n",
" format=\"%(levelname)-4s [%(filename)s:%(lineno)d] %(message)s\",\n",
" level='INFO',\n",
")\n",
"\n",
"from pybalance.utils import *\n",
"from pybalance.sim import generate_toy_dataset\n",
"from pybalance.propensity import PropensityScoreMatcher, plot_propensity_score_match_distributions\n",
"from pybalance.visualization import (\n",
" plot_numeric_features, \n",
" plot_categoric_features, \n",
" plot_binary_features,\n",
" plot_joint_numeric_distributions,\n",
" plot_per_feature_loss\n",
")"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "8dd76114",
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
" Headers Numeric:
\n",
" ['age', 'height', 'weight']
\n",
" Headers Categoric:
\n",
" ['gender', 'haircolor', 'country', 'binary_0', 'binary_1', 'binary_2', 'binary_3']
\n",
" Populations
\n",
" ['pool', 'target']
\n",
"
| \n", " | age | \n", "height | \n", "weight | \n", "gender | \n", "haircolor | \n", "country | \n", "population | \n", "binary_0 | \n", "binary_1 | \n", "binary_2 | \n", "binary_3 | \n", "patient_id | \n", "
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", "60.807949 | \n", "173.610298 | \n", "77.912924 | \n", "0.0 | \n", "1 | \n", "4 | \n", "pool | \n", "0 | \n", "0 | \n", "1 | \n", "1 | \n", "0 | \n", "
| 1 | \n", "45.810836 | \n", "170.541198 | \n", "112.416988 | \n", "0.0 | \n", "1 | \n", "4 | \n", "pool | \n", "0 | \n", "1 | \n", "0 | \n", "0 | \n", "1 | \n", "
| 2 | \n", "58.876976 | \n", "188.138610 | \n", "108.789013 | \n", "0.0 | \n", "0 | \n", "2 | \n", "pool | \n", "0 | \n", "0 | \n", "1 | \n", "1 | \n", "2 | \n", "
| 3 | \n", "73.398077 | \n", "162.939196 | \n", "65.345017 | \n", "0.0 | \n", "1 | \n", "4 | \n", "pool | \n", "0 | \n", "1 | \n", "1 | \n", "1 | \n", "3 | \n", "
| 4 | \n", "56.890587 | \n", "156.386701 | \n", "78.140295 | \n", "0.0 | \n", "0 | \n", "3 | \n", "pool | \n", "0 | \n", "0 | \n", "1 | \n", "0 | \n", "4 | \n", "
| ... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "
| 995 | \n", "39.662026 | \n", "162.692755 | \n", "54.607476 | \n", "0.0 | \n", "2 | \n", "4 | \n", "target | \n", "0 | \n", "0 | \n", "1 | \n", "1 | \n", "10995 | \n", "
| 996 | \n", "49.130301 | \n", "141.583192 | \n", "103.798145 | \n", "1.0 | \n", "0 | \n", "2 | \n", "target | \n", "1 | \n", "0 | \n", "0 | \n", "0 | \n", "10996 | \n", "
| 997 | \n", "68.035281 | \n", "168.744482 | \n", "56.499644 | \n", "1.0 | \n", "1 | \n", "1 | \n", "target | \n", "0 | \n", "0 | \n", "0 | \n", "1 | \n", "10997 | \n", "
| 998 | \n", "62.044564 | \n", "177.796983 | \n", "75.983973 | \n", "1.0 | \n", "1 | \n", "1 | \n", "target | \n", "0 | \n", "0 | \n", "0 | \n", "1 | \n", "10998 | \n", "
| 999 | \n", "51.243734 | \n", "161.013556 | \n", "86.513956 | \n", "0.0 | \n", "0 | \n", "1 | \n", "target | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "10999 | \n", "
11000 rows × 12 columns
\n", "| \n", " | age | \n", "height | \n", "weight | \n", "gender | \n", "haircolor | \n", "country | \n", "population | \n", "binary_0 | \n", "binary_1 | \n", "binary_2 | \n", "binary_3 | \n", "patient_id | \n", "
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 5781 | \n", "74.382687 | \n", "194.082038 | \n", "118.760023 | \n", "0.0 | \n", "2 | \n", "5 | \n", "pool | \n", "1 | \n", "0 | \n", "0 | \n", "0 | \n", "5781 | \n", "
| 6714 | \n", "66.581290 | \n", "178.545534 | \n", "102.566840 | \n", "0.0 | \n", "2 | \n", "2 | \n", "pool | \n", "1 | \n", "0 | \n", "0 | \n", "0 | \n", "6714 | \n", "
| 9937 | \n", "61.860293 | \n", "159.449219 | \n", "108.945960 | \n", "0.0 | \n", "2 | \n", "4 | \n", "pool | \n", "1 | \n", "0 | \n", "0 | \n", "1 | \n", "9937 | \n", "
| 8223 | \n", "46.656414 | \n", "140.392554 | \n", "65.453208 | \n", "1.0 | \n", "2 | \n", "4 | \n", "pool | \n", "1 | \n", "0 | \n", "0 | \n", "1 | \n", "8223 | \n", "
| 962 | \n", "52.829914 | \n", "137.725077 | \n", "93.206007 | \n", "0.0 | \n", "1 | \n", "2 | \n", "pool | \n", "1 | \n", "1 | \n", "0 | \n", "0 | \n", "962 | \n", "
| ... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "
| 995 | \n", "39.662026 | \n", "162.692755 | \n", "54.607476 | \n", "0.0 | \n", "2 | \n", "4 | \n", "target | \n", "0 | \n", "0 | \n", "1 | \n", "1 | \n", "10995 | \n", "
| 996 | \n", "49.130301 | \n", "141.583192 | \n", "103.798145 | \n", "1.0 | \n", "0 | \n", "2 | \n", "target | \n", "1 | \n", "0 | \n", "0 | \n", "0 | \n", "10996 | \n", "
| 997 | \n", "68.035281 | \n", "168.744482 | \n", "56.499644 | \n", "1.0 | \n", "1 | \n", "1 | \n", "target | \n", "0 | \n", "0 | \n", "0 | \n", "1 | \n", "10997 | \n", "
| 998 | \n", "62.044564 | \n", "177.796983 | \n", "75.983973 | \n", "1.0 | \n", "1 | \n", "1 | \n", "target | \n", "0 | \n", "0 | \n", "0 | \n", "1 | \n", "10998 | \n", "
| 999 | \n", "51.243734 | \n", "161.013556 | \n", "86.513956 | \n", "0.0 | \n", "0 | \n", "1 | \n", "target | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "10999 | \n", "
2000 rows × 12 columns
\n", "| \n", " | age | \n", "height | \n", "weight | \n", "gender | \n", "haircolor | \n", "country | \n", "population | \n", "binary_0 | \n", "binary_1 | \n", "binary_2 | \n", "binary_3 | \n", "patient_id | \n", "
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", "37.519341 | \n", "178.337875 | \n", "57.424543 | \n", "0.0 | \n", "1 | \n", "4 | \n", "target | \n", "1 | \n", "1 | \n", "1 | \n", "1 | \n", "10000 | \n", "
| 1 | \n", "23.722325 | \n", "128.347114 | \n", "102.183004 | \n", "0.0 | \n", "2 | \n", "2 | \n", "target | \n", "0 | \n", "1 | \n", "0 | \n", "1 | \n", "10001 | \n", "
| 2 | \n", "64.523502 | \n", "144.600598 | \n", "90.061948 | \n", "1.0 | \n", "2 | \n", "4 | \n", "target | \n", "1 | \n", "0 | \n", "0 | \n", "1 | \n", "10002 | \n", "
| 3 | \n", "25.377578 | \n", "177.986337 | \n", "82.076883 | \n", "0.0 | \n", "0 | \n", "2 | \n", "target | \n", "1 | \n", "0 | \n", "0 | \n", "1 | \n", "10003 | \n", "
| 4 | \n", "26.922515 | \n", "155.633760 | \n", "76.929413 | \n", "1.0 | \n", "2 | \n", "5 | \n", "target | \n", "1 | \n", "1 | \n", "0 | \n", "1 | \n", "10004 | \n", "
| ... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "... | \n", "
| 9935 | \n", "63.955700 | \n", "164.060497 | \n", "101.344670 | \n", "0.0 | \n", "2 | \n", "3 | \n", "pool | \n", "1 | \n", "1 | \n", "0 | \n", "1 | \n", "9935 | \n", "
| 9944 | \n", "38.089602 | \n", "174.116424 | \n", "90.463362 | \n", "0.0 | \n", "1 | \n", "5 | \n", "pool | \n", "0 | \n", "1 | \n", "1 | \n", "1 | \n", "9944 | \n", "
| 9950 | \n", "42.340454 | \n", "154.759178 | \n", "113.770268 | \n", "1.0 | \n", "2 | \n", "5 | \n", "pool | \n", "0 | \n", "0 | \n", "0 | \n", "0 | \n", "9950 | \n", "
| 9960 | \n", "58.773727 | \n", "158.792184 | \n", "80.576978 | \n", "1.0 | \n", "0 | \n", "5 | \n", "pool | \n", "0 | \n", "1 | \n", "1 | \n", "0 | \n", "9960 | \n", "
| 9983 | \n", "65.077128 | \n", "175.593470 | \n", "75.612613 | \n", "0.0 | \n", "2 | \n", "4 | \n", "pool | \n", "1 | \n", "1 | \n", "0 | \n", "1 | \n", "9983 | \n", "
2000 rows × 12 columns
\n", "