Inference attack¶

The most sensitive information a person holds is often not directly available to an adversary. But it can be inferred from information that is. An inference attack trains a machine learning model on a labelled dataset and uses it to predict private attributes from observable ones. The target does not have to provide the sensitive attribute at all. The model infers it from the trail left by ordinary activity. Profiling sorts observable behaviour into a broader description; an inference attack predicts one particular attribute that was never supplied.

Apps as a proxy¶

The clearest example is political affiliation inferred from smartphone app installations. The operating system knows what is installed. An ordinary third-party app does not: Android filters package visibility by default for apps targeting Android 11 or later, so an app “can’t detect all the apps installed on a device”, and the QUERY_ALL_PACKAGES permission that lifts the restriction is subject to approval on Google Play.

Where a service can observe app presence anyway, through the permissions it does hold, an SDK it ships, or a device-management arrangement, those observations become features. Certain apps correlate statistically with certain political leanings, and a model trained on app lists paired with known affiliations can then predict the affiliation of people whose lists it has never seen.

# Training phase
training_data = [
 {apps: ["weather", "hunting_app", "bible_study", "news_aggregator"],
 label: "conservative"},
 {apps: ["weather", "climate_tracker", "lgbtq_support_app", "news_aggregator"],
 label: "liberal"},
 # ... hundreds or thousands of labelled examples
]

model = LogisticRegression.train(features=app_lists, labels=political_leanings)

# Inference phase (attack):
target = {apps: ["weather", "hunting_app", "sports_scores", "news_aggregator"]}
predicted_label, confidence = model.predict(target.apps)

# The coefficients can be read as well as the prediction: which apps move the
# answer, in which direction, and by how much.
feature_weights = model.coefficients()

The weights tell an analyst which apps are most predictive, and in which direction: installing a hunting app moves the prediction towards one label, a climate tracker towards the other. Such a ranking can also guide which signals to collect next.

Sixty apps¶

greenhouse-attacks inference.py runs this against generated installs rather than a real dataset, because no open dataset pairs app lists with a sensitive attribute. The Facebook likes behind that claim in the literature came through the myPersonality app, which collected them between 2007 and 2012. The app was discontinued that year and banned from Facebook in 2018 when its developers would not agree to an audit, and the corpus went with it.

Generating the data buys something a real corpus cannot. The generator knows the coefficients that produced the labels, so the run tests two things at once: whether the attribute can be predicted, and whether the fitted coefficients match the ones that created it. Across 50,000 people and 60 apps the fitted model recovers them at a correlation of 0.997:

No single app gives the answer away. The loudest one shares 0.087 bits of mutual information with the attribute and the average app 0.008, against an attribute the generator draws balanced, so its entropy is exactly one bit and the comparison is like for like.

Weak signals can still be useful together, and that accumulation is measured here rather than assumed. The measure below is the area under the ROC curve, where 0.5 is a coin and 1.0 is never wrong:

One app is a coin. Sixty weak signals, no one of them decisive, put the attribute within reach.

The app names are placeholders and the effects were planted by the generator, which also sets the conditions the mechanism runs under. The run shows the inference working in those conditions. It says nothing about whether any real application predicts any real attribute.

Useful without being right¶

A model does not need to be right about any particular person to be useful at population scale. A ranking that separates groups more often than not is enough to aim advertising at, or to sort people onto lists, even where a great many of the individual predictions inside it are wrong. A person in that population installed the apps they found useful. They did not provide their political views.

The same pattern can be pointed at other sensitive attributes: health conditions, pregnancy, sexual orientation, financial standing, and anything else nobody records directly while its proxies sit in plain view. In each case the sensitive attribute is not directly observable and the proxy data is.

The Cambridge Analytica affair made this class of data use widely visible. The Information Commissioner’s investigation records the research behind it, in which likes, shares and posts were matched against personality scores, and notes the academics’ claim that as few as 68 Facebook likes sufficed to predict attributes including ethnicity and political affiliation. What that research showed and what the company later deployed are separate questions, and the report is a source for the first. Sensitivity lay in the combination, not in any one like.

Installing less¶

Whatever an app cannot observe cannot become a feature. Android filters package visibility by default for modern apps, and Apple’s tracking controls govern cross-company tracking and access to the advertising identifier, so the platforms now withhold some of what an inference model would want. More fundamentally, installing fewer applications leaves fewer installs for anything watching to count. An app installed for occasional convenience becomes one more feature in a profile its user never intended to create.

Last reviewed: 2026-09-19.