The Label Problem in Industrial Predictive Maintenance

#Abstract

Supervised failure prediction on industrial assets assumes a labelled dataset that well-maintained equipment does not produce. This report characterises four distinct sources of label corruption, sets out three practical responses and the validation evidence each can support, ordered by the author's experience rather than by measurement on a common dataset, and argues that reporting a point accuracy for a failure model trained on a handful of events is misleading regardless of the value.

#The premise that fails

Given a dataset of asset states with failures marked, predicting failure is a conventional supervised problem. On operating industrial data that dataset does not exist, and the reasons are structural rather than a matter of data collection effort.

#Four sources of corruption

Rarity by design. A maintenance programme exists to prevent failures. A facility with an effective programme produces few positive examples, and those it produces are heterogeneous in mechanism.

Censored positives. A technician observes a developing fault and intervenes. The failure does not occur and the record shows routine maintenance. A positive example is recorded as a negative, and the rate of this is unknown.

Timestamp displacement. Work orders carry the date they were raised or closed rather than the onset of the condition. The displacement can be weeks, which matters when aligning a label to a sensor window.

Unstructured description. The actual failure mechanism is in free text written by many people over decades, with inconsistent vocabulary.

The second is the most damaging, because it is a systematic bias toward the negative class that is invisible in the data and correlated with exactly the conditions a model should detect.

#Three responses

Change the target. Predict a measurable proxy with adequate positive examples, such as an operating parameter crossing a threshold or a derived health indicator changing. Less directly valuable and it produces a model that can be validated, which is the point.

Anomaly detection. Model normal operation and flag deviation. This requires far less labelled data and detects change rather than predicting failure. It generates false positives, which are tolerable only where review cost is low.

Reconstruct labels. Work with maintenance staff to rebuild failure histories from work orders, comments and recollection. Slow, unglamorous, and the highest value activity on such a project. That data work dominates practical machine learning effort is well documented [2] [1].

#Drift is structural here

Industrial equipment is maintained, modified and replaced, and operating conditions vary with season, product and demand. A model trained on last year describes a system that no longer exists in the same form, which is concept drift in its standard sense [3].

Monitoring input distribution is therefore mandatory rather than advisable, and it is the leading indicator, because output quality metrics require ground truth that arrives late or never.

#On reported accuracy

A model claiming to predict failure thirty days ahead, validated against a small number of historical events, carries very wide uncertainty on that claim. Reporting a single accuracy figure invites a decision the evidence does not support.

The computational science community has been direct that scaling from individually calibrated models to fleets requires uncertainty quantification that largely does not yet exist [5] [6]. The practical consequence adopted here is that a model output presented to an operator carries an interval, and a model that cannot produce one is not presented.

It is worth noting that the original per-unit formulation in the aerospace structural life literature made its value proposition concrete in a way most industrial projects do not: replacing a fleet-average decision with a per-unit decision, measurable in inspection intervals [7]. Projects that cannot state their decision in those terms tend to struggle to demonstrate return.

#Limitations

The four corruption sources are drawn from work on a limited number of facilities and are not claimed to be exhaustive or to have representative frequencies. A quantitative study of censored positives, which would require access to maintenance records and staff recollection across several sites, would be the most useful follow-up and has not been performed.

No comparison of the three responses on a common dataset is offered, so the ordering reflects the author's experience rather than measurement [4].