LFI Conf 23 | Christine Jefferies | The role of monitoring in proactive safety and performance mgmt

Christine Jefferies, PhD Candidate, The Ohio State University

The role of monitoring in proactive safety and performance management

This talk will discuss the differences among reactive, responsive, and proactive approaches to safety and performance monitoring. In many safety critical domains, monitoring of the process being managed is a core function to ensuring safe operations. While monitoring in the software engineering domain typically looks at technical performance of the system, in this context we include the socio-technical system, that is, monitoring the work being done and monitoring the system's ability to deliver work.

The monitoring capability of a proactive management system must be informed by at least three types of monitoring (Balkin, 2023). Firstly, capability must exist to monitor for changes in the system, which goes beyond monitoring technical performance to include, through reflective system analysis, monitoring for the violation of requirements and assumptions made during system design. Secondly, the socio-technical system must monitor the ability of the system to detect, interpret, and communicate changes to decision-making agents, which includes continuous confirmatory testing that the sampling process would in fact collect and distribute desired and useful information regarding changes to the appropriate parts of the system. Thirdly, monitoring system adaptive capacity, which Woods & Rayo (2022, p. 16) define as “the potential for adjusting patterns of activities to handle future changes in the kinds of events, opportunities and disruptions experienced." Finally, forecasting the system’s ability to respond to the types, frequency, and severity of changes and challenges detected by the monitoring system would require an additional meta-monitoring capability to detect and sensemake patterns of how adaptive capacity is created, preserved, and degraded.

The talk draws on the existing literature base, as well as my ongoing research in the domains of healthcare, aviation, and defense acquisitions. In it, we will explore requirements and design considerations for proactive performance monitoring programs as well as various monitoring indicators to support a proactive approach, to help inform the field of software engineering.

Learning from Incidents (LFI) is a community challenging conventional views and reshaping how the software industry thinks about incidents, software reliability, and the critical role people play in keeping their systems running.In today’s economy, software organizations can’t afford to not learn from incidents.

LFI Conference is made possible by the financial and planning support of the Jeli team. Nora Jones, Founder and CEO of Jeli, founded the LFI community and website as a way to show organizations how to get more ROI out of their most powerful investments -- their incidents.