Third, plan for continuous improvement. Odds are high that most of the underlying systems monitored will evolve over time for one reason or another. In parallel, the monitoring system must evolve to continue meeting expectations.
A monitoring system that can only handle 10Mb/s will face a virtually impossible task if the underlying system is upgraded to gigabit speeds and it can't sample the data fast enough. Furthermore, these systems must be reviewed over time to ensure that they still align with operator requirements. For example, filters may need to be added or modified in order to screen out unnecessary "noise" that the operators are contending with that didn't initially exist (providing proper analysis is done to determine why the noise exists of course).
Fourth, treat monitoring as an important activity and have the appropriate engineering resources and processes, such as change advisory boards set up to review, approve and schedule changes.
Monitoring must evolve from a haphazard afterthought to a critical application with specified service levels identifying timeliness, accuracy, uptime and security. For all monitoring, and especially SCADA systems, there must be effective communication between functional groups to ensure the systems are designed, secured and maintained appropriately.
Summary
Complex systems require increasingly sophisticated monitoring systems. Care must be taken to design secure systems that meet requirements and are perceived as accurate by the operators.
If a monitoring system is perceived as not adding value, operators will depend on it less and less. This, in turn, creates a fertile environment for security breaches, accidents and all types of inefficiencies. With that in mind, monitoring systems must consistently evolve from afterthoughts to well engineered systems to ensure expectations are met.