Signals before Single Pane
There is a mantra in the silicon valley; “Measure, or it’s not meaningful.” But what does this mean in practice?
It assumes a reliable source: such as measuring profit or loss. The data source is a bank, and the measure is a simple calculation.
How to Measure the Metrics that Matter, By Chris Brahm, Mark Kovac, and Peter Guarraia, covers this topic well. It boils down to signals processed into metrics. This creates two important questions.
Where and how do we obtain the best signals?
Are the chosen signals designed to answer the question?
Real success comes from asking another question. What do we want to discover about our solution, product, service, or company?
Asking the question a different way, you create a different outcome. Today, business tends to measure from obvious and readily available data. In the worst case, they take what ever data they can find and convert it into pretty pictures for management to consume.
A better method is to model what you want to know and find ways to create signals. This gives you two new avenues.
In band, and out of band. In-band data can be something like S.M.A.R.T data on a hard drive. Out of band data can be something like sensors in the case to monitor vibration and other external factors.
Traditionally we look for data, such as S.M.A.R.T. to convert into information or knowledge. An entire industry relies on this data knowing it’s limits. So if data is a critical building block, why do we accept limits like this? Cost.
Cost tends to drive every discussion when the cry for information rises — setting up a comprehensive system after the fact is costly. Sometimes impossible. We accept data from anything that will send it to answer the question at hand or solve the problem discovered. If the solution becomes complex, such as an out of band solution to verify, validate or simply discover new signals, the answer is normally work with what you have and do the best you can.
A better solution is defining our outcomes and invest early in signal definition. Correct signals generate data early and often, allowing us to decide what data works for our knowledge solution. Who doesn’t want a choice to make better decisions early and often.
The single pane of glass is the holy grail of knowledge management. Management has mostly subscribed to this quest, and companies have made billions being the Sherpa for them. After all this investment in human and monetary capital, who wants to say it’s fundamentally broken? No one.
Examples are everywhere. BITS covered it well for cybersecurity, and F5 covered the issues for networking. Another Medium author on Myplanet created more musings on the topic. There are literally thousands of examples. They all conclude the same thing; not yet and maybe never.
Why is this a problem? Data is not the problem. Signal generation is. In cybersecurity and networking, the core problem is speeds and feeds. No one can capture the full flow of information through a network. North to South, maybe. East to West, no. There are technology and vendor limits to the equipment or data supplied. Data is not always available or accurate. Most importantly, we ask the questions long after the solutions are in place.
Having designed more than a few machine learning / AI security products and infrastructures, I can tell you everyone cherry-picks signal sources as data. Or, they find what they can to manage. You can use the most exotic of technologies to filter what you want to work on, but you are still filtering. You can acquire all the data available, but still have no secondary source-sideband of signals to improve your resolution. If you are forced to compromise, you have compromised everything.
What’s the solution? Define a process, product, or company with signals built in. Be focused on execution with an eye to the future. Signals that can be generated early, should be included. If you find this to expensive, difficult, or simply not clear at the time, go ahead and use my now, near, or never list to tack where you are and plan for the future you know is coming.
That’s it, use it, be decisive.
