The future of cyber security: autonomous system of systems

Taking over the bulk of human tasks

‘In the race against malicious actors in the digital domain, humans are a complicating factor. In Security Operations Centres, where computer and network activity in an organisation is monitored, it’s still mainly people who are at the controls. But there’s so much coming at them that we have to accept that security systems are going to become autonomous.

This goes a step further than automation, because autonomy means that, step by step, machines are going to be taking over people’s assessment tasks to a great extent,’ says Berry Vetjens, Market Director ICT, Strategy, and Policy at TNO.

Self-learning and self-repairing

The response devised by TNO to the increasing number and intensity of cyber attacks has been dubbed ‘Athena’. It is a self-learning and self-repairing system that can perform a wide range of security tasks simultaneously.

Take, for example, a ransomware attack on a large energy company that invades its internal systems and threatens to bring down not only its operations, but also the energy supply to hundreds of thousands of customers.

The planned Athena system would detect the intrusion at a very early stage; assess at lightning speed what measures are most effective to protect the company, customers, and other affected parties; and succeed in repelling the attack without the customer even noticing, while the business processes remained intact.

Systems that no longer involve humans

‘Many people are worried by the idea of handing over ever more human thoughts and actions to autonomous systems by means of AI, but the uncomfortable truth is that not doing so is perhaps even more worrying specifically when it comes to cybersecurity,’ says Berry.

‘Large companies with critical infrastructure are becoming aware that autonomous cyber security is inevitable. It may start with the frontrunners in cyber security, such as financial institutions, but soon it will also be used in other sectors. These will not only be vulnerable sectors such as healthcare, high tech, the manufacturing industry, national and local government, but – in time – SMEs as well.

Systems that no longer involve humans are on their way and it’s time for us to work with scientific partners, industry, and government to design these systems. There’s no time to lose.’

Changing the rules during the game

‘We chose the term checkmate with a question mark as the title of the paper because it’s an extremely complex game against invisible and often elusive opponents. You’re playing chess against computers, so to speak.

Twenty years after Deep Blue, IBM’s chess computer that played against Kasparov, Google subsidiary DeepMind launched the chess engine AlphaZero, delivering unprecedented performance through the application of AI and machine learning. That said, programming chess and other games is easy in a way because the rules of the game are laid down. In cyber security, the rules keep changing during the game and our opponents constantly flout the rules. That’s what makes it all so complex.’

Going further than has ever been imagined

Cyber security operators are like chess players in that it all comes down to speed, the ability to process ever-increasing amounts of dynamic data from a growing number of sources, and precision. The huge amount of data needed to detect an attack in time and the speed required to respond appropriately have now become far too much for the human brain to handle.

‘Athena will be revolutionary. With it, we’re going further than we ever imagined we would. We’ve always had the idea that automation would involve joint control by humans and computers. “Human in the loop” is making way for “human before the loop”. Humans design the system, which then thinks and acts on its own. However, this must be done with the utmost care,’ Berry says.

Ethical, legal and societal

According to Berry, the process is about much more than technology alone, complicated as that already is. From an ethical point of view, the system must meet high values laid down in consultation between experts, policymakers, and stakeholders. This must also involve accountability. Processing a lot of data affects people’s privacy.

There must be legally binding agreements on this. Societally, there are issues of accessibility for SMEs, healthcare, and local authorities. It must be ensured that it is not only large companies that can afford autonomous cyber security. The question of who will be assigned responsibility for what still requires much discussion as well.

Combining technologies and interests

‘Designing such a system of systems is incredibly complex. You have to combine different technologies related to cyber security and AI, address legal and ethical issues, and reconcile the interests of diverse public and private parties. With our knowledge in all these areas, we want to give clear direction to this process in the coming period,’ Berry concludes.