Operational Data Fusion
How do you make different entities sharing the same network communicate with one another when each entity relies on different protocols and data types? To resolve that issue, Rafael has developed the Philosopher's Stone, a real-time operational intelligence system
Ami Rojkes Dombe
| 04/05/2018
Illustration: Bigstock
The trend of military organizations adopting Big Data has intensified over the last few years owing to the fact that armed forces worldwide are interested in switching to Network Centric Warfare (NCW). States and defense industries are investing substantial resources in the development of communication, command and control systems based on which platforms, troops, and weapon systems may be linked together in a single network enabling a whole military organization to be commanded with a key click.
One of the challenges associated with the attempt to link multiple elements in a single network is the "flattening" of the data. How can you make so many entities sharing the same network to communicate with one another when each entity relies on different protocols and data types? To resolve this gap, Rafael Advanced Defense Systems has developed the system designated The Philosopher's Stone. According to sources at Rafael, this system supports dozens of different databases and is capable of converting all data types into a single common language.
Systems for the World of Intelligence
"This platform enables the user to take an extensive range of structured and unstructured data from an extensive range of databases and data types, then flatten and index them," explains Yodi B., head of Rafael's Big Data line. "This capability enables optimal data fusion. You concentrate all of the data in a single data pool, then run multiple algorithms on that pool."
The Philosopher's Stone system has admitted Rafael into a very small market of industries, worldwide, capable of consolidating data from a variety of platforms for the benefit of the military world and the intelligence world. One of the most famous companies in this exclusive club is the American company Palantir, known for its close association with the US intelligence agencies.
"The Philosopher's Stone is a real-time operational intelligence system," explains B. "It enables analysts and operations specialists to analyze the data available at the same time. If you want to catch a certain person at an airport, the coupling of the analyst and the operations specialist around the same data in real time, using the same language, will enable you to close the loop promptly."
The people at Rafael explain that the system is a data fusion platform of sorts. Whereas defense and commercial clients worldwide are unwilling to give up the assets they had already acquired, then in most cases, integration for an existing setup of information systems will be required. "The key word is open architecture," says B. "The client wants you to provide him with a data fusion and database consolidation capability, but he does not want you to see his data. We can provide them with the capabilities without gaining access to the data."
In order to apply this consolidation capability to existing systems, the system is implemented on the basis of micro-services. Simplified, it is an open architecture that enables small units of software to communicate with one another through an API interface. In this way, a substantial, complex process may be broken down into small units to enable prompt integration with existing software systems.
Another capability the Philosopher's Stone system delivers to the user's desk is the option of allowing a large number of users to work on the system simultaneously. In a military organization or an intelligence agency, hundreds or even thousands of analysts may be working simultaneously, with each one running different inquiries opposite different databases. The ability to handle thousands of users and substantial volumes of data simultaneously is a challenge for data fusion systems.
"The Philosopher's Stone system provides the organization with a growth option and with the ability to handle substantial amounts of data. We can handle petabytes of data, store and process those data in real time," says B. "The system is sold to states and commercial organizations. One should bear in mind that there are commercial organizations around the world that produce more data than states. One international bank, where the system is installed, produces terabytes of data each year. It is also obligated by regulation to keep the data for years retrospectively. In the context of the number of users, we developed a modular product. The client decides how much hardware and bandwidth he wants to allocate to the system, and the number of users will be determined accordingly. There are clients who have thousands and tens of thousands of users."
Between the Security Realm & the Civilian Realm
At Rafael, they also have medium and small commercial businesses in mind and understand that not everyone has the option of acquiring a substantial system. "We have identified a trend of uploading data to the cloud, so our development effort is headed in that direction, too – a system as a service," explains B. "There is a lot of similarity in the analysis of data between the security realm and the civilian realm. We have realized that many of the things we developed are suitable for the civilian market, too."
Some parts of the Philosopher's Stone system drew their inspiration from internal solutions by Rafael in a format that would suit clients with similar needs around the world. "It took us years to develop this system," says B. "Initially, we worked with dozens of terabytes, and later on we expanded. Our internal IT organ participated in the development process. This is an excellent example of the way in which an internal IT organ can contribute to the company's revenue. The system underwent a process of adaptation to the civilian market vis-à-vis the Ministry of Defense, and today the solution is not under supervision, which enables us to compete with it anywhere in the world."
How do you make different entities sharing the same network communicate with one another when each entity relies on different protocols and data types? To resolve that issue, Rafael has developed the Philosopher's Stone, a real-time operational intelligence system
Illustration: Bigstock
The trend of military organizations adopting Big Data has intensified over the last few years owing to the fact that armed forces worldwide are interested in switching to Network Centric Warfare (NCW). States and defense industries are investing substantial resources in the development of communication, command and control systems based on which platforms, troops, and weapon systems may be linked together in a single network enabling a whole military organization to be commanded with a key click.
One of the challenges associated with the attempt to link multiple elements in a single network is the "flattening" of the data. How can you make so many entities sharing the same network to communicate with one another when each entity relies on different protocols and data types? To resolve this gap, Rafael Advanced Defense Systems has developed the system designated The Philosopher's Stone. According to sources at Rafael, this system supports dozens of different databases and is capable of converting all data types into a single common language.
Systems for the World of Intelligence
"This platform enables the user to take an extensive range of structured and unstructured data from an extensive range of databases and data types, then flatten and index them," explains Yodi B., head of Rafael's Big Data line. "This capability enables optimal data fusion. You concentrate all of the data in a single data pool, then run multiple algorithms on that pool."
The Philosopher's Stone system has admitted Rafael into a very small market of industries, worldwide, capable of consolidating data from a variety of platforms for the benefit of the military world and the intelligence world. One of the most famous companies in this exclusive club is the American company Palantir, known for its close association with the US intelligence agencies.
"The Philosopher's Stone is a real-time operational intelligence system," explains B. "It enables analysts and operations specialists to analyze the data available at the same time. If you want to catch a certain person at an airport, the coupling of the analyst and the operations specialist around the same data in real time, using the same language, will enable you to close the loop promptly."
The people at Rafael explain that the system is a data fusion platform of sorts. Whereas defense and commercial clients worldwide are unwilling to give up the assets they had already acquired, then in most cases, integration for an existing setup of information systems will be required. "The key word is open architecture," says B. "The client wants you to provide him with a data fusion and database consolidation capability, but he does not want you to see his data. We can provide them with the capabilities without gaining access to the data."
In order to apply this consolidation capability to existing systems, the system is implemented on the basis of micro-services. Simplified, it is an open architecture that enables small units of software to communicate with one another through an API interface. In this way, a substantial, complex process may be broken down into small units to enable prompt integration with existing software systems.
Another capability the Philosopher's Stone system delivers to the user's desk is the option of allowing a large number of users to work on the system simultaneously. In a military organization or an intelligence agency, hundreds or even thousands of analysts may be working simultaneously, with each one running different inquiries opposite different databases. The ability to handle thousands of users and substantial volumes of data simultaneously is a challenge for data fusion systems.
"The Philosopher's Stone system provides the organization with a growth option and with the ability to handle substantial amounts of data. We can handle petabytes of data, store and process those data in real time," says B. "The system is sold to states and commercial organizations. One should bear in mind that there are commercial organizations around the world that produce more data than states. One international bank, where the system is installed, produces terabytes of data each year. It is also obligated by regulation to keep the data for years retrospectively. In the context of the number of users, we developed a modular product. The client decides how much hardware and bandwidth he wants to allocate to the system, and the number of users will be determined accordingly. There are clients who have thousands and tens of thousands of users."
Between the Security Realm & the Civilian Realm
At Rafael, they also have medium and small commercial businesses in mind and understand that not everyone has the option of acquiring a substantial system. "We have identified a trend of uploading data to the cloud, so our development effort is headed in that direction, too – a system as a service," explains B. "There is a lot of similarity in the analysis of data between the security realm and the civilian realm. We have realized that many of the things we developed are suitable for the civilian market, too."
Some parts of the Philosopher's Stone system drew their inspiration from internal solutions by Rafael in a format that would suit clients with similar needs around the world. "It took us years to develop this system," says B. "Initially, we worked with dozens of terabytes, and later on we expanded. Our internal IT organ participated in the development process. This is an excellent example of the way in which an internal IT organ can contribute to the company's revenue. The system underwent a process of adaptation to the civilian market vis-à-vis the Ministry of Defense, and today the solution is not under supervision, which enables us to compete with it anywhere in the world."
