Efficient Messaging - Network Optional Warfare
All Projects
Efficient Messaging
Navy networks afloat are very different than networks ashore. Bandwidth is a precious and finite resource, latency can be huge, connectivity can be intermittent, environmental effects dominate, channels are limited in varying ways, and mobile relays are rare. Manned and unmanned naval systems need efficient messaging for networks afloat - but rarely have it. Failing to properly utilize communications capacity directly limits tactical effectiveness.
Efficient messaging is needed to take maximum advantage of severely constrained data links. The key to our strategies for achieving efficient messaging is first to use of Extensible Markup Language (XML) for structured data languages, and then use EXI for compressing XML. Since XML provides a flexible and validatable way to define regular data structures for any language, it provides a practical opportunity to compatibly capture and convert all manner of diverse data formats used for military messaging. The economics of Web technologies are undeniable and usually provide industry-wide best practices as well. As a result, this use of open standards is scalable and repeatable, avoiding the "stove pipes" which commonly prevent system-wide interoperability between Navy platforms and coalition partners.
"Efficiency" means both size and speed. EXI has demonstrated compaction that always meets or beats the most commonly used compression techniques (zip and gzip). Additionally, because EXI decompression goes straight into memory rather than string characters, which then require significant additional parsing, decoding EXI is many times faster than other techniques. This approach also reduces memory requirements and power consumption on small devices. Because Navy tactical traffic is usually highly structured and highly numeric, EXI provides major advantages that might well impact all afloat Navy communications. Alternative bit-centric compression schemes cannot take full advantage of those characteristics.
"Efficiency" is compatible with Data-Centric Security. Demonstrated thesis work has shown that digital signature (for authentication) and XML Encryption (privacy and access control) can coexist with efficient compression, when applied in the correct order. Such interoperability for Information Assurance (IA) Is necessary when working with coalition partners, and also for safeguarding data within deployed unmanned systems that are beyond the reach of network-centric security.
Efficient XML Interchange (EXI)
The World Wide Web Consortium (W3C) supervises these open standardization efforts with contributions by many companies, universities, agencies and individuals to continually improve these capabilities.
NPS student-driven research continues to provide significant explorations of XML-based compression and decompression techniques. Many opportunities are emerging.
- Article. Debich, Steven, Bruce Hill, Scot Miller, and Don Brutzman, "Being Efficient with Bandwidth," Professional Notes, U.S. Naval Institute Proceedings , volume 140 number 7, July 2014, pp. 76-78 (also original version). This article describes the potential role of compressed XML and the efficient use of bandwidth for afloat units.
-
Debich, Steven, The Role of Efficient XML Interchange (EXI) in Navy Wide-Area Network (WAN) Optimization, Master’s Thesis, Naval Postgraduate School, Monterey California, March 2015. Also available: catalog , slideset ( .pdf ), thesis.
- Hill, Bruce, Evaluation of Efficient XML Interchange (EXI) for Large Datasets and as an Alternative to Binary JSON Encodings, Master’s Thesis, Naval Postgraduate School, Monterey California, March 2015. Also available: catalog , slideset ( .pdf ), thesis .
- Snyder, Sheldon L., Efficient XML Interchange (EXI) Compression and Performance Benefits: Development, Implementation and Evaluation , Master’s Thesis, Naval Postgraduate School, Monterey California, March 2010. Also available: poster.
-
Williams, Jeffrey Scott, Document-based Message-centric Security using XML Authentication and Encryption for Coalition and Interagency Operations , Master’s Thesis, Naval Postgraduate School, Monterey California, September 2009.
Currently NPS participates in EXI working-group activities as liaison member for Web3D Consortium work improving the X3D Graphics Compressed Binary Encoding (CBE).
OpenEXI Open-Source Java Implementation
We support open Web standards and actively contribute to the OpenEXI project.
Prior NPS work
We have been interested in the potential of binary compression of XML for some time. When the World Wide Web Consortium (W3C) first held a workshop examining compression possibilities for XML, we contributed a paper documenting our design work. We're happy to report that the final W3C EXI Recommendation compatibly includes a large number of capabilities and requirements explored by our work on Cross-Format Schema Protocol XFSP (later renamed XML Schema-Based Compression, XSBC).
- W3C Workshop on Binary Interchange of XML Information Item Sets , 24-26 September 2003, Santa Clara California. Resources include Call for Participation and Workshop Report. Included:
-
Brutzman, Don and Don McGregor, XML Binary Serialization using Cross-Format Schema Protocol (XFSP) and XML Compression Considerations for Extensible 3D (X3D) Graphics , 24 September 2003, position paper.
-
- Serin, Ekrem, Design and Test of the Cross-Format Schema Protocol (XFSP) for Networked Virtual Environments , Master’s Thesis, Naval Postgraduate School, Monterey California USA, March 2003.
Much of our work was originally performed in order to improve data communications for modeling and simulation. It is extremely interesting to find that these design patterns are directly applicable to data streams for robotics, command & control (C2), and modeling & simulation (M&S) as well. Chronological list of key references follows.
- DeVos, Daniel A., XML Tactical Chat (XTC): the Way Ahead for Navy Chat, Master’s Thesis, Naval Postgraduate School, Monterey California USA, September 2007.
- Armold, Adrian D., XML Tactical Chat (XTC) Extensible Messaging and Presence Protocol for Command and Control Applications, Master’s Thesis, Naval Postgraduate School, Monterey California USA, September 2006.
- McGregor, Don, Don Brutzman, Adrian Armold, Curtis L. Blais, Mark Falash and Eytan Pollack, DIS-XML: Moving DIS to Open Data Exchange Standards, Simulation Interoperability Standards Organization (SISO) Simulation Interoperability Workshop (SIW), 2006.
-
Brutzman, D., McGregor, D., DeVos, D. A., Lee, C. S., Amsden, S., and Blais, C., et al., XML-Based Tactical Chat (XTC): Requirements, Capabilities and Preliminary Progress, Technical Report No. NPS-MV-2004-001, Naval Postgraduate School, Monterey California, 2004.
- Fischer, William D., Enhancing Network Communication in NPSNET-V Virtual Environments using XML-Described Dynamic Behavior Protocols (DBP) , Master's Thesis, Naval Postgraduate School, Monterey California, September 2001.
W3C Efficient XML Interchange (EXI) Working Group
This group is where the serious efforts and successes occur. No single company, even the biggest, has the ability to match the combined expertise of W3C experts from many companies and institutions working together within an open process.
W3C Working Group public page (and member page) XML Binary Characterization Working Group (predecessor effort)
Key references (among many):
- XML Binary Characterization Use Cases (especially Sensor Processing and Communication and Military Information Interoperability)
- XML Binary Characterization summarizing several years of preparatory work to clarify properties and metrics for proper measurement
- Efficient XML Interchange Measurements Note (especially Test Data Classification) compared candidate technologies prior to distilling the EXI design
-
Efficient XML Interchange Evaluation (especially Use Case Requirements diagram showing how design decisions reconciled a multitude of requirements)
- Efficient XML Interchange (EXI) Primer, W3C Working Draft
- XML in 10 Points provides an excellent overview of how and why XML works. Proposed addition:
Efficient XML Interchange (EXI) further improves these capabilities. Smaller compression enables bigger data on the Web, while faster decompression saves both memory and power consumption which can lead to the Internet of Things becoming the Web of Things.
More to follow
Stay tuned! Much work has been completed and more is under way. Detailed information and additional topics are being added on this site. Recommendations include
- Additional open-source/commercial EXI code bases and implementations
- Test suites and quantitative results for compaction (always best) and decompression speedup (7-12x faster)
- XML Security for coalition-friendly information assurance (IA):
- use of XML Encryption standard for information security, and
- use of XML Digital Signature standard for identity authentication.
- Test-suite measurements showing relevance to Military Information Interoperability use case
- Test-suite measurements showing relevance to Sensor Processing and Communication use case and Internet of Things (making it the Web of Things)
- Use of adaptive Forward Error Correction (FEC) to reduce the need for retransmission in noisy signaling environments
- Reimers, Stephen P, Towards Internet Protocol over seawater (IP/SW): forward error correction (FEC) using Hamming codes for reliable acoustic telemetry, Masters Thesis, Naval Postgraduate School, Monterey California, September 1995.
Contact
All questions and comments are welcome, please let us know what you think.