I wanted to follow up after my previous note on messaging. We’ve been exploring this recently at 3DR and would like to expand the discussion. Our goal is to identify a solution to the issues we’ve experienced first hand, and to promote compatibility & interoperability.
Good question! Our primary concern is to resolve issues we have encountered in the messaging layer of our systems. A quick recap of the most important points:
- insufficient reliability/delivery & performance on lossy transports
- not able to easily discover devices/services
- not possible/easy to specify message delivery config per client (ie, different clients want different stream rates, different reliability settings, etc)
- don’t reinvent the wheel
We think these are quite general concerns and, as such, are good candidates for a general solution.
After exploring a variety of options, we've been focusing on RTPS (Real-Time Publish Subscribe) as a pub/sub and message delivery system. I've created a brief summary of RTPS here.
RTPS addresses the concerns above, and has eased many issues that we've encountered during the development of previous systems. We are focused on IP-based systems at the moment, given our immediate requirements, but RTPS can work on any transport capable of delivering datagrams.
To date, we’ve prototyped systems that show promise on both Linux and OS X host machines, embedded Linux targets, and mobile targets (iOS and Android). The implementations we have been using/consulting so far (sorry for the lack of direct links, but Discourse has limited the number of links I can post...):
- C++: Fast-RTPS, by eProsima, available on github
- C: freertps, by OSRF, available on github. specifically designed with the ability to run on MCU class devices)
- Java: jRTPS, available on sourceforge
The current lack of broader platform support (especially Python) is a concern, but I think it's possible and likely to improve over time.
Work on Mavlink 2
The Mavlink 2 proposal addresses some critical issues that we have definitely experienced on our mavlink based systems. These are important steps forward, and help ensure that the fleet of deployed mavlink-based vehicles continues to evolve and be well supported. However, it does not meet several of our goals above, which represent functionality that is important enough for us to consider an alternative solution.
As RTPS does not specify a payload format, we think of it mainly as a message delivery mechanism, corresponding roughly to the parts of mavlink outside of the
payload field of the message. There is an opportunity to identify a common payload format and message set, which appears to be aligned with Mavlink 2 efforts to add flexibility to the packet framing format.
More message definition detail in a follow up message!
What about DDS?
DDS is the middleware system selected for ROS2 and, as such, we’re quite interested in it. In our investigations, however, we did not encounter any DDS systems that were approachable enough that we felt confident we could meaningfully modify and contribute to.
In fact, RTPS is the underlying pub/sub system and wire protocol that powers DDS. It represents a simpler (though not totally simple!) implementation and interface, and provides the benefits that are relevant to our stated goals.
This page captures some of the ROS2 evaluation process. Specifically, I would call out the concluding paragraph as one that I agree with (and holds true for RTPS as well as DDS):
After working with DDS and having a healthy amount of skepticism about the ethos, community, and licensing, it is hard to come up with any real technical criticisms. While it is true that the community surrounding DDS is very different from the ROS community or the ZeroMQ community, it appears that DDS is just solid technology on which ROS could safely depend. There are still many questions about exactly how ROS would utilize DDS, but they all seem like engineering exercises at this point and not potential deal breakers for ROS.
We’ve been pleased with the results of our RTPS prototyping to date:
- Pub/sub with support for per-client QoS provides valuable flexibility in plumbing data through our systems
- Performance looks good
- Platform support is good enough for now
We plan on getting a prototype system up and running shortly that demonstrates the following via RTPS:
- telem and control between a mobile app (iOS and Android) and controller
- telem and RC control between a controller and drone
- onboard communication to and from the autopilot and onboard computer
We’ll share out details here as they become available.
As mentioned above, we’d be delighted to discuss the possibility of collaboration and/or interoperability. If you’re interested, or just have questions, please don’t hesitate to comment here or get in touch!