6. Transports

eProsima Fast RTPS implements an architecture of pluggable transports. Current version implements five transports: UDPv4, UDPv6, TCPv4, TCPv6 and SHM (shared memory). By default, when a Participant is created, one built-in UDPv4 transport is configured.

You can add custom transports using the attribute rtps.userTransports.

C++

    //Create a descriptor for the new transport.
    auto custom_transport = std::make_shared<UDPv4TransportDescriptor>();
    custom_transport->sendBufferSize = 9216;
    custom_transport->receiveBufferSize = 9216;

    //Disable the built-in Transport Layer.
    participant_attr.rtps.useBuiltinTransports = false;

    //Link the Transport Layer to the Participant.
    participant_attr.rtps.userTransports.push_back(custom_transport);

XML

<transport_descriptors>
    <transport_descriptor>
        <transport_id>my_transport</transport_id>
        <type>UDPv4</type>
        <sendBufferSize>9216</sendBufferSize>
        <receiveBufferSize>9216</receiveBufferSize>
    </transport_descriptor>
</transport_descriptors>

<participant profile_name="my_transport">
    <rtps>
        <userTransports>
            <transport_id>my_transport</transport_id>
        </userTransports>
        <useBuiltinTransports>false</useBuiltinTransports>
    </rtps>
</participant>

All Transport configuration options can be found in the section Transport descriptors.

6.1. Shared memory Transport (SHM)

The shared memory transport enables fast communications between entities running in the same processing unit/machine, relying on the shared memory mechanisms provided by the host operating system.

SHM transport provides better performance than other transports like UDP / TCP, even when these transports use loopback interface. This is mainly due to the following reasons:

  • Large message support: Network protocols need to fragment data in order to comply with the specific protocol and network stacks requirements. SHM transport allows the copy of full messages where the only size limit is the machine’s memory capacity.

  • Reduce the number of memory copies: When sending the same message to different endpoints, SHM transport can directly share the same memory buffer with all the destination endpoints. Other protocols require to perform one copy of the message per endpoint.

  • Less operating system overhead: Once initial setup is completed, shared memory transfers require much less system calls than the other protocols. Therefore there is a performance/time consume gain by using SHM.

When two participants on the same machine have SHM transport enabled, all communications between them are automatically performed by SHM transport only. The rest of the enabled transports are not used between those two participants.

To enable SHM transport in a Participant, you need to add the SharedMemTransportDescriptor to the rtps.userTransports attribute (C++ code) or define a transport_descriptor of type SHM in the XML file (see examples below).

C++

    // Create a descriptor for the new transport.
    std::shared_ptr<SharedMemTransportDescriptor> shm_transport = std::make_shared<SharedMemTransportDescriptor>();

    // Link the Transport Layer to the Participant.
    participant_attr.rtps.userTransports.push_back(shm_transport);

XML

<transport_descriptors>
    <!-- Create a descriptor for the new transport -->
    <transport_descriptor>
        <transport_id>shm_transport</transport_id>
        <type>SHM</type>
    </transport_descriptor>
</transport_descriptors>

<participant profile_name="SHMParticipant">
    <rtps>
        <!-- Link the Transport Layer to the Participant -->
        <userTransports>
            <transport_id>shm_transport</transport_id>
        </userTransports>
    </rtps>
</participant>

SHM configuration parameters:

  • segment_size: The size of the shared memory segment in bytes. A shared memory segment is created by each participant. Participant’s writers copy their messages into the segment and send a message reference to the destination readers.

  • port_queue_capacity: Each participant with SHM transport enabled listens on a queue (port) for incoming SHM message references. This parameter specifies the queue size (in messages).

  • healthy_check_timeout_ms: With SHM, Readers and writers use a queue to exchange messages (called Port). If one of the processes involved crashes while using the port, the structure can be left inoperative. For this reason, every time a port is opened, a healthy check is performed. If the attached listeners don’t respond in healthy_check_timeout_ms milliseconds, the port is destroyed and created again.

  • rtps_dump_file: Full path, including the file name, of the protocol dump_file. When this string parameter is not empty, all the participant’s SHM traffic (sent and received) is traced to a file. The output file format is tcpdump text hex, and can be read with protocol analyzer applications such as Wireshark.

6.2. TCP Transport

Unlike UDP, TCP transport is connection oriented and for that Fast-RTPS must establish a TCP connection before sending the RTPS messages. Therefore TCP transport can have two behaviors, acting as a server (TCP Server) or as a client (TCP Client). The server opens a TCP port listening for incoming connections and the client tries to connect to the server. The server and the client concepts are independent from the RTPS concepts: Publisher, Subscriber, Writer, and Reader. Any of them can operate as a TCP Server or a TCP Client because these entities are used only to establish the TCP connection and the RTPS protocol works over it.

To use TCP transports you need to define some more configurations:

You must create a new TCP transport descriptor, for example TCPv4. This transport descriptor has a field named listening_ports that indicates to Fast-RTPS in which physical TCP ports our participant will listen for input connections. If omitted, the participant will not be able to receive incoming connections but will be able to connect to other participants that have configured their listening ports. The transport must be added to the userTransports list of the participant attributes. The field wan_addr can be used to allow incoming connections using the public IP in a WAN environment or the Internet. See WAN or Internet Communication over TCP/IPv4 for more information about how to configure a TCP Transport to allow or connect to WAN connections.

C++

    //Create a descriptor for the new transport.
    auto tcp_transport = std::make_shared<TCPv4TransportDescriptor>();
    tcp_transport->add_listener_port(5100);
    tcp_transport->set_WAN_address("80.80.99.45");

    //Disable the built-in Transport Layer.
    participant_attr.rtps.useBuiltinTransports = false;

    //Link the Transport Layer to the Participant.
    participant_attr.rtps.userTransports.push_back(tcp_transport);

XML

<transport_descriptors>
    <transport_descriptor>
        <transport_id>tcp_transport</transport_id>
        <type>TCPv4</type>
        <listening_ports>
            <port>5100</port>
        </listening_ports>
        <wan_addr>80.80.99.45</wan_addr>
    </transport_descriptor>
</transport_descriptors>

<participant profile_name="TCPParticipant">
    <rtps>
        <userTransports>
            <transport_id>tcp_transport</transport_id>
        </userTransports>
        <useBuiltinTransports>false</useBuiltinTransports>
    </rtps>
</participant>

To configure the participant to connect to another node through TCP, you must configure and add a Locator to its initialPeersList that points to the remote listening port.

C++

    auto tcp2_transport = std::make_shared<TCPv4TransportDescriptor>();

    //Disable the built-in Transport Layer.
    participant_attr.rtps.useBuiltinTransports = false;

    //Set initial peers.
    Locator_t initial_peer_locator;
    initial_peer_locator.kind = LOCATOR_KIND_TCPv4;
    IPLocator::setIPv4(initial_peer_locator, "80.80.99.45");
    initial_peer_locator.port = 5100;
    participant_attr.rtps.builtin.initialPeersList.push_back(initial_peer_locator);

    //Link the Transport Layer to the Participant.
    participant_attr.rtps.userTransports.push_back(tcp2_transport);

XML

<transport_descriptors>
    <transport_descriptor>
        <transport_id>tcp2_transport</transport_id>
        <type>TCPv4</type>
    </transport_descriptor>
</transport_descriptors>

<participant profile_name="TCP2Participant">
    <rtps>
        <userTransports>
            <transport_id>tcp2_transport</transport_id>
        </userTransports>
        <builtin>
            <initialPeersList>
                <locator>
                    <tcpv4>
                        <address>80.80.99.45</address>
                        <physical_port>5100</physical_port>
                    </tcpv4>
                </locator>
            </initialPeersList>
        </builtin>
        <useBuiltinTransports>false</useBuiltinTransports>
    </rtps>
</participant>

A TCP version of helloworld example can be found in this link.

6.2.1. WAN or Internet Communication over TCP/IPv4

Fast-RTPS is able to connect through the Internet or other WAN networks when configured properly. To achieve this kind of scenarios, the involved network devices such as routers and firewalls should add the rules to allow the communication.

For example, to allow incoming connections through our NAT, Fast-RTPS must be configured as a TCP Server listening to incoming TCP connections. To allow incoming connections through a WAN, the TCP descriptor associated must indicate its public IP through its field wan_addr.

C++

    //Create a descriptor for the new transport.
    auto tcp_transport = std::make_shared<TCPv4TransportDescriptor>();
    tcp_transport->add_listener_port(5100);
    tcp_transport->set_WAN_address("80.80.99.45");

    //Disable the built-in Transport Layer.
    participant_attr.rtps.useBuiltinTransports = false;

    //Link the Transport Layer to the Participant.
    participant_attr.rtps.userTransports.push_back(tcp_transport);

XML

<transport_descriptors>
    <transport_descriptor>
        <transport_id>tcp_transport</transport_id>
        <type>TCPv4</type>
        <listening_ports>
            <port>5100</port>
        </listening_ports>
        <wan_addr>80.80.99.45</wan_addr>
    </transport_descriptor>
</transport_descriptors>

<participant profile_name="TCPParticipant">
    <rtps>
        <userTransports>
            <transport_id>tcp_transport</transport_id>
        </userTransports>
        <useBuiltinTransports>false</useBuiltinTransports>
    </rtps>
</participant>

In this case, configuring the router (which public IP is 80.80.99.45) is mandatory to allow the incoming traffic to reach the TCP Server. Typically a NAT routing with the listening_port 5100 to our machine is enough. Any existing firewall should be configured as well.

In the client side, it is needed to specify the public IP of the TCP Server with its listening_port as initial_peer.

C++

    auto tcp2_transport = std::make_shared<TCPv4TransportDescriptor>();

    //Disable the built-in Transport Layer.
    participant_attr.rtps.useBuiltinTransports = false;

    //Set initial peers.
    Locator_t initial_peer_locator;
    initial_peer_locator.kind = LOCATOR_KIND_TCPv4;
    IPLocator::setIPv4(initial_peer_locator, "80.80.99.45");
    initial_peer_locator.port = 5100;
    participant_attr.rtps.builtin.initialPeersList.push_back(initial_peer_locator);

    //Link the Transport Layer to the Participant.
    participant_attr.rtps.userTransports.push_back(tcp2_transport);

XML

<transport_descriptors>
    <transport_descriptor>
        <transport_id>tcp2_transport</transport_id>
        <type>TCPv4</type>
    </transport_descriptor>
</transport_descriptors>

<participant profile_name="TCP2Participant">
    <rtps>
        <userTransports>
            <transport_id>tcp2_transport</transport_id>
        </userTransports>
        <builtin>
            <initialPeersList>
                <locator>
                    <tcpv4>
                        <address>80.80.99.45</address>
                        <physical_port>5100</physical_port>
                    </tcpv4>
                </locator>
            </initialPeersList>
        </builtin>
        <useBuiltinTransports>false</useBuiltinTransports>
    </rtps>
</participant>

The combination of the above configurations in both TCP Server and TCP Client allows a scenario similar to the represented by the following image.

../../_images/TCP_WAN.png

IPLocator

IPLocator is an auxiliary static class that offers methods to ease the management of IP based locators, as UDP or TCP. In TCP, the port field of the locator is divided into physical and logical port. The physical port is the port used by the network device, the real port that the operating system understands. The logical port can be seen as RTPS port, or UDP’s equivalent port (physical ports of UDP, are logical ports in TCP). Logical ports normally are not necessary to manage explicitly, but you can do it through IPLocator class. Physical ports instead, must be set to explicitly use certain ports, to allow the communication through a NAT, for example.

        Locator_t locator;
        // Get & Set Physical Port
        uint16_t physical_port = IPLocator::getPhysicalPort(locator);
        IPLocator::setPhysicalPort(locator, 5555);

        // Get & Set Logical Port
        uint16_t logical_port = IPLocator::getLogicalPort(locator);
        IPLocator::setLogicalPort(locator, 7400);

        // Set WAN Address
        IPLocator::setWan(locator, "80.88.75.55");

NOTE

TCP doesn’t support multicast scenarios, so you must plan carefully your network architecture.

6.2.2. TLS over TCP

Fast-RTPS allows configuring a TCP Transport to use TLS (Transport Layer Security) by setting up TCP Server and TCP Client properly.

TCP Server

C++

        auto tls_transport = std::make_shared<TCPv4TransportDescriptor>();

        using TLSOptions = TCPTransportDescriptor::TLSConfig::TLSOptions;
        tls_transport->apply_security = true;
        tls_transport->tls_config.password = "test";
        tls_transport->tls_config.cert_chain_file = "server.pem";
        tls_transport->tls_config.private_key_file = "serverkey.pem";
        tls_transport->tls_config.tmp_dh_file = "dh2048.pem";
        tls_transport->tls_config.add_option(TLSOptions::DEFAULT_WORKAROUNDS);
        tls_transport->tls_config.add_option(TLSOptions::SINGLE_DH_USE);
        tls_transport->tls_config.add_option(TLSOptions::NO_SSLV2);

XML

<transport_descriptors>
    <transport_descriptor>
        <transport_id>tls_transport_server</transport_id>
        <type>TCPv4</type>
        <tls>
            <password>test</password>
            <private_key_file>serverkey.pem</private_key_file>
            <cert_chain_file>server.pem</cert_chain_file>
            <tmp_dh_file>dh2048.pem</tmp_dh_file>
            <options>
                <option>DEFAULT_WORKAROUNDS</option>
                <option>SINGLE_DH_USE</option>
                <option>NO_SSLV2</option>
            </options>
        </tls>
    </transport_descriptor>
</transport_descriptors>

TCP Client

C++

        auto tls_transport = std::make_shared<TCPv4TransportDescriptor>();

        using TLSOptions = TCPTransportDescriptor::TLSConfig::TLSOptions;
        using TLSVerifyMode = TCPTransportDescriptor::TLSConfig::TLSVerifyMode;
        tls_transport->apply_security = true;
        tls_transport->tls_config.verify_file = "ca.pem";
        tls_transport->tls_config.verify_mode = TLSVerifyMode::VERIFY_PEER;
        tls_transport->tls_config.add_option(TLSOptions::DEFAULT_WORKAROUNDS);
        tls_transport->tls_config.add_option(TLSOptions::SINGLE_DH_USE);
        tls_transport->tls_config.add_option(TLSOptions::NO_SSLV2);

XML

<transport_descriptors>
    <transport_descriptor>
        <transport_id>tls_transport_client</transport_id>
        <type>TCPv4</type>
        <tls>
            <verify_file>ca.pem</verify_file>
            <verify_mode>
                <verify>VERIFY_PEER</verify>
            </verify_mode>
            <options>
                <option>DEFAULT_WORKAROUNDS</option>
                <option>SINGLE_DH_USE</option>
                <option>NO_SSLV2</option>
            </options>
        </tls>
    </transport_descriptor>
</transport_descriptors>

More TLS related options can be found in the section Transport descriptors.

6.3. Listening locators

eProsima Fast RTPS divides listening locators into four categories:

  • Metatraffic Multicast Locators: these locators are used to receive metatraffic information using multicast. They usually are used by built-in endpoints, like the discovery of built-in endpoints. You can set your own locators using attribute rtps.builtin.metatrafficMulticastLocatorList.

            // This locator will open a socket to listen network messages on UDPv4 port 22222 over multicast address 239.255.0.1
            eprosima::fastrtps::rtps::Locator_t locator;
            IPLocator::setIPv4(locator, 239, 255, 0, 1);
            locator.port = 22222;
    
            participant_attr.rtps.builtin.metatrafficMulticastLocatorList.push_back(locator);
    
  • Metatraffic Unicast Locators: these locators are used to receive metatraffic information using unicast. They usually are used by built-in endpoints, like the discovery of built-in endpoints. You can set your own locators using attribute rtps.builtin.metatrafficUnicastLocatorList.

            // This locator will open a socket to listen network messages on UDPv4 port 22223 over network interface 192.168.0.1
            eprosima::fastrtps::rtps::Locator_t locator;
            IPLocator::setIPv4(locator, 192, 168, 0, 1);
            locator.port = 22223;
    
            participant_attr.rtps.builtin.metatrafficUnicastLocatorList.push_back(locator);
    
  • User Multicast Locators: these locators are used to receive user information using multicast. They are used by user endpoints. You can set your own locators using attribute rtps.defaultMulticastLocatorList.

            // This locator will open a socket to listen network messages on UDPv4 port 22224 over multicast address 239.255.0.1
            eprosima::fastrtps::rtps::Locator_t locator;
            IPLocator::setIPv4(locator, 239, 255, 0, 1);
            locator.port = 22224;
    
            participant_attr.rtps.defaultMulticastLocatorList.push_back(locator);
    
  • User Unicast Locators: these locators are used to receive user information using unicast. They are used by user endpoints. You can set your own locators using attributes rtps.defaultUnicastLocatorList.

            // This locator will open a socket to listen network messages on UDPv4 port 22225 over network interface 192.168.0.1
            eprosima::fastrtps::rtps::Locator_t locator;
            IPLocator::setIPv4(locator, 192, 168, 0, 1);
            locator.port = 22225;
    
            participant_attr.rtps.defaultUnicastLocatorList.push_back(locator);
    

By default eProsima Fast RTPS calculates the listening locators for the built-in UDPv4 network transport using well-known ports. These well-known ports are calculated using the following predefined rules:

Ports used

Traffic type

Well-known port expression

Metatraffic multicast

PB + DG * domainId + offsetd0

Metatraffic unicast

PB + DG * domainId + offsetd1 + PG * participantId

User multicast

PB + DG * domainId + offsetd2

User unicast

PB + DG * domainId + offsetd3 + PG * participantId

These predefined rules use some values explained here:

  • DG: DomainId Gain. You can set this value using attribute rtps.port.domainIDGain.

  • PG: ParticipantId Gain. You can set this value using attribute rtps.port.participantIDGain. The default value is 2.

  • PB: Port Base number. You can set this value using attribute rtps.port.portBase. The default value is 7400.

  • offsetd0, offsetd1, offsetd2, offsetd3: Additional offsets. You can set these values using attributes rtps.port.offsetdN. Default values are: offsetd0 = 0, offsetd1 = 10, offsetd2 = 1, offsetd3 = 11.

Both UDP and TCP unicast locators support to have a null address. In that case, eProsima Fast RTPS understands to get local network addresses and use them.

Both UDP and TCP locators support to have a zero port. In that case, eProsima Fast RTPS understands to calculate well-known port for that type of traffic.

6.4. Initial peers

According to the RTPS standard (Section 9.6.1.1), each participant must listen for incoming Participant Discovery Protocol (PDP) discovery metatraffic in two different ports, one linked with a multicast address, and another one linked to a unicast address (see Discovery). Fast-RTPS allows for the configuration of an initial peers list which contains one or more such address-port pairs corresponding to remote participants PDP discovery listening resources, so that the local participant will not only send its PDP traffic to the default multicast address-port specified by its domain, but also to all the address-port pairs specified in the initial-peers list.

A participant’s initial peers list contains the list of address-port pairs of all other participants with which it will communicate. It is a list of addresses that a participant will use in the unicast discovery mechanism, together or as an alternative to multicast discovery. Therefore, this approach also applies to those scenarios in which multicast functionality is not available.

According to the RTPS standard (Section 9.6.1.1), the participants’ discovery traffic unicast listening ports are calculated using the following equation: 7400 + 250 * domainID + 10 + 2 * participantID. Thus, if for example a participant operates in Domain 0 (default domain) and its ID is 1, its discovery traffic unicast listening port would be: 7400 + 250 * 0 + 10 + 2 * 1 = 7412. By default eProsima Fast RTPS uses as initial peers the Metatraffic Multicast Locators.

The following constitutes an example configuring an Initial Peers list with one peer on host 192.168.10.13 with participant ID 1 in domain 0.

C++

    Locator_t initial_peers_locator;
    IPLocator::setIPv4(initial_peers_locator, "192.168.10.13");
    initial_peers_locator.port = 7412;
    participant_attr.rtps.builtin.initialPeersList.push_back(initial_peers_locator);

XML

<participant profile_name="initial_peers_example_profile" is_default_profile="true">
    <rtps>
        <builtin>
            <initialPeersList>
                <locator>
                    <udpv4>
                        <address>192.168.10.13</address>
                        <port>7412</port>
                    </udpv4>
                </locator>
            </initialPeersList>
        </builtin>
    </rtps>
</participant>

6.5. Whitelist Interfaces

There could be situations where you want to block some network interfaces to avoid connections or sending data through them. This can be managed using the field interface whitelist in the transport descriptors, and with them, you can set the interfaces you want to use to send or receive packets. The values on this list should match the IPs of your machine in that networks. For example:

C++

    UDPv4TransportDescriptor descriptor;
    descriptor.interfaceWhiteList.emplace_back("127.0.0.1");

XML

<transport_descriptors>
    <transport_descriptor>
        <transport_id>CustomTransport</transport_id>
        <type>UDPv4</type>
        <interfaceWhiteList>
            <address>127.0.0.1</address>
        </interfaceWhiteList>
    </transport_descriptor>
<transport_descriptor>

6.6. Tips

Disabling all multicast traffic

C++

    // Metatraffic Multicast Locator List will be empty.
    // Metatraffic Unicast Locator List will contain one locator, with null address and null port.
    // Then eProsima Fast RTPS will use all network interfaces to receive network messages using a well-known port.
    Locator_t default_unicast_locator;
    participant_attr.rtps.builtin.metatrafficUnicastLocatorList.push_back(default_unicast_locator);

    // Initial peer will be UDPv4 addresss 192.168.0.1. The port will be a well-known port.
    // Initial discovery network messages will be sent to this UDPv4 address.
    Locator_t initial_peer;
    IPLocator::setIPv4(initial_peer, 192, 168, 0, 1);
    participant_attr.rtps.builtin.initialPeersList.push_back(initial_peer);

XML

<participant profile_name="disable_multicast" is_default_profile="true">
    <rtps>
        <builtin>
            <metatrafficUnicastLocatorList>
                <locator/>
            </metatrafficUnicastLocatorList>
            <initialPeersList>
                <locator>
                    <udpv4>
                        <address>192.168.0.1</address>
                    </udpv4>
                </locator>
            </initialPeersList>
        </builtin>
    </rtps>
</participant>

Non-blocking write on sockets

For UDP transport, it is possible to configure whether to use non-blocking write calls on the sockets.

C++

    //Create a descriptor for the new transport.
    auto non_blocking_UDP_transport = std::make_shared<UDPv4TransportDescriptor>();
    non_blocking_UDP_transport->non_blocking_send = false;

    //Disable the built-in Transport Layer.
    participant_attr.rtps.useBuiltinTransports = false;

    //Link the Transport Layer to the Participant.
    participant_attr.rtps.userTransports.push_back(non_blocking_UDP_transport);

XML

<transport_descriptors>
    <transport_descriptor>
        <transport_id>non_blocking_transport</transport_id>
        <type>UDPv4</type>
        <non_blocking_send>false</non_blocking_send>
    </transport_descriptor>
</transport_descriptors>

<participant profile_name="non_blocking_transport">
    <rtps>
        <userTransports>
            <transport_id>non_blocking_transport</transport_id>
        </userTransports>
        <useBuiltinTransports>false</useBuiltinTransports>
    </rtps>
</participant>

XML Configuration

The XML profiles section contains the full information about how to setup Fast RTPS through an XML file.