15.5.1. Tuning allocations

Allocating and deallocating memory implies some non-deterministic time consuming operations. Therefore, most real-time systems need to operate in a way that all dynamic memory is allocated during the application initialization, avoiding memory management operations in the main loop.

If users provide maximum sizes for the data and collections that Fast DDS keeps internally, memory for these data and collections can be preallocated during entity initialization. In order to choose the correct size values, users must be aware of the topology of the whole domain. Specifically, the number of DomainParticipants, DataWriters, and DataReaders must be known when setting their configuration.

The following sections describe how to configure allocations to be done during the initialization of the entities. Although some examples are provided on each section as reference, there is also a complete example use case.

15.5.1.1. Parameters on the participant

Every DomainParticipant holds an internal collection with information about every local and remote peer DomainParticipants that has been discovered. This information includes, among other things:

  • A nested collection with information of every DataWriter announced on the peer DomainParticipant.

  • A nested collection with information of every DataReader announced on the peer DomainParticipant.

  • Custom data configured by the user on the peer DomainParticipant, namely, UserDataQosPolicy, PartitionQosPolicy, and PropertyPolicyQos.

By default, these collections are fully dynamic, meaning that new memory is allocated when a new DomainParticipant, DataWriter, or DataReader is discovered. Likewise, the mentioned custom configuration data parameters have an arbitrary size. By default, the memory for these parameters is allocated when the peer DomainParticipant announces their value.

However, DomainParticipantQos has a member function allocation(), of type ParticipantResourceLimitsQos, that allows configuring maximum sizes for these collections and parameters, so that all the required memory can be preallocated during the initialization of the DomainParticipant.

15.5.1.1.1. Limiting the number of discovered entities

ParticipantResourceLimitsQos provides three data members to configure the allocation behavior of discovered entities:

  • participants configures the allocation of the collection of discovered DomainParticipants.

  • readers configures the allocation of the collection of DataWriters within each discovered DomainParticipant.

  • writers configures the allocation of the collection of DataReaders within each discovered DomainParticipant.

By default, a full dynamic behavior is used. Using these members, however, it is easy to configure the collections to be preallocated during initialization, setting them to a static maximum expected value, as shown in the example below. Please, refer to ResourceLimitedContainerConfig for a complete description of additional configuration alternatives given by these data members.

C++

DomainParticipantQos qos;

// Fix the size of discovered participants to 3
// This will effectively preallocate the memory during initialization
qos.allocation().participants =
        eprosima::fastrtps::ResourceLimitedContainerConfig::fixed_size_configuration(3u);

// Fix the size of discovered DataWriters to 1 per DomainParticipant
// Fix the size of discovered DataReaders to 3 per DomainParticipant
// This will effectively preallocate the memory during initialization
qos.allocation().writers =
        eprosima::fastrtps::ResourceLimitedContainerConfig::fixed_size_configuration(1u);
qos.allocation().readers =
        eprosima::fastrtps::ResourceLimitedContainerConfig::fixed_size_configuration(3u);

XML

<?xml version="1.0" encoding="UTF-8" ?>
<profiles xmlns="http://www.eprosima.com/XMLSchemas/fastRTPS_Profiles">
    <participant profile_name="participant_profile_qos_entity_resource_limit">
        <rtps>
            <allocation>
                <!-- Limit to 3 participants -->
                <total_participants>
                    <initial>3</initial>
                    <maximum>3</maximum>
                    <increment>0</increment>
                </total_participants>

                <!-- Limit to 3 readers per participant -->
                <total_readers>
                    <initial>3</initial>
                    <maximum>3</maximum>
                    <increment>0</increment>
                </total_readers>

                <!-- Limit to 1 writer per participant -->
                <total_writers>
                    <initial>1</initial>
                    <maximum>1</maximum>
                    <increment>0</increment>
                </total_writers>
            </allocation>
        </rtps>
    </participant>
</profiles>

Warning

Configuring a collection as fixed in size effectively limits the number of peer entities that can be discovered. Once the configured limit is reached, any new entity will be ignored. In the given example, if a fourth peer DomainParticipant appears, it will not be discovered, as the collection of discovered DomainParticipants is already full.

15.5.1.1.2. Limiting the size of custom parameters

data_limits inside ParticipantResourceLimitsQos provides three data members to configure the allocation behavior of custom parameters:

If these sizes are configured to something different than zero, enough memory will be allocated for them for each participant and endpoint. A value of zero implies no size limitation, and memory will be dynamically allocated as needed. By default, a full dynamic behavior is used.

C++

DomainParticipantQos qos;

// Fix the size of the complete user data field to 256 octets
qos.allocation().data_limits.max_user_data = 256u;
// Fix the size of the complete partitions field to 256 octets
qos.allocation().data_limits.max_partitions = 256u;
// Fix the size of the complete properties field to 512 octets
qos.allocation().data_limits.max_properties = 512u;

XML

<?xml version="1.0" encoding="UTF-8" ?>
<profiles xmlns="http://www.eprosima.com/XMLSchemas/fastRTPS_Profiles">
    <participant profile_name="participant_profile_qos_parameter_resource_limit">
        <rtps>
            <allocation>
                <max_partitions>256</max_partitions>
                <max_user_data>256</max_user_data>
                <max_properties>512</max_properties>
            </allocation>
        </rtps>
    </participant>
</profiles>

Warning

If the data fields announced by the remote peer do not fit on the preallocated memory, an error will be triggered during the processing of the announcement message. This usually means that the discovery messages of a remote peer with too large data fields will be discarded, i.e., peers with too large data fields will not be discovered.

15.5.1.2. Parameters on the DataWriter

Every DataWriter holds an internal collection with information about every DataReader to which it matches. By default, this collection is fully dynamic, meaning that new memory is allocated when a new DataReader is matched. However, DataWriterQos has a data member writer_resource_limits(), of type WriterResourceLimitsQos, that allows configuring the memory allocation behavior on the DataWriter.

WriterResourceLimitsQos provides a data member matched_subscriber_allocation of type ResourceLimitedContainerConfig that allows configuring the maximum expected size of the collection of matched DataReader, so that it can be preallocated during the initialization of the DataWriter, as shown in the example below. Please, refer to ResourceLimitedContainerConfig for a complete description of additional configuration alternatives given by this data member.

C++

DataWriterQos qos;

// Fix the size of matched DataReaders to 3
// This will effectively preallocate the memory during initialization
qos.writer_resource_limits().matched_subscriber_allocation =
        eprosima::fastrtps::ResourceLimitedContainerConfig::fixed_size_configuration(3u);

XML

<?xml version="1.0" encoding="UTF-8" ?>
<profiles xmlns="http://www.eprosima.com/XMLSchemas/fastRTPS_Profiles">
    <data_writer profile_name="writer_profile_qos_resource_limit">
        <!-- Limit to 3 matching readers -->
        <matchedSubscribersAllocation>
            <initial>3</initial>
            <maximum>3</maximum>
            <increment>0</increment>
        </matchedSubscribersAllocation>
    </data_writer>
</profiles>

Warning

Configuring the collection of matched DataReaders as fixed in size effectively limits the number of DataReaders to be matched. Once the configured limit is reached, any new DataReader will be ignored. In the given example, if a fourth (potentially matching) DataReader appears, it will not be matched, as the collection is already full.

15.5.1.3. Parameters on the DataReader

Every DataReader holds an internal collection with information about every ReaderResourceLimitsQos to which it matches. By default, this collection is fully dynamic, meaning that new memory is allocated when a new DataWriter is matched. However, DataReaderQos has a data member reader_resource_limits(), of type ReaderResourceLimitsQos, that allows configuring the memory allocation behavior on the DataReader.

ReaderResourceLimitsQos provides a data member matched_publisher_allocation of type ResourceLimitedContainerConfig that allows configuring the maximum expected size of the collection of matched DataWriters, so that it can be preallocated during the initialization of the DataReader, as shown in the example below. Please, refer to ResourceLimitedContainerConfig for a complete description of additional configuration alternatives given by this data member.

C++

DataReaderQos qos;

// Fix the size of matched DataWriters to 1
// This will effectively preallocate the memory during initialization
qos.reader_resource_limits().matched_publisher_allocation =
        eprosima::fastrtps::ResourceLimitedContainerConfig::fixed_size_configuration(1u);

XML

<?xml version="1.0" encoding="UTF-8" ?>
<profiles xmlns="http://www.eprosima.com/XMLSchemas/fastRTPS_Profiles">
    <data_reader profile_name="reader_profile_qos_resource_limit">
        <!-- Limit to 1 matching writer -->
        <matchedPublishersAllocation>
            <initial>1</initial>
            <maximum>1</maximum>
            <increment>0</increment>
        </matchedPublishersAllocation>
    </data_reader>
</profiles>

Warning

Configuring the collection of matched DataWriters as fixed in size effectively limits the number of DataWriters to be matched. Once the configured limit is reached, any new DataWriter will be ignored. In the given example, if a fourth (potentially matching) DataWriter appears, it will not be matched, as the collection is already full.

15.5.1.4. Full example

Given a system with the following topology:

Allocation tuning example topology

Participant P1

Participant P2

Participant P3

Topic 1 publisher

Topic 1 subscriber

Topic 2 subscriber

Topic 1 subscriber

Topic 2 publisher

Topic 1 subscriber

Topic 2 subscriber

  • The total number of DomainParticipants is 3.

  • The maximum number of DataWriters per DomainParticipant is 1

  • The maximum number of DataReaders per DomainParticipant is 2.

  • The DataWriter for topic 1 matches with 3 DataReaders.

  • The DataWriter for topic 2 matches with 2 DataReaders.

  • All the DataReaders match exactly with 1 DataWriter.

We will also limit the size of the parameters:

The following piece of code shows the set of parameters needed for the use case depicted in this example.

C++

// DomainParticipant configuration
//////////////////////////////////
DomainParticipantQos participant_qos;

// We know we have 3 participants on the domain
participant_qos.allocation().participants =
        eprosima::fastrtps::ResourceLimitedContainerConfig::fixed_size_configuration(3u);
// We know we have at most 2 readers on each participant
participant_qos.allocation().readers =
        eprosima::fastrtps::ResourceLimitedContainerConfig::fixed_size_configuration(2u);
// We know we have at most 1 writer on each participant
participant_qos.allocation().writers =
        eprosima::fastrtps::ResourceLimitedContainerConfig::fixed_size_configuration(1u);

// We know the maximum size of partition data
participant_qos.allocation().data_limits.max_partitions = 256u;
// We know the maximum size of user data
participant_qos.allocation().data_limits.max_user_data = 256u;
// We know the maximum size of properties data
participant_qos.allocation().data_limits.max_properties = 512u;


// DataWriter configuration for Topic 1
///////////////////////////////////////
DataWriterQos writer1_qos;

// We know we will only have three matching subscribers
writer1_qos.writer_resource_limits().matched_subscriber_allocation =
        eprosima::fastrtps::ResourceLimitedContainerConfig::fixed_size_configuration(3u);

// DataWriter configuration for Topic 2
///////////////////////////////////////
DataWriterQos writer2_qos;

// We know we will only have two matching subscribers
writer2_qos.writer_resource_limits().matched_subscriber_allocation =
        eprosima::fastrtps::ResourceLimitedContainerConfig::fixed_size_configuration(2u);


// DataReader configuration for both Topics
///////////////////////////////////////////
DataReaderQos reader_qos;

// We know we will only have one matching publisher
reader_qos.reader_resource_limits().matched_publisher_allocation =
        eprosima::fastrtps::ResourceLimitedContainerConfig::fixed_size_configuration(1u);

XML

<?xml version="1.0" encoding="UTF-8" ?>
<profiles xmlns="http://www.eprosima.com/XMLSchemas/fastRTPS_Profiles">
    <participant profile_name="participant_alloc_qos_example">
        <rtps>
            <allocation>
                <!-- We know we have 3 participants on the domain -->
                <total_participants>
                    <initial>3</initial>
                    <maximum>3</maximum>
                    <increment>0</increment>
                </total_participants>
                <!-- We know we have at most 2 readers on each participant -->
                <total_readers>
                    <initial>2</initial>
                    <maximum>2</maximum>
                    <increment>0</increment>
                </total_readers>
                <!-- We know we have at most 1 writer on each participant -->
                <total_writers>
                    <initial>1</initial>
                    <maximum>1</maximum>
                    <increment>0</increment>
                </total_writers>
                <max_partitions>256</max_partitions>
                <max_user_data>256</max_user_data>
                <max_properties>512</max_properties>
            </allocation>
        </rtps>
    </participant>

    <data_writer profile_name="alloc_qos_example_pub_for_topic_1">
        <!-- we know we will have three matching subscribers -->
        <matchedSubscribersAllocation>
            <initial>3</initial>
            <maximum>3</maximum>
            <increment>0</increment>
        </matchedSubscribersAllocation>
    </data_writer>

    <data_writer profile_name="alloc_qos_example_pub_for_topic_2">
        <!-- we know we will have two matching subscribers -->
        <matchedSubscribersAllocation>
            <initial>2</initial>
            <maximum>2</maximum>
            <increment>0</increment>
        </matchedSubscribersAllocation>
    </data_writer>

    <data_reader profile_name="alloc_qos_example_sub">
        <!-- we know we will only have one matching publisher -->
        <matchedPublishersAllocation>
            <initial>1</initial>
            <maximum>1</maximum>
            <increment>0</increment>
        </matchedPublishersAllocation>
    </data_reader>
</profiles>