MTU (Maximum Transmission Unit) is a critical networking parameter that defines the maximum size of a network packet that can be transmitted over a network interface. In general, larger MTUs can result in higher network performance by reducing the number of packets required to transmit a given amount of data. However, when network packets exceed the MTU of the underlying network infrastructure, they must be fragmented into smaller packets before transmission.
This is especially important for UDP (User Datagram Protocol) packets, which do not have built-in fragmentation capabilities. In contrast to TCP (Transmission Control Protocol), which can automatically segment large packets into smaller ones, UDP relies on the network infrastructure to fragment packets if necessary. As a result, it is crucial to carefully manage MTUs when using UDP to avoid packet fragmentation and ensure reliable network performance.
In any VoIP/IMS network, there are some cases for interconnecion to other servers or providers that just accept SIP over UDP! Yeah, I know, It is ridiculous for me as well, but they are more powerful and you should accept when they declare “we can’t enable TCP” ! OK, so I am talking about UDP fragmentation in SIP signaling and some interesting hints that I found.
In VoIP networks, usually SIP signaling messages don’t have much headers and supported codecs by UAC or UAS are limited, that means the packet size usually is not big enough to exceed MTU! but in IMS network situation is different, there are different X-CSCF or other elements in the path, and each of them like to say we are doing very amazing task, so they inject a few headers. Furthermore, AMR, EVS, and other codecs have various attributes that make the SIP INVITE message larger.
I said larger, but larger than what? In the RFC 3261 section 18.1.1 it says SIP over UDP message size should be lower than 1300 bytes and if you are carrying larger than this, you have to use TCP.
In short, in IMS, this usually occurs when SIP messages have to be sent over UDP; these messages are bigger than 1300 bytes and would need to be fragmented if the MTU is 1500 or smaller than message size.
In Wireshark, how can you see fragmentation?
For example, when you are sending a SIP INVITE to other server over UDP, when it exceeds the MTU, you will receive an ICMP message Destination unreachable (Fragmentation needed) that inside it you can find the expected MTU size. Following that, the server fragments the messages and forwards them.
In SIP signaling, re-transmission mechanism helps to sending fragmented message to destination. This was the mechanism that we didn’t have in our Lambda functions but fragmentation forced us to implement it quickly.
So in next try you can see fragmentation in Wireshark like this:
Okay, so we do fragmentation, but other ways can be used to prevent fragmentation and consequent retransmission. First we can adjust correct MTU for the network interface that is linked to the destination UDP servers! If your MTU is higher than your adjacent link MTU, and you are doing fragmentation for all requests, so why you need that?? decrease it or set it to value that fit for your next hop.
You can check the MTU and the set it easily:
ifconfig <Interface_name> mtu <mtu_size> up
|
|
To persist your network MTU setting for Amazon Linux 2, add the following line to the /etc/sysconfig/network-scripts/ifcfg-eth0 file:
MTU=1500
change the value to whatever you want. For other OS, better to google.
Bad news: if you are using the ECS Fargate, you can NOT change the MTU size. Default value is 9001 (Jumbo frames)!
In our case, as we are using Fargate, so I tried to explore other solutions and found that Linux does keep a history of the learned MTU sizes for each network path, which is referred to as the “Path MTU Discovery (PMTUD)” mechanism. This is done to avoid repeated fragmentation of packets and improve network performance.
When fragmentation is needed, the sender reduces the packet size and retransmits it until it reaches the destination without fragmentation. During this process, the sender also updates its PMTUD cache to remember the MTU of the path. However, this PMTUD cache is not permanent and can expire after a certain period of time, usually around 10 minutes. If you send another packet with a size larger than the current path MTU after the cache has expired, the process will start over and the router will send an ICMP message again, causing the packet to be dropped.
To check the current PMTUD cache status on Linux, you can use the ip route show command and look for entries with the “mtu” field.
|
|
Ok, good. I tried to create a route to destination with specific MTU. Oh no you need privileged mode in container to do that and this is not possible in Fargate.
The cache expiration time can be configured using the net.ipv4.mtu_expires. Default value is 600 seconds.
|
|
When you run a container, it shares the host kernel, which manages the networking stack and related parameters such as MTU caching. So, if you need to change the net.ipv4.mtu_expires parameter for a specific application running in a container, you may be able to modify the application’s configuration or code to adjust its behavior. However, if you need to change the parameter globally for all applications running in a container, you will need to modify the host operating system’s configuration instead. In short, forget this option for ECS too!
In conclusion, try to use TCP in SIP network and if you can’t afford it, try to adjust your MTU, otherwise you should pay the fragmentation cost!
Usefule Links:
[1] How to change MTU size in Linux - https://linuxhint.com/how-to-change-mtu-size-in-linux/
[2] Fargate task networking - https://docs.aws.amazon.com/AmazonECS/latest/userguide/fargate-task-networking.html#fargate-task-networking-considerations
[3] Network maximum transmission unit (MTU) for your EC2 instance - https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/network_mtu.html
[4] Docker run –sysctl - https://docs.docker.com/engine/reference/commandline/run/#sysctl