As more and more business-critical applications move to the cloud, ensuring that your network infrastructure can handle the traffic and deliver optimal performance is becoming increasingly important. Linux, being one of the most popular operating systems used for hosting servers, provides several built-in tools to fine-tune the network performance. In this blog post, we will explore the use of Qdiscs and Filters to improve network performance on Linux.
Queue is an important data structure in an operating system that is used to hold items that are waiting to be processed. By using a queue, the operating system can manage and prioritize the processing of these items, which can help to improve the performance and efficiency of the system.
In an operating system, a queue is a data structure used to store a collection of items that are waiting to be processed. The items in the queue are typically processed in a first-in, first-out (FIFO) order, which means that the item that was added to the queue first will be the first item to be processed. Queues are used in many different parts of an operating system, such as in process scheduling, input/output (I/O) scheduling, and network traffic management.
In process scheduling, a queue is used to hold a list of processes that are waiting to be executed by the CPU. When a process is finished executing, the operating system will choose the next process from the queue to be executed.
In I/O scheduling, a queue is used to hold a list of I/O requests that are waiting to be processed by a device driver. When an I/O request is completed, the device driver will choose the next request from the queue to be processed.
In network traffic management, a queue is used to hold a list of packets that are waiting to be transmitted on a network interface. The packets are typically prioritized based on their importance or the type of traffic they represent. For example, real-time traffic such as voice and video may be given a higher priority than non-real-time traffic such as email or file transfers.
Qdiscs, or queuing disciplines, are a set of kernel modules that provide various algorithms for managing network traffic on Linux. Qdiscs are implemented in the kernel and can be used to shape, prioritize, and manage the flow of network traffic on a per-interface basis.
There are several types of Qdiscs available on the Linux kernel, each with a specific purpose for managing network traffic. Here are some of the most commonly used types:
- pfifo_fast: A simple FIFO (First-In, First-Out) Qdisc that provides no traffic prioritization or shaping. This is the default Qdisc used by many Linux distributions.
- htb: Hierarchical Token Bucket Qdisc that allows for hierarchical classification and prioritization of traffic.
- tbf: Token Bucket Filter Qdisc that uses a token bucket algorithm to shape traffic to a specified rate.
- sfq: Stochastic Fairness Queue Qdisc that provides fair access to the network for multiple flows.
- fq_codel: Fair Queuing with Controlled Delay Qdisc that prioritizes traffic based on flow and reduces packet queuing delay.
- red: Random Early Detection Qdisc that randomly drops packets to prevent network congestion.
- cake: Common Applications Kept Enhanced Qdisc that combines several queuing algorithms to provide low-latency and high-throughput networking.
Here’s an example of how to create a simple HTB (Hierarchical Token Bucket) Qdisc that limits the outgoing traffic to 1Mbps on the eth0 interface:
tc qdisc add dev eth0 root handle 1: htb default 10
tc class add dev eth0 parent 1: classid 1:1 htb rate 1mbit ceil 1mbit
tc class add dev eth0 parent 1:1 classid 1:10 htb rate 1mbit ceil 1mbit
In this example, the tc qdisc command is used to add a new root qdisc with a handle of 1: and a default class of 10. The tc class command is used to create two new classes (1:1 and 1:10) and set their rates to 1Mbps. The parent parameter is used to specify the parent class for each class.
Filters, on the other hand, are used to match packets based on certain criteria and direct them to a specific qdisc. Filters can be attached to a Qdisc or a class and can be used to classify traffic based on source or destination IP addresses, port numbers, protocol types, and more. Here’s an example of how to add a simple filter that matches packets based on their source IP address and directs them to class 1:10:
tc filter add dev eth0 parent 1: protocol ip prio 1 u32 match ip src 192.168.0.1 flowid 1:10
In this example, the tc filter command is used to add a new filter to the eth0 interface. The parent parameter is used to specify the parent qdisc or class, and the protocol parameter is used to match packets of a specific protocol type (in this case, IP). The u32 match ip src parameter is used to match packets based on their source IP address. The flowid parameter is used to direct matching packets to the class with ID 1:10.
Using Qdiscs and Filters together can provide fine-grained control over network traffic on a Linux system. For example, you can use Qdiscs to shape the outgoing traffic and Filters to prioritize certain types of traffic over others. Here’s an example of how to create a Qdisc and filter that prioritizes SSH traffic over other traffic:
sudo tc qdisc add dev eth0 root handle 1: htb default 12
sudo tc class add dev eth0 parent 1: classid 1:1 htb rate 1mbit ceil 1mbit prio 1
sudo tc filter add dev eth0 protocol ip parent 1: prio 1 u32 match ip protocol 6 0xff match ip dport 22 0xffff flowid 1:1
With these commands, any SSH traffic on the “eth0” network interface will be prioritized over other traffic, such as web browsing or file transfers. The Qdisc and filter can be modified or removed as needed to suit the needs of a particular network environment.
Limiting Network Traffic is also possible via the combined use of qdiscs and filters. In the following example we have checked the implementation of qdiscs along with filters to rate limit network traffic for specified filters. The filter we used for demo was only b/w two IP addresses. We used iperf for testing network speed before and after application of qdiscs on filters.
root@corelive:/home/test# ip -brie -c a
lo UNKNOWN 127.0.0.1/8 ::1/128
eth1 UP fe80::1cae:4aff:fee0:f408/64
eth0 UP 172.17.7.10/30 fe80::78f1:77ff:fe5d:5799/64
root@corelive:/home/test# iperf -c 172.21.34.34
------------------------------------------------------------
Client connecting to 172.21.34.34, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[ 3] local 172.17.7.10 port 44416 connected with 172.21.34.34 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.0 sec 507 MBytes 425 Mbits/sec
root@corelive:/home/test# tc qdisc add dev eth0 root handle 1: htb
root@corelive:/home/test# tc class add dev eth0 parent 1: classid 1:1 htb rate 100000
root@corelive:/home/test# tc filter add dev eth0 protocol ip parent 1: prio 1 u32 match ip src 172.17.7.10 match ip dst 172.21.34.34 flowid 1:1
root@corelive:/home/test# iperf -c 172.21.34.34
------------------------------------------------------------
Client connecting to 172.21.34.34, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[ 3] local 172.17.7.10 port 44418 connected with 172.21.34.34 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-13.8 sec 226 KBytes 135 Kbits/sec
root@corelive:/home/test# tc class change dev eth0 parent 1: classid 1:1 htb rate 15728640
root@corelive:/home/test# iperf -c 172.21.34.34
------------------------------------------------------------
Client connecting to 172.21.34.34, TCP port 5001
TCP window size: 85.0 KByte (default)
------------------------------------------------------------
[ 3] local 172.17.7.10 port 44420 connected with 172.21.34.34 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0-10.1 sec 18.2 MBytes 15.2 Mbits/sec
It can be used for limiting overall traffic as well, which will be done via following example. The key part of this command is the u32 match u32 0 0 parameter. This matches all packets, regardless of their source or destination IP address.
tc filter change dev eth0 protocol ip parent 1: prio 1 u32 match u32 0 0 flowid 1:1
Summary: This blog post provides an overview of using Qdiscs and Filters on a Linux system to fine-tune network performance. Qdiscs are kernel modules that provide algorithms for managing network traffic on a per-interface basis. Filters are used to match packets based on certain criteria and direct them to a specific Qdisc or class. The post includes examples of how to create and configure Qdiscs and Filters on a Linux system, including a simple HTB Qdisc and a filter that prioritizes SSH traffic over other traffic. By using Qdiscs and Filters together, network administrators can have fine-grained control over network traffic and optimize network performance on a Linux system.