In the world of computers and embedded systems, two common terms we often hear about are latency and throughput. These terms are used when discussing the speed of a system and revolves around the process of sending data from one location to the next, indicating the time it takes to process and transmit the data.
Both of these terms are sometimes thought to be interchangeable; and while they do have an important relationship with each other, they are quite different in their meanings and how they influence the time to process data.
In this blog, we will go over the concepts of latency and throughput, as well as discuss their relationship when it comes to transmitting and processing data in an embedded network or computer system.
What is Latency?
Latency is the amount of time it takes to complete an operation and indicates how long it takes for data packets to travel from one point in a system to another. This concept is measured in units of time, such as milliseconds, microseconds, and nanoseconds. When it comes to latency, the lower the latency, the better — meaning as close to 0 as possible. Having a lower latency means having less of a delay when sending data within a system, making the system run faster and resulting in a positive user experience. A higher latency can cause an increase in webpage load times or disrupt video or audio steams. Even a slight increase in latency can cause a significant delay in system performance.
While having a lower latency is ideal, there will generally always be some latency with multiple systems connected together simply because of the way data travels. System latency is most often the result of the amount of time it takes for an endpoint to respond to a read request and the amount of time it takes a packet to be transmitted across a device.
What is Throughput?
Throughput is the number of data packets that are able to reach their destination within a specific period of time. In other words, it is the measurement of the rate at which data is processed and transferred from one location to another in a given timeframe. Throughput is measured in bits per second (bps) including megabits per second (Mbps) or gigabits per second (Gbps).
Having a higher throughput is ideal as this means the system is capable of processing more data at a time, enhancing system performance.
In embedded systems or computer networks, throughput can be affected by a number of factors, including bus or network congestion, packet loss/errors, and the protocol used.
Relationship Between Latency vs Throughput
Both latency and throughput are factors that affect a network or system’s performance. As mentioned before, when it comes to the speed of a system, latency is considered to be the delay in processing data while throughput is how much data can be sent in a given period of time.
A higher latency means packets will take a longer amount of time to reach their destination, resulting in slower devices and applications. Similarly, the lower the amount of throughput, the lower the number of packets being processed in a specific time period.
When thought of together, both have an effect on data transfer and speed: latency measures how quickly data is exchanged between different points in a system. The higher the latency, the longer the exchanges take place. The level of latency determines the maximum throughput of the data exchange, or how much data can be transmitted during a transaction. Fundamentally, it’s better to maximize throughput and minimize latency.
Latency vs Throughput vs Bandwidth
In addition to latency and throughput, it’s also important to discuss how bandwidth is involved in the overall speed and performance of a system. Bandwidth has a significant effect on the efficiency of latency and throughput. It measures the amount of data that is able to pass through a network at a given time. Typically, systems with higher bandwidth capacities perform better than systems with lower bandwidth.
Like throughput, bandwidth is measured in bits per second (bps), such as megabits per second (Mbps) and gigabits per second (Gbps). Although bandwidth determines the amount of data that can be transferred within a system, interferences in throughput can cause lower-quality system performance. For example, if throughput is hindered by packet loss, jitter, or latency, the network or system will likely experience delays even with a high bandwidth.
Like previously mentioned, a high latency can impact the throughput within a system regardless of the amount of bandwidth available. For example, a network with high bandwidth may have components that process their various tasks slowly, while a lower-bandwidth network may have faster components, resulting in higher overall throughput.
Calculating Latency in USB Systems
When developing USB systems, engineers may experience instances where there is a high latency within the system due to various factors. This can be intensified when a hub is introduced into the system, causing increased delays in data transmission across the bus. To help determine how low or high the latency is within a USB system that incorporates a hub, developers can utilize two or more Total Phase Beagle USB 5000 v2 SuperSpeed Protocol Analyzers to concurrently capture data at multiple points in the USB device chain and subsequently measure the hub latency using the Hub Latency View feature.
Image by We provide more information on this feature here: Data Center Software Series: Hub Latency View
For additional information on this feature or how it can benefit your USB developments, please email sales@totalphase.com.
Conclusion
Latency and throughput are important concepts when understanding network and system performance. They both affect the speed of a system, but in different ways. Data transmission can be hindered by a high latency, due to data delays, and a lower throughput due to less data being processed at a time. These factors together influence how quickly data is able to reach its destination in a system. Therefore, it’s important for embedded engineers to be knowledgeable of the relationship between the two.