Improve Network Path Analysis Beyond Traceroute With Zscaler Digital Experience

With applications and end users as distributed as they’ve become, most organizations have adjusted their workforces to hybrid environments. With this adjustment, IT must ensure excellent end user experiences no matter where users or applications are located. This means understanding each user’s environment—even those not under IT’s control.. Without the proper tools in place, troubleshooting takes longer and undue burden is placed on network operations teams lacking the ability to quickly pinpoint issues. 

In this blog, we’ll dive into a useful networking troubleshooting tool, “traceroute”, or “tracert” on Microsoft Windows.

What is traceroute?

Traceroute is a tool built into many different devices and operating systems, such as Microsoft Windows, Macs, or routers and switches. Traceroute is a network diagnostic tool which provides the path packets take between a device and a destination. This destination can be anywhere from a website on the internet to local devices on the network (e.g., printers). Traceroute indicates the number of hops between the two points on the network and provides the round trip time (RTT) for each one of the hops.

Why use traceroute?

Traceroute is used for troubleshooting network problems related to connectivity, congestion, or performance. Herein, it indicates where packets are dropped or delayed. Additionally, it can be used by network operations teams to analyze the route(s) packets take to ensure they’re on the optimal path. In some cases, it can help network engineers understand if hops along the path are oversubscribed, which would result in a poor end user experience.

Traceroute example

What happens when packets travel over the internet? They typically traverse many layer 3 devices (routers) before eventually reaching their destination. Tools such as traceroute can help identify an issue, but keep in mind that it is NOT scalable to manually run traceroute from each end user’s device. 

Here is a typical output from a traceroute:

“(8.8.8.8)” is the destination, “64 hops max” is the maximum number of hops, and “52 byte packets” indicates the size of the packets. In the traceroute above, “1.252 ms”, “0.685 ms”, and “0.676 ms” are the response times, in milliseconds, to the three packets sent to the first hop. The first three hops in this output indicate a local network, and eventually, the hops travel over different network providers. It takes 8 hops to reach the end destination.

You can adjust traceroute settings based on the device you’re using. In this example, setting the “-m” flag sets the maximum number of hops traceroute will use before ending. 

prompt> traceroute -m 255 8.8.8.8

Traceroutes leverage time to live (TTL), which is normally used to prevent routing loops and general overconsumption of network resources. The TTL is typically set to a value, and each router decreases the TTL by 1. If the TTL reaches 0, it’s dropped. 

Traceroute works by sending a TTL=1, which makes it to the first hop and the router signals that it dropped the packet. Then, traceroute increases the TTL=2 and so on until the packet reaches its destination. In some cases, networks block these types of packets. 

Additionally, packets don’t always take the same route. Sometimes, the routes change based on a router's availability. For instance, if a router is unavailable, the packet would have to take another route to reach its destination. Fortunately, there are typically many routes to a destination. 

However, not all routes are optimal for the best user experience. This makes troubleshooting even more difficult, as a user could have issues over a particular route. 

Manual traceroutes lack full context

Imagine a user is having application response issues, but the network operations team doesn’t see any latency issues after analyzing the route from the user’s device to the destination.What happened? It appears that the user’s issues have been resolved, but the user calls the service desk again the next morning with a similar issue. Again, when the NetOps team manages to get access to the device and check the route, they don’t see any issues. 

In this case, getting access to the machine or getting the end user to run a traceroute requires coordination and time is wasted. Depending on the issue, the operations team may catch it. For example, if it’s the Wi-Fi or internal network, it might be easier to identify. However, if it’s an ISP related issue they may or may not catch it, as those routes typically switch within milliseconds. With manual tools such as traceroute, they are good point-in-time solutions, but don’t provide insights across a user’s experience in real time, which only serves to increase troubleshooting times.

Traceroute also shows IP addresses and sometimes DNS names to help identify the routes. However, it doesn’t show the geographical information. With easily accessible geographical data, it’s easier to visualize the different hops. For example, if a user is located in Florida, US, and they’re accessing a SaaS application whose traffic routes to Europe then back to the US, network operations can quickly discern that there’s something wrong with the route.

Ensure flawless end user experiences with Zscaler Digital Experience (ZDX)

ZDX helps IT teams monitor digital experiences from an end user perspective to optimize performance and rapidly fix application, network, and device issues. In particular, one of the key ZDX capabilities for network operations teams is rapid visualization where network paths cause performance issues. NetOps teams are given the power to track metrics across Wi-Fi, local ISPs, and corporate and vendor networks to easily spot latency or packet losses between hops. 

Let’s examine how ZDX helps you quickly triage network performance issues. Below is an example of a path from an end user to a destination. ZDX captures network path data at higher frequencies, lets you choose a point in time when a user indicated they were having performance issues, and offers visibility into the Cloud Path to pinpoint the root cause.

ZDX dashboard showing the overall Cloud Path

Drill into the hops to see the actual latency within ZDX

ZDX Cloud Path latency chart to identify issues

Cloud Path probes are used to collect the following metrics:

• Hop Count: The number of hops between each hop point on the path.
• Packet Loss: the % of packet loss at each hop point on the path.
• Latency (Average, Minimum, Maximum, and Standard Deviation, aka Jitter): This is the roundtrip path time measured in milliseconds.

ZDX Cloud Path command line view

Network hops block ICMP traffic, now what?

There are instances when network hops such as routers or firewalls block ICMP packets, limiting the response. ZDX uses Adaptive Mode to select the best protocol for the cloud path to reach its destination. Adaptive mode tries the TCP, UDP, and ICMP protocols for each run and picks the best available protocol for the probe.

Configuring Adaptive Mode within ZDX

Overall, traceroute is a valuable tool, but it presents clear challenges when it comes to troubleshooting an end user’s experience. Namely, it lacks historical information, reverse traceroutes are difficult, choosing the optimal protocol is time consuming, and it doesn’t scale for global organizations. Network operations teams can benefit from tools such as ZDX to quickly analyze an end user’s experience and provide root cause. 

To learn more about ZDX, sign up for a demo!