<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[salvy's DSE Chronicles]]></title><description><![CDATA[salvy's DSE Chronicles]]></description><link>https://salvy.hashnode.dev</link><generator>RSS for Node</generator><lastBuildDate>Mon, 21 Sep 2026 00:03:32 GMT</lastBuildDate><atom:link href="https://salvy.hashnode.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[Diagnosing Latency: A Systematic Method for "Why is Google Slow?"]]></title><description><![CDATA[Identifying the Problem
In network engineering, user complaints are often subjective, nuanced, and hazy. A report of "Google is slow" presents a classic diagnostic challenge. The perceived latency could be rooted in multiple layers of the network sta...]]></description><link>https://salvy.hashnode.dev/diagnosing-latency-a-systematic-method-for-why-is-google-slow</link><guid isPermaLink="true">https://salvy.hashnode.dev/diagnosing-latency-a-systematic-method-for-why-is-google-slow</guid><category><![CDATA[networking]]></category><category><![CDATA[diagnostics]]></category><category><![CDATA[technical-deep-dive]]></category><category><![CDATA[distributed systems]]></category><dc:creator><![CDATA[salvy snr]]></dc:creator><pubDate>Sat, 13 Sep 2025 11:34:55 GMT</pubDate><enclosure url="https://cdn.hashnode.com/res/hashnode/image/upload/v1757756060097/216965fc-be21-40f8-ab0c-80e1e1cd3a3c.png" length="0" type="image/jpeg"/><content:encoded><![CDATA[<h2 id="heading-identifying-the-problem">Identifying the Problem</h2>
<p>In network engineering, user complaints are often subjective, nuanced, and hazy. A report of "Google is slow" presents a classic diagnostic challenge. The perceived latency could be rooted in multiple layers of the network stack: local DNS caching, ISP routing, backbone congestion, or server-side application delays.</p>
<p>This article documents my first-ever diagnostic procedure performed in response to such a complaint. The goal is to provide a blueprint for systematically isolating the source of network latency, using only standard command-line tools available on most endpoints. We will proceed from the network layer up to the application layer, eliminating potential causes one by one.</p>
<h2 id="heading-the-diagnostic-toolkit">The Diagnostic Toolkit</h2>
<p>The investigation employed four core utilities:</p>
<ol>
<li><p><code>dig</code> (Domain Information Groper): Used for interrogating DNS nameservers to assess <a target="_blank" href="https://threat.media/definition/what-is-dns-latency/">resolution latency</a> at different levels of the DNS hierarchy.</p>
</li>
<li><p><code>ping</code>: To measure baseline <a target="_blank" href="https://www.cloudflare.com/learning/cdn/glossary/round-trip-time-rtt/">Round-Trip Time (RTT)</a> and assess packet loss to a specific IP address. The <code>-l</code> flag on Windows was used to test with larger payloads.</p>
</li>
<li><p><code>tracert</code> (Windows) or <code>traceroute</code> (UNIX): Used to map the network path from the source to the destination, identifying each hop and measuring latency to each intermediate router.</p>
</li>
<li><p><code>curl</code>: Used with specific timing flags to break down the HTTP/S request process into distinct stages (DNS, TCP, TLS) and measure the latency of each.</p>
</li>
</ol>
<h2 id="heading-phase-1-dns-resolution-analysis">Phase 1: DNS Resolution Analysis</h2>
<p>The first step in any web request is Domain Name System (DNS) resolution. Delays here would manifest as a long wait before any connection is attempted.</p>
<h3 id="heading-step-11-testing-cached-resolution-latency">Step 1.1: Testing Cached Resolution Latency</h3>
<p>The first test checks the latency of a DNS query that is likely cached by the local OS or stub resolver.</p>
<p><strong>Command</strong>: <code>dig google.com</code></p>
<p><strong>Methodology</strong>: The query time was recorded over ten (10) consecutive attempts to establish an average.</p>
<p><strong>Result</strong>: Query times (in ms): 46, 15, 0, 31, 15, 15, 40, 15, 0, 0.</p>
<p><strong>Finding</strong>: The average latency for a cached query was <strong>17.7ms</strong>. This is considered excellent performance and effectively rules out the local DNS cache as the source of significant delay.</p>
<h3 id="heading-step-12-testing-uncached-resolution-with-a-public-resolver">Step 1.2: Testing “Uncached” Resolution with a Public Resolver</h3>
<p>This test bypasses the local cache by directly querying a public recursive resolver, Google's <code>8.8.8.8</code>.</p>
<p><strong>Command</strong>: <code>dig google.com @8.8.8.8</code></p>
<p><strong>Result</strong>: Query times (in ms): 78, 62, 70, 46, 43, 62, 60, 46, 67, 46.</p>
<p><strong>Finding</strong>: The average latency was <strong>58.0ms</strong>. This is a reasonable latency for a recursive query and is unlikely to be the sole cause of a user-perceived "slow" experience.</p>
<h3 id="heading-step-13-testing-resolution-from-an-authoritative-source">Step 1.3: Testing Resolution from an Authoritative Source</h3>
<p>To understand the full DNS chain, I queried one of the domain's authoritative nameservers directly. This is not a normal user operation but is informative.</p>
<p><strong>Command</strong>: <code>dig google.com @ns1.google.com</code></p>
<p><strong>Result</strong>: Query times (in ms): 159, 159, 190, 155, 154, 155, 176, 188, 165, 174.</p>
<p><strong>Finding</strong>: The average latency was <strong>167.4ms</strong>. This is slower, as expected, because authoritative nameservers are not optimised for direct end-user queries. The more important finding was the discrepancy between this and the <code>@8.8.8.8</code> query, suggesting the public resolver had a warm cache.</p>
<h3 id="heading-step-14-measuring-resolver-internal-processing-time">Step 1.4: Measuring Resolver Internal Processing Time</h3>
<p>To see the true "uncached" performance of the resolvers themselves, I used the <code>+stats</code> flag to see the resolution time reported by the resolver.</p>
<p><strong>Command (Google DNS)</strong>: <code>dig @8.8.8.8 +stats</code></p>
<p><strong>Result</strong>: Resolver query times (in ms): 145, 132, 137, 130, 158, 172, 167, 144, 147, 139.</p>
<p><strong>Finding</strong>: Average internal resolution time for <code>@8.8.8.8</code> was <strong>147.1ms</strong>.</p>
<p><strong>Command (Cloudflare DNS)</strong>: <code>dig @1.1.1.1 +stats</code></p>
<p><strong>Result</strong>: Resolver query times (in ms): 175, 256, 217, 227, 168, 291, 171, 239, 201, 174.</p>
<p><strong>Finding</strong>: Average internal resolution time for <code>@1.1.1.1</code> was <strong>221.9ms</strong>.</p>
<h3 id="heading-phase-1-conclusion">Phase 1 Conclusion</h3>
<p>DNS resolution shows expected latencies. The cached time is fast (<strong>&lt;20ms</strong>), and even full recursive resolution times (<strong>~150-220ms</strong>) are insufficient to explain a severely "slow" feeling on their own. The investigation must move higher up the stack.</p>
<h2 id="heading-phase-2-network-latency-and-path-analysis">Phase 2: Network Latency and Path Analysis</h2>
<p>With the IP address resolved (<code>216.58.223.206</code>), I measured the raw network latency to the host.</p>
<h3 id="heading-step-21-icmp-echo-ping-to-target-ip">Step 2.1: ICMP Echo (Ping) to Target IP</h3>
<p><strong>Command</strong>: <code>ping -n 10 216.58.223.206</code></p>
<p><strong>Result</strong>: Average RTT over 10 pings: <strong>~53ms</strong>.</p>
<p><strong>Command (with payload)</strong>: <code>ping -n 10 -l 1000 216.58.223.206</code></p>
<p><strong>Result</strong>: Average RTT with 1000-byte payload: <strong>~57ms</strong>.</p>
<p><strong>Finding</strong>: The base network latency is moderate and stable. The negligible increase with a larger payload suggests the path is not suffering from major congestion or fragmentation issues. This is not low enough to explain a severe slowdown.</p>
<h3 id="heading-step-22-traceroute-to-target-ip">Step 2.2: Traceroute to Target IP</h3>
<p>Command: <code>tracert 216.58.223.206</code></p>
<p><strong>Result</strong>: The path was consistent across multiple runs. Key hops from a representative run:</p>
<ul>
<li><p><strong>Hop 1</strong>: 172.20.10.1 (Local Gateway) - <strong>&lt;1ms</strong></p>
</li>
<li><p><strong>Hop 2</strong>: 10.18.0.252 (ISP First Hop) - Variance: <strong>53ms to 258ms</strong></p>
</li>
<li><p><strong>Hop 3</strong>: 10.169.131.249 (ISP Core) - Variance: <strong>33ms to 154ms</strong></p>
</li>
<li><p><strong>Hop 11</strong>: 108.170.238.151 (Google Network) - Variance: <strong>45ms to 230ms</strong></p>
</li>
<li><p><strong>Hops 12-13</strong>: 172.253.76.173 -&gt; 216.58.223.206 - Stable at <strong>~42-66ms</strong></p>
</li>
</ul>
<p><strong>Finding</strong>: The traceroute revealed significant jitter (latency variation) at early ISP hops (2 and 3) and within Google's network (Hop 11). This indicates intermittent congestion or traffic shaping, which would contribute to an inconsistent user experience. However, the final latency to the destination remained in the <strong>50-60ms</strong> range.</p>
<h2 id="heading-phase-3-application-layer-timing">Phase 3: Application Layer Timing</h2>
<p>The final and most crucial test measures the entire process of establishing a secure HTTP connection, which is what a browser does.</p>
<p>Step 3.1: HTTP/S Timing Breakdown</p>
<p><strong>Command</strong>: <code>curl -o /dev/null -s -w "DNS: %{time_namelookup}s\nConnect: %{time_connect}s\nTLS: %{time_appconnect}s\nTTFB: %{time_starttransfer}s\nTotal: %{time_total}s\n" https://google.com</code></p>
<p><strong>Result</strong>:</p>
<p><code>DNS Lookup: 0.088364 s (88.4 ms)</code></p>
<p><code>TCP Connect: 0.293722 s (293.7 ms)</code></p>
<p><code>TLS Handshake: 0.516852 s (516.9 ms)</code></p>
<p><code>Time to First Byte (TTFB): 0.782226 s (782.2 ms)</code></p>
<p><code>Total Time: 0.783087 s (783.1 ms)</code></p>
<p>Finding: This output is definitive. The DNS lookup was fast (<strong>88.4ms</strong>). The overwhelming majority of the delay (<strong>~800ms</strong>) was consumed by the TCP connection (<a target="_blank" href="https://learn.microsoft.com/en-us/troubleshoot/windows-server/networking/three-way-handshake-via-tcpip">TCP three-way handshake</a>) and the <a target="_blank" href="https://www.cloudflare.com/learning/ssl/what-happens-in-a-tls-handshake/">TLS handshake</a> before any application data could be requested or received.</p>
<h2 id="heading-synthesis-and-conclusion">Synthesis and Conclusion</h2>
<p>The evidence leads to a clear and specific diagnosis:</p>
<p><strong>DNS Resolution</strong>: Ruled out as the primary cause. Performance was within expected parameters.</p>
<p><strong>Raw Network Latency</strong>: Ruled out as the sole cause. While moderate (<strong>~53ms</strong>), it does not account for the multi-second perceived delay.</p>
<p><strong>Network Path Jitter</strong>: Identified as a contributing factor. Significant latency variation at early ISP hops (<strong>10.18.0.252, 10.169.131.249</strong>) explains inconsistent performance but is not the root cause.</p>
<p><strong>TCP/TLS Handshake Latency</strong>: Identified as the root cause. The combined <strong>~810ms</strong> delay in establishing a connection indicates the client is being routed to a geographically distant or congested Google edge server. The TLS handshake, involving multiple round trips, is highly sensitive to high latency.</p>
<h2 id="heading-recommendations">Recommendations</h2>
<p><strong>For the End User</strong>: The issue is likely related to ISP routing policies. Using a VPN could force a connection through a different path to a more optimal Google <a target="_blank" href="https://en.wikipedia.org/wiki/Point_of_presence">Point of Presence (PoP)</a>, potentially reducing handshake latency.</p>
<p><strong>For the ISP</strong>: The jitter at hops 2 and 3 should be investigated, as it degrades network quality. Furthermore, <a target="_blank" href="https://www.cloudflare.com/learning/security/glossary/what-is-bgp/">BGP</a> peering relationships with Google's network should be reviewed to ensure local traffic is routed to the nearest edge location.</p>
<p>This case study demonstrates that subjective user experience can be deconstructed into objective data. A methodical, layer-by-layer approach is indispensable for transforming a vague complaint into a precise and actionable network diagnosis.</p>
]]></content:encoded></item></channel></rss>