E-commerce
August 26, 2026
Your customer swears that the checkout page crashed and that their card was never charged. Your availability probes show green all day, yet it is impossible to find any trace of their attempt. The problem? An online store does not produce a single server log but six, each recording a different stage of the same request. Without a shared identifier or precise timing fields, it is impossible to piece them together and tell at which layer the request stopped. This article details the six layers that write logs in an e-commerce stack, the essential fields to reconstruct a customer journey after the fact, and the question that each layer is the only one equipped to resolve. No tool recommendations: what we record is decided before where we send it.
Summary
The six layers of logs of an e-commerce site
A merchant site writes logs at six distinct levels, each layer capturing a different slice of the same client request. First, the CDN or the edge, which responds from the cache, blocks, or forwards to the origin. Next, the web server (Nginx, Apache) which writes an access line. Then the application which validates the cart and records its business decisions. The database generates a slow query log if you enable it. The application then calls third-party services (payment gateway, logistics) and these responses are only traced if you choose to record them. Finally, the site's internal search captures what visitors typed.
Each layer writes its own timestamp. A request that fails at one level may therefore be absent from the next, which complicates reconstruction. The two layers most often forgotten are the business application logs (order creation, validation, payment denial) and the internal search logs. Depending on the hosted platform, you retrieve certain levels by default but rarely the complete stack. A hosted checkout funnel gives you the response from the gateway but nothing below it.
Convert over 2,000 customers on average per month with Qstomy.
The world’s 1st Shopify AI dedicated to customer conversion
Empowering 200+ e-commerce merchants
Which question is each layer solely responsible for deciding?
The CDN answers the question: did the request even reach us? It records cached responses, blocks, and rate limiting before the traffic reaches the origin. The web server says what the server returned and how quickly. The application reveals which command failed and on which business rule. The database indicates whether the delay came from the data layer (slow query, connection error, lock wait). Outbound third-party calls show what the gateway actually answered, not just what the application interpreted.
Finally, internal search reveals what visitors searched for without finding, a key signal for detecting out-of-stock items or catalog issues. Each layer holds a piece of the puzzle; none can piece together the entire journey alone.
The three fields that decide what can be reconstructed
Three fields determine your ability to diagnose a failed payment: the request duration, the status returned by the upstream, and an identifier that traverses all layers. The Combined Log Format, which is the default on most Nginx and Apache installations, records none of these. It contains seven fields from the Common Log Format plus the referrer and the User-Agent, but ignores timing, upstream status, and any correlation identifier.
Explicitly add the request duration and upstream response time separately, to distinguish your own queue from a slow gateway. Log the upstream status next to the status sent to the client: an edge can transform a 502 into a custom error page that registers as 200. Generate a X-Request-ID at the edge and write it to every layer. Clearly separate the two types of identity: the session identifier says which visitor, the request identifier says which attempt.
IP addresses and timestamps: common pitfalls
Behind a CDN, the IP address that your web server sees by default is that of your own edge, not the client's. Configure the server to read X-Forwarded-For, but trust only the portion added by your edge: a client can send the rest of the header themselves. For timestamps, always write in UTC in ISO 8601 format (year-month-day T hour:minute:second Z), so that sorting events across layers becomes an arithmetic operation rather than a timezone interpretation.
A misconfigured time offset can make an application request appear before the corresponding access line in the web server, making correlation impossible. Keep the same timestamp format everywhere and synchronize server clocks via NTP.
Server logs and JavaScript analytics: two different questions
Client-side analytics like GA4 record what a browser successfully executed. Server logs record what the server was actually requested to do. The requests that matter most during an incident are precisely those that analytics never see. Client-side collection requires the page to load and the script to execute. A request that returns a 5xx, times out, or is cut off mid-response generates no GA4 event. Neither does a visitor who rejects the consent banner or blocks the script.
Logs also capture traffic that no browser produced: bots, scrapers, API clients, mobile apps, autonomous shopping agents, and incoming webhooks like payment callbacks. Analytics read intent and sequence (products viewed, step abandoned). Logs read delivery and failure (what was requested, what was returned, how long it took). An analytics dashboard can look perfectly normal even when the checkout funnel is broken for a subset of customers who never triggered the events that would have revealed the drop.
What level of detail for application logs?
Application logs record business events rather than HTTP traffic. A store can log all requests and still not know which order failed if the application does not trace validations, payment declines, or inventory errors. Record at least order creation (with the identifier), validation failures (relevant business rule), gateway responses (amount, currency, decline code), and fulfillment API calls (shipping status, address error).
Avoid logging sensitive data (full card numbers, CVV, passwords) even in development. Mask or hash personal information if your jurisdiction requires it. Structure logs in JSON to facilitate parsing: a free-text message is difficult to query two months later. Always include the request ID and session ID to link the application event to the web access line.
The database logs we forget to enable
The slow query log in MySQL, PostgreSQL, or others is often not enabled by default. It records queries whose execution exceeds a configurable threshold, connection errors, and lock waits. Without it, it is impossible to know whether a slow checkout process is caused by a poorly indexed join or a lock on the inventory table. Set a realistic threshold (for example, 500 ms) and adjust it according to your load: a threshold that is too low floods the logs, while a threshold that is too high hides emerging issues.
Also log refused connections (saturated pool) and transaction timeouts. These events often signal a traffic spike or a connection leak in the application code. If you use a Redis or Memcached cache, enable its own logs to track evictions, connection failures, and hit rates.
Why log outgoing calls to gateways?
The payment gateway may return a refusal, a timeout, or a technical error that your application interprets and translates into a user message. If you only log the final message ("payment refused"), you lose the exact code returned by the gateway (insufficient funds, expired card, 3D Secure failed, network error). Save the outgoing request (amount, currency, method) and the complete response (HTTP status, business code, message). Anonymize the last four digits of the card if the gateway returns them, and mask the rest.
For fulfillment API calls, track address errors, shipping refusals, and timeouts. This data helps distinguish a problem with the carrier from a bug in your integration. Add the order ID and request ID to link the event to the customer session.
Zero-result internal searches, an invisible signal
The site's internal search captures what visitors typed, how many results appeared, and which searches returned nothing. A zero-result search indicates either an out-of-stock item, or a problem with synonyms or the catalog. It is a signal of purchase intent that you miss if you do not log it. Record the search query, the number of results, the result page viewed, and whether the visitor clicked on a product.
Anonymize queries if they contain proper names or accidentally pasted email addresses. Aggregate frequent zero-result searches to detect missing products or mislabeled variations. An AI chatbot can transform these empty searches into recommendations for similar products or alerts for merchandising.
How long should logs be kept and where should they be stored?
The retention period depends on your regulatory obligations (GDPR, PCI-DSS) and your storage capacity. Keep at least 30 days online to diagnose recent incidents, then archive to cold storage (S3 Glacier, Azure Archive) for audits and long-term analysis. Payment logs fall under PCI-DSS if you handle card data yourself; most merchants delegate this responsibility to the hosted gateway and only retain metadata (amount, status, decline code).
Centralize the six layers into a log management system (ELK, Splunk, Datadog, Loki) to correlate events by request ID. Without centralization, reconstructing a journey requires logging into six different servers and manually cross-referencing timestamps. Encrypt logs in transit and at rest if you store personal information in them. Set alerts on 5xx error rates, gateway timeouts, and zero-result search spikes.
Qstomy and monitoring: understanding after-sales and conversion
Qstomy is a Shopify AI agent that guides visitors toward purchasing (recommendations, upsell and cross-sell, shopping cart, order tracking, customer service). More than a hundred merchants use it to convert, reassure, and build loyalty. Unlike server logs that diagnose technical failures, Qstomy records customer intentions: questions about delivery times, return policy requests, product comparisons. These interactions reveal frictions that analytics do not capture (visitors hesitating over size, worried about delivery times, looking for a promo). By cross-referencing Qstomy conversation logs with the site's technical indicators, you can distinguish between drop-offs due to a bug and those due to an unresolved doubt.
Qstomy also answers questions about offers available online only or in-store only, guiding the customer to the right channel without friction. While server monitoring tells you if the checkout funnel responded, Qstomy tells you why the visitor did not click Order. The two sources complement each other: one for technical availability, the other for relationship availability.
Checklist, in brief, and frequently asked questions
Startup Checklist:
Enable the slow query log of your database with a threshold of 500 ms.
Add request_time and upstream_response_time to the Nginx or Apache log format.
Generate a X-Request-ID at the edge and propagate it to all layers.
Log gateway responses (HTTP status, business code, message) in the application.
Record internal searches with zero results to detect missing products.
Centralize the six layers in a log management system to correlate by identifier.
Set up alerts on 5xx error rates, payment timeouts, and spikes in empty searches.
In short: An online store writes logs at six levels (CDN, web server, application, database, third-party calls, internal search). Without a shared identifier, request duration, and upstream status, it is impossible to reconstruct a failed customer journey. Server logs capture what was requested and delivered; JavaScript analytics capture what managed to execute. The two answer different questions and complement each other. Even a site built for free should enable the minimum: slow query log, request timing, and structured application logs.
Frequently Asked Questions:
Can we rely solely on web server logs?
No. The web server records what it returned but is unaware of why the application rejected the order, why the database was slow, or what the gateway responded. You will know that a request took three seconds but not which layer consumed those three seconds.
How do you correlate logs if each layer has its own timestamp?
Generate a unique request identifier (X-Request-ID) at the entry point (CDN or load balancer) and write it in every log. Then centralize the six sources and query by this identifier. Timestamps are used to order events within the same request.
Do internal search logs reveal sensitive information?
Sometimes. A visitor might accidentally paste an email address or a card number into the search field. Anonymize or hash strings that look like personal data before archiving, especially if you are subject to GDPR.
What is the difference between identifying a session and identifying a request?
The session identifier (cookie, JWT) tells which visitor or account is making multiple requests. The request identifier tells which specific attempt, useful when a visitor refreshes the page or retries a payment. Both are necessary: session for path analysis, request for debugging.
Is a paid monitoring tool needed from the start?
No. Start by enabling native logs (access log, slow query log, application logs in JSON) and store them locally. Even on a free platform, you can export to an S3 bucket or a Git repository. Upgrade to a centralized tool when manual correlation takes too long, generally after a few thousand orders per month.
Enzo
August 26, 2026
