> ## Documentation Index
> Fetch the complete documentation index at: https://www.towbar.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Service analytics

> See requests, website pageviews, and optional visitor estimates with Scout Agent.

Scout analytics shows how much traffic reaches a public service and which pages receive it. Start with request counts. Add a small script to your website when you also want pageviews. Collection is off until you enable it.

## Turn on request analytics

Install or update Scout Agent from the server’s Scout settings. Add this to your service file, then deploy the service:

```yaml theme={"system"}
analytics:
  enabled: true
```

Open **Services → your service → Analytics** and select **HTTP requests**. Visit the service; data usually appears within a minute. You can see request counts, errors, response times, and the most requested paths.

Requests include API calls, images, scripts, and bots, so request counts do not tell you how many people visited. Analytics is available for single-container services with a public primary domain.

## Count website pageviews

To count pages viewed in a browser, add `pageviews: true` and deploy:

```yaml theme={"system"}
analytics:
  enabled: true
  pageviews: true
```

Put this script once in your site’s shared page template:

```html theme={"system"}
<script defer src="/.well-known/towbar-analytics/script.js"></script>
```

Select **Website pageviews** in Analytics. The script counts the first page and navigation within your site, including single-page applications. It is served from your own service domain. It does not measure clicks, conversions, or engagement time. Blocked scripts and disabled JavaScript reduce the count.

## Optional visitor and session estimates

Pageviews work without identifying visitors. Turn on visitor estimates only if you need to distinguish repeat pageviews from separate browser visits and the setting fits your site’s privacy and consent choices:

```yaml theme={"system"}
analytics:
  enabled: true
  pageviews: true
  visitorIdentity: true
```

This adds random IDs in the browser’s local storage. Counts represent browser identities, not people: shared browsers, blocked storage, and deleted IDs affect the estimate. IDs are never derived from an IP address or a browser fingerprint. Leave `visitorIdentity` out to keep visitor and session estimates off.

## Technical details

The sections below explain collection limits, privacy, storage, and how to diagnose missing data. They are useful when operating a busy service or reviewing what analytics stores.

### Settings and troubleshooting

Retention defaults to 30 days and accepts `7`, `30`, or `90`. `pageviews` and `visitorIdentity` default to `false`. Use `excludePaths` to leave sensitive routes out of both request and pageview collection:

```yaml theme={"system"}
analytics:
  enabled: true
  pageviews: true
  retentionDays: 7
  excludePaths:
    - /account
    - /checkout
    - /api/health
```

Exclusions match path prefixes. Set them before collection starts: paths can contain personal information even when query strings are removed.

Compose workloads, datastores, preview deployments, and redirect hostnames are not included. Requests served entirely by a CDN cache never reach the service and do not appear in request analytics.

Scout needs free loopback ports 9468 (UDP) and 9469 (TCP). If they are unavailable, analytics is disabled and performance reporting continues. Scout sends batches about every 30 seconds; allow up to a minute for initial configuration and collection. A warning in Analytics means Scout needs attention on the service’s server.

### Browser behavior and identity storage

The script reports the initial page and pathname changes through `pushState`, `replaceState`, and browser back/forward navigation. Changes limited to a query string or fragment do not create another pageview. A page restored from the browser’s back/forward cache counts as a new pageview. Scroll depth and bounce rate are not measured.

Allow your own origin in your Content Security Policy’s `script-src` and `connect-src`. The script honors Do Not Track and Global Privacy Control. Privacy signals, offline browsers, and failed requests reduce reported pageviews. Public analytics endpoints cannot prove that every event came from a human.

When visitor identity is enabled, cryptographically random visitor IDs expire after 30 days and session IDs expire after 30 minutes without a reported pageview. Scout hashes each ID with the service ID before upload, so stored identities are scoped to one service. Removing `visitorIdentity` stops accepting new identity data; existing history expires with retention. Removing the script stops browser collection.

### What each source measures

| Dimension             | Caddy request analytics                                                | Browser pageviews                                         |
| --------------------- | ---------------------------------------------------------------------- | --------------------------------------------------------- |
| Counts and trends     | Requests, including assets, API calls, and bots                        | Reported pageviews and SPA pathname changes               |
| Path                  | Request pathname                                                       | Page pathname                                             |
| Referrer              | Referrer hostname when sent by the client                              | Referrer hostname when available                          |
| Response              | HTTP status, method, response bytes                                    | Not measured                                              |
| Latency               | Total Caddy request duration, mean, histogram, approximate percentiles | Not measured                                              |
| Country               | Not collected                                                          | Local lookup of the verified client IP                    |
| Browser and device    | Not collected                                                          | Coarse user-agent family; bot classification is heuristic |
| Visitors and sessions | Not collected                                                          | Optional random browser IDs                               |

Referrer paths, queries, and credentials are discarded. Missing referrers are **Unknown**, not proof of a direct visit. Percentiles use the upper bound of the containing latency bucket: 10, 50, 100, 200, 500 ms, 1 and 2.5 seconds, followed by an unbounded bucket over 2.5 seconds. The first bucket is strictly below 10 ms; later bounded buckets include their upper limit. Caddy duration includes proxy handling and response transfer; it is not application CPU time or browser page-load time.

Country lookup is available for browser pageviews only. Caddy’s [network log writer](https://caddyserver.com/docs/caddyfile/directives/log#net) falls back to stderr when its socket fails. Including the remote peer or Cloudflare client-IP headers in those records could therefore persist raw addresses in system logs before Scout could discard them. Request analytics removes those fields before logging and does not estimate country. The pageview endpoint passes the address directly to Scout for an in-memory lookup without adding it to the analytics log.

Pageview country lookup uses the direct connection address for ordinary services. For `cloudflare-dns`, Scout accepts Cloudflare’s client-IP headers only when the connection comes from a published Cloudflare network. For `cloudflare-tunnel`, it accepts those headers only from the running, Towbar-managed tunnel container belonging to that service. The local Docker snapshot must be less than a minute old. Unverified Cloudflare connections, missing headers, private addresses, and addresses belonging to Cloudflare itself are Unknown. Original IPv6 addresses are preferred when Cloudflare uses pseudo IPv4.

Cloudflare network ranges ship with Scout; newly added ranges remain untrusted until an agent update. Other CDNs are not recognized and may appear as the connection’s country. Cloudflare Workers can change the client address, so country estimates are not a security or identity signal. See [Cloudflare’s header behavior](https://developers.cloudflare.com/fundamentals/reference/http-headers/). IP geolocation is approximate; it does not locate an individual.

### Local geolocation database

Scout downloads [DB-IP Country Lite](https://db-ip.com/db/download/ip-to-country-lite) over HTTPS on initial startup and stores an MMDB file locally under `/var/lib/towbar-monitoring/geo/`. No visitor IP is sent to DB-IP or Towbar’s control plane. The Countries header’s info icon identifies DB-IP as the provider and links to this documentation. IP Geolocation is provided by [DB-IP](https://db-ip.com).

DB-IP publishes Lite editions monthly under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/). A durable Temporal workflow requests a refresh every 24 hours. Agents check the current edition, bound compressed and expanded download sizes, validate MMDB structure and build metadata, reject older editions, and atomically replace the file. An already installed current-month edition needs no download. Failed refreshes retry after an hour and keep the last valid database. Without a valid file, country is Unknown and other analytics continues.

We compared [ip-location-db](https://github.com/sapics/ip-location-db), whose original country datasets publish daily under PDDL but whose documented upstream geofeeds can lack explicit licenses, and [MaxMind GeoLite](https://dev.maxmind.com/geoip/geolite2-free-geolocation-data/), which has additional account and license obligations. DB-IP provides a direct provider download and clear attribution requirement without an account or license key. A daily check does not imply daily source releases.

### Privacy, reliability, and retention

Caddy removes the request object and response headers before emitting its analytics log. It keeps only the pathname, method, sanitized referrer host, status, duration, byte count, and service identity. Browser requests are validated against the configured service and exact HTTPS Origin. The IP is used transiently for local country lookup and discarded. Scout never uploads raw IP addresses, headers, query strings, fragments, or raw user agents.

Caddy sends records over loopback UDP so analytics does not depend on a remote collector being available. This is best-effort observation: UDP loss and Scout downtime cannot be reconstructed. Caddy may emit the same sanitized record to its system log when a network writer fails. Scout accepts at most 1,000 incoming records per second and bounds each pending batch to 512 dimension combinations across the server; new combinations beyond that limit are dropped and reported. Pending records expire after one minute if the performance collector stops producing snapshots. An interrupted process can lose the pending batch. High-volume services with many unique paths or visitor IDs can therefore undercount.

Accepted batches use Scout’s existing bounded, private retry directory: up to 10 MiB for one hour. Uploads use the server’s revocable credential; replayed sample IDs do not duplicate metrics or analytics. The control plane validates current service ownership and manifest opt-in again before storing a batch. Revoking Scout’s credential rejects uploads. Configuration older than two minutes stops local analytics acceptance. Removing analytics from the manifest stops server acceptance immediately after synchronization; redeploy to remove its Caddy routes.

Reports require `scout.read` and are scoped to the authenticated workspace. Data is retained for the manifest’s duration and deleted by daily Temporal maintenance. Shorter retention applies to report queries immediately and to stored rows on the next sweep. Deleting a service cascades its analytics history. Backups follow their own retention policy. Aggregates and optional pseudonymous identity hashes live in Towbar’s PostgreSQL database; no third-party analytics service receives them.

### Filter by path

Select **Filters** beside the Analytics title. Choose **Path**, then **is** for an exact path or **starts with** for a prefix, enter the path, and select **Apply**. The button shows the number of active filters. Filters stay active when switching between HTTP requests and pageviews or changing the time range. Use **Clear filters** in the modal to reset them. You can also hover a row in **Paths** and select its filter icon to add an exact-path condition immediately. An already active exact-path condition is not added twice.

All conditions must match. For example, `/docs` as a prefix includes `/docs`, `/docs/api`, and `/docs-old`; use `/docs/` to match only paths beneath that directory. Matching is case-sensitive and treats `%` and `_` literally. Enter paths without query strings or fragments. Filters scope totals, trends, breakdowns, and the previous period consistently. A previous period with no matching activity is shown as unavailable.

### API

`GET /v1/core/apps/{appId}/analytics?kind=request&days=7` returns request totals, trends, ranked dimensions, and a latency histogram. Reports also include `comparison` totals and an aligned trend for the immediately preceding period of equal length. Comparison is unavailable when that period exceeds configured retention or has no collected samples. Tables describe only the selected period. Use `kind=pageview` for browser analytics. `days` accepts 1–90 and is clamped to service retention. Browser-only response dimensions are absent from request reports, and latency fields are not meaningful for pageviews.

Pass an optional `filters` query parameter containing a URL-encoded JSON array of up to eight conditions:

```json theme={"system"}
[{ "field": "path", "operator": "startsWith", "value": "/docs/" }]
```

Each condition has a field, operator, and value. Supported operators are `equals` and `startsWith`; `path` is currently the only supported field. Conditions use AND, and the response echoes the applied `filters`. Invalid fields, operators, or values return HTTP 400. Each path value is limited to 256 characters. Omitting `filters` returns the unfiltered report.

The collection format follows [Caddy’s structured access log](https://caddyserver.com/docs/caddyfile/directives/log) and [log\_append](https://caddyserver.com/docs/caddyfile/directives/log_append). Browser navigation follows the [History API](https://developer.mozilla.org/en-US/docs/Web/API/History/pushState).