System Logic
TraceLog operates based on the relationship between and .
Probes (Sensors)
Origin points of tests. They can be the application server itself or Remote Probes installed on external infrastructures.
Targets
Monitored destinations (e.g., domains, IP addresses). TraceLog maps the exact path to them.
Events
Notifications automatically generated when the system detects routing changes (hops) or unexpected deviations.
Operation Workflow
Register Targets
Add the destinations you want to monitor. Use our "Known Targets" database to quickly import popular services.
Manage TargetsGenerate SENSOR_KEY
Each remote probe needs a unique key for authentication. Generate them in the Network > Probes menu.
Manage ProbesBulk Import
Save time using the bulk import tool. You can paste a list of IPs/Hosts or select popular services (Google, Cloudflare, etc.) for immediate monitoring.
Go to Targets & ImportProbe Deployment
Use the Docker container for Linux servers and, if your operation already runs Agente NOC, consume TraceLog telemetry through the optional compatibility between both products.
Technical GuideTeam Management
TraceLog is natively multi-tenancy, allowing you to organize your structure into different Teams for data isolation and collaboration.
Total Isolation
Probes and Targets registered in one team are not visible to other teams, ensuring privacy between projects or clients.
Member Assignment
Invite collaborators to your team via email. Define who can only view or who can manage the infrastructure.
Smart Triggers
Avoid "false positives" by configuring triggers based on consecutive occurrences and confirmation windows — every target answers three questions: does it arrive? does it arrive well? which way does it arrive?
Offline Detection
Define how many consecutive Ping failures (100% loss) must occur before triggering an Offline Alert.
Online Recovery
Determine how many consecutive successes are needed to consider the target stable and generate a recovery event.
Performance Degradation
Set the latency threshold (ms), the loss limit (%) and the window (minutes). The first sample above either threshold creates a suspected event; the whole window violating (or two probes agreeing) confirms it; the event is marked recovered after the stability period without violation.
Trace depth (max TTL)
Limits how many hops the probe walks (up to 255). It is a measurement parameter, not a trigger: the hop count is now an attribute of Route Change and exceeding it becomes an observation, with no alert.
Maintenance Windows
Schedule maintenance periods to silence notifications during planned updates. Unlike disabling a target, the maintenance window continues to collect telemetry, allowing you to analyze network behavior during the intervention, but without triggering team alerts.
How it works
- Email/Telegram notifications are suspended.
- Webhooks are not triggered.
- Telemetry collection (latency/route) remains active.
Native Webhooks
Integrate TraceLog with any external system. We POST a JSON to the configured URL whenever a relevant event happens. The "event" field only emits the 6 current types (target_offline, target_online, route_change, final_host_change, routing_loop, performance_degradation).
HMAC Security
Each request includes an "X-Tracelog-Signature" header containing the HMAC-SHA256 hash of the payload, signed with your Secret.
{ "event": "performance_degradation", "target": "8.8.8.8", "details": { "metric": "loss", "status": "confirmed" } }Compatibility with legacy names
Subscriptions using the legacy names are still accepted and normalised on save: packet_loss and high_latency become performance_degradation; hop_count_exceeded becomes route_change. load_balancing is no longer deliverable — it is an observation, not an event — and a subscription that only had that type is rejected with a clear message.
Analysis and Intelligence
Strategic Dashboard
Gain macro visibility over your entire network. A unified view of performance trends enables proactive responses before customers notice service degradation.
Hops Differential
Identify silent changes in transit providers through the ASN history at each hop of the route.
PDF Evidences
Export full reports directly from the Dashboard. Ideal for opening tickets with transit providers. The PDF includes periodic charts, hop matrix, and the consolidated ASN of each hop to prove where the failure occurred.
Focus on Troubleshooting
Reduce MTTR (Mean Time To Repair). Identify if the problem is local, in the transit provider, or at the destination datacenter by visually analyzing the route breakdown.
Comparative Analysis
Comparative Analysis shows the same target as seen by multiple probes at once. Because each probe observes the destination from a different network, comparing the curves reveals whether a degradation lives at the destination, in shared transit, or inside a single probe network.
1. Select the target
Pick the destination in the Target selector. Until a target is chosen, the page only displays the Connectivity Matrix.
2. Choose the probes
Toggle the probe chips to control which series appear. Each probe keeps a fixed color across the chart, stat cards and legend. With no chip selected, all probes are shown.
3. Pick metric and filters
Switch between Latency, Jitter, Packet Loss and Hops. Narrow the data by IP Version (IPv4/IPv6), Transport (ICMP/TCP/UDP) and period. Live mode reloads the series every 30 seconds.
Per-probe stat cards
Each compared probe gets a card with the average, P95 and peak of the selected metric for the period, in the same color as its line on the chart.
Events on the timeline
Vertical dashes mark events on the chart. The summary chips count events by type — click a chip to filter the list, and click an entry in "Latest events" to highlight that exact moment on the chart.
How to interpret the comparison
- All probes degrade at the same time: the problem is at the destination or in a transit segment shared by every path — escalate to the destination or the common carrier.
- Only one probe degrades: the problem is local to that probe network (uplink, peering or last mile). The destination remains healthy for everyone else.
- Probes differ in absolute values but move together: that is geography, not an incident — compare trends, not raw milliseconds.
Matrix as the entry point
The Connectivity Matrix lists the health score of every target and probe pair, worst pairs first. Click a cell to open the comparison of that exact pair — the fastest starting point when you do not know where to look.
Open Comparative AnalysisHealth Score
Every target and probe pair receives a health score from 0 to 100, recalculated whenever a new measurement or protocol check arrives. The score starts at 100 and each factor below subtracts points. Scores are kept separately per IP version and transport protocol, so an IPv6 issue never hides behind a healthy IPv4 path.
Packet loss penalty
Each 1% of packet loss in the most recent sample removes 1 point, capped at 50 points.
Latency above baseline
When latency runs above the learned baseline, the score loses points in proportion to the excess — twice the baseline removes about 20 points, capped at 25. Baselines are learned from the last 100 samples of the pair (minimum of 5).
Failed protocol check
A failed health check (ICMP, TCP connect, HTTP, TLS, DNS or UDP) removes 35 points. A successful check removes nothing.
Normal 71-100
The pair operates within its usual behavior.
Degraded 41-70
Measurable degradation — inspect the pair in Comparative Analysis.
Critical 0-40
Severe loss, sustained latency or failing checks — act now.
The score reflects the most recent state of the pair, not a historical average. The same ranges drive the Connectivity Matrix and the "Critical now" strip on the NOC Wallboard.
Advanced Resources & Metrics
Precise Jitter Calculation
TraceLog calculates Jitter using the RFC 3550 standard, which represents the mean absolute difference between consecutive delays. This is critical for real-time applications like VoIP and Video Streaming, where variation matters more than absolute latency.
Trace depth and hop count
Trace depth (max TTL, up to 255) sets how far the probe walks. The hop count is no longer an alert: it travels as an attribute of Route Change (before/after/delta) and, when it exceeds the configured depth, it is recorded as the hop_count_exceeded observation.
Conditional Thresholds
Set advanced criteria: receive alerts only if latency exceeds 200ms OR packet loss is greater than 10% for the full configured window, such as 5 continuous minutes. This eliminates noise from transient oscillations.
Multi-Protocol (v4/v6) Parity
Monitor the same target via both IPv4 and IPv6 simultaneously from the same Probe. Analyze path differences and compare performance between protocols in real-time.
Routing Intelligence (Loops vs Load Balancing)
TraceLog automatically tells load balancing behaviour from critical routing anomalies. Consecutive repetitions, diamond patterns and final-hop rotation inside the same ASN are load balancing observations (ECMP is normal). Only repeated sequences (A->B->A->B->A->B) are flagged as Routing Loop; the same IP answering 10+ times is a router answering for the remaining TTLs and becomes the repeated_hop observation.
Event Hierarchy and Coexistence
A single trace may generate more than one event when distinct causes happen together. Example: if intermediate hops changed and the final destination also moved, TraceLog can emit both Route Change and Final Host Change. If only the final hop changed, it does not degrade to a generic Route Change.
Event Glossary & Intelligence Criteria
TraceLog classifies network incidents into six types, organised in three families: availability (does it arrive?), quality (does it arrive well?) and path (which way does it arrive?). Every event is fired by specific technical criteria and carries status, confidence and evidence to separate confirmed incidents from suspected or merely observed behaviour.
Availability — does it arrive?
100% loss (or the return below it) over N consecutive samples, evaluated per probe and per protocol. The target only shows offline on the map when any probe/protocol pair is offline.
- Alvo Offline
- Alvo Online
Quality — does it arrive well?
A single quality event: latency and/or loss above the target limits. The violated metric lives in details.metric (latency, loss or both) and the event moves from suspected to confirmed and recovered.
- Degradação de Desempenho
Path — which way does it arrive?
Change in the ASN sequence of intermediate hops, final host moving to another ASN and routing loop by repeated sequence. Load balancing and hop count excess stay as observations.
- Mudança de Rota
- Mudança de Host Final
- Loop de Roteamento
Confidence, Status and Evidence
Every event keeps its technical type, but details_json also includes status, confidence, confidence_level and evidence. Confirmed means the signal was validated by persistence, a strong topology criterion or consecutive samples. Suspected means the signal is real in the sample but still needs persistence or another probe to be treated as definitive. Recovered marks a degradation that went through the stability period without violating. Observed is informational behaviour, such as load balancing, that explains the route without being treated as an incident on its own.
Target Offline
Criteria: 100% packet loss detected.
Logic: The system waits for a certain amount of consecutive failed checks (defined in triggers) before flagging the host as down. This avoids "flap" alerts during brief network blips.
Target Online (Recovery)
Criteria: Packet loss drops below 100% (restoration of connectivity).
Logic: Triggered when the host returns to a responsive state. Similar to offline detection, it respects a "stability" threshold of consecutive successful checks.
Performance Degradation
Criterion: latency AND/OR packet loss above the limits set on the target — the single quality event (it replaces High Latency and Packet Loss).
Logic: The violated metric lives in details.metric (latency, loss or both), with the measured values, the thresholds and the affected hops. 100% loss is not degradation — it is availability (Target Offline). The event has its own lifecycle:
- Suspected: first sample above the threshold (what used to be High Latency or Packet Loss). Notified only when "Also notify suspected events" is on.
- Confirmed: the whole window (condition_duration_minutes) violates the threshold — with at least two samples and a real span between them — or a second probe sees the same degradation within the confirmation window. Repetition on the same probe never confirms.
- Recovered: after recovery_stability_minutes without any violation.
Route Change
Criterion: the ASN sequence (transit) of the intermediate hops changed — or intermediate IPs changed without ASN evidence that they stayed on the same carrier. It carries old_hop_count, new_hop_count and hop_count_delta.
Logic: Detects routing instability or "flapping" when the transit really changed. Info severity by default; raised to warning when a Performance Degradation or Target Offline of the same target exists within the confirmation window (in both directions), with the correlated event in details.correlated_event_id. Hops that swap IP inside the same ASN — even one hop longer or shorter, as in ECMP — are a load balancing observation, never a Route Change; a private hop never counts as transit and a timed-out hop (???) is not a difference. If only the final hop changed, it is Final Host Change. If the IP changed but the hop ASN is unknown, the Route Change is still logged, but only as suspected. A destination that stops answering does not create a Route Change; that is availability.
Final Host Changes
Criteria: Change in the last valid hop with a real final destination/ASN change.
Logic: Used when the last valid hop moves to another ASN (or an unknown ASN). A single sample is marked suspected because classic traceroute/MTR does not always prove that the application final host changed; persistence, multiple probes or protocol agreement strengthen the diagnosis. If the destination merely rotates inside the same ASN, TraceLog records a load balancing observation instead of Final Host Change.
Routing Loop
Critical Priority
Sequence Loop: The only loop detector: an IP pattern (e.g. A->B->A->B) repeating 2 or more times (A-B A-B A-B).
Repeated hop (observation): A single IP answering 10 or more times in the same trace is NOT a loop — it is a router answering for all the remaining TTLs (MPLS/NAT). It becomes the repeated_hop observation, with no alert.
Anti-FP Logic: Private IP ranges and short consecutive repetitions are suppressed to avoid false positives from internal balancers or ECMP behaviour.
Observations vs Incidents
TraceLog stores technical observations (network_observations) separately from actionable events. An observation never triggers e-mail, Telegram or a webhook — it explains the path in the measurement detail. A signal becomes an incident when it persists, reaches the destination, affects real per-protocol checks or shows up on multiple probes. Today these are observations:
- load_balancing — ECMP is normal: diamond patterns, consecutive repetitions, destination or internal-hop rotation inside the same ASN. Recorded when "Record Load Balancing Observations" is on for the target.
- hop_count_exceeded — the destination was reached with more hops than the configured trace depth. The hop count is an attribute of Route Change; on its own it gives the NOC nothing to act on.
- repeated_hop — the same IP answering 10 or more times (the former "distributed loop" criterion).
- intermediate_hop_loss — an intermediate hop dropping ICMP while the following hops answer (rate limiting, not real loss).
Event Hierarchy Summary
- Only the last hop changed: same ASN = load balancing observation; different ASN/final destination = suspected Final Host Change until it persists or shows in more evidence.
- Intermediate hops changed: same ASN path with ASN evidence = load balancing observation; ASN path changed = confirmed Route Change; IPs changed without ASN evidence = suspected Route Change.
- The transit (ASN path) of the intermediate hops changed and the final destination changed too: Route Change coexists with Final Host Change in the same test.
- Routing loop takes priority over the load balancing observation when the route itself shows a real cyclic sequence (A->B->A->B).
- Route Change with a Performance Degradation or Target Offline of the same target in the same window: the change is raised to warning and points to the correlated event; both remain separate events.
Alert Customization
TraceLog allows you to be notified instantly when critical events occur, ensuring a rapid response to network incidents.
Email Channels
Configure multiple email destinations separated by commas. Ideal for NOC distribution lists or support teams.
Telegram Integration
Receive real-time alerts on your mobile. You will need a Bot Token (generated by @BotFather) and the destination Chat ID.
How to Test your Configuration
- Access the Settings > Notifications menu at the top of the page.
- Enable the desired channel (Email or Telegram).
- Fill in the necessary data.
- Click the "Test Notification" button. The system will send an immediate example event.
Notification Criteria
- All Events: Any detected change generates an alert.
- Route Change (Internet): Indicates rerouting in the intermediate public internet path (Transit/ASN).
- Performance Degradation (quality): Latency or packet loss above the target thresholds: suspected on the first sample, confirmed when the whole window violates. Load balancing is an observation and never notifies.
- Final Host Change (App): DNS failover or server migration when the final destination really moved to another ASN/destination context.
- Combined Conditions: More than one alert may be generated from the same test when distinct causes coexist.
Problem Resolution (Troubleshooting)
Probe does not report data
- Check if the SENSOR_KEY in the container/agent is correct.
- Ensure the host has access to port 80/443 of the TraceLog server.
- In Docker, use "docker logs [container_id]" to see connection errors.
Incomplete Route (Asterisks)
This occurs when an intermediate router blocks the ICMP or UDP protocol from the probe. TraceLog will continue trying to map other hops to complete the overview.
Target always Offline
Check if the destination firewall allows ICMP (Ping) packets. Some infrastructures (like AWS/Azure) require explicit configuration in the Security Group.
False Packet Loss (Rate Limiting)
Some hosts (like Cloudflare or DDoS protected IPs) drop ICMP packets when the frequency is high. If you see intermittent loss, try switching the Target Protocol to TCP or UDP in the target settings.
Agente NOC Compatibility
Agente NOC is an optional compatibility layer. TraceLog remains responsible for probes, telemetry, dashboards, and historical analysis while the agent can consume this data for customers who already use it.
Probe Re-deployment
If you move a sensor to a new location or network, use the "Clear History" action in the sensor menu to reset metrics without generating a new API Key.