Data Processing and Freshness
Reveal doesn’t just analyze raw shipment data; it makes sure the data powering your insights is clean, accurate, and up-to-date. Every completed shipment is reprocessed through a rigorous data pipeline, so the results reflect what actually happened, giving you a reliable foundation for decisions.
How It Works
Reveal reprocesses shipments after they’re complete to clean up the data and give you a clear view of what really happened. Our data cleaning filters out approximately 20% of raw shipments that would otherwise distort metrics.
What Data Gets Included
- Completed Shipments Only: Reveal waits until the full shipment journey is finished to capture complete context.
- Regular Updates: Shipments are reprocessed about every 6 hours and re-checked for up to 30 days to capture any late-arriving data.
- Multi-Use Device Shipments: If a device is used for a new shipment or you manually mark a shipment complete, Reveal stops adding new data to the original shipment and starts aggregating data for the new shipment so your results stay clean and accurate.
How Shipment Data Gets Cleaned
- Location Smoothing: Removes errant GPS points and collapses tight clusters to show the most likely path, improving the accuracy of distance and speed metrics.
- Reconstructed Events: Confirms true departure, transit, and arrival times using geofences when available, or proximity to the provided address (starting at 1 km, expanding up to 5 km).
- Invalid Shipments Identified: Shipments that never reach within 5km of a valid location are treated as invalid. The shipment can be excluded using the “invalid shipments” toggle to ensure clean data.
- Backfilling: Once a shipment is complete, Reveal adds back sensor data from the 24 hours prior to the shipment’s scheduled start. This ensures we capture condition data at the origin that can be used for deeper analysis.
- Stage Windows & Metrics: Defines shipment stages (At Origin, In Transit, At Leg Transitions, and At Destination) and calculates key metrics including duration, distance, speed, temperature, time out of range, on-time performance, and alerts, based on the cleaned, smoothed timeline.
Data Cleaning In Action
Example A: Shipment Never Arrived. Without cleaning: A shipment that never arrives would show 10+ days of excess duration and temperature excursions, skewing metrics. With cleaning: The shipment is excluded from the analysis, so your metrics remain accurate.
Example B: Premature Arrival. Without cleaning: A shipment is prematurely marked as “arrived” 3 days earlier than its actual time. With cleaning: The arrival is only recorded when the shipment enters the correct geofence, ensuring an accurate duration.
Example C: Incorrect Start Date. Without cleaning: The first leg of a journey is not included, making the total shipment duration look shorter than it really was. With data cleaning: Reveal includes the full shipment journey in the analysis because the tracker was turned on at the correct origin (point 1).
Filtering Data for Clearer Analysis
Reveal includes filters to help you focus on specific shipments or specific parts of a shipment’s journey that matter most. You can also use advanced settings to swap between units for temperature, distance, and time, giving you flexibility in how you view and compare performance
Global Filters: Apply across all pages, and stay active as you switch between views:
- Date range (defaults to last 90 days)
- Comparison period (for % deltas)
- Exclude Invalid Shipments (on by default).
- Commodity and Commodity Category (defaults to all)
- Count(s) (defaults to all)
Page Filters: Apply only to the current page you’re viewing:
- Lane Aggregation: Group by City (recommended for precision) or Country (useful for high-level global supply chains).
- Origin/Destination: Choose which shipment endpoints you want to include.
- Shipment Stage: Focus metrics on a specific part of shipments’ journey:
- At Shipment Origin: Before the first departure.
- In Transit: Between departures and arrivals at all legs, origin, and destination.
- At Shipment Leg Transition: During handoffs between legs.
- At Shipment Destination: After the final arrival.
- Anytime: The full journey window.
Key Points to Know
- Out of the Box Clean, Accurate Data: Reveal provides insights you can trust. After completion, shipment data is reprocessed to smooth GPS points, reconstruct events, and remove invalid shipments.
- Continuous Updates: Shipments are refreshed approximately every 6 hours and checked for up to 30 days to capture late-arriving data.
- Stage Windows & Metrics: See performance across origin, in transit, leg transitions, and destination by leveraging available filters.
- Filters for Precision: Use global and page-specific filters—including lane aggregation, origin/destination, and shipment stage—to focus your analysis on the right shipments.
Best Practices
To get the most accurate insights:
- Keep Data Cleaning On: Leave Exclude Invalid Shipments turned on. This setting is enabled by default to ensure your metrics are clean and accurate
- Enter Accurate Shipment Data: Enter precise addresses, geofences, and scheduled arrival times to ensure reliable metrics. Read more about locations to make this easy.
- Include Carrier Information: When creating shipments, be sure to add carrier details to shipment legs to unlock detailed performance insights.
- Apply Stage Filters: Focus analysis on the relevant journey period (At Origin, In Transit, At Destination) for meaningful insights. Stage filters help you zero in on the most relevant time period—for example, evaluating a carrier’s in-transit temperature compliance—without mixing in conditions at the origin or destination.
Need Help?
If you have additional questions, reach out to support@tive.com for assistance.