- If you are logged into the Analytics dashboard, then a web-socket has been activated between your browser and GA. For fast lookup, there's probably an in-memory hash of GA IDs, web-socket IDs.
SO:
- User visits your webpage / performs some action worth analyzing.
- GA script gets executed; sends request to GA server.
- GA server acknowledges reception and schedules jobs for:
-- storing the hit for associated GA-ID.
-- sending info of hit to websocket associated with GA-ID.
Likely by processing different data at different times.
E.g. big aggregates and time series are likely updated on a lower interval whilst near-time event data is probably stored in fast recall storage so that you can satisfy near-time queries immediately.
Otherwise, their long view queries are always sampled. They control the fields available so they can make sure samples are properly stratified in order to be representative.
IMO, it's just good old fashioned good engineering. It could be done with no special tech whatsoever. E.g. sharded Postgres DBs and network storage would do just fine.
Pre aggregated reports. Custom reports have `using x out of y sessions`. Custom reports using bigquery or something similar and more compact/specialized.
10 comments
[ 3.2 ms ] story [ 35.6 ms ] thread- If you are logged into the Analytics dashboard, then a web-socket has been activated between your browser and GA. For fast lookup, there's probably an in-memory hash of GA IDs, web-socket IDs.
SO:
- User visits your webpage / performs some action worth analyzing.
- GA script gets executed; sends request to GA server.
- GA server acknowledges reception and schedules jobs for:
-- storing the hit for associated GA-ID.
-- sending info of hit to websocket associated with GA-ID.
E.g. big aggregates and time series are likely updated on a lower interval whilst near-time event data is probably stored in fast recall storage so that you can satisfy near-time queries immediately.
Otherwise, their long view queries are always sampled. They control the fields available so they can make sure samples are properly stratified in order to be representative.
IMO, it's just good old fashioned good engineering. It could be done with no special tech whatsoever. E.g. sharded Postgres DBs and network storage would do just fine.
You could look at pre-calculation, subsetting the data, in-memory databases, dedicated servers, non-blocking requests, etc.