Practice an Analytics Instrumentation Interview Before a Launch
Prepare analytics instrumentation from product questions, exact event meanings, minimal properties and controlled prelaunch verification.
TL;DR
- In this fictional document-sharing launch, choose analytics from the decision the team needs to make.
- Define event meaning, minimum useful properties, and the authoritative success event.
- Verify the contract before launch and assign ownership for changes and failures afterward.
Begin with a launch question
In this fictional interview, a team is launching a document-sharing feature and asks you to add analytics. The initial request is to “track everything.” The interviewer wants to know how you would design the instrumentation.
Start with the product decisions the data should support. The team may need to know whether users can create a share, whether recipients can open it and where failures occur. Those questions suggest a small event contract. They do not automatically justify recording document contents or every pointer movement.
Separate product analytics from operational logs and security records. The systems may share some identifiers under approved rules, but their purposes, access and retention can differ. A vague analytics request should not become a universal data collection mandate.
Specify event meaning before event names
Choose events that correspond to clear states. A user clicking Share is an attempt; a server creating an authorised share is a different event. A recipient viewing the page is different from successfully accessing the document.
| Product question | Candidate event | Boundary to define |
|---|---|---|
| Do users attempt sharing? | Share action initiated | Client interaction, not success |
| Are shares created? | Share created | Durable authorised result |
| Can recipients use the link? | Share access completed | Actual permitted document access |
| Where does the flow fail? | Share operation failed | Defined error class, minimal data |
The exact names are illustrative. What matters is a definition that different engineers implement consistently. A dashboard cannot repair an event whose meaning varies between clients.
Choose minimum useful properties
For each event, list the property, purpose and permitted values. A schema version, platform or safe operation identifier may help analysis. Full document text, recipient addresses or unrestricted error payloads may be unnecessary for the stated questions.
Decide how users and sessions are represented under the product's approved privacy and consent model. Do not invent an identity-stitching policy during implementation. If anonymous and authenticated activity cannot be linked reliably or appropriately, keep that limitation visible in the analysis.
Define which timestamps are included and what they mean. Client occurrence time and server receipt time can differ. If offline events are possible, downstream analysis needs a deliberate rule rather than an assumption that arrival order describes the journey.
Assign an authoritative success event
If both client and server emit “share created,” the funnel may double count successful operations. Choose one authoritative source for the state transition, or explicitly model the two observations as different events.
Retries also need a defined identity policy. A user repeating a failed action may create a new attempt, while a transport retry of the same event should not necessarily create another analytical occurrence. Explain the distinction before introducing deduplication.
For the fictional feature, a server-side durable share record can support the success event. The client interaction still matters for understanding abandoned or failed attempts, but it should not be treated as evidence that the share was created.
Verify the contract before launch
Write a small set of test journeys: successful share, validation failure, permission denial, repeated click, network interruption and recipient access after the share is revoked. Inspect the emitted events and properties for each.
Compare event counts with the underlying operational records in a controlled test. If three shares were created, explain why the analytics view should show three successes and how long ingestion may take. Do not accept a dashboard that merely displays some activity as proof of correct instrumentation.
Check negative expectations too: no document body in the event payload, no success event on denied access and no accidental production event from an isolated test environment. These checks directly support the contract rather than mirroring implementation details.
If the experiment or launch uses multiple client versions, test their event schemas together. An older client may omit a new property or emit a previous event name. Decide how analysis recognises that difference instead of interpreting missing fields as genuine user behaviour or silently excluding the older population.
Plan ownership after release
Give each event definition an owner and a change process. If the product later changes what “share completed” means, version or migrate the contract deliberately. Quietly reusing a name for a different behaviour can make historical comparisons misleading.
Monitor basic data quality such as missing required properties, unexpected values and unexplained divergence from authoritative counts. A collection pipeline can be technically available while the events it receives are semantically wrong.
Our system design preparation guide can help connect the event contract to the broader feature design. Use the mock interview strategy to rehearse a changed requirement, such as supporting offline share attempts.
End with the decision questions, event definitions, minimal properties and verification cases. A strong instrumentation answer makes the eventual analysis interpretable. It does not confuse collecting more data with learning more about whether the feature works.