learning-it by example

#266 Data modelling 6.4 min

Event Data Modelling: Events, Sessions, Late Arrivals

Two late clicks made an append-only job report 7 sessions; rebuilt from event time there were 4.

1. In event data, what is the difference between event time and received (processing) time?
2. In the clickstream example, a job that only sessionised the new batch and appended the result reported 7 sessions. What did rebuilding from event time give?
3. How does the session query decide where a new session starts for a user?
4. What happens in the example when event e03 is delivered a second time with the same event id?
5. The example rebuilds every session from scratch each time. What limit does the video state about this?

Practice only: nothing is saved and no score counts towards anything. Each answer comes with the reason.

  1. Predict the sessions

    Copy the code folder first. Read the events in events.py and predict how many sessions each user has after batch 1, using a gap of more than 30 minutes. Run python run.py first to check, then predict the result of python run.py late and run it.

    A hint

    User 1 has a break between 09:20 and 10:05.

  2. Change the session gap

    In sessions.py, change GAP_MINUTES from 30 to 45. Predict how the batch 1 session count changes, then run python run.py first and python run.py late. When I ran it, batch 1 gave 2 sessions, appending gave 6 and rebuilding from event time still gave 4.

    A hint

    User 1's break between 09:20 and 10:05 is exactly 45 minutes.

  3. Make one late event on time

    In events.py, change the received time of e08 in BATCH_2 from 10:58 to 09:36. Predict how many late events are reported and how the append count changes, then run python run.py late. When I ran it, only e09 was reported late, appending said 6 and rebuilding still said 4.

    A hint

    The append job only looks at rows received after 10:30.

  4. Add a new user event

    In events.py, add the event ("e12", "u3", "10:55", "10:55", "item") to BATCH_2, after the e03 line. Predict whether user 3 gets a second session, then run python run.py late. When I ran it, 6 were delivered and 5 were new, and user 3 had one session from 10:50 to 10:55 with 2 events.

    A hint

    A session starts a new one only when the gap is more than 30 minutes.

These tasks are for you to try on your own machine. Nothing is checked or saved.

What this does not show

Eleven made-up events in SQLite, rebuilt in full each time, with no lateness cut off, one time zone and no speed measured.

Up next in Data modelling

More in Data modelling

All Data modelling videos

Related videos