learning-it by example

#151 Python for data 6.6 min

Regex for Data Cleaning: Extract IDs and Phones in Python

Seven made-up messy notes: two regex patterns pull out 5 order ids and 5 phone numbers, and miss 1.

1. What three steps does the regex video use to clean messy text?
2. On seven made-up support notes, how many order ids and phone numbers did the two patterns find, and what was missed?
3. The video's phone pattern missed a number with the same digits as the others. What does that show?
4. Why does the video use re.fullmatch rather than re.search to check that a string is exactly five digits?
5. The order id pattern has one group around the five digits. Why does the video's order_ids function put back an ORD- prefix?

Practice only: nothing is saved and no score counts towards anything. Each answer comes with the reason.

  1. Spot the greedy match

    Copy the code folder first. In run.py, in the traps section, change the greedy pattern r"\(.*\)" to r"\([^)]*\)". Predict whether the first line of python run.py traps will still return one long match, then run it and compare it with the non-greedy line.

    A hint

    [^)]* means any characters except a closing bracket.

  2. Accept four-digit ids

    In clean.py change \d{5} in the ORDER pattern to \d{4,5}. Predict what happens to note 5, which has the short id ORD-1023, then run python run.py ids. On my machine note 5 then gave ['ORD-1023'] and the unique list grew to six ids. Decide whether that loosening is what you want.

    A hint

    The video rejects the short id on purpose.

  3. Teach the pattern a new spelling

    Add a new last note to NOTES in notes.py: "Please check Ord # 10250". Predict whether python run.py ids finds it, then run it. Next change [\s#-]? in the ORDER pattern so that it allows a space then a hash, and run again. On my machine the original pattern found nothing in the new note.

    A hint

    [\s#-]? allows only one separator character. Think about what * would allow.

  4. Rescue the missed phone

    Note 5 (0770 090 0999) is missed because its digits are grouped 4, 3, 4. In clean.py write a second compiled pattern for that grouping, with three groups, and change phones() so that it returns numbers found by either pattern in the same +44 form. Run python run.py phones and check that note 5 now gives a number and the other notes give what they gave before.

    A hint

    The second pattern needs a 0, then 7 and two more digits, then three digits, then four digits, with an optional space or dash between them.

These tasks are for you to try on your own machine. Nothing is checked or saved.

What this does not show

Seven made-up notes and patterns for only the formats shown, so the run says nothing about real, larger or multilingual text.

Up next in Python for data

More in Python for data

All Python for data videos

Related videos