#151 Python for data 6.6 min
Regex for Data Cleaning: Extract IDs and Phones in Python
Seven made-up messy notes: two regex patterns pull out 5 order ids and 5 phone numbers, and miss 1.
Spot the greedy match
Copy the code folder first. In run.py, in the traps section, change the greedy pattern r"\(.*\)" to r"\([^)]*\)". Predict whether the first line of python run.py traps will still return one long match, then run it and compare it with the non-greedy line.
A hint
[^)]* means any characters except a closing bracket.
Accept four-digit ids
In clean.py change \d{5} in the ORDER pattern to \d{4,5}. Predict what happens to note 5, which has the short id ORD-1023, then run python run.py ids. On my machine note 5 then gave ['ORD-1023'] and the unique list grew to six ids. Decide whether that loosening is what you want.
A hint
The video rejects the short id on purpose.
Teach the pattern a new spelling
Add a new last note to NOTES in notes.py: "Please check Ord # 10250". Predict whether python run.py ids finds it, then run it. Next change [\s#-]? in the ORDER pattern so that it allows a space then a hash, and run again. On my machine the original pattern found nothing in the new note.
A hint
[\s#-]? allows only one separator character. Think about what * would allow.
Rescue the missed phone
Note 5 (0770 090 0999) is missed because its digits are grouped 4, 3, 4. In clean.py write a second compiled pattern for that grouping, with three groups, and change phones() so that it returns numbers found by either pattern in the same +44 form. Run python run.py phones and check that note 5 now gives a number and the other notes give what they gave before.
A hint
The second pattern needs a 0, then 7 and two more digits, then three digits, then four digits, with an optional space or dash between them.
These tasks are for you to try on your own machine. Nothing is checked or saved.
What this does not show
Seven made-up notes and patterns for only the formats shown, so the run says nothing about real, larger or multilingual text.
Up next in Python for data
More in Python for data
Related videos
- #149 Missing Data in pandas: Drop, Fill or Model It? Python for data
- #204 Tag PII Columns in Python, Including a False Positive Data governance
- #321 Support Bundle Without Secrets: Redact and Check It Forward deployed engineer interviews
- #325 Parse a Messy CSV Live: Interview Round in Python Forward deployed engineer interviews
