Python Regular Expression Tutorial
Discover the power of regular expressions with this tutorial. You will work with the re library, deal with pattern matching, learn about greedy and non-greedy matching, and much more!
Regular Expressions, often shortened as regex, are a sequence of characters used to check whether a pattern exists in a given text (string) or not. If you've ever used search engines, search and replace tools of word processors and text editors - you've already seen regular expressions in use. They are used at the server side to validate the format of email addresses or passwords during registration, used for parsing text data files to find, replace, or delete certain string, etc. They help in manipulating textual data, which is often a prerequisite for data science projects involving text mining.
This tutorial will walk you through the important concepts of regular expressions with Python. You will start with importing re - Python library that supports regular expressions. Then you will see how basic/ordinary characters are used for performing matches, followed by wild or special characters. Next, you'll learn about using repetitions in your regular expressions. You'll also learn how to create groups and named groups within your search for ease of access to matches. Next, you'll get familiar with the concept of greedy vs. non-greedy matching.
This already seems like a lot, and hence, there is a handy summary table included to help you remember what you've seen so far with short definitions. Do check it out!
This tutorial also covers some very useful functions provided by the re library, such as: compile(), search(), findall(), sub() for search and replace, split(), and some more. You will also learn about compilation flags that you can use to make your regex better.
In the end, there is a case study - where you can put your knowledge to use! So let's regex...
Regular Expressions in Python
In Python, regular expressions are supported by the re module. That means that if you want to start using them in your Python scripts, you have to import this module with the help of import:
import reThe re library in Python provides several functions that make it a skill worth mastering. You will see some of them closely in this tutorial.
Basic Patterns: Ordinary Characters
You can easily tackle many basic patterns in Python using ordinary characters. Ordinary characters are the simplest regular expressions. They match themselves exactly and do not have a special meaning in their regular expression syntax.
Examples are 'A', 'a', 'X', '5'.
Ordinary characters can be used to perform simple exact matches:
pattern = r"Cookie"
sequence = "Cookie"
if re.match(pattern, sequence):
print("Match!")
else:
print("Not a match!")Most alphabets and characters will match themselves, as you saw in the example.
The match() function returns a match object if the text matches the pattern. Otherwise, it returns None. The re module also contains several other functions, and you will learn some of them later on in the tutorial.
For now, let's focus on ordinary characters!
Do you notice the r at the start of the pattern Cookie?
This is called a raw string literal. It changes how the string literal is interpreted. Such literals are stored as they appear.
For example, \ is just a backslash when prefixed with an r rather than being interpreted as an escape sequence. You will see what this means with special characters. Sometimes, the syntax involves backslash-escaped characters, and to prevent these characters from being interpreted as escape sequences; you use the raw r prefix.
TIP: You don't actually need it for this example; however, it is a good practice to use it for consistency.
Wild Card Characters: Special Characters
Special characters are characters that do not match themselves as seen but have a special meaning when used in a regular expression. For simple understanding, they can be thought of as reserved metacharacters that denote something else and not what they look like.
Let's check out some examples to see the special characters in action...
But before you do, the examples below make use of two functions namely: search() and group().
With the search function, you scan through the given string/sequence, looking for the first location where the regular expression produces a match.
The group function returns the string matched by the re. You will see both these functions in more detail later.
Back to the special characters now.
. - A period. Matches any single character except the newline character.
re.search(r'Co.k.e', 'Cookie').group()^ - A caret. Matches the start of the string.
This is helpful if you want to make sure a document/sentence starts with certain characters.
re.search(r'^Eat', "Eat cake!").group()
## However, the code below will not give the same result. Try it for yourself:
# re.search(r'^eat', "Let's eat cake!").group()$ - Matches the end of string.
This is helpful if you want to make sure a document/sentence ends with certain characters.
re.search(r'cake$', "Cake! Let's eat cake").group()
## The next search will return the NONE value, try it:
# re.search(r'cake$', "Let's get some cake on our way home!").group()[abc] - Matches a or b or c.
[a-zA-Z0-9] - Matches any letter from (a to z) or (A to Z) or (0 to 9).
TIP: Characters that are not within a range can be matched by complementing the set. If the first character of the set is ^, all the characters that are not in the set will be matched.
re.search(r'[0-6]', 'Number: 5').group()## Matches any character except 5
re.search(r'Number: [^5]', 'Number: 0').group()
## This will not match and hence a NONE value will be returned
#re.search(r'Number: [^5]', 'Number: 5').group()