Regular Expressions

The Weekly Iteration

29 minutes

Video

Want to see the full-length video right now for free?

Sign In with GitHub for Free Access

Notes

In this episode of the Weekly Iteration, Chris is joined by Matthew Mongeau, aka Goose, to discuss regular expressions. Learn how to work with them, where they fit, and perhaps more importantly where they don't.

Regex Introduction

If you're new to regular expressions, check out these resources to get up to speed:

Interactive Regex Testing Utilities

  • Rubular - Ruby regex testing tool
  • Scriptular - Similar to Rubular, but using JS
  • Reggyapp - OS X app with multiple language support

Regex Visualization Utilities

  • Debugex - See the parsed state machine representation of your regex
  • Regexper - Similar to Debugex, but with more friendly summaries.

Support

  • Ruby 1.9+ has great support thanks to the Oniguruma engine
  • Vim has a relatively complete regex implementation but uses a very non-standard syntax that defaults to treating most characters literally. Use \v for "very no-magic" mode, a more standard regex interpretation. :h \v for more info.
  • JavaScript has solid and mostly standard implementation, although not as complete as Vim & Ruby. Try XRegExp for more advanced features.
  • Git log log -i -E --grep for case insensitive extended regex search of commit message bodies.

Ruby Tricks

String#scan

Use String#scan to slice all matches to a pattern out of a target string.

sample = <<-PHONE
  You can call me on my cell: (555) 555-5123, or my
  home phone: (555) 555-6457 after 7pm
PHONE

phone_number_pattern = /\(\d{3}\)\s\d{3}-\d{4}/

sample.scan(phone_number_pattern).inspect
#=> ["(555) 555-5123", "(555) 555-6457"]

gsub with a block

Use String#gsub to replace all matches to a pattern, optionally yielding the matches to a block for more complex replacement behavior.

sample = "Hello world, this is a test string."

word_pattern = /\w+/

substitutions = {
  "world" => "planet",
  "test" => "experiment",
  "a" => "an"
}

sample.gsub!(word_pattern) do |word|
  puts "Processing '#{word}'"
  substitutions.fetch(word, word)
end
#=> "Hello planet, this is an experiment string."

Named Captures

Use named captures to label the capture groups in a pattern and produce a more expressive and maintainable pattern.

# url_pattern = /^(https?:\/\/)?(\w+\.)?(\w+)\.(com|org|net|biz)$/

url_pattern = %r{
  ^
  (?<protocol>https?:\/\/)?
  (?<subdomain>\w+\.)?
  (?<domain>\w+)\.
  (?<tld>com|org|net|biz)
  $
}x

urls = [
  "http://google.com",
  "https://www.google.com",
  "www.google.com",
  "google.com"
]

urls.each do |url|
  matches = url.match(url_pattern)

  puts matches.inspect
end

# <MatchData "http://google.com" protocol:"http://" subdomain:nil domain:"google" tld:"com">
# <MatchData "https://www.google.com" protocol:"https://" subdomain:"www." domain:"google" tld:"com">
# <MatchData "www.google.com" protocol:nil subdomain:"www." domain:"google" tld:"com">
# <MatchData "google.com" protocol:nil subdomain:nil domain:"google" tld:"com">

Proper String Anchoring

When needing to ensure you match the entirety of a string, use \A and \z rather than ^ and $. Proper string anchoring is needed to ensure security to avoid script injections. See the Rails Guide to Security for additional detail.

dangerous_email = "me@example.com\n<script>alert('pwned');</script>"

# line anchored
# valid_email_pattern = /^\w+@\w+\.\w+$/

# string anchored
valid_email_pattern = /\A\w+@\w+\.\w+\z/

def validate_email(email, pattern)
  if email =~ pattern
    puts "\nValid\n\n"
    puts "email '#{email}' is valid!"
  else
    puts "\nNOT VALID!!!!\n\n"
    puts "email '#{email}' is not valid!!!!"
  end
end

validate_email(dangerous_email, valid_email_pattern)
#=> NOT VALID

Using Comments to Document Your Pattern

You can use a special regex construct to use comments to document your pattern.

When Not To Use

While Regular Expressions are an incredibly powerful tool, they fall short of a vast number of use cases. Instead, look to parsers and lexers to more intelligently and reliable split up a complex input based on multiple and more subtle rules.

Additional References