Forgot your password?
typodupeerror
Programming IT Technology

(Useful) Stupid Regex Tricks? 516

Posted by ScuttleMonkey
from the hope-you-like-reading-lots-of-random-characters dept.
careysb writes to mention that in the same vein as '*nix tricks' and 'VIM tricks', it would be nice to see one on regular expressions and the programs that use them. What amazingly cool tricks have people discovered with respect to regular expressions in everyday life as a developer or power user?"
This discussion has been archived. No new comments can be posted.

(Useful) Stupid Regex Tricks?

Comments Filter:
  • by rallymatte (707679) * on Monday November 10, 2008 @10:20AM (#25703249)
    To filter a string to make sure it's a valid ip address this regexp is quite useful.
    /^((25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9][0-9]|[0-9])\.){3}(25[0-5]|2[0-4][0-9]|1[0-9][0-9]|[1-9][0-9]|[0-9])$/

    And this one for mac addresses
    /^[0-9a-fA-F]{2}:[0-9a-fA-F]{2}:[0-9a-fA-F]{2}:[0-9a-fA-F]{2}:[0-9a-fA-F]{2}:[0-9a-fA-F]{2}$/
  • by Anonymous Coward on Monday November 10, 2008 @10:22AM (#25703303)
    (Useful) Stupid * Tricks

    Yes sir, that will guarantee a front page story. You better head back to the drawing board if it doesn't fit that pattern. Next week: (Useful) Stupid Starcraft Tricks.
    • by Malevolent Tester (1201209) on Monday November 10, 2008 @10:52AM (#25703785) Journal

      Next week: (Useful) Stupid Starcraft Tricks.

      You can assign a building, building add-on, or a group of up to 12 units to a single key. To do this, select what you want to assign, then hold down Control and select a number on the keyboard between 0-9. Then, when you want to select what you assigned, simply press the number of the group that you want. Pressing a group number twice will center the screen on the group.

      • Re: (Score:3, Funny)

        by Anpheus (908711)

        Did you get that from the tips that pop up every game when you first install StarCraft?

      • Re: (Score:3, Interesting)

        On fastest:

        Stasis your oponents' fleet with your arbiter and start a 15 second countdown. On zero, your teammate nukes the stasis. Wait 30 seconds for the nuke to come down right on your open-mouthed oponents' fleet.

        To add insult to injury, if you manage to stasis both their and your ships, you can recall them out right before the nuke hits.

    • by McWilde (643703) on Monday November 10, 2008 @10:54AM (#25703813) Homepage

      That doesn't look right...
      Try:

      /^\(Useful\) Stupid \w+ Tricks$/

      Also, I noticed that the previous stupid tricks stories ended with a question mark, but this one doesn't. So:

      /^\(Useful\) Stupid \w+ Tricks\??$/

    • Blasphemy (Score:3, Informative)

      by Shohat (959481)
      There are no Stupid Starcraft Tricks.
    • Re: (Score:3, Interesting)

      by Talderas (1212466)

      You can permanently cloak zerg units that can burrow if you control an arbiter. By burrowing the zerg unit just as it enters the arbiter's cloaking field radius, the zerg will become permanently cloaked.

  • New Slashot Section (Score:5, Interesting)

    by Frankie70 (803801) on Monday November 10, 2008 @10:24AM (#25703329)

    Maybe we should have a new section for "Useful Stupid Tricks" on Slashdot.

    • Next installment: "Useful Stupid Manager Tricks"
      • If it's understandable by a manager, than it's stupid...
      • and if it gives you your much deserved salary hike, than it's useful
  • format c:*.*
  • ARGH!!!! (Score:2, Funny)

    by soapdog (773638)
    You see yourself in digg.com. You are likely to be eaten by a grue.
    • Re: (Score:2, Insightful)

      by Anonymous Coward

      So clearly, Slashdot's shit never stank?

      No, seriously, why the bitching? Did you expect the site to just keep reporting dry stories about incremental Linux kernel upgrades for its entire existence? You expected a website to never change and never update with the times? Just because it's old doesn't mean it's sacred.

  • Regex Support (Score:2, Interesting)

    by Extremus (1043274)
    I have used regex in the past, mainly for keeping long SQL scripts. The problem is the lack of full support for regex in most of editors. IMO the best (for windows, at least) is the EditPad Pro [editpadpro.com].
  • How about (Score:4, Funny)

    by cbiltcliffe (186293) on Monday November 10, 2008 @10:30AM (#25703409) Homepage Journal

    Stupid (Useful) Ask Slashdot tricks?

    I'm not sure whether these are legitimate, or just a "I don't know what the hell I'm doing, so let's see if I can get someone else to show me how to do my job, under the guise of sharing information."

    I'd like to say the former, but my cynicism is making me lean to the latter.....

  • by mutende (13564) <klaus@seistrup.dk> on Monday November 10, 2008 @10:32AM (#25703443) Homepage Journal
    Beautiful regexp that validates RFC 822 addresses: Mail-RFC822-Address.html [ex-parrot.com]
  • I use this to remove formatting that is included in the reports spit out from the mainframe -

    cat REPORT_NAME | sed 's/[^a-z0-9,.-]//gi' > REPORT.out

    It uses a few commands to accomplish this but I figured I would include the entire command line for completeness. It keeps all letters, numbers, ',', '.', and '-'. If you need other characters you can always add them to the regular expression.

  • Windows (Score:4, Informative)

    by jgtg32a (1173373) on Monday November 10, 2008 @10:32AM (#25703451)
    MS Office does support regexp while not as good as Perl regex, they are very helpful.

    Link to and excel .bas addon for regexp, which helped me alot.
    Don't forget to add the lib {tools->References->MS VBA Scrip regexp 5.5}

    http://www.tmehta.com/regexp/using_functions.htm [tmehta.com]
  • by Anonymous Coward on Monday November 10, 2008 @10:32AM (#25703455)

    please validate using the rfc and not your sketchy interpretation of an e-mail address. /.*@.*\..*/ will not cut it.

    Try instead
    ([^\\x00-\\x20\\x22\\x28\\x29\\x2c\\x2e\\x3a-\\x3c\\x3e\\x40\\x5b-\\x5d\\x7f-\\xff]+|\\x22([^\\x0d\\x22\\x5c\\x80-\\xff]|\\x5c[\\x00-\\x7f])*\\x22)(\\x2e([^\\x00-\\x20\\x22\\x28\\x29\\x2c\\x2e\\x3a-\\x3c\\x3e\\x40\\x5b-\\x5d\\x7f-\\xff]+|\\x22([^\\x0d\\x22\\x5c\\x80-\\xff]|\\x5c\\x00-\\x7f)*\\x22))*\\x40([^\\x00-\\x20\\x22\\x28\\x29\\x2c\\x2e\\x3a-\\x3c\\x3e\\x40\\x5b-\\x5d\\x7f-\\xff]+|\\x5b([^\\x0d\\x5b-\\x5d\\x80-\\xff]|\\x5c[\\x00-\\x7f])*\\x5d)(\\x2e([^\\x00-\\x20\\x22\\x28\\x29\\x2c\\x2e\\x3a-\\x3c\\x3e\\x40\\x5b-\\x5d\\x7f-\\xff]+|\\x5b([^\\x0d\\x5b-\\x5d\\x80-\\xff]|\\x5c[\\x00-\\x7f])*\\x5d))*

    See the original at http://www.iamcal.com/publish/articles/php/parsing_email/

    • by Timmmm (636430) on Monday November 10, 2008 @10:49AM (#25703719)

      Mmmmm readable.

    • Re: (Score:3, Insightful)

      by ais523 (1172701)

      Does that thing allow nested comments, and escaping inside them? It doesn't look like it, it isn't recursive. (I have some in the email address I typically put online, ais523(524\)(525)x)@bham.ac.uk; that could be a good test for your email client, and is useful because I've never come across a spambot that can parse it.)

      Recent versions of Perl and Python regices allow you to write recursively; that probably qualifies as a stupid regex trick, especially as it makes them more computationally powerful so t

  • by Pahalial (580781) on Monday November 10, 2008 @10:35AM (#25703501)
    Saw this one recently, by Andrew Savige. He did use a Perl module to generate the regex itself, but even so!

    http://search.cpan.org/dist/Acme-EyeDrops/lib/Acme/EyeDrops.pm#99_Bottles_of_Beer [cpan.org]

    (I would quote the final result but /. won't allow that many "junk" characters.. let's hope that doesn't cripple this entire discussion.)
  • Regex Bill (Score:5, Funny)

    by Anonymous Coward on Monday November 10, 2008 @10:36AM (#25703517)
    Why couldn't Bill try out his regular expressions?

    His mom wouldn't let him play with matches.

  • (?:<thing>foo)

    Where you can then access the matched substring ("foo" in this case) by the tag/label "thing" (access syntax depends on language).

    It's pretty spiffy if you need order independent matching.
  • For that annoying user who never shuts up... /[.]//

    What's that? Cat got your tongue, troll?

  • by Gulthek (12570) on Monday November 10, 2008 @10:41AM (#25703593) Homepage Journal

    Here's a chunk of perl script I wrote (years ago) that determines if $text matches any of the styles of library call number that I've ever encountered.

    Slashcode is interestingly interpreting my formatting, but you should get the gist.


    $text =~ /
            ^[A-Z]+ # starts with at least one capital letter
            \s? # followed by an optional space
            \d+ # followed by one or more digits /x
        or $text =~ /
            ^\d+ # starts with one or more digits
            \. # followed by a single decimal /x
        or $text =~ /
            \d+ # starts with one or more digits
            \s # and a space /x
        or $text =~ /
            Thesis # starts with "Thesis" .+ # with one or more characters of any kind
            \d{4} # then four numbers - year
            \s+ # separated by at least one space
            [A-Z]+ # from one or more capital letters
            \d+ # followed by one or more numbers /xi # case ignored here in case we run into THESIS or thesis
        or $text =~ /
            \d+ # starts with one or more digits
            \- # connected with a dash
            \d+ # to one or more following digits /x
        or $text =~ /
            \d+ # starts with one or more digits
              # followed by a space
            [A-Z]* #followed by zero or more capital letters
        \d+ # followed by one or more digits /x

  • to filter ask slashdot posts for 'tricks' articles.
  • by darkvizier (703808) on Monday November 10, 2008 @10:44AM (#25703649)
    I've never found regexes to be useful at all. I prefer to write my own parsers from scratch in assembly language, or conway's game of life [wikipedia.org], if I'm feeling m/(ambitious|artistic|autistic|masochistic)/.

    But even an artist gets lazy sometimes.
  • Here's one I came up with recently:

    If you want to get documentation out of both CWEB and Doxygen, write the Doxygen comments in the source files like @=//! Comment for Doxygen.@> to prevent ctangle from stripping the comment out, then use sed 's/@=\/.*@>//g' input.w > output.w to strip those comments out so they don't end up in the output from cweave.

  • by gzipped_tar (1151931) on Monday November 10, 2008 @10:53AM (#25703803) Journal
    This regex matches a number: interger or float, scientific notation or plain, plus or minus...

    [-+]?(?:\b[0-9]+(?:\.[0-9]*)?|\.[0-9]+\b)(?:[eE][-+]?[0-9]+\b)?
  • use Regex::Common; (Score:5, Insightful)

    by oneiros27 (46144) on Monday November 10, 2008 @10:54AM (#25703815) Homepage
    use Regex::Common qw(URI net);
    $text_with_urls =~ m/$RE{URI}/;
    $text_with_ips =~ m/$RE{net}{IPv4}/;
  • by cerberusss (660701) on Monday November 10, 2008 @10:56AM (#25703851) Homepage Journal
    To remove trailing whitespace from a textfile (vim regex, don't know if the \s will work in other regex dialects):

    /\s\+$//e

  • by DerCed (155038) on Monday November 10, 2008 @11:00AM (#25703925)

    I wonder why such FAQs are still posted on a site like Slashdot. We now have a great repository for exactly this kind of questions:
    http://stackoverflow.com/questions/tagged?tagnames=regex&sort=votes&pagesize=15 [stackoverflow.com]

  • I often use sed to split a delimited line into multiple lines. E.g.:

    echo $PATH | sed 's/:/\
    /g'

  • by gpuk (712102)

    Cal Henderson's routine is the best RFC compliant regex I have ever found to verify an email address:

    http://code.iamcal.com/php/rfc822/ [iamcal.com]

  • Be lazy! (Score:5, Interesting)

    by subreality (157447) on Monday November 10, 2008 @11:03AM (#25703973)

    OK, you asked for stupid tricks, but this one's just plain lazy.

    Between bash and grep, there are quite a lot of special characters that you have to escape... Or just ignore with dots!

    /I.do.this.frequently..(even.with.parenthases).,.because.sometimes.my....backslash..key.is.tired/

    A couple neat things happened: The extra dot after frequently is matching an inline paren. The paren in the PATTERN right next to it starts the mark of an atom, closed by its brother. The comma is because I put one outside the paren (here represented as the dot to the left of the comma) as is my style. Also note the literal backslash, just before you see the word backslash in hidden parenthesis.

    Why not add quotes to match the spaces easily? I get a word or two in, and I find I naturally switch to using dots. These are throwaways for single tries through grep. For production code, I hone in carefully on the parts that I'm dead sure I can anchor to, escaped by any means needed, before carefully choosing my atom to match as tightly as possible, so it'll error out if my data has gone wrong.

    Even in a simple case like this, half the fun is in explaining it. :)

  • by doti (966971) on Monday November 10, 2008 @11:07AM (#25704037) Homepage


          my $re = '';
            $re = qr/
                    \{ (?:
                            (?> [^{}]+ ) # nao-chaves
                    |
                            (??{ $re }) # sub-bloco de chaves
                    )* \} /xs;

  • email validation... (Score:2, Interesting)

    by Ramley (1168049)
    This was always useful when appropriate: /^[\w.|-]+@(?:[\w.|-]{2,63}\.)+[a-z]{2,6}$/ Validates a valid email address (rfc 5322) -- although not taking into account an IP address (user@192.168.1.2)
    • by jeremyp (130771) on Monday November 10, 2008 @02:30PM (#25708027) Homepage Journal

      Your regex doesn't allow + signs in the name part.

      Nor, I would suspect would it handle quoted strings e.g. "Jeremy P"@example.com is technically a valid RFC 822 address.

      And having just looked up the RFC 5322 spec which you quote, I see there are more cases you fail to take acount of e.g.

      Jeremy P <jeremyp@example.com>

      Also, what makes you think upper case in domain names is invalid? jeremyp@example.COM fails validation.

  • by ianare (1132971) on Monday November 10, 2008 @11:11AM (#25704111)
    SSN
    ^(?!000)([0-6]\d{2}|7([0-6]\d|7[012]))([ -]?)(?!00)\d\d\3(?!0000)\d{4}$

    US phone with or without parentheses
    ^\([0-9]{3}\)\s?[0-9]{3}(-|\s)?[0-9]{4}$|^[0-9]{3}-?[0-9]{3}-?[0-9]{4}$

    ISO Date (19th to 21st century only)
    ^((18|19|20)\d\d)-(0[1-9]|1[012])-(0[1-9]|1[0-9]|2[0-9]|3[01])$
    • Re: (Score:3, Funny)

      by jonaskoelker (922170)

      ISO Date (19th to 21st century only)
      ^((18|19|20)\d\d)-(0[1-9]|1[012])-(0[1-9]|1[0-9]|2[0-9]|3[01])$

      This regexp is ISO certified. The certificate is valid until 2009-02-31.

  • by Janek Kozicki (722688) on Monday November 10, 2008 @11:13AM (#25704167) Journal

    I was wondering with my friend someday if it's possible with regex to select a pattern which occurs twice or more times repeatedly in single line but is separated by undefined characters. For example I want to select only lines in which the same pattern "[FB][ot]o" occurs exactly two times (in example below . is any character, for clarity):
     
    ...Foo... - is not selected
    ...Foo...Bto... - is not selected
    ...Bto...Bto... - is selected

    a normal /[FB][ot]o.*[FB][ot]o/ would select the second and third case. But I only want the third case. The first occurrence would define my pattern, and second occurrence must exactly match it. Magic stuff like this is not working: /\([FB][ot]o\).*\1/ although that seems to be the closest description of what we wanted.

    • by natebarney (987940) on Monday November 10, 2008 @11:30AM (#25704453)

      Magic stuff like this is not working: /\([FB][ot]o\).*\1/ although that seems to be the closest description of what we wanted.

      In perl, I did /([FB][ot][o]).*\1/ and it seemed to work as you wanted. Also, if you're using a regex engine that supports lazy (non-greedy) quantifiers like perl does, I would use them in this case. It reduces backtracking. In perl, put a ? after the *.

    • Re: (Score:3, Informative)

      by Moebius Loop (135536)

      In most regex engines, you should be able to do this with backreferences. I don't use them often, but I think something like this would work:

      /^(.*?)([FB][ot]o)((.+?)\2)+(.*?)$/

      I think the reason the example you gave using \1 didn't work is because the .* was too greedy, and ate up the rest of the pattern before the \1 got a chance to match. Also, when you're doing full line matching, it's always good to think about ^/$ and whether you're using any multiline modifiers.

  • Handy links (Score:3, Informative)

    by Kozz (7764) on Monday November 10, 2008 @11:14AM (#25704185)
    While I'm not providing any specific trick per say, on topic are a few useful links:
    http://www.regular-expressions.info/ [regular-expressions.info] - this one is handy for regex info particularly in Javascript which I use so infrequently I need to know how to match, capture, substitute, etc.
    http://perldoc.perl.org/perlre.html [perl.org] - plenty of regex info there which is Perl specific, but of course extends to many other similar implementations
    • Re: (Score:3, Informative)

      by Pope (17780)

      I was recently trying to come up with a regex for some renaming file thingy recently, and I found I could easily state is pseudo-code what I wanted to do, but looking through and regex sites/tips/FAQs quickly went from "here's a very simple match test" to "going to the moon", with little in-between, which is what I was after.

      However, I eventually found Reggy [apple.com] for OS X, a handy little tool for testing regexes, so all was not lost.

  • by hansamurai (907719) <hansamurai@gmail.com> on Monday November 10, 2008 @11:47AM (#25704857) Homepage Journal

    Does anyone know if the Luhn Algorithm can be implemented in regex only?

    http://en.wikipedia.org/wiki/Luhn_algorithm [wikipedia.org]

    (sorry if I double post this... I swear I posted it 10 minutes ago)

    • Re: (Score:3, Insightful)

      by phantomfive (622387)
      Yes, actually, (despite what the other posters have said), you can, but it will be very complicated since you will be implementing something like your own multiplier in regex.

      The simplest way to do it, of course, is to just list all valid Luhn Algorithm numbers. something like (.....384848583 | 938484845 | 8383838383......). Of course, this is probably not what you are looking for, because you will be listing a lot of numbers, and if your Luhn number is too big, then it won't be in your list.

      So, as fo
  • by cerberusss (660701) on Monday November 10, 2008 @11:56AM (#25705021) Homepage Journal
    This regex goes through my enormous list of girlfriends' telephone numbers, and makes a selection based on the area code I'm currently in!

    #$%^&*(&^%{{}}{/\/\||```

    (No, that's not a regex at all. And no, I don't even have a single girlfriend.)

  • by Bandman (86149) <bandman@NOSPaM.gmail.com> on Monday November 10, 2008 @12:02PM (#25705147) Homepage

    I have to parse files with bash sometimes, and I use these:

    ^# = line with a leading comment
    ^$ = empty line

    They're simple, but work usually. You can make them a lot more bullet proof by adding in blank checking between the characters, but it seems to work.

    cat httpd.conf | grep -v \^\# | grep -v \^\$ | less

    makes httpd.conf a lot more readable.

  • OK, I'll play... (Score:3, Informative)

    by PRMan (959735) on Monday November 10, 2008 @12:14PM (#25705369)

    Bad filename character for Windows (if it matches, the filename is invalid):

    [*<>=+"\\/,.:;]

    E-mail (use case insensitive):

    ^\s*[\w-~&$+']+(\.[\w-]+)*@(?<domain>[\w-]+\.)+(?<tld>[0-9]{1,3}|aero|arpa|biz|com|coop|edu|gov|info|int|museum|net|org|[a-z]{2})\s*$

    GUID (use case insensitive):

    ^\{?[0-9a-f]{8}-?([0-9a-f]{4}-?){3}[0-9a-f]{12}\}?$

    IP on local private network:

    ^127\.|^10\.|^192\.168\.|^172\.1[6-9]\.|^172.2\d\.|^172.3[01]\.|^169\.254

    Removes .NET named capture syntax so that a .NET Regex string can be used elsewhere (such as Javascript) (replace with nothing):

    \?\<\w+\>

    Flame away about how horrible it is that I missed some edge case that even nobody on Slashdot has ever heard of, but they work well for me and hopefully for you too.

    Now, if you actually find a common case that I missed, I would appreciate the help...

  • valid utf-8 string (Score:3, Interesting)

    by Danny Rathjens (8471) <`slashdot2' `at' `rathjens.org'> on Monday November 10, 2008 @12:26PM (#25705611)
    Here is the crazy regex to detect a valid UTF-8 string. :)
    /^(
         [\x09\x0A\x0D\x20-\x7E]            # ASCII
       | [\xC2-\xDF][\x80-\xBF]             # non-overlong 2-byte
       |  \xE0[\xA0-\xBF][\x80-\xBF]        # excluding overlongs
       | [\xE1-\xEC\xEE\xEF][\x80-\xBF]{2}  # straight 3-byte
       |  \xED[\x80-\x9F][\x80-\xBF]        # excluding surrogates
       |  \xF0[\x90-\xBF][\x80-\xBF]{2}     # planes 1-3
       | [\xF1-\xF3][\x80-\xBF]{3}          # planes 4-15
       |  \xF4[\x80-\x8F][\x80-\xBF]{2}     # plane 16
      )*$/x

    This can crash perl if the string being checked is too big. :D
    So it's usually better to just let perl attempt to decode anything non-ascii as utf8 and see if it fails or not. (And hope all the utf8 parsing exploits have been fixed :)
    eval { $param = decode( 'utf8', $param, Encode::FB_CROAK) if $param =~ /[^\x00-\x7E]/ };
    $param = decode( 'iso-8859-1', $param, Encode::FB_CROAK) if $@; # utf8 decode of non-ascii text failed so treat as latin1
  • by ShatteredArm (1123533) on Monday November 10, 2008 @01:42PM (#25707155)
    I came up with a Regex that can be used to match literally anything (yes, anything!). It is, therefore, the most flexible regex ever concocted. Here it is:

    .*
    • Re: (Score:3, Funny)

      by Lord Ender (156273)

      I was able to simplify your regex somewhat so that it still matches everything, but takes up half the space:

      .

  • by IorDMUX (870522) <mark.zimmerman3@ ... minus physicist> on Monday November 10, 2008 @05:37PM (#25711601) Homepage

    Some people, when confronted with a problem, think "I know, I'll use regular expressions." Now they have two problems.

    -- Jamie Zawinski

Scientists will study your brain to learn more about your distant cousin, Man.

Working...