regex - Python regular expression again - match url

8
votes

I have such regexp:

 re.compile(r"((https?):((//)|(\\\\))+[\w\d:#@%/;$()~_?\+-=\\\.&]*)", re.MULTILINE|re.UNICODE)

But that doesn't include hashbangs (#!). What I need to change, to get it working? I know I can add ! to group with #@% etc, but that will select something like 

Check this out: http://example.com/something/!!!

and I want to avoid that.

5
pythonregex
How about checking out the RFC for URI syntax (ietf.org/rfc/rfc3986.txt)? It will show you that the bang can only be used in certain ways otherwise it has to be escaped. Good question. - Ray Toal
I hope you're not trying to use this regex to match URLs requested by a browser: if so, you should realise that the part after the hash is not sent in a normal client request. - Daniel Roseman
No. I'm parsing user input and make links shorter and safer for users (we have full control, we can block link, domain etc.). And with original regex there was ourshortdomain.foo/urlhash/#!/twitter/something ;) - ThomK

5 Answers

18
votes

Don't try to make your own regular expression for matching URLs, use someone else's who has already solved such problems, like this one.

3
votes

It could be very long but in practice mine works pretty good. Please try this one ((http|https)\:\/\/)?[a-zA-Z0-9\.\/\?\:@\-_=#]+\.([a-zA-Z]){2,6}([a-zA-Z0-9\.\&\/\?\:@\-_=#])*

It matches all of the example below 

http://wwww.stackoverflow.com
abc.com
http://test.test-75.1474.stackoverflow.com/
stackoverflow.com/
stackoverflow.com
[email protected]
http://www.example.com/etcetc
www.example.com/etcetc
example.com/etcetc
user:[email protected]/etcetc
(www.itmag.com)
example.com/etcetc?query=aasd
example.com/etcetc?query=aasd&dest=asds
http://stackoverflow.com/questions/6427530/regular-expression-pattern-to-
match-url-with
www/[email protected]
[email protected].
[email protected] 
[email protected]
3
votes

This is a common problem, use default libraries.

For python use urlparse

1
votes

I'll admit that I'm a little bit worried about an application that requires a regex like that to match URLs. That said, this seems to work for me:

((https?):((//)|(\\\\))+([\w\d:#@%/;$()~_?\+-=\\\.&](#!)?)*)
0
votes

Based on this link we can use the library validators

For example:

import validators

valid=validators.url('https://codespeedy.com/')
if valid==True:
    print("Url is valid")
else:
    print("Invalid url")