Ruby Programming Language :: Problem with open-uri


	Problem with open-uri Hello, I'm trying to use open-uri on yahoo mail's login page. If I do the following: #!/usr/bin/env ruby require 'open-uri' require 'uri' f = open("http://mail.yahoo.com") -------- I get the following error: /usr/lib/ruby/1.8/open-uri.rb:174:in `open_loop': redirection forbidden: http://mail.yahoo.com/ -> https://login.yahoo.com/config/login_verify2?&.src=ym (RuntimeError) from /usr/lib/ruby/1.8/open-uri.rb:132:in `open_uri' from /usr/lib/ruby/1.8/open-uri.rb:528:in `open' from /usr/lib/ruby/1.8/open-uri.rb:30:in `open' from ./readForms.rb:11 ------- I've tried to use mechanize instead, but I can't seem to figure out how to emulate the base_uri functionality in open-uri. Any ideas? Regards, jon -- Posted via http://www.ruby-forum.com/. On 6/3/07, Jonathan Nichols <jnich@alumni.princeton.edu> wrote: - Hide quoted text - > Hello, > I'm trying to use open-uri on yahoo mail's login page. If I do the > following: > #!/usr/bin/env ruby > require 'open-uri' > require 'uri' > f = open("http://mail.yahoo.com") > -------- > I get the following error: > /usr/lib/ruby/1.8/open-uri.rb:174:in `open_loop': redirection forbidden: > http://mail.yahoo.com/ -> > https://login.yahoo.com/config/login_verify2?&.src=ym (RuntimeError) > from /usr/lib/ruby/1.8/open-uri.rb:132:in `open_uri' > from /usr/lib/ruby/1.8/open-uri.rb:528:in `open' > from /usr/lib/ruby/1.8/open-uri.rb:30:in `open' > from ./readForms.rb:11 > ------- > I've tried to use mechanize instead, but I can't seem to figure out > how to emulate the base_uri functionality in open-uri. Any ideas? > Regards, > jon You're being redirected. Try the redirect url (and you can leave the ?&.src=ym off depending on what you want to do) require 'open-uri' document = open( "https://login.yahoo.com/config/login_verify2" ).read Hi Todd, Thanks for the quick response. Unfortunately, I need to have the ability to take a simple uri and follow the redirections. open-uri works for some uris, but not for yahoo for some reason. If you do open("https://mail.google.com"), it will follow the redirects and successfully download the html at the redirected uri, which is https://www.google.com/accounts/ServiceLogin. this looks like a bug in open-uri to me, as mechanize (and firefox) is able to follow the redirects for yahoo. I can't find a way to report bugs for open-uri. Anybody know how to do this? Regards, jon - Hide quoted text - Todd Benson wrote: > On 6/3/07, Jonathan Nichols <jnich@alumni.princeton.edu> wrote: >> f = open("http://mail.yahoo.com") >> from /usr/lib/ruby/1.8/open-uri.rb:30:in `open' >> from ./readForms.rb:11 >> ------- >> I've tried to use mechanize instead, but I can't seem to figure out >> how to emulate the base_uri functionality in open-uri. Any ideas? >> Regards, >> jon > You're being redirected. Try the redirect url (and you can leave the > ?&.src=ym off depending on what you want to do) > require 'open-uri' > document = open( "https://login.yahoo.com/config/login_verify2" ).read -- Posted via http://www.ruby-forum.com/. I think I figured this out. The following is the code that generates the exception: ---- def OpenURI.redirectable?(uri1, uri2) # :nodoc: # This test is intended to forbid a redirection from http://... to # file:///etc/passwd. # However this is ad hoc. It should be extensible/configurable. uri1.scheme.downcase == uri2.scheme.downcase \|\| (/\A(?:http\|ftp)\z/i =~ uri1.scheme && /\A(?:http\|ftp)\z/i =~ uri2.scheme) end ---- This code is telling us that if there's a redirection, it will only work if the protocols are the same (http -> http, or https -> https), or if both the first and the second uris are either http or ftp. This logic prohibits a redirect from http to https, which is what the yahoo mail uri does. Changing the last line to (/\A(?:http\|ftp\|https)\z/i =~ uri1.scheme && /\A(?:http\|ftp\|https)\z/i =~ uri2.scheme) will do the trick. Seems like this fix should get incorporated into open-uri. Anyone out there on the open-uri team listening? -jon - Hide quoted text - Jonathan Nichols wrote: > Hi Todd, > Thanks for the quick response. Unfortunately, I need to have the > ability to take a simple uri and follow the redirections. open-uri > works for some uris, but not for yahoo for some reason. If you do > open("https://mail.google.com"), it will follow the redirects and > successfully download the html at the redirected uri, which is > https://www.google.com/accounts/ServiceLogin. this looks like a bug in > open-uri to me, as mechanize (and firefox) is able to follow the > redirects for yahoo. > I can't find a way to report bugs for open-uri. Anybody know how to > do this? > Regards, > jon > Todd Benson wrote: >> On 6/3/07, Jonathan Nichols <jnich@alumni.princeton.edu> wrote: >>> f = open("http://mail.yahoo.com") >>> from /usr/lib/ruby/1.8/open-uri.rb:30:in `open' >>> from ./readForms.rb:11 >>> ------- >>> I've tried to use mechanize instead, but I can't seem to figure out >>> how to emulate the base_uri functionality in open-uri. Any ideas? >>> Regards, >>> jon >> You're being redirected. Try the redirect url (and you can leave the >> ?&.src=ym off depending on what you want to do) >> require 'open-uri' >> document = open( "https://login.yahoo.com/config/login_verify2" ).read -- Posted via http://www.ruby-forum.com/.