New Features
Add header reference methods to Mechanize::File so that a reponse object gets compatible with Net::HTTPResponse.
Mechanize#click accepts a regexp or string to click a button/link in the current page. It works as expected when not passed a string or regexp.
Provide a way to only follow permanent redirects (301) automatically: agent.redirect_ok = :permanent GH #73
Bug Fixes:
Fixed a bug where Referer is not sent when accessing a relative URI starting with “http”.
Fix handling of Meta Refresh with relative paths. GH #39
New Features:
An optional verb may be passed to Mechanize#get GH #26
The WWW constant is deprecated. Switch to the top level constant Mechanize
SelectList#option_with and options_with for finding options
Bug Fixes:
Rescue errors from bogus encodings
7bit content-encoding support. Thanks sporkmonger! GH #2
Fixed a bug with iconv conversion. Thanks awesomeman! GH #9
meta redirects outside the head are not followed. GH #13
Form submissions work with nil page encodings. GH #25
Fixing default values with serialized cookies. GH #3
Checkboxes and fields are sorted by page appearance before submitting. #11
Bug Fixes:
Do not apply encoding if encoding equals 'none' Thanks Akinori MUSHA!
Fixed Page#encoding= when changing the value from or to nil. Made it return the assigned value while at it. (Akinori MUSHA)
Custom request headers may be supplied WWW::Mechanize#request_headers RF #24516
HTML Parser may be set on a per instance level WWW::Mechanize#html_parser RF #24693
Fixed string encoding in ruby 1.9. RF #2433
Rescuing Zlib::DataErrors (Thanks Kelley Reynolds)
Fixing a problem with frozen SSL objects. RF #24950
Do not send a referer on meta refresh. RF #24945
Fixed a bug with double semi-colons in Content-Disposition headers
Properly handling cookies that specify a path. RF #25259
New Features:
Mechanize#submit and Form#submit take arbitrary headers(thanks penguincoder)
Bug Fixes:
Fixed a bug with bad cookie parsing
Form::RadioButton#click unchecks other buttons (RF #24159)
Fixed problems with Iconv (RF #24190, RF #24192, RF #24043)
POST parameters should be CGI escaped
Made Content-Type match case insensitive (Thanks Kelly Reynolds)
Non-string form parameters work
New Features:
Encoding may be specified for a page: Page#encoding=
Bug Fixes:
m17n fixes. ありがとう konn!
Fixed a problem with base tags. ありがとう Keisuke
HEAD requests do not record in the history
Default encoding to ISO-8859-1 instead of ASCII
Requests with URI instances should not be polluted RF #23472
Nonce count fixed for digest auth requests. Thanks Adrian Slapa!
Fixed a referer issue with requests using a uri. RF #23472
WAP content types will now be parsed
Rescued poorly formatted cookies. Thanks Kelley Reynolds!
Deprecations
WWW::Mechanize::List is gone!
Mechanize uses Nokogiri as it's HTML parser but you may switch to Hpricot by using WWW::Mechanize.html_parser = Hpricot
Bug Fixes:
Nil check on page when base tag is used #23021
Deprecations
WWW::Mechanize::List will be deprecated in 0.9.0, and warnings have been added to help you upgrade.
Bug Fixes:
Stopped raising EOF exceptions on HEAD requests. ありがとう：HIRAKU Kuroda
Fixed exceptions when a logger is set and file:// requests are made.
Made Mechanize 1.9 compatible
Not setting the port in the host header for SSL sites.
Following refresh headers. Thanks Tim Connor!
Cookie Jar handles cookie domains containing ports, like 'mydomain.com:443' (Thanks Michal Ochman!)
Fixing strange uri escaping problems [#22604]
Making content-type determintation more robust. (thanks Han Holl!)
Dealing with links that are query string only. [#22402]
Nokogiri may be dropped in as a replacement.
WWW::Mechanize.html_parser = Nokogiri::HTML
Making sure the correct page is added to the history on meta refresh.
Mechanize#get requests no longer send a referer unless they are relative requests.
Bug Fixes:
Setting the port number on the host header.
Fixing Authorization headers for picky servers
Bug Fixes:
Making sure logger is set during SSL connections.
Bug Fixes:
Doh! I was accidentally setting headers twice.
Bug Fixes:
Fixed problem with nil pointer when logger is set
New Features:
Lifecycle hooks. Mechanize#pre_connect_hooks, Mechanize#post_connect_hooks
file:/// urls are now supported
Added Mechanize::Page#link_with, frame_with for searching for links using
criteria.
Implementing PUT, DELETE, and HEAD requests
Bug Fixes:
Fixed an infinite loop when content-length and body length don't match.
Only setting headers once
Adding IIS authentication support
Bug Fixes:
Fixed bug when receiving a 304 response (HTTPNotModified) on a page not cached in history.
#21428 Default to HTML parser for 'application/xhtml+xml' content-type.
Fixed an issue where redirects were resending posted data
New Features:
Page#form_with takes a criteria hash.
Page#form is changed to Page#form_with
Mechanize#get takes custom http headers. Thanks Mike Dalessio!
Form#click_button submits a form defaulting to the current button.
Form#set_fields now takes a hash. Thanks Tobi!
Mechanize#redirection_limit for setting the max number of redirects.
Bug Fixes:
Added more examples. Thanks Robert Jackson.
#20480 Making sure the Host header is set.
#20672 Making sure cookies with weird semicolons work.
Fixed bug with percent signs in urls. d.hatena.ne.jp/kitamomonga/20080410/ruby_mechanize_percent_url_bug
#21132 Not checking for EOF errors on redirect
Fixed a weird gzipping error.
#21233 Smarter multipart boundry. Thanks Todd Willey!
#20097 Supporting meta tag cookies.
New Features:
Added support for reading Mozilla cookie jars. Thanks Chris Riddoch!
Moving text, password, hidden, int to default. Thanks Tim Harper!
Mechanize#history_added callback for page loads. Thanks Tobi Reif!
Mechanize#scheme_handlers callbacks for handling unsupported schemes on links.
Bug Fixes:
Ignoring scheme case bugs.debian.org/cgi-bin/bugreport.cgi?bug=470642
Not encoding tildes in uris. Thanks Bruno. [#19380]
Resetting request bodys when retrying form posts. Thanks Bruno. [#19379]
Throwing away keep alive connections on EPIPE and ECONNRESET.
Duplicating request headers when retrying a 401. Thanks Hiroshi Ichikawa.
Simulating an EOF error when a response length is bad. Thanks Tobias Gruetzmacher. rubyforge.org/tracker/index.php?func=detail&aid=19178&group_id=1453&atid=5711
Defaulting option tags to the inner text. rubyforge.org/tracker/index.php?func=detail&aid=19976&group_id=1453&atid=5709
Supporting blank strings for option values. rubyforge.org/tracker/index.php?func=detail&aid=19975&group_id=1453&atid=5709
Fixed a bug when fetching files and not pages. Thanks Mat Schaffer!
doh!
Pages are now yielded to a blocks given to WWW::Mechanize#get
WWW::Mechanize#get now takes hash arguments for uri parameters.
WWW::Mechanize#post takes an IO object as a parameter and posts correctly.
Fixing a strange zlib inflate problem on windows
Handling gzipped responses with no Content-Length header
Added iPhone to the user agent aliases. [#17572]
Fixed a bug with EOF errors in net/http. [#17570]
Handling 0 length gzipped responses. [#17471]
Removed Ruby 1.8.2 support
Changed parser to lazily parse links
Lazily parsing document
Adding verify_callback for SSL requests. Thanks Mike Dalessio!
Fixed a bug with Accept-Language header. Thanks Bill Siggelkow.
Detecting single quotes in meta redirects.
Adding pretty inspect for ruby versions > 1.8.4 (Thanks Joel Kociolek) rubyforge.org/tracker/index.php?func=detail&aid=13150&group_id=1453&atid=5709
Fixed bug with file name in multipart posts rubyforge.org/tracker/?func=detail&aid=15594&group_id=1453&atid=5709
Posting forms relative to the originating page. Thanks Mortee.
Added a FAQ rubyforge.org/tracker/?func=detail&aid=15772&group_id=1453&atid=5709
Made digest authentication work with POSTs.
Made sure page was HTML before following meta refreshes. rubyforge.org/tracker/index.php?func=detail&aid=12260&group_id=1453&atid=5709
Made sure that URLS with a host and no path would default to '/' for history purposes. rubyforge.org/tracker/index.php?func=detail&aid=12368&group_id=1453&atid=5709
Avoiding memory leaks with transact. Thanks Tobias Gruetzmacher! rubyforge.org/tracker/index.php?func=detail&aid=12057&group_id=1453&atid=5711
Fixing a problem with # signs in the file name. Thanks Tobias Gruetzmacher! rubyforge.org/tracker/index.php?func=detail&aid=12510&group_id=1453&atid=5711
Made sure that blank form values are submitted. rubyforge.org/tracker/index.php?func=detail&aid=12505&group_id=1453&atid=5709
Mechanize now respects the base tag. Thanks Stephan Dale. rubyforge.org/tracker/index.php?func=detail&aid=12468&group_id=1453&atid=5709
Aliasing inspect to pretty_inspect. Thanks Eric Promislow. rubyforge.org/pipermail/mechanize-users/2007-July/000157.html
Updating UTF-8 support for urls
Adding AREA tags to the links list. rubyforge.org/pipermail/mechanize-users/2007-May/000140.html
WWW::Mechanize#follow_meta_refresh will allow you to automatically follow meta refresh tags. [#10032]
Adding x-gzip to accepted content-encoding. Thanks Simon Strandgaard rubyforge.org/tracker/index.php?func=detail&aid=11167&group_id=1453&atid=5711
Added Digest Authentication support. Thanks to Ryan Davis and Eric Hodel, you get a gold star!
Keep alive can be shut off now with WWW::Mechanize#keep_alive
Conditional requests can be shut off with WWW::Mechanize#conditional_requests
Monkey patched Net::HTTP#keep_alive?
Moved last request time. Thanks Max Stepanov
Added WWW::Mechanize::File#save
Defaulting file name to URI or Content-Disposition
Updating compatability with hpricot
Added more unit tests
Fixed a bug with keep-alive requests
fixed problem with cookie paths
Removing hpricot overrides
Fixed a bug where alt text can be nil. Thanks Yannick!
Unparseable expiration dates in cookies are now treated as session cookies
Caching connections
Requests now default to keep alive
Fixed bug where html entities weren't decoded
Updated mechanize history to deal with redirects
Copying headers to a hash to prevent memory leaks
Speeding up page parsing
Aliased fields to elements
Adding If-Modified-Since header
Added delete_field! to form. Thanks to Sava Chankov
Updated uri escaping to support high order characters. Thanks to Henrik Nyh.
Better handling relative URIs. Thanks to Henrik Nyh
Now handles pipes in URLs rubyforge.org/tracker/?func=detail&aid=7140&group_id=1453&atid=5709
Now escaping html entities in form fields. rubyforge.org/tracker/?func=detail&aid=7563&group_id=1453&atid=5709
Added MSIE 7.0 user agent string
Adding the “redirect_ok” method to Mechanize to stop mechanize from following redirects.
http://rubyforge.org/tracker/index.php?func=detail&aid=6571&group_id=1453&atid=5712
Added protected method Mechanize#set_headers so that subclasses can set custom headers. rubyforge.org/tracker/?func=detail&aid=7208&group_id=1453&atid=5712
Aliased Page#referer to Page#page
Fixed a bug when clicking relative urls rubyforge.org/pipermail/mechanize-users/2006-November/000035.html
Fixing a bug when bad version or max age is passed to Cookie::parse rubyforge.org/pipermail/mechanize-users/2006-November/000033.html
Fixing a bug with response codes. [#6526]
Fixed bug [#6548]. Input type of 'button' was not being added as a button.
Fixed bug [#7139]. REXML parser calls hpricot parser by accident
Added keys and values methods to Form
Added has_value? to Form
Added a has_field? method to Form
The add_field! method on Form now creates a field for you on the form. Thanks to Mat Schaffer for the patch. rubyforge.org/pipermail/mechanize-users/2006-November/000025.html
Fixed a bug when form actions have html ecoded entities in them. rubyforge.org/pipermail/mechanize-users/2006-October/000019.html
Fixed a bug when links or frame sources have html encoded entities in the href or src.
Fixed a bug where '#' symbols are encoded rubyforge.org/forum/message.php?msg_id=14747
Added a yield to Page#form so that dealing with forms can be more DSL like.
Added the parsed page to the ResponseCodeError so that the parsed results can be accessed even in the event of an error. rubyforge.org/pipermail/mechanize-users/2006-September/000007.html
Updated documentation (Thanks to Paul Smith)
Added a method to Form called “submit”. Now forms can be submitted by calling a method on the form.
Added a click method to links
Added an REXML pluggable parser for backwards compatability. To use it, just do this:
agent.pluggable_parser.html = WWW::Mechanize::REXMLPage
Fixed a bug with referrers by adding a page attribute to forms and links.
Fixed a bug where domain names were case sensitive. tenderlovemaking.com/2006/09/04/road-to-ruby-mechanize-060/#comment-53
Fixed a bug with URI escaped links. rubyforge.org/pipermail/mechanize-users/2006-September/000002.html
Fixed a bug when options in select lists don't have a value. Thanks Dan Higham
Code in lib/mechanize/form_elements.rb is incorrect.
Fixed a bug with loading text in to links. rubyforge.org/pipermail/mechanize-users/2006-September/000000.html
Changed main parser to use hpricot
Made WWW::Mechanize::Page class searchable like hpricot
Updated WWW::Mechanize#click to support hpricot links like this: @agent.click (page/“a”).first
Clicking a Frame is now possible: @agent.click (page/“frame”).first
Removed deprecated attr_finder
Removed REXML helper methods since the main parser is now hpricot
Overhauled cookie parser to use WEBrick::Cookie
Added WWW::Mechanize#trasact for saving history state between in a transaction. See the EXAMPLES file. Thanks Johan Kiviniemi.
Added support for gzip compressed pages
Forms can now be accessed like a hash. For example, to set the value of an input field named 'name' to “Aaron”, you can do this:
form['name'] = "Aaron"
Or to get the value of a field named 'name', do this:
puts form['name']
File uploads will now read the file specified in FileUpload#file_name
FileUpload can use an IO object in FileUpload#file_data
Fixed a bug with saving files on windows
Fixed a bug with the filename being set in forms
Mechanize#click will now act on the first element of an array. So if an array of links is passed to WWW::Mechanize#click, the first link is clicked. That means the syntax for clicking links is shortened and still supports selecting a link. The following are equivalent:
agent.click page.links.first agent.click page.links
Fixed a bug with spaces in href's and get's
Added a tick, untick, and click method to radio buttons so that radiobuttons can be “clicked”
Added a tick, untick, and click method to check boxes so that checkboxes can be “clicked”
Options on Select lists can now be “tick”ed, and “untick”ed.
Fixed a potential bug conflicting with rails. Thanks Eric Kolve
Updated log4r support for a speed increase. Thanks Yinon Bentor
Added inspect methods and pretty printing
Fixed a bug with input names that are nil
Added a warning when using attr_finder because attr_finder will be deprecated in 0.6.0 in favor of method calls. So this syntax:
@agent.links(:text => 'foo')
should be changed to this:
@agent.links.text('foo')
Added support for selecting multiple options in select tags that support multiple options. See WWW::Mechanize::MultiSelectList.
New select list methods have been added, select_all, select_none.
Options for select lists can now be “clicked” which toggles their selection, they can be “selected” and “unselected”. See WWW::Mechanize::Option
Added a method to set multiple fields at the same time, WWW::Mechanize::Form#set_fields. Which can be used like so:
form.set_fields( :foo => 'bar', :name => 'Aaron' )
Fixed bug with file uploads
Added performance tweaks to the cookie class
Added pluggable parsers. (Thanks to Eric Kolve for the idea)
Changed namespace so all classes are under WWW::Mechanize.
Updating Forms so that fields can be used as accessors (Thanks Gregory Brown)
Added WWW::Mechanize::File as default object used for unknown content types.
Added 'save_as' method to Mechanize::File, so any page can be saved.
Adding 'save_as' and 'load' to CookieJar so that cookies can be saved between sessions.
Added WWW::Mechanize::FileSaver pluggable parser to automatically save files.
Added WWW::Mechanize::Page#title for page titles
Added OpenSSL certificate support (Thanks Mike Dalessio)
Removed support for body filters in favor of pluggable parsers.
Fixed cookie bug adding a '/' when the url is missing one (Thanks Nick Dainty)
Fixed bug with no action in forms. Thanks to Adam Wiggins
Setting a default user-agent string
Added house cleaning to the cookie jar so expired cookies don't stick around.
Added new method WWW::Form#field to find the first field with a given name. (thanks to Gregory Brown)
Added WWW::Mechanize#get_file for fetching non text/html files
Added support for proxies
Added a uri field to WWW::Link
Added a error class WWW::Mechanize::ContentTypeError
Added image alt text to link text
Added an visited? method to WWW::Mechanize
Added Array#value= which will set the first value to the argument. That allows syntax as such: form.fields.name('q').value = 'xyz' Before it was like this: form.fields.name('q').first.value = 'xyz'
Added support for multiple values of the same name
Updated build_query_string to take an array of arrays (Thanks Michal Janeczek)
Added WWW::Mechanize#body_filter= so that response bodies can be preprocessed
Added WWW::Page#body_filter= so that response bodies can be preprocessed
Added support for more date formats in the cookie parser
Fixed a bug with empty select lists
Fixing a problem with cookies not handling no spaces after semicolons
Fixed error in method signature, basic_authetication is now basic_auth
Fixed bug with encoding names in file uploads (Big thanks to Alex Young)
Added options to the select list
Added syntactic sugar for finding things
Fixed bug with HttpOnly option in cookies
Fixed a bug with cookie date parsing
Defaulted dropdown lists to the first element
Added unit tests
Added support for iframes
Made mechanize dependant on ruby-web rather than narf
Added unit tests
Fixed a bunch of warnings
Added support for file uploading
Added support for frames (Thanks Gabriel)
Added more unit tests
Fixed some bugs
Added more unit tests
Added a cookie jar with better cookie support, included expiration of cookies and general cookie security.
Updated mechanize to use built in net/http if ruby version is new enough.
Added support for meta refresh tags
Defaulted form actions to 'GET'
Fixed various bugs
Added more unit tests
Added a response code exception
Thanks to Brian Ellin (brianellin@gmail.com) for: Added support for CA files, and support for 301 response codes