which is not threadsafe. It means you can’t use it while developing multithreaded Rails applications(especially for JRuby). But several monkeypatches can do the trick.<\/p>\n\n\n\n
Workaround<\/h2>\n\n\n\n
When we discovered this limitation almost a year ago, we had next options:<\/p>\n\n\n\n
- development without code reloading and restart server after every change<\/li>
- fix autoload threadsafe issue<\/li>
- create some workaround<\/li><\/ul>\n\n\n\n
As any RoR developer, I can’t imagine my life without automatic code
reloading, so server restart was not an option.
Fixing autoload is a nontrivial task. Corresponded bug was reported 4
years ago, and still wasn’t fixed.<\/p>\n\n\n\nSo, the only choice we had is to create some workaround. Solution
quickly came to mind – simply don’t use multithreading in development mode. We created fallback which made our application logic work in single-threaded mode for development, and in multithreaded mode for production.<\/p>\n\n\n\nFix of autoload<\/h2>\n\n\n\n
After 4 years of struggling, autoload was fixed in ruby-trunk<\/a>. It’s
scheduled to be delivered with Ruby 1.9.4 at the beginning of 2013. Luckily for us, patch was ported into JRuby 1.6.6, so we don’t have to wait till Christmas.<\/p>\n\n\n\nrvm install jruby-1.6.7\n<\/pre>\n\n\n\n
Then I disabled our fallback for development mode, and … found out that server crashed repeatedly across different places. Switching to production mode instantly fixed all exceptions. It looks like threadsafety of autoload was not the only issue.<\/p>\n\n\n\n
Digging into Rails guts<\/h2>\n\n\n\n
Every exception was different, but in most cases it was database
related issue. E.g. some objects in database were missing, some were not updated, deadlocking of SQL-queries and so on.<\/p>\n\n\n\nI started with grep-ing database query logs. Very soon quite weird behavior was observed: bunch of SQL-queries from one controller action were handled by different database connections. But they all should be issued by only one AR-connection!<\/p>\n\n\n\n
After this finding, the main suspect became infamous ActiveRecord::ConnectionAdapters::ConnectionPool. Again! In case you
don’t know, ConnectionPool is not threadsafe in Rails < 3.2.4, so we already had
monkeypatch<\/a> for it.
After adding some traces to ConnectionPool .checkin<\/em> and .checkout<\/em>
methods, we got next flow:<\/p>\n\n\n\n![\"\"](\"\/\/railsware.com\/blog\/wp-content\/uploads\/2012\/11\/code_reload_blog_pic1_v21.png\" "\\"code_reload_blog_pic1_v2\\"")
<\/figure>\n\n\n\nBut why was connection checked-in in the middle of job processing?
ConnectionPool<\/em> has 3 working strategies<\/a>, and
we used the simplest one. Connection is checked-out on demand, and when controller action is done, all connections used by
current thread are checked-in by clear_active_connections!<\/em> method.<\/p>\n\n\n\nBut in development mode something checks-in connections while they are actively used. Such behavior may not affect some applications, but with active usage of database transactions, which are bound to db-connections, disaster is guaranteed.<\/p>\n\n\n\n
There are just a few methods in ActiveRecord, which checkin\/checkout connections. Putting several additional traces I got next flow:
![\"\"](\"\/\/railsware.com\/blog\/wp-content\/uploads\/2012\/11\/code_reload_blog_pic2_v2.png\" "\\"code_reload_blog_pic2_v2\\"")
Connections were checked-in by clear_reloadable_connections! which is
called after processing of every request in development mode. Here is how it looks:<\/p>\n\n\n\ndef clear_reloadable_connections!\n #this part checkin all checked-out connections even used by other threads!\n @reserved_connections.each do |name, conn|\n checkin conn\n end\n @reserved_connections = {}\n #this part disconnects all conections which requires reloading.\n @connections.each do |conn|\n conn.disconnect! if conn.requires_reloading?\n end\n @connections.delete_if do |conn|\n conn.requires_reloading?\n end\nend\n<\/pre>\n\n\n\n