Monday, December 31, 2007

Refactoring: We are not alone!

http://refactormycode.com
Read this blog, it will make you smile!

I did some testing on the refactorings suggested in:
http://refactormycode.com/codes/2-ruby-simple-loop

testing.rb

def prof(t = 1, output = :puts)
starting_time = Time.now
t.times { yield }
dlta = (Time.now - starting_time).to_f/t
case output
when :ret
"### time: #{dlta.to_s}(s) . req/s: #{(1/dlta)}"
else
puts "### time: #{dlta.to_s}(s) . req/s: #{(1/dlta)}"
end
end

times = []
times << ["(1..10).each do |i|
puts i
end",
prof(1000,:ret) {
(1..10).each do |i|
puts i
end
}]

times << ["for i in (1..10)
puts i
end
",prof(1000,:ret){
for i in (1..10)
puts i
end
}]

times << ["(1..10).each { |i| puts i }",prof(1000,:ret){
(1..10).each { |i| puts i }
}]

#times << ["puts (1..10).to_a * '\n'",prof(1000,:ret){
# puts (1..10).to_a * "\n"
#}]

#times << ["puts (1..10).to_a",prof(1000,:ret){
# puts (1..10).to_a
#}]

times << ["1.upto(10) { |i| puts i }",prof(1000,:ret){
1.upto(10) { |i| puts i }
}]

times << ["puts 1,2,3,4,5,6,7,8,9,10",prof(1000,:ret){
puts 1,2,3,4,5,6,7,8,9,10
}]

times << ["10.times {|i| p i+1}",prof(1000,:ret){
10.times {|i| p i+1}
}]

puts '* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *'
times.collect { |t|
puts t
puts '* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *'
}

Two of the suggestions:

  • puts (1..10).to_a
  • puts (1..10).to_a * "\n"
don't really work, so I left them out of the tests)

And the results are ...

* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
(1..10).each do |i|
puts i
end
### time: 0.000165467(s) . req/s: 6043.50112106946
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
for i in (1..10)
puts i
end
### time: 0.00018765(s) . req/s: 5329.07007727152
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
(1..10).each { |i| puts i }
### time: 0.00020241(s) . req/s: 4940.46736821303
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
1.upto(10) { |i| puts i }
### time: 0.000216612(s) . req/s: 4616.54940631175
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
puts 1,2,3,4,5,6,7,8,9,10
### time: 0.000164026(s) . req/s: 6096.59444234451
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
10.times {|i| p i+1}
### time: 0.000208756(s) . req/s: 4790.28147693958
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

Sunday, December 30, 2007

Why I love Ruby on Rails and what should be said on it's performance , part 1


In less than a week, you can establish a working model for your final product.

It is, however, very unwise to consider this is the end of the process.

Ruby on Rails allows ultra-fast development but all this comes in exchange to performance and efficiency.

At the end of the first development period you find yourself with a great working copy but with very low efficiency and as soon as your site starts growing you have to add more servers, more memory, more everything.

What can and should be done in this stage is the infamous 're-factoring' procedure with high attention on performance.

For example for Article and User where each user has many articles and each article
belongs to a user ( 1 -to- many relationship)

If we want to get an article's user email we railslishly write:

Article.find(123).user.email

Which feeds our logs with:

Article Load (0.002447) SELECT * FROM articles WHERE (articles.`id` = 123)
User Load (0.002856) SELECT * FROM users WHERE (users.`id` = 8)

This isn't a problem if you have a small application, this isn't a lot ah? 2 are 100% more SQL queries than only 1 query...

Actually, it's okay to live with it as long as you don't need to grow. When you do, you have to learn SQL.

The truth is even basic SQL can save you here (something PHP or ASP developers would never brag about...):

select * from articles inner join users on users.id = articles.user_id where articles.id = 123

Word is around that Ruby on Rails is slow because of Ruby. The truth for large scale websites is that Rails has much more to do with performance than Ruby.

Let's try a Tiny Profiling Utility on the the scenario described above, where in both cases I want to get an author email according according to it's ID:

>> prof(1000) { Article.find_by_sql('select * from articles inner join users on users.id = articles.user_id where articles.id = 123').first['email'] }
time: 0.004523629(s) . req/s: 221.061453094407
=> nil
>> prof(1000) { Article.find(123).user.email }
time: 0.005518112(s) . req/s: 181.221403262565
=> nil
>>

For one simple action we see a difference of around 20% in speed. Take into consideration that more time consumed by the SQL server is more time other queries are waiting in line, etc. etc.
Let's do the previous test for two articles (123 and 567):

>> prof(1000) {
a = Article.find_by_sql('select * from articles inner join users on users.id = articles.user_id where articles.id = 567 or articles.id = 123')
a[0]['email'] + a[1]['email']
}
time: 0.004959934(s) . req/s: 201.615586013846
=> nil
>> prof(1000) {
a = Article.find(567,123)
a[0].user.email + a[1].user.email
}
time: 0.008703669(s) . req/s: 114.894075130844=> nil

Now that's around 40% difference... imagine what would happen in the case of Many-to-Many connection...

The truth is that eager loading in Rails, should solver this issue - but for some reason it's slower... :
>> prof(1000) {
a = Article.find(567,123, :include => :user)
a[0].user.email+a[1].user.email
}
time: 0.006641206(s) . req/s: 150.575061216291
=> nil

Rails is a great framework and Ruby is a wonderful language, use them and learn some SQL too, they work much faster together...Start using it now: http://rubyonrails.com

Don't forget your sql... http://w3schools.com/sql/default.asp

enjoy!

Friday, December 28, 2007

Mongrel: A recipe for Success, part 1

Take two different good things and combine them together.

Examples:

Wii - Ninetendo's winning game console

Nintendo took two simple technologies:

  1. The Wiimote (http://www.x-arcade.com/newsletter/Wii%20Dupe.shtml)
    The second technology that brings it all together is the use of motion sensors. This could be accomplished with the use of Gyros, as Nintendo did buy a sizable amount of shares in the company Gyration, Inc back in September of 2001. But we do know it does use iMEMS® solid state accelerometers that detect motion by way of acceleration which were provided by Analog Devices, Inc. and/or STMicroelectronics. After much prying around at E3, I was told (by Nintendo's Developer Support) that there are three accelerometers at work in the Wiimote. There are also at least two accelerometers used in the nunchuck add-on to detect motion in it as well.
  2. The Machine (http://en.wikipedia.org/wiki/Wii)
Instead of reinventing the wheel (http://en.wikipedia.org/wiki/PlayStation_3_hardware) they took two good technologies and mashed them up.

http://www.msnbc.msn.com/id/16115967/

Ruby 1.9.0

Have you heard it was released?

http://www.ruby-lang.org/en/news/2007/12/25/ruby-1-9-0-released/

howto install:

autoconf
./configure --prefix=/usr/local/ruby1.9
make
sudo make install



For a long while we were expecting this product that should increase Ruby performance tremendously.
Mind you, I had yet seen a web application running slow because of Ruby. Most of the cases it was misusage of database and memory structures, the major down-pit Rails developers fall into (newbies as well as experts).

Ruby 1.9 is here anyway (dev. version), let's give it a try!

Thursday, December 27, 2007

Tiny profiling utility


def prof(t = 1)
  starting_time = Time.now
  t.times { yield }
  dlta = (Time.now - starting_time).to_f
  puts "time: #{dlta.to_s}(s) . req/s: #{(1/dlta)}"
end

  • Use prof to measure how long a function or any block of code takes.
  • Use parameter t>1 to run the block code t times and return the average time per test
Examples:

>> prof { Flight.find_first }
time: 0.002032(s) . req/s: 492.125984251969
=> nil
>> prof { Flight.find_first }
time: 0.00201(s) . req/s: 497.512437810945
=> nil
>> prof(10) { Flight.find_first }
time: 0.001351(s) . req/s: 740.19245003701
=> nil
>> prof(100) { Flight.find_first }
time: 0.0012727(s) . req/s: 785.731122809775
=> nil
>> prof(1000) { Flight.find_first }
time: 0.001415021(s) . req/s: 706.703292742652
=> nil
>> prof(10000) { Flight.find_first }
time: 0.0013922995(s) . req/s: 718.236270285237
=> nil
>>

ActiveRecord::Base#exists?

class ActiveRecord::Base
def self.exists?(conditions, unique_id = nil)
count1(conditions, unique_id)>0
end
end

Returns true/false if at least one row that follows ceratin conditions is found

  • Can only be used along with count1
  • unique_id - used when you still want to allow a certain raw (defined by id...) to be left out of the count. Useful for validations, ie:

    User.exists? ['nick != ?',user.nick], user.id


ActiveRecord::Base#count1 - fastest counter ever


class ActiveRecord::Base
def self.count1(conditions, unique_id = nil)
connection.execute("select count(1) from #{table_name} where #{sanitize_sql(conditions)} #{'and id<>'+unique_id.to_s if unique_id}").fetch_row.first.to_i
end
end

  • count(1) is the fastest way, as far as I know, for mysql to count table elements.
    Why should we use the relatively expensive count(*) when we can use this modest counter?
  • conditions - can come as ['title = ?','canon'] which than sanitized by sanitize_sql to 'title = "canon"', for example.
  • All this code can be put in environment.rb and than it extends your ActiveRecord::Base objects, thus it will exetend any Model in your railsish system.
  • unique_id - used when you still want to allow a certain raw (defined by id...) to be left out of the count. Useful for validations, ie:

    User.count1(['nick != ?',user.nick], user.id)>0

Everything here applies to Mysql 5.0.14
(never tested on anything else...)

Kudos to Boris Peterbarg