peach-0.5.1/0000755000004100000410000000000012055513773012635 5ustar www-datawww-datapeach-0.5.1/README0000644000004100000410000000351512055513773013521 0ustar www-datawww-dataParallel Each (for ruby with threads) It is pretty common to have iterations over Arrays that can be safely run in parallel. With multicore chips becoming pretty common, single threaded processing is about as cool as Pog. Unfortunately, standard Ruby hates real threads pretty hardcore at the present time; however, for some ruby projects alternate VMs like JRuby do give multicores some lovin'. Peach exists to make this power simple to use with minimal code changes. Functions like map, each, and delete_if are often used in a functional, side-effect free style. If the operation in the block is computationally intense, performance can often be gained by multithreading the process. That's where Peach comes in. In the simplest case, you are one letter away from harnessing the power of parallelism and unlocking the secret of a guilt-free tan. At this stage, the goggles are purely optional. Using Peach Suppose you are going about your day job hacking away at code for the WOPR when you stumble upon the code: cities.each {|city| thermonuclear_war(city)} Clearly, the only winning move is to declare war in parallel. With Peach, the new code is: require 'peach' cities.peach {|city| thermonuclear_war(city)} Requiring peach.rb monkey patches Array into submission. Currently Peach provides peach, pmap, and pdelete_if. Each of these functions takes an optional argument n, which represents the desired number of worker threads with the default being one thread per Array element. For cheaper operations on a large number of elements, you probably want to set n to something reasonably low. (0...10000).to_a.pmap(4) {|x| process(x)} Constructing the threads and adding on a few layers of indirection does add a bit of overhead to the iteration especially on MRI. Keep this in mind and remember to benchmark when unsure. peach-0.5.1/test/0000755000004100000410000000000012055513773013614 5ustar www-datawww-datapeach-0.5.1/test/test_helper.rb0000644000004100000410000000037012055513773016457 0ustar www-datawww-data$:.unshift(File.dirname(__FILE__) + '/../lib') $:.unshift(File.dirname(__FILE__)) unless $:.include?(File.dirname(__FILE__)) || $:.include?(File.expand_path(File.dirname(__FILE__))) require 'rubygems' require 'test/unit' require 'shoulda' peach-0.5.1/test/peach_test.rb0000644000004100000410000000241312055513773016260 0ustar www-datawww-datarequire File.join(File.dirname(__FILE__), "test_helper") require File.join(File.dirname(__FILE__), "..", "lib", "peach") require 'thread' class PeachTest < Test::Unit::TestCase [:peach, :pmap, :pselect].each do |f| context "Parallel function #{f}" do normal_f = f.to_s[1..-1].to_sym setup do @data = [1, 2, 3, 5, 8]*1001 @block = lambda{|i| i**2} end should "return the same result as #{normal_f}" do assert_equal @data.send(normal_f, &@block), @data.send(f, 100, &@block) end should "return the same result as #{normal_f} for empty list" do assert_equal [].send(normal_f, &@block), [].send(f, nil, &@block) end end end [:peach, :pmap, :pselect].each do |f| context "#{f}" do [nil, 101, 99, 51, 49, 20, 5, 1].each do |pool| should "invoke the block exactly once for each array item," + " with thread pool size of #{pool.inspect}" do source_array = (0...100).to_a q = Queue.new() source_array.send(f, pool) {|i| q.push(i)} result_array = [] until q.empty? result_array << q.pop end assert_equal source_array, result_array.sort end end end end end peach-0.5.1/LICENSE0000644000004100000410000000203612055513773013643 0ustar www-datawww-dataCopyright (c) 2008 Ben Hughes Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. peach-0.5.1/Rakefile0000644000004100000410000000030512055513773014300 0ustar www-datawww-datarequire 'bundler' Bundler::GemHelper.install_tasks require 'rake' require 'rake/testtask' Rake::TestTask.new(:test) do |t| t.libs << 'lib' t.pattern = 'test/*_test.rb' t.verbose = true end peach-0.5.1/peach.gemspec0000644000004100000410000000121512055513773015261 0ustar www-datawww-data# -*- encoding: utf-8 -*- $:.push File.expand_path("../lib", __FILE__) require "peach/version" Gem::Specification.new do |s| s.name = 'peach' s.version = Peach::VERSION s.authors = ['Ben Hughes'] s.email = 'ben@pixelmachine.org' s.summary = 'Parallel Each and other parallel things' s.homepage = 'http://peach.rubyforge.org' s.files = `git ls-files`.split("\n") s.test_files = `git ls-files -- {test,spec,features}/*`.split("\n") s.executables = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) } s.require_paths = ["lib"] s.add_development_dependency('rake') s.add_development_dependency('shoulda') end peach-0.5.1/metadata.yml0000644000004100000410000000407012055513773015141 0ustar www-datawww-data--- !ruby/object:Gem::Specification name: peach version: !ruby/object:Gem::Version version: 0.5.1 prerelease: platform: ruby authors: - Ben Hughes autorequire: bindir: bin cert_chain: [] date: 2012-07-03 00:00:00.000000000 Z dependencies: - !ruby/object:Gem::Dependency name: rake requirement: !ruby/object:Gem::Requirement none: false requirements: - - ! '>=' - !ruby/object:Gem::Version version: '0' type: :development prerelease: false version_requirements: !ruby/object:Gem::Requirement none: false requirements: - - ! '>=' - !ruby/object:Gem::Version version: '0' - !ruby/object:Gem::Dependency name: shoulda requirement: !ruby/object:Gem::Requirement none: false requirements: - - ! '>=' - !ruby/object:Gem::Version version: '0' type: :development prerelease: false version_requirements: !ruby/object:Gem::Requirement none: false requirements: - - ! '>=' - !ruby/object:Gem::Version version: '0' description: email: ben@pixelmachine.org executables: [] extensions: [] extra_rdoc_files: [] files: - .gitignore - Gemfile - Gemfile.lock - LICENSE - README - Rakefile - bn/peach_bn.rb - bn/peach_test.rb - lib/peach.rb - lib/peach/version.rb - peach.gemspec - test/peach_test.rb - test/test_helper.rb - web/Peach.sketch.png - web/index.html homepage: http://peach.rubyforge.org licenses: [] post_install_message: rdoc_options: [] require_paths: - lib required_ruby_version: !ruby/object:Gem::Requirement none: false requirements: - - ! '>=' - !ruby/object:Gem::Version version: '0' segments: - 0 hash: 2784794470947016613 required_rubygems_version: !ruby/object:Gem::Requirement none: false requirements: - - ! '>=' - !ruby/object:Gem::Version version: '0' segments: - 0 hash: 2784794470947016613 requirements: [] rubyforge_project: rubygems_version: 1.8.24 signing_key: specification_version: 3 summary: Parallel Each and other parallel things test_files: - test/peach_test.rb - test/test_helper.rb peach-0.5.1/Gemfile0000644000004100000410000000004612055513773014130 0ustar www-datawww-datasource "http://rubygems.org" gemspec peach-0.5.1/bn/0000755000004100000410000000000012055513773013234 5ustar www-datawww-datapeach-0.5.1/bn/peach_test.rb0000644000004100000410000000121412055513773015676 0ustar www-datawww-datarequire 'peach' require 'benchmark' def fac(n) if n == 0 return 1 else n*fac(n-1) end end puts "PEACH TEST" puts "each:" def each_test (0...1000).to_a.sort_by{rand}.each do |x| fac(x) end end puts Benchmark.measure { each_test } puts "peach:" def peach_test (0...1000).to_a.sort_by{rand}.peach(4) do |x| fac(x) end end puts Benchmark.measure { peach_test } puts "map:" def map_test (0...1000).to_a.sort_by{rand}.map do |x| fac(x) end end puts Benchmark.measure { map_test } puts "pmap:" def pmap_test (0...1000).to_a.sort_by{rand}.pmap(4) do |x| fac(x) end end puts Benchmark.measure { pmap_test } peach-0.5.1/bn/peach_bn.rb0000644000004100000410000000241712055513773015324 0ustar www-datawww-data#Benchmark for Peach #Count intrawiki links in wikipedia data require 'peach' require 'benchmark' require 'digest/md5' puts "PEACH BENCHMARK" puts "Wikipedia Processing" puts #Read a small slice of the Wikipedia XML file fn = "peach_bn_data.txt" puts "Reading in dataset #{fn}" puts "Dataset is #{File.size(fn)/1024} kb" dataset = "" puts Benchmark.measure("read dataset") { dataset = File.read(fn) } puts "Splitting dataset into articles" articles = [] puts Benchmark.measure("split dataset") { articles = dataset.scan(/.*?<\/text>/m) articles.delete_if {|x| /#redirect/i.match(x) } } puts "Found #{articles.size} articles" puts puts "BEGIN REAL BENCHMARK" puts puts "map:" links1 = [] for i in (1...5) puts Benchmark.measure { links1 = articles.map do |article| article.scan(/\[\[[\w -']+?\]\]/m) #.each do |link| # Digest::MD5.hexdigest(article) #end end } end puts "Found #{links1.flatten.size} links" puts puts "pmap:" links2 = [] for i in (1...5) puts Benchmark.measure { links2 = articles.pmap(6) do |article| article.scan(/\[\[[\w -']+?\]\]/m) #.each do |link| # Digest::MD5.hexdigest(link) #end end } end puts "Found #{links2.flatten.size} links" p links2 - links1 puts "END" peach-0.5.1/.gitignore0000644000004100000410000000000412055513773014617 0ustar www-datawww-datapkg peach-0.5.1/lib/0000755000004100000410000000000012055513773013403 5ustar www-datawww-datapeach-0.5.1/lib/peach.rb0000644000004100000410000000253012055513773015010 0ustar www-datawww-data# monkey patch Enumerable by reopening it. Enumerable.send(:include, Peach) # doesn't seem to work as it should. module Enumerable def _peach_run(pool = nil, &b) pool ||= $peach_default_threads || count pool = 1 unless pool >= 1 div = (count/pool).to_i # should already be integer div = 1 unless div >= 1 # each thread better do something! threads = [] each_slice(div).with_index do |slice, idx| threads << Thread.new(slice) do |thread_slice| yield thread_slice, idx, div end end threads.each{|t| t.join } self end def peach(pool = nil, &b) _peach_run(pool) do |thread_slice, idx, div| thread_slice.each{|elt| yield elt} end end def pmap(pool = nil, &b) result = Array.new(count) lock = Mutex.new _peach_run(pool) do |thread_slice, idx, div| thread_slice.each_with_index do |elt, offset| local_result = yield elt lock.synchronize do result[(idx*div)+offset] = local_result end end end result end def pselect(pool = nil, &b) results, result = [],[] lock = Mutex.new _peach_run(pool) do |thread_slice, idx, div| local_result = thread_slice.select(&b) lock.synchronize do results[idx] = local_result end end results.each {|x| result += x if x} result end end peach-0.5.1/lib/peach/0000755000004100000410000000000012055513773014463 5ustar www-datawww-datapeach-0.5.1/lib/peach/version.rb0000644000004100000410000000004512055513773016474 0ustar www-datawww-datamodule Peach VERSION = "0.5.1" end peach-0.5.1/Gemfile.lock0000644000004100000410000000046612055513773015065 0ustar www-datawww-dataPATH remote: . specs: peach (0.5.1) GEM remote: http://rubygems.org/ specs: rake (0.9.2.2) shoulda (3.0.1) shoulda-context (~> 1.0.0) shoulda-matchers (~> 1.0.0) shoulda-context (1.0.0) shoulda-matchers (1.0.0) PLATFORMS ruby DEPENDENCIES peach! rake shoulda peach-0.5.1/web/0000755000004100000410000000000012055513773013412 5ustar www-datawww-datapeach-0.5.1/web/index.html0000644000004100000410000002321212055513773015407 0ustar www-datawww-data Peach - Parallel Each
Peach

Parallel Each (for ruby with threads)

It is pretty common to have iterations over Arrays that can be safely run in parallel. With multicore chips becoming pretty common, single threaded processing is about as cool as Pog. Unfortunately, standard Ruby hates real threads pretty hardcore at the present time; however, for some ruby projects alternate VMs like JRuby do give multicores some lovin'. Peach exists to make this power simple to use with minimal code changes.

Functions like map, each, and delete_if are often used in a functional, side-effect free style. If the operation in the block is computationally intense, performance can often be gained by multithreading the process. That's where Peach comes in. In the simplest case, you are one letter away from harnessing the power of parallelism and unlocking the secret of a guilt-free tan. At this stage, the goggles are purely optional.

Using Peach

Suppose you are going about your day job hacking away at code for the WOPR when you stumble upon the code: 

cities.each {|city| thermonuclear_war(city)}

Clearly, the only winning move is to declare war in parallel. With Peach, the new code is: 

require 'peach'

cities.peach {|city| thermonuclear_war(city)}

Requiring peach.rb monkey patches Array into submission. Currently Peach provides peach, pmap, and pdelete_if. Each of these functions takes an optional argument n, which represents the desired number of worker threads with the default being one thread per Array element. For cheaper operations on a large number of elements, you probably want to set n to something reasonably low. 

(0...10000).to_a.pmap(4) {|x| process(x)}

Constructing the threads and adding on a few layers of indirection does add a bit of overhead to the iteration especially on MRI. Keep this in mind and remember to benchmark when unsure.

Syntax (without all the words)

require 'peach'

[1,2,3,4].peach{|x| f(x)} #Spawns 4 threads, => [1,2,3,4]
[1,2,3,4].pmap{|x| f(x)} #Spawns 4 threads, => [f(1),f(2),f(3),f(4)]
[1,2,3,4].pdelete_if{|x| x > 2} #Spawns 4 threads, => [3,4]


[1,2,3,4].peach(2){|x| f(x)} #Spawns 2 threads, => [1,2,3,4]
[1,2,3,4].pmap(2){|x| f(x)} #Spawns 2 threads, => [f(1),f(2),f(3),f(4)]
[1,2,3,4].pdelete_if(2){|x| x > 2} #Spawns 2 threads, => [3,4]

FAQ

Q: I use normal ruby (MRI 1.8 or 1.9), will Peach confer superpowers and great performance upon my code?
A: No, on MRI your code will be slightly slower because of the increased overhead for Thread creation. MRI is singlethreaded so Peach will not make it magically parallel.

Q: Why should I switch to JRuby to get the benefits of Peach?
A: Switching to JRuby for code that needs better performance is a good idea even without Peach. JRuby is insanely fast and a good idea. The multithreading and ability to use this humble utility is just another feature.

Q: Benchmarks?
A: I am pretty bad at benchmarking code, but I do have a simple test comparing performance between map and pmap on MRI and JRuby. Headius helped in the preparation of these materials. Check it out. JRuby on Java 1.6 is even faster. If you come up with any benchmarks, do let me know.

Eat a Peach

Peach is distributed as a gem from github, so:
gem install schleyfox-peach --source=http://gems.github.com.

peach-0.5.1/web/Peach.sketch.png0000755000004100000410000001710412055513773016426 0ustar www-datawww-dataPNG  IHDR@]usRGB pHYs  tIME&*g࿵IDATx˕8Hre|e2Je0TyTf T$I$ϋz///Jhں`hgUZ}H 5#ȚgItӠ&3m0Hsx1Oa gx~q\߮Wk|pRwyWW7Y6s;r/.uWˣܥ|=}<}qG8XݖGhWgz%^}?u%'ģp$"%w{[K`wUw)Tf>!wܓw@_]0Km^. tPIwC^d_닅^.՜Cf_^5IScy#}pt{;mws[Rk^V{R NMzyxˊ2V_. &W+}Fzi_F7pu/ _{%U."<: p5Wp]pQ7 }3.{,_` ̑έ-:ʾryBw$)0.%LO>${ N }p4b. `~~. S/,gUH_0~ytLKE,Kۨ(/// *@p]ź\l@ wA1#Z5f0}N w8b  ܅xe ȋ4D߽De$)ٗ#.L_./p2N_L2pIwA}~%p9wb `Ⱦ fjG+L -(i$.. @X].$5./ F}F*:a#dK#t|Vo* fpjSn0w@ +V<ȞO y3.[qOg[K ȻԺ'zT gc q8[b^_\./KKRqW,3܅ş_.D_>-N:=$ĥXcu3t|o.6It ݝ1]ˀ\/Oe%kyT|eqqc-UeqI  q'.ԧ4lc0]V-#7f?Jc{VKɖ  q@(GZ.&{âf TNb.̔n){MnwF\U2*?&c$`0`NMyh-y ?K wRKAzKKsrϧC_ \avQ%.zR7tXJ/ԙsU .+K(E~Pz=Z݈g4X\:+ h-?EܥAt}Z#-`鋁/]P:TodrRde_+f~gkyHwP\DS0W? rst1n@"ύ -=wiiM˸t֬\jlEZ:rS}?Rsc&6 j0OIw!.=cFkz]H)6K[q9\e?`Kxz|".w_Ⲓ5O K~/ !R848%,gk2T\ޅ`x]/ >E}..cS /[lCzv(Ve."{b `X_--".qppe:}FQtf3=rӎ}ĥd6ШpΜK݋&y׏9xƤ^TIy<44L?贸g%N-7sܥB,qkJCoLAYr p$N)=tPFdr^nUQu }2 yKԀR']Jdt`iq)%oKy? I\uZKZjd-׆)—?.e*qik 6S鑾p”}F{tqsM'8Ee_ڪm/taj?S]v'x(]֙:{p8_\߲ryq {tnZp<']dDY$N\!n,v'J1npE`tRRtt~wKg_+# KD!TJm:K(*/. qo{Dϻ̽Xqiȭ]!cg+֛V˻I5HJb  t [b.͇.,.G ."eXL깦Y*qi;@]3j[TU㔸GajRq9sF0.m$Z\p>R5-FZ$1 &h q.1r~\IK.m{jKNɅuq=W*~?sc f@oaW:ҥ]j*./{]Ʃ/.gϹz`-oa0zHR}#qy{%[:0޴S/<#ky@7]&]. r$.k:i-y?>᳨胱U>_di--ܮhS M ^X.t%,|DIZR2G'pZ"|m4JIť5FWBd_4%~Fk&]ٮe`%WxۏZxˣTDVﰿ+U>{c2J1X}F߻?qH2h ܥS)Rq98fqi]@b QYs #K=u/jL.^Ғ.˸ bh !ӻKUKm=î8CŤ %7hbf-=H\\" z^.;ؤ22'!1Xy#K6q'U%Tg`Fkc=!+}Vi'.] X&c;nt.뛈tiHk t۸KSH6F,4)š.R/t\E\2 .n%qIVUUFR0$,.Mrqi/7#s`ܘgt^* uN*!5H jlcq B-,]3{'.C&w=SsⲌ,Z{1-qa):+UUtm [W]0c.qIUno.B q9粣.5O5tW5 K\lK1bq1PPrSqmw.(ug49ul%.-utO pD%Z`>'u `9.]:2ZdťD sm .AeKm{`;/.I2dC@;M=gK؁F`KzN\sEe\ wi&[  &*QW1.ot]Jz$.fGE@Ebw?"0^2ųtQI `'wi N\J婸0f}FW2l#qnK,b.;5.!m^P>R4A^^QfE+ˎortS] fP'32ťi'~w]qޕ4twOwxY\2Vؿa_.e9VӸ˾A\.ԭ_tHQ,_L.꤆l`JT/.S &.#pGN\)_* ZpO_b3y2I݂vd%:OЍ}FƍO .;٤(y_8pGF^!ȔYak,ayeQm gtKmf֯h[-\;Y+vC|#.wzQ^ ήlln*a-Mc Q>zϾUGKϺ.bEf,nK qyZn ]2]tS\ y.#E'hr,.[WQgqz|MM<<}LJ[<.=XL/0\\Rut=h}SI nmҁ/q(ok]^K7`7f} mTW\n:]"=Fs' wjŷwi"z4F\‹ G12K7o;gI=iy#2|t-8X&nPX.ƃ7uۏ}hM>k< ƸsqO~Მk]Fk:)hfJ>@qI?u$^nƀ/>}O(]Pf SKy]DŽW2]Ų3j} u.eK/Y K5}/.\\.Cٵ_ 9͚U&.Yw/}%No#Aj9s߻@qwYH_2z4X[\8xF25cҝ#.c۸fY.2N7Fbf6.다k8ԃ!/7GTw@_ֶ̰)/s=xٵzt7o,-xb]f՗ u 6R-Ԛ!wOpY(tiˆ Ҟ^`Aw)՗E-WҭElۗ]wYG_N?P}E`ʒ(.)tx?fI2w3G2` ,p 8s7q=Df\ NQ iiq i-ebe٧WaI>/Yz1-6a˨ M:. R 22"2uVtk:e'ӯ?F6z\Y4L>g. O]2|(.,GYCիm]KJQ`C$8IZJn_*e~#TX9h\GFxARSsv߫-:]b򛕺p6Of-tklĩJHlMIY0Ճ 2`.Ru߂;ɏ9ThX1.  dC2@TAI;v]B7D.!D{D{tZa8/#+u+cmT:kԞgȻk>jfy¶4~u1-lk7pj ^6CDӀQp()٭gD7,mt=m_;h_hK6eg%%dIMD^'