Skip to content

Implement Statement#to_a in C - #768

Open
jhawthorn wants to merge 2 commits into
sparklemotion:mainfrom
jhawthorn:perf-results
Open

jhawthorn wants to merge 2 commits into
sparklemotion:mainfrom
jhawthorn:perf-results

Conversation

@jhawthorn

Copy link
Copy Markdown
Member

Previously to_a was Enumerable#to_a which requires a call to step and done for each row. It's faster to just accumulate the full result.

require "sqlite3"
require "ips"

db = SQLite3::Database.new(":memory:")
hdb = SQLite3::Database.new(":memory:", results_as_hash: true)
[db, hdb].each do |d|
  d.execute("create table t (id integer primary key, i integer, s text)")
  d.transaction { 100.times { |k| d.execute("insert into t values (?, ?, ?)", [k, k * 7, "name-#{k}"]) } }
end
stmt = db.prepare("select * from t")

IPS.ips do |x|
  x.report("Statement#to_a")      { stmt.reset!; stmt.to_a }
  x.report("execute")             { db.execute("select * from t") }
  x.report("execute (as hash)")   { hdb.execute("select * from t") }
end

Before:

$ ruby -I/work/sqlite3-base/lib to_a.rb
      Statement#to_a:    40.241k i/s (± 8.7%, GC  8.9%)
             execute:    39.403k i/s (± 8.1%, GC  9.1%)
   execute (as hash):    16.734k i/s (± 6.8%, GC 17.1%)

Summary
  Statement#to_a ran
    1.02 ± 0.12 times faster than execute
    2.40 ± 0.26 times faster than execute (as hash)

After:

$ ruby -I/work/sqlite3-ruby/lib to_a.rb
      Statement#to_a:    52.023k i/s (± 6.4%, GC 11.3%)
             execute:    46.407k i/s (± 7.8%, GC 10.4%)
   execute (as hash):    33.971k i/s (± 6.9%, GC 10.5%)

Summary
  Statement#to_a ran
    1.12 ± 0.11 times faster than execute
    1.53 ± 0.14 times faster than execute (as hash)

Statement#to_a came from Enumerable, so each row went through #each, which
used Kernel#loop and called #step and #done? from Ruby. Rails' SQLite3
adapter calls Statement#to_a for every query.
Database#execute without a block calls ResultSet#to_a, which went through
Enumerable and #next. It now calls Statement#to_a.

For results_as_hash, each row was built with columns.zip(row).to_h, which
allocated the zipped Array and an Array for each column pair before the
Hash. HashResultSet#to_a now builds each Hash from the row in C.

This keeps the zip behaviour: duplicate column names take the last value.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant