Skip to content

openssl-master: rake test_fips failures about keys #926

Closed

Description

@junaruga

I executed the current latest master branch dfbbac6 with the latest OpenSSL master branch openssl/openssl@b20da23 in my forked repository today. I am seeing the failures in the FIPS tests below.

https://github.com/junaruga/ruby-openssl/actions/runs/16680199093/job/47216748820

$ sed -e "s|OPENSSL_DIR|$HOME/openssl|" tool/openssl_fips.cnf.tmpl > tmp/openssl_fips.cnf
$ export OPENSSL_CONF=$(pwd)/tmp/openssl_fips.cnf
$ bundle exec rake debug
$ bundle exec rake test_fips TESTOPTS="-v --no-show-detail-immediately" OSSL_TEST_ALL=1
...
  1) Error: test_check_key(OpenSSL::TestEC): OpenSSL::PKey::PKeyError: Could not parse PKey
/home/runner/work/openssl/openssl/test/openssl/test_pkey_ec.rb:102:in `read'
/home/runner/work/openssl/openssl/test/openssl/test_pkey_ec.rb:102:in `test_check_key'
      99:     if aws_lc? # AWS-LC automatically does key checks on the parsed key.
     100:       assert_raise(OpenSSL::PKey::PKeyError) { OpenSSL::PKey.read(ec_key_data) }
     101:     else
  => 102:       key4 = OpenSSL::PKey.read(ec_key_data)
     103:       assert_raise(OpenSSL::PKey::ECError) { key4.check_key }
     104:     end
     105: 
Error: OpenSSL::PKey::PKeyError: Could not parse PKey
/home/runner/work/openssl/openssl/test/openssl/test_pkey_ec.rb:102:in `read'
/home/runner/work/openssl/openssl/test/openssl/test_pkey_ec.rb:102:in `test_check_key'

  2) Error: test_dh_compute_key(OpenSSL::TestEC): OpenSSL::PKey::ECError: EC_KEY_generate_key: unable to create drbg
/home/runner/work/openssl/openssl/test/openssl/test_pkey_ec.rb:180:in `generate'
/home/runner/work/openssl/openssl/test/openssl/test_pkey_ec.rb:180:in `test_dh_compute_key'
     177:   end
     178: 
     179:   def test_dh_compute_key
  => 180:     key_a = OpenSSL::PKey::EC.generate("prime256v1")
     181:     key_b = OpenSSL::PKey::EC.generate(key_a.group)
     182: 
     183:     pub_a = key_a.public_key
Error: OpenSSL::PKey::ECError: EC_KEY_generate_key: unable to create drbg
...
278 tests, 686 assertions, 0 failures, 170 errors, 0 pendings, 24 omissions, 0 notifications
33.0709% passed

Activity

  1. junaruga commented on Aug 1, 2025

    @junaruga
    MemberAuthor

    The following CI passed with the openssl/openssl@b8c46cb.
    https://github.com/ruby/openssl/actions/runs/16656447633/job/47142524897

    So, maybe a commit on OpenSSL master branch between the b8c46cba5fa0894fc026f1d965beec8089b79dbf (passed) and b20da2328018107414fe896e59e7d4d6c8af8174 (failed) caused the Ruby OpenSSL's test failures.

  2. rhenium commented on Aug 2, 2025

    @rhenium
    Member

    I can't reproduce the errors on my box with OpenSSL b20da2328018107414fe896e59e7d4d6c8af8174 compiled with the fips provider, and all tests passed as expected.

    The error is also weird. require "openssl" succeeds, and during it, a DH PEM is decoded.

    DH_ffdhe2048 = OpenSSL::PKey::DH.new <<-_end_of_pem_
    -----BEGIN DH PARAMETERS-----
    MIIBCAKCAQEA//////////+t+FRYortKmq/cViAnPTzx2LnFg84tNpWp4TZBFGQz
    +8yTnc4kmz75fS/jY2MMddj2gbICrsRhetPfHtXV/WVhJDP1H18GbtCFY2VVPe0a
    87VXE15/V8k1mE8McODmi3fipona8+/och3xWKE2rec1MKzKT0g6eXq8CrGCsyT7
    YdEIqUuyyOP7uWrat2DX9GgdT0Kj3jlN9K5W7edjcrsZCwenyO4KbXCeAvzhzffi
    7MA0BM0oNC9hkXL+nOmFg/+OTxIy7vKBg8P+OxtMb61zO7X8vC7CIAXFjvGDfRaD
    ssbzSibBsu/6iGtCOGEoXJf//////////wIBAg==
    -----END DH PARAMETERS-----
    _end_of_pem_

    However, loading the exact same PEM fails later (dh2048_ffdhe2048.pem has the identical content as the heredoc in lib/openssl/ssl.rb).

    def test_params
    dh = Fixtures.pkey("dh2048_ffdhe2048")
    assert_kind_of(OpenSSL::BN, dh.p)
    assert_equal(dh.p, dh.params["p"])
    assert_kind_of(OpenSSL::BN, dh.g)
    assert_equal(dh.g, dh.params["g"])
    assert_nil(dh.pub_key)
    assert_nil(dh.params["pub_key"])
    assert_nil(dh.priv_key)
    assert_nil(dh.params["priv_key"])
    dhkey = OpenSSL::PKey.generate_key(dh)
    assert_equal(dh.params["p"], dhkey.params["p"])
    assert_kind_of(OpenSSL::BN, dhkey.pub_key)
    assert_equal(dhkey.pub_key, dhkey.params["pub_key"])
    assert_kind_of(OpenSSL::BN, dhkey.priv_key)
    assert_equal(dhkey.priv_key, dhkey.params["priv_key"])
    end

    https://github.com/junaruga/ruby-openssl/actions/runs/16680199093/job/47216748820#step:11:683

  3. junaruga commented on Aug 4, 2025

    @junaruga
    MemberAuthor

    I was able to reproduce this issue with openssl/openssl@b20da23 on my local.

    $ OPENSSL_CONF=/home/jaruga/.local/openssl-3.6.0-dev-fips-debug-b20da23280/ssl/openssl_fips.cnf \
      /home/jaruga/.local/openssl-3.6.0-dev-fips-debug-b20da23280/bin/openssl list -providers
    Providers:
      base
        name: OpenSSL Base Provider
        version: 3.6.0
        status: active
      fips
        name: OpenSSL FIPS Provider
        version: 3.6.0
        status: active
    
    $ which ruby
    ~/.local/ruby-3.5.0dev-debug-8d14d6ea2d/bin/ruby
    
    $ bundle list
    Gems included by the bundle:
      * date (3.4.1)
      * erb (5.0.2)
      * forwardable (1.3.3)
      * power_assert (2.0.5)
      * prime (0.1.4)
      * psych (5.2.6)
      * rake (13.3.0)
      * rake-compiler (1.3.0)
      * rdoc (6.14.2)
      * singleton (0.3.0)
      * stringio (3.1.7)
      * test-unit (3.7.0)
      * test-unit-ruby-core (1.0.7)
    Use `bundle info` to print more detailed information about a gem
    
    $ OPENSSL_CONF=$HOME/.local/openssl-3.6.0-dev-fips-debug-b20da23280/ssl/openssl_fips.cnf \
      bundle exec rake test_fips
    /home/jaruga/.local/ruby-3.5.0dev-debug-8d14d6ea2d/lib/ruby/3.5.0+2/openssl/ssl.rb:24: warning: already initialized constant OpenSSL::SSL::SSLContext::DEFAULT_PARAMS
    /home/jaruga/.local/ruby-3.5.0dev-debug-8d14d6ea2d/lib/ruby/3.5.0+2/openssl/ssl.rb:24: warning: previous definition of DEFAULT_PARAMS was here
    ...
    E
    ===============================================================================
    Error: test_check_key(OpenSSL::TestEC): OpenSSL::PKey::PKeyError: Could not parse PKey
    /home/jaruga/var/git/ruby/openssl/test/openssl/test_pkey_ec.rb:102:in 'OpenSSL::PKey.read'
    /home/jaruga/var/git/ruby/openssl/test/openssl/test_pkey_ec.rb:102:in 'OpenSSL::TestEC#test_check_key'
          99:     if aws_lc? # AWS-LC automatically does key checks on the parsed key.
         100:       assert_raise(OpenSSL::PKey::PKeyError) { OpenSSL::PKey.read(ec_key_data) }
         101:     else
      => 102:       key4 = OpenSSL::PKey.read(ec_key_data)
         103:       assert_raise(OpenSSL::PKey::ECError) { key4.check_key }
         104:     end
         105:-
    ===============================================================================
    E
    ===============================================================================
    Error: test_dh_compute_key(OpenSSL::TestEC): OpenSSL::PKey::ECError: EC_KEY_generate_key: unable to create drbg
    /home/jaruga/var/git/ruby/openssl/test/openssl/test_pkey_ec.rb:180:in 'OpenSSL::PKey::EC.generate'
    /home/jaruga/var/git/ruby/openssl/test/openssl/test_pkey_ec.rb:180:in 'OpenSSL::TestEC#test_dh_compute_key'
         177:   end
         178:-
         179:   def test_dh_compute_key
      => 180:     key_a = OpenSSL::PKey::EC.generate("prime256v1")
         181:     key_b = OpenSSL::PKey::EC.generate(key_a.group)
         182:-
         183:     pub_a = key_a.public_key
    ===============================================================================
    ...
    276 tests, 682 assertions, 0 failures, 169 errors, 0 pendings, 24 omissions, 0 notifications
    32.9365% passed
    -------------------------------------------------------------------------------
    34.49 tests/s, 85.22 assertions/s
    rake aborted!
    Command failed with status (1)
    /home/jaruga/var/git/ruby/openssl/Rakefile:22:in 'block in <top (required)>'
    /home/jaruga/var/git/ruby/openssl/bundle/ruby/3.5.0+2/gems/rake-13.3.0/exe/rake:27:in '<top (required)>'
    /home/jaruga/.local/ruby-3.5.0dev-debug-8d14d6ea2d/bin/bundle:25:in 'Kernel#load'
    /home/jaruga/.local/ruby-3.5.0dev-debug-8d14d6ea2d/bin/bundle:25:in '<main>'
    Tasks: TOP => test_fips_internal
    (See full trace by running task with --trace)
    
  4. junaruga commented on Aug 4, 2025

    @junaruga
    MemberAuthor

    The following test always fails with the openssl/openssl@b20da23.

    $ OPENSSL_CONF=$HOME/.local/openssl-3.6.0-dev-fips-debug-b20da23280/ssl/openssl_fips.cnf \
      bundle exec rake test_fips TEST=test/openssl/test_pkey_ec.rb TESTOPTS="--name=test_check_key -v"
    /home/jaruga/.local/ruby-3.5.0dev-debug-8d14d6ea2d/lib/ruby/3.5.0+2/openssl/ssl.rb:24: warning: already initialized constant OpenSSL::SSL::SSLContext::DEFAULT_PARAMS
    /home/jaruga/.local/ruby-3.5.0dev-debug-8d14d6ea2d/lib/ruby/3.5.0+2/openssl/ssl.rb:24: warning: previous definition of DEFAULT_PARAMS was here
    /bin/gmake install sitearchdir=../../../../lib sitelibdir=../../../../lib target_prefix=
    /bin/install -c -m 0755 openssl.so ../../../../lib
    cp tmp/x86_64-linux/openssl/3.5.0/openssl.so tmp/x86_64-linux/stage/lib/openssl.so
    Loaded suite /home/jaruga/var/git/ruby/openssl/bundle/ruby/3.5.0+2/gems/rake-13.3.0/lib/rake/rake_test_loader
    Started
    OpenSSL::TestCase:
      OpenSSL::PKeyTestCase:
        OpenSSL::TestEC:
          test_check_key:																							E
    ===============================================================================================================================================================================================================================
    Error: test_check_key(OpenSSL::TestEC): OpenSSL::PKey::PKeyError: Could not parse PKey
    /home/jaruga/var/git/ruby/openssl/test/openssl/test_pkey_ec.rb:102:in 'OpenSSL::PKey.read'
    /home/jaruga/var/git/ruby/openssl/test/openssl/test_pkey_ec.rb:102:in 'OpenSSL::TestEC#test_check_key'
          99:     if aws_lc? # AWS-LC automatically does key checks on the parsed key.
         100:       assert_raise(OpenSSL::PKey::PKeyError) { OpenSSL::PKey.read(ec_key_data) }
         101:     else
      => 102:       key4 = OpenSSL::PKey.read(ec_key_data)
         103:       assert_raise(OpenSSL::PKey::ECError) { key4.check_key }
         104:     end
         105:
    ===============================================================================================================================================================================================================================
    : (0.013514)
    
    Finished in 0.014728665 seconds.
    -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    1 tests, 10 assertions, 0 failures, 1 errors, 0 pendings, 0 omissions, 0 notifications
    0% passed
    -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    67.89 tests/s, 678.95 assertions/s
    rake aborted!
    Command failed with status (1)
    /home/jaruga/var/git/ruby/openssl/Rakefile:22:in 'block in <top (required)>'
    /home/jaruga/var/git/ruby/openssl/bundle/ruby/3.5.0+2/gems/rake-13.3.0/exe/rake:27:in '<top (required)>'
    /home/jaruga/.local/ruby-3.5.0dev-debug-8d14d6ea2d/bin/bundle:25:in 'Kernel#load'
    /home/jaruga/.local/ruby-3.5.0dev-debug-8d14d6ea2d/bin/bundle:25:in '<main>'
    Tasks: TOP => test_fips_internal
    (See full trace by running task with --trace)
    
  5. junaruga commented on Aug 4, 2025

    @junaruga
    MemberAuthor

    The following test always fails with the openssl/openssl@b20da23.

    Sorry after compiling, the failure disappeared. Weird.

  6. junaruga commented on Aug 4, 2025

    @junaruga
    MemberAuthor

    I am working on the following branch.

    https://github.com/junaruga/ruby-openssl/tree/wip/fips-failures-debug

    I found the following test.rb fails with error on the openssl-master, but passes on the openssl-3.5.0.

    https://github.com/junaruga/ruby-openssl/actions/runs/16729044811/job/47352316360

    $ cat test.rb
    require 'openssl'
    
    key_data = <<~EOF
    -----BEGIN EC PRIVATE KEY-----
    MHcCAQEEIP+TT0V8Fndsnacji9tyf6hmhHywcOWTee9XkiBeJoVloAoGCCqGSM49
    AwEHoUQDQgAEBkhhJIU/2/YdPSlY2I1k25xjK4trr5OXSgXvBC21PtY0HQ7lor7A
    jzT0giJITqmcd81fwGw5+96zLcdxTF1hVQ==
    -----END EC PRIVATE KEY-----
    EOF
    OpenSSL::PKey.read(key_data)
    
    $ OPENSSL_CONF="$OPENSSL_DIR/ssl/openssl_fips.cnf" \
      bundle exec rake debug
    /home/jaruga/.local/ruby-3.5.0dev-debug-8d14d6ea2d/lib/ruby/3.5.0+2/openssl/ssl.rb:24: warning: already initialized constant OpenSSL::SSL::SSLContext::DEFAULT_PARAMS
    /home/jaruga/.local/ruby-3.5.0dev-debug-8d14d6ea2d/lib/ruby/3.5.0+2/openssl/ssl.rb:24: warning: previous definition of DEFAULT_PARAMS was here
    OpenSSL::OPENSSL_VERSION: OpenSSL 3.6.0-dev
    OpenSSL::OPENSSL_LIBRARY_VERSION: OpenSSL 3.6.0-dev
    OpenSSL::OPENSSL_VERSION_NUMBER: 30600000
    OpenSSL::LIBRESSL_VERSION_NUMBER: undefined
    FIPS enabled: true
    Providers: fips, base
    
    $ OPENSSL_CONF="$OPENSSL_DIR/ssl/openssl_fips.cnf" \
      bundle exec ruby -I./lib test.rb
    test.rb:10:in 'OpenSSL::PKey.read': Could not parse PKey (OpenSSL::PKey::PKeyError)
    	from test.rb:10:in '<main>'
    
  7. junaruga commented on Aug 4, 2025

    @junaruga
    MemberAuthor

    In my case with the above test.rb, I confirmed the test.rb passed with the the commit openssl/openssl@dc5cd6f (dc5cd6f70a0eeb30e272fe885a64f3e3d76b5e42). CI log. So, I ran the git bisect for the b20da2328018107414fe896e59e7d4d6c8af8174 (failed) and dc5cd6f70a0eeb30e272fe885a64f3e3d76b5e42 (passed). The bisect.sh code is here.

    $ pwd
    /home/jaruga/git/openssl
    $ git bisect start b20da2328018107414fe896e59e7d4d6c8af8174 dc5cd6f70a0eeb30e272fe885a64f3e3d76b5e42
    $ git bisect run /path/to/bisect.sh
    
    $ git bisect log
    # bad: [b20da2328018107414fe896e59e7d4d6c8af8174] Revert "Pairwise check for DH keys import as part of FIPS"
    # good: [dc5cd6f70a0eeb30e272fe885a64f3e3d76b5e42] rsa: expose pairwise consistency test API
    git bisect start 'b20da2328018107414fe896e59e7d4d6c8af8174' 'dc5cd6f70a0eeb30e272fe885a64f3e3d76b5e42'
    # bad: [32ff539daf83cccc15a159fe214cac66acc80fec] changes: add note about PCT on key import to the FIPS provider
    git bisect bad 32ff539daf83cccc15a159fe214cac66acc80fec
    # bad: [a177798e0b8f3c7ff66b2a609924aafeb66b8b12] ec (fips): add PCT for key import
    git bisect bad a177798e0b8f3c7ff66b2a609924aafeb66b8b12
    # good: [58ab3b0ffeea2b4c30f8794d281650ce7bfec6d2] ecx (fips): add PCT for key import
    git bisect good 58ab3b0ffeea2b4c30f8794d281650ce7bfec6d2
    # first bad commit: [a177798e0b8f3c7ff66b2a609924aafeb66b8b12] ec (fips): add PCT for key import
    

    Then I found the openssl/openssl@a177798 was the first bad (failed) commit in my case with the test.rb reading EC PRIVATE KEY.

    Seeing the commits of the openssl/openssl, there are similar commits for the other algorithms below. I assume that these commits caused the test failures for the other algorithms

    57230da2bd0be6b219cec2995832034b1e09e0e9 rsa (fips): add PCT for key import
    a177798e0b8f3c7ff66b2a609924aafeb66b8b12 ec (fips): add PCT for key import
    58ab3b0ffeea2b4c30f8794d281650ce7bfec6d2 ecx (fips): add PCT for key import
    
  8. changed the title [-]openssl-master: rake test_fips failures[/-] [+]openssl-master: rake test_fips failures about keys[/+] on Aug 5, 2025
  9. rhenium commented on Aug 5, 2025

    @rhenium
    Member

    Thanks for looking into this.

    Then I found the openssl/openssl@a177798 was the first bad (failed) commit in my case with the test.rb reading EC PRIVATE KEY.

    This commit would explain the behavior. The PEM block from test_check_key is specially crafted so that it fails a pair-wise consistency test, and OpenSSL self-destructs itself when it encounters this which seems like a requirement by FIPS 140.

  10. junaruga commented on Aug 5, 2025

    @junaruga
    MemberAuthor

    This commit would explain the behavior. The PEM block from test_check_key is specially crafted so that it fails a pair-wise consistency test, and OpenSSL self-destructs itself when it encounters this which seems like a requirement by FIPS 140.

    I found the issue ticket openssl/openssl#26572 managing the changes. I still don't understand how to fix our tests by the OpenSSL's changes.

    For example, in the case of the test/openssl/test_pkey_ec.rb#test_check_key, how can we change the value of the ec_key_data?

  11. rhenium commented on Aug 12, 2025

    @rhenium
    Member

    I think that would mean tests using invalid keys have to be skipped. #930

  12. junaruga commented on Aug 13, 2025

    @junaruga
    MemberAuthor

    I think that would mean tests using invalid keys have to be skipped. #930

    Thank you for the fix!
    I was surprised that skipping the only 2 tests (test/openssl/test_pkey_dh.rb#test_params_ok?, and test/openssl/test_pkey_ec.rb#test_check_key) makes all the tests pass.

    Below is my test by reverting the fixed commit 25ad8f4 on the latest master branch commit 58cf287. You see many failures except for these 2 tests. Why did this happen? If unnecessary tests failed for some reasons, I think it's better to improve the tests for only essential tests to fail.

    https://github.com/junaruga/ruby-openssl/actions/runs/16933565029/job/47984681154#step:11:357

  13. rhenium commented on Aug 27, 2025

    @rhenium
    Member

    The OpenSSL commit caused OpenSSL to enter into an error state and stop working entirely. I think that behavior can change, looking at the openssl/openssl discussion: https://redirect.github.com/openssl/openssl/issues/28326

    Since it appears that it's a FIPS requirement to do a pair-wise check on loading keys, these test cases in ruby/openssl would not be meaningful in the FIPS mode anyway.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions