Home » Articles » Announcements »Unique Product Features » KVS Automated Testing for KMIP Servers

KVS Automated Testing for KMIP Servers

By jsusoy - February 15, 2019 @ 11:33 am

P6R’s KMIP Verification Suite (KVS) automates testing of a KMIP server for compliance with any of the defined OASIS KMIP protocol versions, including those defined in OASIS KMIP profiles. P6R has been using this tool for the past several years in at least 4 separate formal OASIS interops. The result is a very mature and easy to use tool. We’ve used it to perform interop testing, with one person running the tests, on as many as 10 servers in only days. It can be used for running single or multiple test-cases from the command line, integrating with a CI system to automate testing for every build.

Highlights

  • Comprehensive coverage. KVS implements all of the defined OASIS KMIP test cases. This is over 2600 test cases, all message formats (TTLV/XML/JSON), and all KMIP versions. These tests are actually used in formal interop testing.
  • Very detailed logging. KVS provides logging of tests and KMIP protocol interactions in a variety of formats (eg. Test TTLV, but log in XML with TTLV hex dump). Detail is important to help find out what going on when a test fails. Supported log formats are TTLV, XML and JSON. Logs provide a source of proof that server is working or not. Provides separate logs for protocol logging and test output to make finding problems easier.
  • Flexible. Can easily run just one, a subset, all tests in a category, or all tests. Tests can be excluded if that is more convenient. Multiple different output formats making integration much easier.
  • Interoperable. KVS can provide jUnit style output allowing integration with a variety of tools. Runs from the command line allowing it to be scripted, or run from a variety of tools including CI systems.

Reduces QA and Development Costs

Customers that are using it are realizing huge QA cost reductions, enabling them to concentrate their engineering efforts on their server instead of writing, verifying and maintaining test cases.

“The KVS test suite has been a great asset for interoperability testing” stated John Leiseboer, CTO of QuintessenceLabs. “It works seamlessly, delivering both a significant time and resource saving, as well as a more robust process.”

P6R maintains and enhances KVS, adding test cases as they evolve in the standard. This is a significant effort. In the latest interop, around 180 new test cases were added to test the KMIP 2.0 protocol version and this does not yet cover all the new KMIP 2.0 features.

Enables Test-Driven/Test-First Development (TDD)

KVS is not just for QA. KVS enables Test-Driven Development (TDD) by providing the test cases for KMIP features that have yet to be implemented. Developers then write the code to fulfill those test cases. A huge benefit of this practice is the ability to refactor code with confidence. Developers can easily verify refactored code by running the tests periodically as they change code.

SKC Compatibility

P6R’s Secure KMIP Client (SKC) is the foundation of KVS and so another benefit of using KVS is proving compatibility with P6R’s KMIP Client.

KVS is currently shipping and available for purchase. P6R is an active  member of the OASIS KMIP Technical Committee.

"KVS Automated Testing for KMIP Servers" was published on February 15th, 2019 and is listed in Announcements, Unique Product Features.

Follow comments via the RSS Feed | Leave a comment | Trackback URL


Leave Your Comment