README
¶
Easy Input-Output Test Specifications in YAML
Often testing requires a large number of inputs with specific outputs --- particularly when testing parsers and grammar implementations. Such tests are also usually required across different language implementations with different testing suites and approaches. For testing such specifications it makes sense to maintain inputs and outputs in a language-agnostic way. This is one such approach.
YAML is used as the format for the test file because of its strong support for multi-line data. Specifications --- which are maintained by human hands --- are a particularly good use case for YAML structured data.
Any number or fields may be included, but all key names must begin with an initial capital letter. The following are common and expected:
Name- name or title of the specVersion- semantic version of the spec being testedSource- URL of YAML files that define the specIssues- URL to report issues with the specDiscuss- URL of place to discuss the specNotes- any notes about the specificationDate- date of the specificationLicense- license of the spec
One of the following is required although both are allowed:
Tests- array of testsSections- array of sections containing tests
If both are included the non-grouped tests will be checked first.
The content of Tests can be a simple array of tests, or an array of
test groups. All tests must have an I and O property and can
optionally add an N property as well:
Must:
I: Any string, number, or boolean containing the input.O: Any string, number, or boolean containing the required output.
Optional:
N: Short note about what is being tests, often from the specification.
Here's an example from the CommonMark specification. The first uses the array format:
Name: CommonMark
Version: v1.0.0
Source: https://gitlab.com/commonmark/commonmark-spec
Issues: https://gitlab.com/commonmark/commonmark-spec/issues
Discuss: https://talk.commonmark.org/
Notes: CommonMark is a clarification version of Markdown.
Tests:
- I: '\tfoo\tbaz\t\tbim\n'
O: '<pre><code>foo\tbaz\t\tbim\n</code></pre>\n'
N: 'tabs are equal to four spaces'
- I: ' \tfoo\tbaz\t\tbim\n'
O: '<pre><code>foo\tbaz\t\tbim\n</code></pre>\n'
And the same as a map with keys matching the sections:
Sections:
- Name: Tabs
Notes: Tests related to use of the tab character.
Tests:
- I: '\tfoo\tbaz\t\tbim\n'
O: '<pre><code>foo\tbaz\t\tbim\n</code></pre>\n'
N: 'tabs are equal to four spaces'
- I: ' \tfoo\tbaz\t\tbim\n'
O: '<pre><code>foo\tbaz\t\tbim\n</code></pre>\n'
See tests for usage examples.
Go (golang) Reference Implementation
A reference implementation in Go has been included for reference and import inclusion:
package mytest
import (
"testing"
"github.com/rwxrob/tinout"
)
// ...
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type CheckMethod ¶
CheckMethod is any function that takes a test and returns true if the test passed. The value it got when testing can be stored in t.Got and t.Passed should be set to true if passed.
type Section ¶
type Section struct {
Name string `yaml:"Name,omitempty"`
Notes string `yaml:"Notes,omitempty"`
Tests []Test `yaml:"Tests,omitempty"`
}
Section of tests.
type Spec ¶
type Spec struct {
Name string `yaml:"Name,omitempty"`
Version string `yaml:"Version,omitempty"`
Source string `yaml:"Source,omitempty"`
Issues string `yaml:"Issues,omitempty"`
Discuss string `yaml:"Discuss,omitempty"`
Notes string `yaml:"Notes,omitempty"`
Date string `yaml:"Date,omitempty"`
License string `yaml:"License,omitempty"`
Tests []Test `yaml:"Tests,omitempty"`
Sections []Section `yaml:"Sections,omitempty"`
}
Spec represents a test specification of inputs and outputs.
func (*Spec) Check ¶
func (s *Spec) Check(ok CheckMethod) *Test
Check takes a function as an argument that takes in CheckMethod function. It then calls the CheckMethod on all the s.Tests and s.Sections[].Tests until it finds the first one that does not pass and returns a pointer to it. Returns nil if all pass.
Source Files