Open Document Format Interoperability and Conformance (OIC) TC WikiThis is the initial draft process, welcome TC members outstanding remarks, we hope it simple,operable and efficient. This process is serving for creating and maintaining ODF test suites, one of OIC TC critical deliverables. It is based on at least the following assumptions:
- Granularity of each test document is defined
see the proposal Test document types
- TC subversion structure is defined
- on going, may be structured according to 4 types of test documents: atomic test document, complex test document, derived test document, real life test document
- Test meta data for test suite is defined
see the proposal Test metadata
- Utility to submit test case is ready
- plain mail template is proposed by Bart, may be tuned further
- template2xml.groovy script is proposed by Bart, may be tuned further
- doctest2html.xslt is proposed by Bart, may be tuned further
The process is describing how TC create and maintain the ODF test suites, detailed steps are below(now only consider the scenario that contribute one test case per time, will consider the scenario that contribute a batch of test cases per time later):
- Step 1: Contributor prepares one test case, each test case includes one or more test documents(e.g. for different MIME types) as well as the test meta data
- Step 2: Contributor submits test case to TC test suite repository, the state of test case is marked as "draft", the detailed steps include:
- 2.1 Contributor sends a mail to TC mailing list with the plain mail template(will paste the link here after the template is tuned finally), as well as let TC members be aware his/her willing to contribute test case
- 2.2 Test case coordinator(any volunteer TC member, maybe the contributor himself if the contributor is TC member) runs the template2xml.groovy script to generate the XML file for meta data from the filled plain mail template
- 2.3 Test case coordinator does a XSLT transformation using doctest2html.xslt to generate the HTML file for the meta data based on the previous XML file
- 2.4 Test case coordinator submits the test document, the txt file from plain mail template, the generated XML file, and the generated HTML file to the corresponding folders at Subversion repository
- Step 3: Test case coordinator sends a mail to TC mailing list, provides the test case link in the Subversion test suite repository, and lets TC members be aware the test case is ready for review
- Step 4: TC members review the test case, send comments through mailing list,the review contents shall at least include:
- 4.1 The test document shall not contain the extra info, e.g. specific vendor info
- 4.2 The XML schema in the test document shall be valid. This can be done by some validation tool, e.g. Jing or ODF Validator
- 4.3 The test document shall pass the semantics verification. First, refer to ODF specification, verify whether the test case complies with the semantics interpretation. Second,compare whether the behavior is consistent when the test document is opened and rendered by different existed ODF applications. If both the first and second conditions are satisfied, ready to accept the test case, otherwise, through analysis, it may be caused by:
- 4.3.1 Test case is not clear or unreasonable, then request contributor to modify test case
- 4.3.2 Test case is correct, but one or more of ODF applications have defects, then ready to accept the test case, at the same time, record the ODF application defects through TC Jira, add the Jira link to the test case. This tracking data can be used for interoperability testing and analysis.
- 4.3.3 Semantics description in ODF specification is ambiguous, then submit proposal to ODF TC, the test case is pending until getting clear answer from ODF TC
- Step 5: Contributor requests to discuss/approve the contribution on the next TC meeting at least 2 days earlier
- Step 6: TC meeting discuss/approve the contribution, similar with step 4, may take one of the actions: (1)accept the test case, change the state of test case as "approved";(2)request contributor to modify test case;(3)submit proposal to ODF TC
- Step 7: TC chair or secretary moves the approved test case to the corresponding place at the Subversion test suite repository.
- Step 8: Repeat step 4-7 until the contribution is accepted.
-- See also http://www.oasis-open.org/committees/download.php/31636/TestDocumentTypes-wd-1.odt -- bart.hanssens 2009-03-25 13:15:38