Doctests guide

Doctests are generated by recursing the source files and looking for Example: Sphinx directives in the comments. The corresponding Sphinx block, as defined by its indentation, represents a single doctest. A doctest is composed of a series of statements.

Each statement is composed of a command and its expected output. The first line of the statement has the following structure:

>>> command-text [...] [% [comment-text] [doctest: flags]]

where:

• The first three characters, >>> define the start of a statement.

• The command text command-text is MATLAB/Octave code.

• The command text is optionally terminated by ..., in which case the command text of the next line will be concatenated.

• An optional comment, which will be stripped from the constructed doctest. However, only on the first line of a statement, the comment may contain a doctest: substring, and the part of the comment following that substring will be interpreted as statement flags.

If a statement line representing a comment has its command part ending with ..., the next line is of the form:

`` command-text […] [% [comment-text]]``

and such lines are concatenated to the first as long as they end with .... Flags are only parsed on the first line of a statement.

A statement is followed by the expected output, which can be empty. The expected test output ends

1. when the next line is no longer a comment line,

2. when another comment line starts with >>>,

3. when two blank comment lines are provided.

The command-text can have different forms, and the expected output is interpreted accordingly.

Expected output

In expected test output, blank lines are ignored. Errors are not captured, and will crash the test handler.

Test files are written in the MoXUnit format, using functions and subfunctions as the rest of the RepLAB test suite; helpers functions are used to verify the test output.

Our inspiration for the framework comes from http://doc.sagemath.org/html/en/developer/coding_basics.html#writing-testable-examples.

We distinguish the following three kinds of statements.

Silent statements

A silent statement finishes with a semicolumn. It can be of the form statement; or a = statement; or [a, b] = statement;.

In that case, the text written to the standard output/error streams is captured and compared to the data provided by the doctest.

REPL statements

A REPL statement is a non-assigning statement which does not finish with a semicolumn: it is of the form statement, and is expected to returns a single output. If the command does not return any output (as would an assert do, for example), it should be followed by a semicolon in the form of a silent statement instead.

There are two forms of expected output for REPL statements. The long form is used when the command displays text on the standard output:

>>> verbose_f()
    Log: computing the f value
    ans =
      1

where the ans = line separates what the code displays on the standard output (before the ans =) line, and what the returned value is (here, 1).

The short form is used when the code does not display anything on the standard output. In that case, the expected output is just the returned value::

>>> silent_f()
    1

We diverge from the Matlab output conventions in one case. When we display the value of a variable, without assignment, we do not repeat the variable name. The following would be a valid Matlab session:

>> x = 2;
>> x
x =
     2

whereas the corresponding doctest would be:

>>> x = 2;
>>> x
    2

Assignment statements

A command of an assignment statement is of one of the forms:

• a = statement, in which case the value of the variable a is captured and compared to the doctest data;

• [a, b] = statement, in which case several variables are captured and compared to the doctest data.

These statements do not finish with a semicolumn. The expected output contains first the text written by the command to the standard output, if any, before the value of each of the variables, where the value of each variable is preceded by a varname = line.

For example::

>>> a = verbose_f()
    Log: computing the f value
    a =
      1
>>> a = 2
    a =
      2
>>> [a, b] = deal(1, 2)
    a =
      1
    b =
      2

Note that, in the case of a single output and no text output on the standard output, the short form is also accepted:

>>> a = silent_f()
    1

Values in expected output

For now, we support the following types in the expected output.

Strings

Strings are char row vectors. In the expected output, they can be described in two ways:

• They are written between single quotes as in 'between quotes', and quotes in the string are doubled (as in the standard Matlab string syntax: 'String with a ''double'' quote'). In that case, the string must not contain newline characters.

• They are written without a beginning single quote, and they can be multiline.

Scalars

• double expected values are parsed using eval and compared for equality (the NaN case is not supported yet)

• replab.cyclotomic expected scalars are written as expressed by the replab.cyclotomic.num2str method.

• vpi expected values are written on one line as a base 10 string

• logical expected values are evaluated by eval and converted to the logical type