Help:Pickle/Documentation

Pickle Portal
Home FAQ Documentation Glossary Quick tour with most important points Documentation Examples Hello World (GitHub) Technical Extension:Pickle Software documentation Misc Sources (GitHub) Subpages

This is the original documentation for the module before repackaging. The module is not ready for production.

The purpose of this module is to support behavior-driven development (BDD), a software development process that emerged from test-driven development (TDD), in on-going development of Lua-based modules. It makes an assumption of a test module on a separate page, possibly a subpage, and presentation on another page like the talk page or generated through an API call.

The module mimics some ideas from other BDD libs, like RSpec [1], Jasmine [2], and Busted [3]. There are no clear standard on how to do this, so there are some variation and adaption.

The module is not built for great speed, it uses closures to build a number of small objects, that is methods returns a value that holds a closure. This creates a fairly efficient implementation for simple access, but it is not very efficient for caching larger structures. Some larger structures are put in tables and built once. This is done for Expect where each new instance would otherwise create a lot of methods.

The module avoids as much caching as possible, as this can poison the tests.

Usage

If you have a module like «Module:HelloWorld», the ubiquitous and quite pesky example, coded as something like

local p = {}

function p.helloWorld()
	return "Hi there!"
end

return p

Then on a test page you would test this like like the following

require 'Module:BDD'()
local p = require 'Module:HelloWorld'()

describe('Hello world', function()
	context('On all pages', function()
		it('says hello', function()
			expect('p.helloWorld()', p.helloWorld()):toBe("Hi there!");
		end);
	end);
end);

return result('Module:HelloWorld')

In the first line the test framework is not only loaded, but it is also installed in the global name space. The trailing parenthesis is a call on the loaded table, and it will install additional global functions. It is not necessary to keep a pointer to the test framework, the functions keep their own pointers.

In the second line the module under test is loaded. Usually it is necessary to keep a pointer to the returned value.

The describe function adds a title and evaluates the function from a xpcall (not done yet). The same happens with context and it. These three are really the same, it is only our interpretation that changes.

The expect function adds a comment and the statement that creates the actual value. It then builds a closure and returns a structure in a flow style. This makes it possible to manipulate the instance, before we finally evaluates it and stores the result for later visualization. In this case we simply tests it to see if it is equal.

The final results are returned by a call to result. Like before this can add a title.

The tests can be prepared for for localization into several languages, for example English and Norwegian Bokmål

require 'Module:BDD'()
local p = require 'Module:HelloWorld'()

describe({en='Hello world', nb='Hallo verden'}, function()
	context({en='on all pages', nb='på alle sider'}, function()
		it({en='says hello', nb='sier hallo'}, function()
			expect('p.helloWorld()', p.helloWorld()):toBe("Hi there!");
		end);
	end);
end);

return result({ en='Module:HelloWorld', nb='Module:HelloWorld'})

In this case the languages will be chosen according to the site content language, and if that isn't defined it will use the languages from the fallback chain. If the first argument is a string, then that string will always be used. The language can be changed if the tests are run through the api, which makes it easy to check tests if they are prepared for the requested language.

Usually it is good practice to both use a common language to make the tests readable and reusable, and use a local language to make them readable for the local community at each project.

Further work

At present formatting does not work properly. It is expected to be fixed. ;)

Setup/teardown

There will be a solution whereby setup and teardown is chained. That means that the tests are somewhat isolated from each other at each test level, that is each time we define a describe, context, or it, the environment will be recreated. State can still leak between expectations though.

It is assumed that there will be no global setup and teardown as there is no resource allocation available from pages in Scribunto.

Xpcall

All calls to provided functions should be protected in xpcalls so a single exception does not make the whole test set to fail. A failed function should be marked as such in the report.

Actual function is xpcall, and it will always add a stack trace on our own stack. The entry might be without a message.

This is partially implemented, proper formatting remains unsolved.

Spy on public calls

Note that spying on public calls made before the module is available to the testing regime will not be possible.

Carp

Adds a message to stack without exiting the test, printing the callers name and its arguments.

Cluck

Like Carp, but also prints a stack trace starting one level up.

Croak/Confess

Like Carp, but also stops the running test (the user provided anonymous function). Because it throws an exception it will always trigger a stack trace.

Call would be like [carp|cluck|croak|confess](message, table, method).

Set up of spies are implemented, but they should be removed during teardown. Such cleanup must check if the struct is still available. This may happen if structs are created inside the test environment.

Coverage for public call

It should be possible to register spies for profiling coverage on members of a module. A function should loop over all members and register spies for all of them. Coverage can then be calculated for the public members. This will only give coverage of the entry points, but it seems sufficient to get a coarse number for coverage.

Stub public methods

It should be possible to stub public methods from the test page. This will not include private methods. (Is this something that should be part of a BDD module?)

Module return

At present an explicit return must be done. It seems to be possible to do an implicit return. This makes a slightly less error prone setup procedure.

Several options exist in literature, but it might be some limitations to what is actually allowed in Scribunto. It seems like all options are blocked.

Test Anything Protocol

At some point the Test Anything Protocol (TAP) should be supported. The simplest solution is to use yet another page for that response, but it is also possible to test on the page name. This can be different if we transclude the talk page into some other page, or even constructs the page on the fly through the api. If the transcluded page name is "API" then we switch to a TAP response. This makes it possible to support several formats from a single page.

Support libs

These will be repackaged

• w:no:User:Jeblad/Module:BDD/utils – Utility functions for testing tables and table contents. The functions are roughly equal to the same methods in Yonabas Moses library. [4]
• w:no:User:Jeblad/Module:BDD/view – View to show the result of the tests.
• w:no:User:Jeblad/Module:BDD/localize – Access methods to the localization libraries and the localized strings.
• w:no:Special:Prefixindex/User:Jeblad/Module:BDD/localize/ – The localized strings for each language.

See also

• Module:UnitTests
• Module:ScribuntoUnit