Jump to content

Help:Pickle/Glossary

From mediawiki.org
Translate this page
Languages:
Pickle Portal

Documentation

Examples

Technical

Misc

Glossary for pickle tests is a list of core terms and concepts, both for #spec and #step style tests. Terms are borrowed from the ITIL, ISTQB, RSpec framework, and predicate logic. The different frameworks are linked as necessary. Try to keep close to the definitions, but adapt to this testing environment.

Terms and concepts

After
Marks functions to be run after #describe, #context, and #it. It uses a single queue so all functions will be run in the correct sequence.
Part of the teardown of #test fixture.
Also known as teardown.
AfterAll
Marks functions similar to #after, except it is run only once. Strictly speaking this function is not necessary as code can be added inline. It uses a single queue so all functions will be run in the correct sequence. It uses the same queue as after so it will be run at the correct time in the first test. At later invocations it will not be run.
Part of the teardown of #test fixture.
Around
An alternate to #after that is only run if the test #throws an #exception. If an exception is thrown, and there is an around function registered, then no after function will be run.
Part of the teardown of #test fixture.
Assertion
Test that focuses on the actual code, and assertions about that code, and their falsy #conditions.
Part of the verification of #test fixture.
Before
Marks functions to be run before #describe, #context, and #it. It use a single queue so all functions will be run in the correct sequence.
Part of the setup of #test fixture.
Also known as setup.
BeforeAll
Marks functions similar to #before, except it is run only once. Strictly speaking this function is not necessary as code can be added inline. It uses a single queue so all functions will be run in the correct sequence. It uses the same queue as before so it will be run at the correct time in the first test. At later invocations it will not be run.
Part of the setup of #test fixture.
Carp
A #spy that adds a message to stack without exiting the test, printing the caller's name and its arguments. Partly emulating the carp() function from Perl.
Cluck
A #spy like #carp, but also prints a stack trace starting one level up. Partly emulating the cluck() function from Perl.
Condition
A logical expression that can be evaluated as True or False, e.g., A>B.
Part of the #assertion.
Confess
A #spy like #croak, but also prints a stack trace starting one level up. Partly emulating the confess() function from Perl.
Context
Marks functions used as examples, but also a function context() itself.
See #example for details.
Continuous integration
Agile approach to software development to minimize the duration and effort during each iteration, and at the same time deliver software suitable for release.
This dual objective requires an integration procedure that is reproducible, usually by automated integration and builds – often multiple times each day, and achieved through extensive testing, version control, team policies, and conventions.
Coverage
The degree, expressed as a percentage, to which a specified coverage item has been exercised by a test suite.
Criteria
Decision rules are used to determine whether a test item (function) or feature has passed or failed a given test. In specs it is only used simple pass/fail.
Croak
A #spy like #carp, but also stops the running test (the user provided anonymous function). Partly emulating the croak() function from Perl. Because it throws an exception it will always trigger a stack trace.
Describe
Marks functions used as examples, but also a function describe() itself. See #example for details.
Design
Also known as test design
The process of transforming general test objectives into tangible test conditions and test cases.
The document that describes the implementation details of the test, or results of, the system or in our case the module. The document is a subsection on the /doc subpage for the spec page.
Developer
A person that write code. See #tester
Documentation
Also known as test documentation
The document that describes plans of, or results of, the system or in our case the module. The document is the /doc subpage for the spec page.
Dummy
A necessary minimum implementation of functionality that is only necessary to make some other implementation available for use. Such objects are usually created inside the tests, and lifespan is limited to the tests. A dummy is a type of #test double.
Duration
Time elapsed while executing a test, test run, or the entire test execution of a build or release pipeline. Scribunto uses cpuLimit to limit total duration of a test run.
Examples
The levels in the #describe, #context, and #it ladder. The name and level are a bit arbitrary as they all have nearly the same function. The context is often left out, and only describe and it are used.
Exceptions
Error states reported from within the code. They will be caught by the test framework.
Expectations
Tests that focus on the provided objects and examples, and their truthy states.
Fail
A test is deemed to fail if its actual result does not match its expected result.
Fake
A simpler alternate implementation of functionality that otherwise would be too heavy or difficult to implement or not feasible to use. Such objects are usually created inside the tests, lifespan is limited to the tests and enhances the performance of tests. A fake is a type of #test double.
Feature
An attribute of a component or system specified or implied by requirements documentation (for example reliability, usability, or design constraints).
Fixture
The function defining the test is the test fixture and a frame without a test fixture will be skipped.
Good
See #pass for details.
Harness
A test environment comprised of stubs and drivers needed to execute a test.
It
Marks functions used as examples, but also a function it() itself. See #example for details.
Mock
A simulated object that acts as a strictly controlled replacement for a real object and is used to verify the interaction between the modules and services that is compared with the expectation value which corresponds to the number of times the modules and services must interact with each other while a test is running. A mock is a type of #test double.
Overfitting
Happens when too tight couplings between the test and source code exist. Usually this happens when tests use spies to verify internal data structures in source code.
Pass
A test is deemed to pass if its actual result matches its expected result.
Pending
A frame marked with #skip or #todo will be in a pending final state. This state is an override of #pass ("ok") and #fail ("not ok") and makes it possible to code and test with less noise.
Return
The outermost zero-level that encapsulates the test results and returns them in a formatted fashion.
Setup
See #before for details.
Skip
The system (or optionally the author) can mark a test as skipped, either in the description or in the code. Further processing within the frame will then be terminated and the current state used.
Spec
A type of testing to ensure we build the thing right, usually by writing some kind of unit tests, but can also be more high level.
Spies
Functions that can be registered on other functions, or injected into code to report or alter internal states. Spies are one of #carp, #cluck, #croak, and #confess. Spying on public calls made before the module is available to the testing regime will not be possible. A spy is a type of #test double.
Step
A type of testing to make sure we build the right thing, usually by writing some kind of acceptance tests, which is at the integration level.
Stub
Simulated placeholders for real code, used to isolate and test specific system parts by excluding certain dependencies. A stub is a type of #test double.
Subject
The object under test. It can be explicitly set or set as part of the #examples. It is passed on to #expect.
Teardown
See #after for details.
Test
An activity to verify the proper operation of a system, given that it is executed under specific conditions, and with observed and recorded results. Consists of one or more #test cases.
Test case
A set of input values, execution preconditions, expected results, and execution postconditions, developed for a particular objective or test condition, such as to exercise a particular program path or to verify compliance with a specific requirement. Can also include documentation for how to run the test case.
Test doubles
Code replacing pieces of code that are not tested, and should return only known values.
Test item
The individual element to be tested. Usually, it is one test object and many test items.
Tester
A skilled user who is involved in the testing of a component or system. See #developer
Throw
The test code can run into a statement that either can't be executed and thus "throws" an exception or explicitly "throw" an exception at that point. When the code throws an exception further execution is stopped and a report is initiated.
Todo
The user (or optionally the system) can mark a test as todo, either in the description or in the code. Further processing within the frame will continue, but the final state is set to pending anyhow.

Please try to adhere to common terms from w:International Software Testing Qualifications Board (ISTQB), w:American Software Testing Qualifications Board (ASTQB) and w:Information Technology Infrastructure Library (ITIL).

Retrieved from "https://www.mediawiki.org/w/index.php?title=Help:Pickle/Glossary/en&oldid=6073212"