Meetings/Test framework/2010-07-11
(Redirected from Test framework 07 11 2010)
 | This page is obsolete. It is being retained for archival purposes. It may document extensions or features that are obsolete and/or no longer supported. Do not rely on the information here being up-to-date. See Selenium instead. |
|
Test framework deployment
Develop a Selenium test framework with samples and demos that can be run on the grid.
|
Test Framework Discussion Notes
[edit]Location: Wikimania 2010
Attendees: Ryan Lane, Markus Glaser, Benedikt Kämpgen, Siebrand Mazeland, Priyanka Dhanda
Short Term Goals
[edit]- Formalize the test framework roadmap. Make this visible to the community. (Robla and Priyanka)
- Documentation (Ryan Lane)
- Document the Selenium Grid setup on a public wiki (both hardware and software).
- Make all documentation visible to the openqa community for both feedback and as guidelines to others trying to do the same thing.
- Make a visual representation of all test system interactions - Continuum, PHPUnit, Selenium Grid (Ryan Lane)
- Work in small iterative steps and make these steps visible.
- For example: The first step should be clear documentation and a working example of how to write a test, run it against the grid and see the results.
- Define testing best practices.
- Encourage following code conventions and exporting tests to PHP.
- We should encourage these best practices, but not frown upon tests that don't comply. There should be a process of sanitizing then and pulling them into the codebase.
- Investigate and integrate the Selenium Framework with Apache Continuum. Look into a code coverage tool.
Long Term Goals
[edit]- Find a good reporting strategy for test failures, code convention violations, etc.
- Figure out a process to let community members write IDE tests and then incorporate them into the code base.
- Who should be is responsible for maintaining extension and core tests? Should it be the developer?
- Trigger a subset of tests on each commit and run all tests at some longer interval.
- Have requirements and tests closely tied together through the feature specification page.
- Make it easy for a bug submitter to submit test cases. Same with feature requests.
- This way bug fixers and feature developers can use the test cases as a guideline and improve it as they go.
Outstanding issues with the current Selenium Framework
[edit]- Initial content on the wiki we're testing.
- For now tests should be responsible for content generation and cleanup.
- When the number of tests increase we can figure out a better way to bootstrap data.
- Add a configuration flag to allow tests to be run anonymously (Currently it requires login).
- Dynamically reconfigure the wiki being tested.
- Finalize the structure of the tests in the code base.
- Now the test classes live in <ExtensionName>/tests/selenium. Each class is a suite with multiple test cases.
- We should be able to run tests by a tag name and specify dependencies for each test suite.
- Test job queues.
- This may be easier if we have a configuration to put a wiki in test mode. Ideally it should enable flushing queues through the framework.