Annotations are used to configure tests and suites in a declarative way similar to modern OOP languages. This way, test configuration is stored along with the test logic inside the test package. No configuration files or tables are needed. The annotations names are based on popular testing frameworks such as jUnit. The framework runner searches for all the suitable annotated packages, automatically configures suites, forms suites hierarchy, executes it and reports results in specified formats.
Annotations are interpreted only in package specification and are case-insensitive. It is recommended however, to use the lower-case annotations as described in documentation.
There are two places where annotations may appear:
- at the beginning of the package specification (
- right before a procedure (
Package level annotations need to be separated by at least one empty line from the underlying procedure annotations.
Procedure annotations are defined right before the procedure they reference, no empty lines are allowed.
If a package specification contains
%suite annotation, it is treated as a test package and processed by the framework.
Some annotations accept parameters like
%displayname. The parameters for annotations need to be placed in brackets. Values for parameters should be provided without any quotation marks.
create or replace package test_pkg is -- %suite(Name of suite) -- %suitepath(all.globaltests) -- %beforeall procedure global_setup; -- %afterall procedure global_cleanup; /* Such comments are allowed */ -- %test -- %displayname(Name of a test) procedure some_test; -- %test(Name of another test) -- %beforetest(setup_another_test) -- %aftertest(cleanup_another_test) procedure another_test; -- %test -- %displayname(Name of test) -- %disabled procedure disabled_test; -- %test(Name of test) -- %rollback(manual) procedure no_transaction_control_test; procedure setup_another_test; procedure cleanup_another_test; -- %beforeeach procedure test_setup; -- %aftereach procedure test_cleanup; end test_pkg;
||Package||Mandatory. Marks package as a test suite. Optional suite description can be provided (see
||Package||Similar to java package. The annotation allows logical grouping of suites into hierarchies.|
||Package/procedure||Human-readable and meaningful description of a suite/test.
||Procedure||Denotes that the annotated procedure is a unit test procedure. Optional test description can by provided (see
||Procedure||Denotes that the annotated procedure should be executed once before all elements of the suite.|
||Procedure||Denotes that the annotated procedure should be executed once after all elements of the suite.|
||Procedure||Denotes that the annotated procedure should be executed before each
||Procedure||Denotes that the annotated procedure should be executed after each
||Procedure||Denotes that mentioned procedure should be executed before the annotated
||Procedure||Denotes that mentioned procedure should be executed after the annotated
||Package/procedure||Defines transaction control. Supported values:
||Package/procedure||Used to disable a suite or a test. Disabled suites/tests do not get executed, they are however marked and reported as disabled in a test run.|
It is very likely that the application for which you are going to introduce tests consists of many different packages or procedures/functions. Usually procedures can be logically grouped inside a package, there also might be several logical groups of procedure in a single package or even packages themselves might relate to a common module.
Let’s say you have a complex insurance application that deals with policies, claims and payments. The payment module contains several packages for payment recognition, charging, planning etc. The payment recognition module among others contains a complex
recognize_payment procedure that associates received money to the policies.
If you want to create tests for your application it is recommended to structure your tests similarly to the logical structure of your application. So you end up with something like:
- Integration tests
- Policy tests
- Claim tests
- Payment tests
- Payments recognition
- Payments set off
%suitepath annotation is used for such grouping. Even though test packages are defined in a flat structure the
%suitepath is used by the framework to form them into a hierarchical structure. Your payments recognition test package might look like:
create or replace package test_payment_recognition as -- %suite(Payment recognition tests) -- %suitepath(payments) -- %test(Recognize payment by policy number) procedure test_recognize_by_num; -- %test -- %displayname(Recognize payment by payment purpose) procedure test_recognize_by_purpose; -- %test(Recognize payment by customer) procedure test_recognize_by_customer; end test_payment_recognition;
And payments set off test package:
create or replace package test_payment_set_off as -- %suite(Payment set off tests) -- %suitepath(payments) -- %test(Set off creation test) procedure test_create_set_off; -- %test -- %displayname(Set off annulation test) procedure test_annulate_set_off; end test_payment_set_off;
When you execute tests for your application, the framework constructs a test suite for each test package. Then it combines suites into grouping suites by the
%suitepath annotation value so that the fully qualified path to the
recognize_by_num procedure is
USER:payments.test_payment_recognition.test_recognize_by_num. If any of its expectations fails then the test is marked as failed, also the
test_payment_recognition suite, the parent suite
payments and the whole run is marked as failed.
The test report indicates which expectation has failed on the payments module. The payments recognition submodule is causing the failure as
recognize_by_num has not met the expectations of the test. Grouping tests into modules and submodules using the
%suitepath annotation allows you to logically organize your project’s flat structure of packages into functional groups.
An additional advantage of such grouping is the fact that every element level of the grouping can be an actual unit test package containing a common module level setup for all of the submodules. So in addition to the packages mentioned above you could have the following package.
create or replace package payments as -- %suite(Payments) -- %beforeall procedure set_common_payments_data; -- %afterall procedure reset_common_paymnets_data; end payments;
%suitepath can be provided in three ways:
- schema - execute all tests in the schema
- [schema.]package[.procedure] - execute all tests in the specified test package. The whole hierarchy of suites in the schema is built before all before/after hooks or part suites for the provided suite package are executed as well. Example:
testsis the current schema.
Using automatic rollbacks in tests
By default, changes performed by every setup, cleanup and test procedure are isolated by savepoints. This solution is suitable for use-cases where the code that is getting tested as well as the unit tests themselves do not use transaction control (commit/rollback) or DDL commands.
In general, your unit tests should not use transaction control as long as the code you are testing is not using it too. Keeping the transactions uncommitted allows your changes to be isolated and the execution of tests does not impact others who might be using a shared development database.
If you are in a situation where the code you are testing uses transaction control (common case with ETL code), then your tests probably should not use the default automatic transaction control.
In that case use the annotation
-- %rollback(manual) on the suite level to disable automatic transaction control for the entire suite.
If you are using nested suites, you need to make sure that the entire suite all the way to the root is using manual transaction control.
It is possible with utPLSQL to change the transaction control on individual suites or tests that are part of complex suite. It is strongly recommended not to have mixed transaction control in a suite. Mixed transaction control settings will not work properly when your suites are using shared setup/cleanup with beforeall, afterall, beforeeach or aftereach annotations. Your suite will most likely fail with error or warning on execution. Some of the automatic rollbacks will probably fail to execute depending on the configuration you have.
In some cases it is necessary to perform DDL as part of setup or cleanup for the tests.
It is recommended to move such DDL statements to a procedure with
pragma autonomous_transaction to eliminate implicit commits in the main session that is executing all your tests.
Doing so allows your tests to use the framework’s automatic transaction control and releases you from the burden of manual cleanup of data that was created or modified by test execution.
When you are testing code that performs explicit or implicit commits, you may set the test procedure to run as an autonomous transaction with
Keep in mind that when your tests runs in autonomous transaction it will not see the data prepared in setup procedure unless the setup procedure committed the changes.
Order of execution
When processing the test suite
test_pkg defined in Example of annotated test package, the order of execution will be as follows.
create a savepoint 'beforeall' execute global_setup create savepoint 'beforeeach' execute test_setup execute some_test execute test_cleanup rollback to savepoint 'beforeeach' create savepoint 'beforeeach' execute test_setup execute setup_another_test execute another_test execute cleanup_another_test execute test_cleanup rollback to savepoint 'beforeeach' mark disabled_test as disabled execute test_setup execute no_transaction_control_test execute test_cleanup execute global_cleanup rollback to savepoint 'beforeall'