Skip to content

Feature/tags #906

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 33 commits into from
Jun 6, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
3058610
Added tags for tests and tags for suite.
lwasylow Apr 14, 2019
3495490
Switch to use VARCHAR2 instead of UT_VARCHAR2_ROWS()
lwasylow Apr 19, 2019
d2f9796
Switch to use VARCHAR2 instead of UT_VARCHAR2_ROWS()
lwasylow Apr 19, 2019
777a2af
Merge branch 'feature/tags' of github.com:utPLSQL/utPLSQL into featur…
lwasylow Apr 19, 2019
1a941a8
Adding Tests to tags #suitebuilder
lwasylow Apr 20, 2019
3a8ba8f
Change annotation to --%tags
lwasylow Apr 21, 2019
8ebb8b8
Fixing empty tags to use xmlgen
lwasylow Apr 21, 2019
a42e244
Adding tests for ut.run
lwasylow Apr 23, 2019
5ee8641
Added tests for ut.run procedure
lwasylow Apr 24, 2019
176519c
Added extra tests for ut run function
lwasylow Apr 24, 2019
b1f4a62
Fixing error message function
lwasylow Apr 24, 2019
dbbf197
Update docs
lwasylow Apr 24, 2019
f714499
Update docs and tests
lwasylow Apr 24, 2019
1774db5
Merge branch 'develop' of github.com:utPLSQL/utPLSQL into feature/tags
lwasylow Apr 24, 2019
a433753
Merge branch 'develop' into feature/tags
lwasylow Apr 24, 2019
4544240
Merge branch 'feature/tags' of github.com:utPLSQL/utPLSQL into featur…
lwasylow Apr 24, 2019
e575404
Fixes to PR comments
lwasylow Apr 25, 2019
632b64d
Fixes to PR comments
lwasylow Apr 25, 2019
66257d1
Updates to PR
lwasylow Apr 25, 2019
ddf3a7f
Phase 2 : switch to varchar_rows()
lwasylow Apr 26, 2019
6d8b099
Phase 2 : switch to varchar_rows()
lwasylow Apr 26, 2019
957ea8b
Remove a 4k tests
lwasylow Apr 28, 2019
2a91551
Added constant
lwasylow Apr 28, 2019
612a30c
Merge branch 'develop' into feature/tags
lwasylow Apr 28, 2019
b508e18
Merge branch 'develop' of github.com:utPLSQL/utPLSQL into feature/tags
lwasylow Apr 28, 2019
43b27a6
Merge branch 'feature/tags' of github.com:utPLSQL/utPLSQL into featur…
lwasylow Apr 28, 2019
b636830
Merge branch 'develop' of github.com:utPLSQL/utPLSQL into feature/tags
lwasylow Apr 28, 2019
d00a0f7
Updates to PR.
lwasylow Apr 29, 2019
3164310
Move query to template to simplify view
lwasylow Apr 29, 2019
58f788b
Merge branch 'develop' of https://github.com/utPLSQL/utPLSQL into fea…
Apr 30, 2019
e5d8011
Merge branch 'develop' of github.com:utPLSQL/utPLSQL into feature/tags
lwasylow May 12, 2019
d61a8c8
Merge remote-tracking branch 'origin/develop' into feature/tags
jgebal Jun 5, 2019
effb420
Renamed `gc_tag` to `gc_tags`
jgebal Jun 5, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
101 changes: 97 additions & 4 deletions docs/userguide/annotations.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ We strongly recommend putting package level annotations at the very top of packa
| `--%disabled` | Package/procedure | Used to disable a suite or a test. Disabled suites/tests do not get executed, they are however marked and reported as disabled in a test run. |
| `--%context(<name>)` | Package | Denotes start of a named context (sub-suite) in a suite package |
| `--%endcontext` | Package | Denotes end of a nested context (sub-suite) in a suite package |
| `--%tags` | Package/procedure | Used to label a test or a suite for purpose of identification |

### Suite

Expand Down Expand Up @@ -823,7 +824,7 @@ See [beforeall](#Beforeall) for more examples.
Indicates specific setup procedure(s) to be executed for a test. The procedure(s) can be located either:
- within current package (package name is optional)
- within another package

The annotation need to be placed alongside `--%test` annotation.

The `--%beforetest` procedures are executed after invoking all `--%beforeeach` for a test.
Expand Down Expand Up @@ -911,7 +912,7 @@ Finished in .015185 seconds
Indicates specific cleanup procedure(s) to be executed for a test. The procedure(s) can be located either:
- within current package (package name is optional)
- within another package

The annotation need to be placed alongside `--%test` annotation.

If a test is marked as disabled the `--%aftertest` procedures are not invoked for that test.
Expand Down Expand Up @@ -1221,6 +1222,98 @@ Finished in .035261 seconds
```



### Tags

Tag is a label attached to the test or a suite path. It is used for identification and execution a group of tests / suites that share same tag.

It allows us to group a tests / suites using a various categorization and place a test / suite in multiple buckets. Same tests can be group with other tests based on the functionality , frequency, type of output etc.

e.q.

```sql
--%tags(batch,daily,csv)
```

or

```sql
--%tags(api,online,json)
```



Tags are defined as a coma separated list. When executing a test run with tag filter applied, framework will find all tests associated with given tags and execute them. Framework applies `OR` logic when resolving a tags so any tests / suites that match at least one tag will be included in the test run.

When a suite gets tagged all of its children will automatically inherit a tag and get executed along the parent. Parent suit tests are not executed. but a suitepath hierarchy is kept.

Sample tag package.

```sql
create or replace package ut_sample_test IS

--%suite(Sample Test Suite)
--%tag(suite1)

--%test(Compare Ref Cursors)
--%tag(test1,sample)
procedure ut_refcursors1;

--%test(Run equality test)
--%tag(test2,sample)
procedure ut_test;

end ut_sample_test;
/

create or replace package body ut_sample_test is

procedure ut_refcursors1 is
v_actual sys_refcursor;
v_expected sys_refcursor;
begin
open v_expected for select 1 as test from dual;
open v_actual for select 2 as test from dual;

ut.expect(v_actual).to_equal(v_expected);
end;

procedure ut_test is
begin
ut.expect(1).to_equal(0);
end;

end ut_sample_test;
/
```

Execution of the test is done by using a parameter `a_tags`

```sql
select * from table(ut.run(a_path => 'ut_sample_test',a_tags => 'suite1'));
select * from table(ut.run(a_tags => 'test1,test2'));
select * from table(ut.run(a_tags => 'sample'));

begin
ut.run(a_path => 'ut_sample_test',a_tags => 'suite1');
end;
/

exec ut.run('ut_sample_test', a_tags => 'sample');
```



Tags should adhere to following rules:

- tags are case sensitive
- tags cannot be an empty string
- tags cannot contain spaces e.g. to create a multi-word `tag` please use underscores,dashes, dots etc. e.g. `test_of_batch`
- tags with empty spaces will be ignored during execution
- tags can contain special characters



### Suitepath

It is very likely that the application for which you are going to introduce tests consists of many different packages, procedures and functions.
Expand Down Expand Up @@ -1346,9 +1439,9 @@ If `--%throws` annotation is specified with arguments and exception raised is no
The framework will raise a warning, when `--%throws` annotation has invalid arguments or when no arguments were provided.

Annotation `--%throws(7894562, operaqk, -=1, -20496, pow74d, posdfk3)` will be interpreted as `--%throws(-20496)`.

Please note that `NO_DATA_FOUND` exception is a special case in Oracle. To capture it use `NO_DATA_FOUND` named exception or `-1403` exception No.

Example:
```sql
create or replace package exc_pkg is
Expand Down
9 changes: 5 additions & 4 deletions docs/userguide/querying_suites.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,21 +22,22 @@ Querying the data from function provides the follwing details:
- `item_line_no` - line_number where annotation identifying the item exists
- `path` - suitepath of the item
- `disabled_flag` - (0/1) indicator if item is disabled by --%disabled annotation

- `tags` - tags associated with suites

To get list of all test suites in current schema
```sql
select * from table(ut_runner.get_suites_info()) where item_type = 'UT_SUITE';
```
```

To get list of all tests for test suite `TEST_STUFF` in current user schema
```sql
select * from table(ut_runner.get_suites_info(USER, 'TEST_STUFF')) where item_type = 'UT_TEST';
```
```

To get a full information about suite `TEST_STUFF` including suite description, all contexts and tests in a suite
```sql
select * from table(ut_runner.get_suites_info(USER, 'TEST_STUFF')) where item_type = 'UT_TEST';
```
```

## Checking if schema contains tests

Expand Down
Loading