1Testing 2======= 3 4Examples below assume cwd at the gradlew script in the top directory of 5the project's checkout. 6 7 8Generic test/ checkup commands 9------------------------------ 10 11Run all unit tests: 12 13gradlew test 14 15Run all verification tasks, including tests: 16 17gradlew check 18 19Run all verification tasks, excluding tests (-x is gradle's generic task 20exclusion mechanism): 21 22gradlew check -x test 23 24Run verification for a selected module only: 25 26gradlew :lucene:core:check # By full gradle project path 27gradlew -p lucene/core check # By folder designation + task 28 29 30Randomization 31------------- 32 33To run tests with the given starting seed pass 'tests.seed' 34property: 35 36gradlew :lucene:misc:test -Ptests.seed=DEADBEEF 37 38There are a lot of other test randomization properties 39available. To list them, their defaults and current values 40run the testOpts task against a project that has tests. 41For example: 42 43gradlew -p lucene/core testOpts 44 45 46Filtering 47--------- 48 49Run tests of lucene-core module: 50 51gradlew -p lucene/core test 52 53Run a single test case (from a single module). Uses gradle's built-in filtering 54(https://docs.gradle.org/current/userguide/java_testing.html#test_filtering): 55 56gradlew -p lucene/core test --tests TestDemo 57 58Run all tests in a package: 59 60gradlew -p lucene/core test --tests "org.apache.lucene.document.*" 61 62Run all test classes/ methods that match this pattern: 63 64gradlew -p lucene/core test --tests "*testFeatureMissing*" 65 66 67Test groups 68----------- 69 70Tests can be filtered by an annotation they're marked with. 71Some test group annotations include: @AwaitsFix, @Nightly 72 73This uses filtering infrastructure on the *runner* (randomizedtesting), 74not gradle's built-in mechanisms (but it can be combined with "--tests"). 75For example, run all lucene-core tests annotated as @Nightly: 76 77gradlew -p lucene/core test -Ptests.filter=@Nightly 78 79Test group filters can be combined into Boolean expressions: 80 81gradlew -p lucene/core test "default and not(@awaitsfix or @nightly)" 82 83 84Reiteration ("beasting") 85------------------------ 86 87Multiply each test case N times (this works by repeating the same test 88within the same JVM; it also works in IDEs): 89 90gradlew -p lucene/core test --tests TestDemo -Ptests.iters=5 91 92Tests tasks will be (by default) re-executed on each invocation because 93we pick a random global tests.seed and because we want them to (force 94the test task to run). If you want to make the 'tests' task obey up-to-date 95gradle rules, fix the seed and turn off up to date trigger: 96 97gradlew -p lucene/core test -Ptests.seed=deadbeef -Ptests.neverUpToDate=false 98 99These properties can be set in your local gradle.properties. To force re-execution 100of tests, even for the same master seed, apply cleanTest task: 101 102gradlew -p lucene/core cleanTest test -Ptests.seed=deadbeef 103 104The 'tests.iters' option should be sufficient for individual test cases 105and is *much* faster than trying to duplicate re-runs of the entire 106test suites. When it is absolutely needed to re-run an entire suite (because 107of randomization in the static initialization, for example), you can do it 108by running the 'beast' task with 'tests.dups' option: 109 110gradlew -p lucene/core beast -Ptests.dups=10 --tests TestPerFieldDocValuesFormat 111 112Note the filter (--tests) used to narrow down test reiterations to a particular 113class. You can use any filter, including no filter at all, but it rarely makes 114sense (will take ages). By default the test tasks generated by the 'beast' mode 115use a random starting seed for randomization. If you pass an explicit seed, this 116won't be the case (all tasks will use exactly the same starting seed): 117 118gradlew -p lucene/core beast -Ptests.dups=10 --tests TestPerFieldDocValuesFormat -Dtests.seed=deadbeef 119 120 121Verbose mode and debugging 122-------------------------- 123 124The "tests.verbose" mode switch enables standard streams from tests 125to be dumped directly to the console. Run your verbose tests explicitly 126specifying the project and test task or a fully qualified task path. Example: 127 128gradlew -p lucene/core test -Ptests.verbose=true --tests "TestDemo" 129 130 131Run GUI tests headlessly with Xvfb (Linux only) 132----------------------------------------------- 133 134GUI test for Luke application launches a window, this might mess up your 135monitor depending on the display manager you are using. In that case, 136you can install Xvfb (X Virtual Frame Buffer) so that the test runs on the 137virtual display and does not open a real window. 138 139# redhat-type OS 140$ sudo yum install Xvfb 141 142# ubuntu/debian-type OS 143$ sudo apt install xvfb 144 145# arch 146$ sudo pacman -S xorg-server-xvfb 147 148 149Profiling slow tests 150-------------------- 151 152The "tests.profile" mode switch turns on a sampling profiler during test execution, 153and prints a simple summary at the end. 154 155For example, top-10 histogram of methods (cpu samples): 156 157gradlew -p lucene/core test -Ptests.profile=true 158 159Alternatively, you can profile heap allocations instead: 160 161gradlew -p lucene/core test -Ptests.profile=true -Ptests.profile.mode=heap 162 163By default, results are computed (deduplicated) on just the method name, folding 164together all events from the same method. To drill down further, you can increase the 165stack size from the default of 1, to get a histogram of stacktraces instead: 166 167gradlew -p lucene/core test -Ptests.profile=true -Ptests.profile.stacksize=8 168 169For big methods, it can also be helpful to include line numbers for more granularity: 170 171gradlew -p lucene/core test -Ptests.profile=true -Ptests.profile.linenumbers=true 172 173Using these additional options will make the results more sparse, so it may be useful 174to increase the top-N count: 175 176gradlew -p lucene/core test -Ptests.profile=true -Ptests.profile.count=100 177 178 179Generating Coverage Reports 180------------------------------ 181 182Running the "coverage" task (or setting the property "tests.coverage" to true) 183will run the tests with instrumentation to record code coverage. 184 185Example: 186 187gradlew -p lucene/core coverage 188open lucene/core/build/reports/jacoco/test/html/index.html 189 190If you want to use test filtering to just check a particular test, specify 191the "test" task explicitly before "coverage": 192 193gradlew -p lucene/core test --tests TestDemo coverage 194open lucene/core/build/reports/jacoco/test/html/index.html 195 196 197External data sets 198------------------ 199 200Some tests may require external (and large) data sets. To see relevant tasks 201that download and extract these data files automatically, run the following: 202 203gradlew tasks --group "Data set download" 204