-
Notifications
You must be signed in to change notification settings - Fork 1.1k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
0ff31a1
commit 31335b4
Showing
6 changed files
with
335 additions
and
48 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
command script import ./tools/lldbhalide.py |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,125 @@ | ||
# Testing | ||
|
||
Halide uses CTest as its primary test platform and runner. | ||
|
||
## Organization | ||
|
||
Halide's tests are organized beneath the top-level `test/` directory. These | ||
folders are described below: | ||
|
||
| Folder | Description | | ||
|----------------------|----------------------------------------------------------------------------------| | ||
| `autoschedulers/$AS` | Test for the `$AS` (e.g. `adams2019`) autoscheduler | | ||
| `common` | Code that may be shared across multiple tests | | ||
| `correctness` | Tests that check correctness of various compiler properties | | ||
| `error` | Tests that expect an exception to be thrown (or `abort()` to be called) | | ||
| `failing_with_issue` | Correctness tests that are associated with a particular issue on GitHub | | ||
| `fuzz` | Fuzz tests. Read more at [FuzzTesting.md](FuzzTesting.md) | | ||
| `generator` | Tests of Halide's AOT compilation infrastructure. | | ||
| `integration` | Tests of Halide's CMake package for downstream use, including cross compilation. | | ||
| `performance` | Tests that check that certain schedules indeed improve performance. | | ||
| `runtime` | Unit tests for the Halide runtime library | | ||
| `warning` | Tests that expected warnings are indeed issued. | | ||
|
||
The tests in each of these directories are given CTest labels corresponding to | ||
the directory name. Thus, one can use `ctest -L generator` to run only the | ||
`generator` tests. The `performance` tests configure CTest to not run them | ||
concurrently with other tests (including each other). | ||
|
||
The vast majority of our tests are simple C++ executables that link to Halide, | ||
perform some checks, and print the special line `Success!` upon successful | ||
completion. There are three main exceptions to this: | ||
|
||
First, the `warning` tests are expected to print a line that reads | ||
`Warning:` and do not look for `Success!`. | ||
|
||
Second, some tests cannot run in all scenarios; for example, a test that | ||
measures CUDA performance requires a CUDA-capable GPU. In these cases, tests are | ||
expected to print `[SKIP]` and exit and not print `Success!` or `Warning:`. | ||
|
||
Finally, the `error` tests are expected to throw an (uncaught) exception that is | ||
not a `Halide::InternalError` (i.e. from a failing `internal_assert`). The logic | ||
for translating uncaught exceptions into successful tests is in | ||
`test/common/expect_abort.cpp`. | ||
|
||
## Debugging with LLDB | ||
|
||
We provide helpers for pretty-printing Halide's IR types in LLDB. The | ||
`.lldbinit` file at the repository root will load automatically if you launch | ||
`lldb` from this directory and your `~/.lldbinit` file contains the line, | ||
|
||
``` | ||
settings set target.load-cwd-lldbinit true | ||
``` | ||
|
||
If you prefer to avoid such global configuration, you can directly load the | ||
helpers with the LLDB command, | ||
|
||
``` | ||
command script import ./tools/lldbhalide.py | ||
``` | ||
|
||
again assuming that the repository root is your current working directory. | ||
|
||
To see the benefit of using these helpers, let us debug `correctness_bounds`: | ||
|
||
``` | ||
$ lldb ./build/test/correctness/correctness_bounds | ||
(lldb) breakpoint set --file bounds.cpp --line 18 | ||
Breakpoint 1: where = correctness_bounds`main + 864 at bounds.cpp:18:12, address = 0x0000000100002054 | ||
(lldb) run | ||
Process 29325 launched: '/Users/areinking/dev/Halide/build/test/correctness/correctness_bounds' (arm64) | ||
Defining function... | ||
Process 29325 stopped | ||
* thread #1, queue = 'com.apple.main-thread', stop reason = breakpoint 1.1 | ||
frame #0: 0x0000000100002054 correctness_bounds`main(argc=1, argv=0x000000016fdff160) at bounds.cpp:18:12 | ||
15 g(x, y) = min(x, y); | ||
16 h(x, y) = clamp(x + y, 20, 100); | ||
17 | ||
-> 18 Var xo("xo"), yo("yo"), xi("xi"), yi("yi"); | ||
19 | ||
20 Target target = get_jit_target_from_environment(); | ||
21 if (target.has_gpu_feature()) { | ||
Target 0: (correctness_bounds) stopped. | ||
(lldb) | ||
``` | ||
|
||
Now we can try to inspect the Func `h`. Without the helpers, we see: | ||
|
||
``` | ||
(lldb) v h | ||
(Halide::Func) { | ||
func = { | ||
contents = { | ||
strong = (ptr = 0x0000600002486a20) | ||
weak = nullptr | ||
idx = 0 | ||
} | ||
} | ||
pipeline_ = { | ||
contents = (ptr = 0x0000000000000000) | ||
} | ||
} | ||
``` | ||
|
||
But if we load the helpers and try again, we get a much more useful output: | ||
|
||
``` | ||
(lldb) command script import ./tools/lldbhalide.py | ||
(lldb) v h | ||
... lots of output ... | ||
``` | ||
|
||
The amount of output here is maybe a bit _too_ much, but we gain the ability to | ||
more narrowly inspect data about the func: | ||
|
||
``` | ||
(lldb) v h.func.init_def.values | ||
... | ||
(std::vector<Halide::Expr>) h.func.init_def.values = size=1 { | ||
[0] = max(min(x + y, 100), 20) | ||
} | ||
``` | ||
|
||
These helpers are particularly useful when using graphical debuggers, such as | ||
the one found in CLion. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.