diff options
author | 2022-04-04 05:59:34 +0000 | |
---|---|---|
committer | 2022-04-04 05:59:34 +0000 | |
commit | b9e18327ea5e620c9205f3d90627eb6504a0a872 (patch) | |
tree | 7a5c98d8a9df87f8a79072f1bb1a48061b8efcbd /testsuite/minitest/src | |
parent | b4eaadcfa5c2600b0337e3a6f5e887a078523c49 (diff) | |
parent | b4181a8a313ef7686da3f64e3947ce96b9788037 (diff) | |
download | cortex-m-b9e18327ea5e620c9205f3d90627eb6504a0a872.tar.gz cortex-m-b9e18327ea5e620c9205f3d90627eb6504a0a872.tar.zst cortex-m-b9e18327ea5e620c9205f3d90627eb6504a0a872.zip |
Merge #355
355: Add on-target tests to CI r=thejpster a=newAM
A week ago in the rust-embedded matrix chat `@adamgreig` mentioned that on-target CI would be helpful for the `cortex-m` and `cortex-m-rt` crates. This is a pull-request to make that happen.
## History: Bootstrapping and `cortex-m-rt`
The first problem I came across was the amount of boilerplate required to build an on-target binary without `cortex-m-rt`.
There are two paths forward here:
1. Introduce a lot of boilerplate in `cortex-m`, which will largely be copy-pasted from `cortex-m-rt`.
2. Relocate `cortex-m-rt` to this workspace.
In (#391) `cortex-m-rt` was relocated to this workspace to fix the bootstrapping problem.
## Test Execution
Tests run with QEMU and physical hardware.
QEMU uses the LM3S6965 Cortex-M3 target because it is widely used.
The physical hardware is a NUCLEO-F070RB connected to a self-hosted runner. In the future more hardware can be added to cover more CPUs.
Due to reliability concerns with self-hosted runners only QEMU will be a required status check in CI, the physical hardware checks will be informational only.
### CI Software
The CI software for running on physical hardware is simply a self-hosted github actions runner. A working demonstration of this can be found [here](https://github.com/newAM/totally-not-a-cortex-m-fork/runs/5345451343?check_suite_focus=true) (the repository does not appear as a fork to work around github actions limitations with forks).
The runner uses [`probe-run`] to execute the tests on embedded hardware. [`probe-run`] was chosen for several reasons:
* Actively maintained by [knurling-rs], with an actively maintained back-end from [`probe-rs`].
* Written in rust. Understanding the code does not require contributors to learn a new language.
* Designed with on-target testing as a primary goal (for use with [`defmt-test`]).
* Automatic unwinding and backtrace display.
## Test Harness
This PR introduces a test harness, `minitest`.
`minitest` is almost identical to [`defmt-test`], the only difference is that it replaces [`defmt`] with [`rtt-target`] because [`defmt`] introduces a dependency cycle on `cortex-m`.
This is harness is very minimal, adding only 327 lines of rust:
```console
$ tokei testsuite/minitest
===============================================================================
Language Files Lines Code Comments Blanks
===============================================================================
Markdown 1 7 0 4 3
TOML 2 41 34 0 7
-------------------------------------------------------------------------------
Rust 3 406 350 5 51
|- Markdown 1 8 0 7 1
(Total) 414 350 12 52
===============================================================================
Total 6 454 384 9 61
===============================================================================
```
The test harness does introduce some abstraction, and may not be suitable for all tests. Lower-level tests are still possible without the harness using `asm::udf` to fail the test, and `asm::bkpt` to exit without failure.
## Reliability and Uptime
I have been doing automatic on-target testing for the [`stm32wlxx-hal`] using [`probe-run`]. Over hundreds of automatic runs spanning several months I have had no failures as a result of external factors (USB connectivity, programming errors, ect.). I do not anticipate on-target CI being perfect, but at the same time I do not anticipate frequent problems.
[`defmt-test`]: https://github.com/knurling-rs/defmt/tree/main/firmware/defmt-test
[`defmt`]: https://ferrous-systems.com/blog/defmt/
[`probe-rs`]: https://probe.rs/
[`probe-run`]: https://github.com/knurling-rs/probe-run
[`rtt-target`]: https://crates.io/crates/rtt-target
[`stm32wlxx-hal`]: https://github.com/stm32-rs/stm32wlxx-hal
[knurling-rs]: https://knurling.ferrous-systems.com/
Co-authored-by: Alex Martens <alex@thinglab.org>
Diffstat (limited to 'testsuite/minitest/src')
-rw-r--r-- | testsuite/minitest/src/export.rs | 13 | ||||
-rw-r--r-- | testsuite/minitest/src/lib.rs | 70 |
2 files changed, 83 insertions, 0 deletions
diff --git a/testsuite/minitest/src/export.rs b/testsuite/minitest/src/export.rs new file mode 100644 index 0000000..4b04fda --- /dev/null +++ b/testsuite/minitest/src/export.rs @@ -0,0 +1,13 @@ +use crate::TestOutcome; +use cortex_m_rt as _; + +pub fn check_outcome<T: TestOutcome>(outcome: T, should_error: bool) { + if outcome.is_success() == should_error { + let note: &str = if should_error { + "`#[should_error]` " + } else { + "" + }; + panic!("{}test failed with outcome: {:?}", note, outcome); + } +} diff --git a/testsuite/minitest/src/lib.rs b/testsuite/minitest/src/lib.rs new file mode 100644 index 0000000..d98fb64 --- /dev/null +++ b/testsuite/minitest/src/lib.rs @@ -0,0 +1,70 @@ +#![no_std] + +use core::fmt::Debug; +pub use minitest_macros::tests; + +/// Private implementation details used by the proc macro. +#[doc(hidden)] +pub mod export; + +mod sealed { + pub trait Sealed {} + impl Sealed for () {} + impl<T, E> Sealed for Result<T, E> {} +} + +/// Indicates whether a test succeeded or failed. +/// +/// This is comparable to the `Termination` trait in libstd, except stable and tailored towards the +/// needs of defmt-test. It is implemented for `()`, which always indicates success, and `Result`, +/// where `Ok` indicates success. +pub trait TestOutcome: Debug + sealed::Sealed { + fn is_success(&self) -> bool; +} + +impl TestOutcome for () { + fn is_success(&self) -> bool { + true + } +} + +impl<T: Debug, E: Debug> TestOutcome for Result<T, E> { + fn is_success(&self) -> bool { + self.is_ok() + } +} + +#[macro_export] +macro_rules! log { + ($s:literal $(, $x:expr)* $(,)?) => { + { + #[cfg(feature = "semihosting")] + ::cortex_m_semihosting::hprintln!($s $(, $x)*); + #[cfg(feature = "rtt")] + ::rtt_target::rprintln!($s $(, $x)*); + #[cfg(not(any(feature = "semihosting", feature="rtt")))] + let _ = ($( & $x ),*); + } + }; +} + +/// Stop all tests without failure. +pub fn exit() -> ! { + #[cfg(feature = "rtt")] + cortex_m::asm::bkpt(); + #[cfg(feature = "semihosting")] + cortex_m_semihosting::debug::exit(cortex_m_semihosting::debug::EXIT_SUCCESS); + + unreachable!() +} + +/// Stop all tests and report a failure. +pub fn fail() -> ! { + #[cfg(feature = "rtt")] + cortex_m::asm::udf(); + #[cfg(feature = "semihosting")] + cortex_m_semihosting::debug::exit(cortex_m_semihosting::debug::EXIT_FAILURE); + + #[cfg(not(feature = "rtt"))] + unreachable!() +} |