bindgen: use bindgen to provide Rust bindings to C - v7 #12461
+3,627
−98
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add a minimal integration of bindgen to the build. This required some refactoring of the C so app-layer-events.h did not also include "rust.h", which causes issues for bindgen, probably related to circular references. AppLayerEventType was chosen as the first step as its an argument type some app-layer functions that we may want to use bindgen to export Rust, and one of the requirements of bindgen might be that C functions should only use datatypes defined in C, and not Rust. Following such a rule also prevents circular dependencies between Rust and C code. As bindgen can only accept one header file, we construct a pseudo-header file of all the headers that need to be consumed. Makefile dependency checking is done to make sure the pseudo-header is only generated as needed to avoid rebuilding every time. Special handling is required for Windows to use the Windows path. Ticket: OISF#7341 bindgen: pre-generate rust bindings to C This simplifies building, as we don't have to worry about path and such under autoconf/automake. It does however mean when we update C headers that are exposed to Rust, we manually have to re-generate the bindings, but this is a check we can do in CI. It also eliminates the need for everyone who wants to build Suricata to have bindgen and clang tools installed.
This lets us remove decode.h from app-layer-events.h as pulling in app-layer-events.h shouldn't result in pulling in dpdk, and other includes not related to app-layer-events. decode.h also doesn't need those forward declarations anymore due to previous changes.
Instead of defining this function pointer in type in Rust, and having it in C signatures, create a type and export it to Rust. To facilitate this, and new header has been creates, "app-layer-types.h", this is to avoid the circular reference of C headers pulling in "rust.h" which are required to generate Rust bindings.
This exposes the C define ALPROTO values to Rust without having to perform some runtime initialization with init_ffi. As app-layer-protos.h was clean of a circular reference to rust.h we could use it directly, it just needed the addition of suricata-common.h.
This required us to remove the auto-generated by header as it can change depending on what version is used. I suppose different versions could generate slightly different output that would cause this error out. In which case we may want to use a custom script that can be a bit smarter output, or simply error out if any of the headers used in the bindings is newer than the generated _sys.rs file.
If we have bindgen, and we're building in-tree, rebuild the bindings and compare to the previous bindings and print a warning if there is a difference. Only a warning for now, as I'm not sure where this might fail, and I don't want it to become a nuisance.
This is an example of how we can use cbindgen -> C for "C" style functions written in Rust, and then back to Rust with bindgen. The idea is to build one Rust module that contains all the C bindings, whether or not the functions are written in Rust or C.
Not supported on 0.69.5 as found in many package managers.
This crate will contain bindgen generated bindings to C.
This allows a plugin, or a library user to interface with the library just using exported C definitions, and not having to use the main suricata crate. It is a common pattern in the Rust ecosystem to have a crate that just contains the bindings like this.
A minimal example of how we should use the sys crate for types and structures that come from C.
|
WARNING:
Pipeline 24336 |
|
stats are from the last next branch. should be fine |
|
A lot is going on here, can this be split in different PRs ? |
catenacyber
reviewed
Jan 23, 2025
| use std; | ||
| use std::ffi::CString; | ||
|
|
||
| pub(super) static mut ALPROTO_DHCP: AppProto = ALPROTO_UNKNOWN; | ||
| pub(super) static ALPROTO_DHCP: AppProto = AppProtoEnum::ALPROTO_DHCP as AppProto; |
catenacyber
reviewed
Jan 23, 2025
rust/src/_sys.rs
Outdated
| @@ -1,5 +1,50 @@ | |||
| #[repr(u32)] | |||
| #[derive(Debug, Copy, Clone, Hash, PartialEq, Eq)] | |||
| pub enum AppProtoEnum { | |||
There was a problem hiding this comment.
I remember doing such a PR once, and then rejected the ticket as we want dynamic app-layer protocols, and this is not what is done in a rust enum...
catenacyber
reviewed
Jan 23, 2025
| #include <stdint.h> | ||
|
|
||
| typedef enum SCAppLayerEventType { | ||
| APP_LAYER_EVENT_TYPE_TRANSACTION = 1, |
There was a problem hiding this comment.
This was in rust, why is it in C now ?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Previous PR: #12451
Changes from previous PR:
suricata-syscrate to expose C bindigns. The mainsuricatacrate uses this crate.AppLayerDecoderEventsand use the generated one fromsuricata-sys.This also shows how
extern "C"functions written in Rust can be re-exported back into Rust and put into thesuricata-syscrate. For now we just re-export theJsonBuilderfunctions.jsonbuildermodule insidesuricata-systhat provided a Rust API around the C JsonBuilder API.NOTE: This re-export of
rust-bindings.hdoes not work withbindgen 0.69.5as found in RHEL/Fedora. And Ubuntu LTS has an even older version. However it does work with the latest version ofbindgenthat can installed with Cargo. If we want to keep working withbindgenfrom package managers we can remove this re-export, and just export curated header files.Ticket: https://redmine.openinfosecfoundation.org/issues/7341