Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cpufeatures: incongruities between ARM64 hwcaps and Rust SHA* target features #395

Closed
tarcieri opened this issue May 6, 2021 · 4 comments · Fixed by #456
Closed

cpufeatures: incongruities between ARM64 hwcaps and Rust SHA* target features #395

tarcieri opened this issue May 6, 2021 · 4 comments · Fixed by #456

Comments

@tarcieri
Copy link
Member

tarcieri commented May 6, 2021

Following up from #393:

The output of rustc --target=aarch64-unknown-linux-gnu --print target-features shows the following cryptography-related target features on ARM64 (which is the same for aarch64-apple-darwin)

    sha2                               - Enable SHA1 and SHA256 support.
    sha3                               - Enable SHA512 and SHA3 support.

This suggests that sha2 implies SHA-1 support, and that sha3 implies SHA(2)-512 support (but curiously, that the aforementioned sha2 does not).

This does not consistent with the Linux hwcaps or the ARM registers they map to (or for that matter, the algorithm families the respective algorithms belong to):

https://www.kernel.org/doc/html/latest/arm64/elf_hwcaps.html

HWCAP_SHA1: Functionality implied by ID_AA64ISAR0_EL1.SHA1 == 0b0001.
HWCAP_SHA2: Functionality implied by ID_AA64ISAR0_EL1.SHA2 == 0b0001.
HWCAP_SHA3: Functionality implied by ID_AA64ISAR0_EL1.SHA3 == 0b0001.
HWCAP_SHA512: Functionality implied by ID_AA64ISAR0_EL1.SHA2 == 0b0010.

On macOS targets, support for sha1 and sha2 is implicit, however support for SHA-512 and SHA-3 intrinsics can be queried through sysctl(3).

Question:

How should cpufeatures handle this mapping? For example, should checking for support for the sha3 target feature using cpufeatures test for support for both SHA(2)-512 and SHA-3, and return false unless both are available?

Sidebar: do the Rust target feature mappings actually make sense here? Should sha3 perhaps be decoupled from SHA(2)-512?

@tarcieri
Copy link
Member Author

I think the short answer to my question is "yes". In the sha3 case we should check for both.

Likewise it would be nice to support the crypto target feature this way. My understanding is it implies aes, sha2, and neon (although NEON is present on all AArch64 targets).

From what I can tell aes implies PMULL support as well, so perhaps we should check for both.

There are also quite a number of additional target features which would benefit the performance of AES implementations. Notably:

    fuse-aes                           - CPU fuses AES crypto operations.
    fuse-arith-logic                   - CPU fuses arithmetic and logic operations.
    fuse-crypto-eor                    - CPU fuses AES/PMULL and EOR operations.

tarcieri added a commit that referenced this issue Jun 2, 2021
Closes #395

The HWCAPs provided by Linux are finer-grained than LLVM target
features. This crate is trying to detect the latter, so we need to group
together these finer grained features detected via HWCAPs.

This commit attempts to group together finer grained HWCAPs into coarser
grained target features, ensuring all of the relevant HWCAPs for a given
target feature are available.

Additionally, this adds support for the `crypto` target feature, which
implies `aes` (and with it PMULL), along with `sha2` and `neon`
(although the latter is always present on `aarch64`)
tarcieri added a commit that referenced this issue Jun 3, 2021
)

Closes #395

The HWCAPs provided by Linux are finer-grained than LLVM target
features. This crate is trying to detect the latter, so we need to group
together these finer grained features detected via HWCAPs.

This commit attempts to group together finer grained HWCAPs into coarser
grained target features, ensuring all of the relevant HWCAPs for a given
target feature are available.

Additionally, this adds support for the `crypto` target feature, which
implies `aes` (and with it PMULL), along with `sha2` and `neon`
(although the latter is always present on `aarch64`)
@newpavlov
Copy link
Member

newpavlov commented Jul 2, 2021

Do you know how Rust toolchain currently handles those features?A bit while ago when I looked into it, I think it had only a general crypto target feature, which weirdly enough had slightly different capabilities on different ARM versions. IIUC the current approach is a fine temporary workaround until Rust handling of ARM target features gets properly stabilized, but ideally I think we should use target features enabled by default by toolchain (e.g. as we do for SSE2 on x86(-64) targets) instead of introducing our own features.

@tarcieri
Copy link
Member Author

tarcieri commented Jul 2, 2021

From what I've gathered, Rust target features map directly to the corresponding LLVM ones.

I think it had only a general crypto target feature, which weirdly enough had slightly different capabilities on different ARM versions.

Yeah, unfortunately I didn't include citations for the mappings. I really should've, my bad. But indeed I saw it gets quite tricky. Here's some info I found via a cursory googling:

https://lists.llvm.org/pipermail/llvm-dev/2018-September/126346.html

On AArch64, crypto means different things for v8.{1,2,3}-a than v8.4-a:

  • v8.{1,2,3}-a: aes and sha2
  • v8.4-a: aes and sha2 as well as sha3 and sm4

Given only the aes, sha2, and sha3 features are supported right now, this means the only feature that poses a potential problem is sha3, which we aren't currently using in downstream crates. It might be good to make a tracking issue about it specifically though.

Also note that only aarch64* targets are supported as these are all ARMv8 features and to my knowledge there are no 32-bit ARMv8 targets currently supported by the Rust compiler, so at least for now we don't have to worry about those.

@newpavlov
Copy link
Member

From what I've gathered, Rust target features map directly to the corresponding LLVM ones.

I think I've seen somewhere a suggestion in the LLVM mailing list to split the crypto feature to finer grained ones. But I am not sure about its prospects.

I really hope that Rust eventually will adopt separate (aes, sha2, sha3, sm3, sm4) target features even if it will mean departing from LLVM a bit. I think made such suggestion in a relevant rust-lang issue, but IIRC it didn't find much traction...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants