Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dataset set postmatch 5576 v14 #12196

Closed

Conversation

catenacyber
Copy link
Contributor

Link to ticket: https://redmine.openinfosecfoundation.org/issues/
https://redmine.openinfosecfoundation.org/issues/5576

Describe changes:

  • detect/dataset: delay set operation after signature full match

SV_BRANCH=OISF/suricata-verify#2093

#12126 rebased after merge of absent keyword

The design is :

  • detect: postmatch can run AppLayerTxMatch callbacks (in its own commit)
  • usage of delay_postmatch: buffers using dataset/set are put at the tail of inspections whatever their progress
  • usage of DETECT_ENGINE_INSPECT_SIG_MATCH_MORE_BUF: dataset may return this new case to tell that ok we match on a multi-buffer, but we still want to try all occurrences of a multi-buffer (instead of returning a simple early match)

How do you like the git history/commits ?

Will be useful for dataset, when it needs to find a transaction
buffer again.
The set operation of dataset keyword was done even if signature
did not fully match, which is not the expected behavior.
We want dataset to behave like flowbits for instance.

This patch changes the behavior of the dataset keyword to do a
match and a post match for the set operation.
The postmatch retrieves the data, using the list identifier
associated to the buffer for this signature.

This avoids to store the buffer(s), when we do not have a
dedicated storage (per signature and per tx) that can own
and clean arbitrary buffers over multiple packets, in
the case the transaction spans over multiple packets
with different tx progresses for instance.
If detection runs on one packet, the InspectionBuffer are
cached and fast to get.
The most expensive case if for multi buffers, where we
need to run detection again, to see which occurences
match all payload keywords and should be added in the dataset.

Ticket: OISF#5576
Copy link

codecov bot commented Dec 2, 2024

Codecov Report

Attention: Patch coverage is 88.23529% with 20 lines in your changes missing coverage. Please review.

Project coverage is 83.15%. Comparing base (ae10fc3) to head (68beac3).
Report is 8 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master   #12196      +/-   ##
==========================================
- Coverage   83.16%   83.15%   -0.01%     
==========================================
  Files         912      912              
  Lines      257066   257197     +131     
==========================================
+ Hits       213797   213884      +87     
- Misses      43269    43313      +44     
Flag Coverage Δ
fuzzcorpus 61.03% <77.05%> (+0.05%) ⬆️
livemode 19.60% <43.52%> (+0.19%) ⬆️
pcap 44.36% <18.82%> (-0.07%) ⬇️
suricata-verify 62.75% <88.23%> (+0.04%) ⬆️
unittests 59.15% <18.23%> (-0.03%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

@suricata-qa
Copy link

WARNING:

field baseline test %
SURI_TLPR1_stats_chk
.app_layer.flow.dcerpc_tcp 40 42 105.0%

Pipeline 23640

@catenacyber
Copy link
Contributor Author

Next in #12230

@catenacyber catenacyber closed this Dec 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants