Skip to content

Commit

Permalink
[ProxySG] Remove default syslog format/framing from TCP and UDP inputs (
Browse files Browse the repository at this point in the history 
#11679)

The ProxySG TCP and UDP inputs were not always using syslog formatting/framing by default, so this removes those as default processors and input configs. The inputs now expect raw ProxySG messages by default.

If there user does configure syslog format or framing, it's still possible to add support for these with customized options in the integration's input configuration.
  • Loading branch information
@mjwolf
mjwolf authored Nov 8, 2024
1 parent f4550b7 commit 6097db7
Show file tree
Hide file tree
Showing 6 changed files with 15 additions and 17 deletions.
16 changes: 8 additions & 8 deletions packages/proxysg/_dev/deploy/docker/sample_logs/syslog/proxysg.log
View file
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
<13>1 2024-03-08T10:14:08+00:00 srvr serverd - - - 2024-03-22 16:16:01 48 10.82.255.36 302 TCP_NC_MISS 1242 969 GET https pixel.tapad.com 443 /idsync/ex/push ?partner_id=2499&partner_device_id=aeb66687-eabe-442e-b11e-79494b740d0d-640ba437-5553&partner_url=https%3A%2F%2Fa.vidoomy.com%2Fapi%2Frtbserver%2Fpbscookie%3Fuid%3Daeb66687-eabe-442e-b11e-79494b740d0d-640ba437-5553%26vid%3D280fa751e99651c4193ef92f6dab0f92%26dspid%3DCEN aeinstein - - pixel.tapad.com - https://vid.vidoomy.com/ OBSERVED "FastwebRes_CallCntr;Web Ads/Analytics" - 142.182.19.21 34.111.113.62 "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36" sha256WithRSAEncryption
<13>1 2024-03-08T10:14:08+00:00 srvr serverd - - - 2024-03-22 16:16:01 1418 10.82.255.36 302 TCP_NC_MISS 1158 1360 GET https p.rfihub.com 443 /cm ?pub=224&in=1&getuid=https%3A//image2.pubmatic.com/AdServer/Pug%3Fvcode%3Dbz0yJnR5cGU9MSZjb2RlPTI3MzkmdGw9MTI5NjAw%26piggybackCookie%3D%24UID%26gdpr%3D0%26gdpr_consent%3D aeinstein - - p.rfihub.com - https://ads.pubmatic.com/ OBSERVED "FastwebRes_CallCntr;Web Ads/Analytics" - 142.182.19.21 199.38.167.131 "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36" sha256WithRSAEncryption
<13>1 2024-03-08T10:14:08+00:00 srvr serverd - - - 2024-03-22 16:16:01 382 10.82.255.36 200 TCP_ACCELERATED 39 356 CONNECT tcp token.rubiconproject.com 443 / - aeinstein - - - - - OBSERVED "FastwebRes_CallCntr;Web Ads/Analytics" - 142.182.19.21 8.43.72.98 "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36" -
<13>1 2024-03-08T10:14:08+00:00 srvr serverd - - - 2024-03-22 16:16:01 1407 10.82.255.36 0 TUNNELED 4897 5784 unknown ssl rtb-csync.smartadserver.com 443 / - aeinstein - - rtb-csync.smartadserver.com - - OBSERVED "Web Ads/Analytics" - 142.182.19.21 23.105.12.150 - -
<13>1 2024-03-08T10:14:08+00:00 srvr serverd - - - 2024-03-22 16:16:01 250 10.82.255.36 200 TCP_ACCELERATED 39 334 CONNECT tcp a.vidoomy.com 443 / - aeinstein - - - - - OBSERVED "Technology/Internet" - 142.182.19.21 212.36.83.245 "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36" -
<13>1 2024-03-08T10:14:08+00:00 srvr serverd - - - 2024-03-22 16:16:01 203 10.82.255.36 200 TCP_ACCELERATED 39 348 CONNECT tcp pixel.quantserve.com 443 / - aeinstein - - - - - OBSERVED "FastwebRes_CallCntr;Web Ads/Analytics" - 142.182.19.21 192.184.67.40 "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36" -
<13>1 2024-03-08T10:14:08+00:00 srvr serverd - - - 2024-03-22 16:16:01 535 10.82.255.36 302 TCP_NC_MISS 298 809 GET https x.bidswitch.net 443 /sync ?ssp=outbrain&user_id=4j-4C6xkdi0fxtAr1zi3vjj5CQOVs0JfyrDcmeLF9_8pS5pxS1tF-2qSfks1f0jg&us_privacy=1---&gdpr=0&gdpr_pd=1&gdpr_consent=&initiator=ob aeinstein - - x.bidswitch.net - https://widgets.outbrain.com/ OBSERVED "Web Ads/Analytics" - 142.182.19.21 35.211.178.172 "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36" sha256WithRSAEncryption
<13>1 2024-03-08T10:14:08+00:00 srvr serverd - - - 2024-03-22 16:16:01 33 10.82.255.36 200 TCP_NC_MISS 947 893 GET https token.rubiconproject.com 443 /khaos.json ?khaos=LERH3RQR-10-H6DO aeinstein - - token.rubiconproject.com application/json;%20charset=UTF-8 https://eus.rubiconproject.com/ OBSERVED "FastwebRes_CallCntr;Web Ads/Analytics" - 142.182.19.21 8.43.72.98 "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36" sha256WithRSAEncryption
2024-03-22 16:16:01 48 10.82.255.36 302 TCP_NC_MISS 1242 969 GET https pixel.tapad.com 443 /idsync/ex/push ?partner_id=2499&partner_device_id=aeb66687-eabe-442e-b11e-79494b740d0d-640ba437-5553&partner_url=https%3A%2F%2Fa.vidoomy.com%2Fapi%2Frtbserver%2Fpbscookie%3Fuid%3Daeb66687-eabe-442e-b11e-79494b740d0d-640ba437-5553%26vid%3D280fa751e99651c4193ef92f6dab0f92%26dspid%3DCEN aeinstein - - pixel.tapad.com - https://vid.vidoomy.com/ OBSERVED "FastwebRes_CallCntr;Web Ads/Analytics" - 142.182.19.21 34.111.113.62 "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36" sha256WithRSAEncryption
2024-03-22 16:16:01 1418 10.82.255.36 302 TCP_NC_MISS 1158 1360 GET https p.rfihub.com 443 /cm ?pub=224&in=1&getuid=https%3A//image2.pubmatic.com/AdServer/Pug%3Fvcode%3Dbz0yJnR5cGU9MSZjb2RlPTI3MzkmdGw9MTI5NjAw%26piggybackCookie%3D%24UID%26gdpr%3D0%26gdpr_consent%3D aeinstein - - p.rfihub.com - https://ads.pubmatic.com/ OBSERVED "FastwebRes_CallCntr;Web Ads/Analytics" - 142.182.19.21 199.38.167.131 "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36" sha256WithRSAEncryption
2024-03-22 16:16:01 382 10.82.255.36 200 TCP_ACCELERATED 39 356 CONNECT tcp token.rubiconproject.com 443 / - aeinstein - - - - - OBSERVED "FastwebRes_CallCntr;Web Ads/Analytics" - 142.182.19.21 8.43.72.98 "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36" -
2024-03-22 16:16:01 1407 10.82.255.36 0 TUNNELED 4897 5784 unknown ssl rtb-csync.smartadserver.com 443 / - aeinstein - - rtb-csync.smartadserver.com - - OBSERVED "Web Ads/Analytics" - 142.182.19.21 23.105.12.150 - -
2024-03-22 16:16:01 250 10.82.255.36 200 TCP_ACCELERATED 39 334 CONNECT tcp a.vidoomy.com 443 / - aeinstein - - - - - OBSERVED "Technology/Internet" - 142.182.19.21 212.36.83.245 "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36" -
2024-03-22 16:16:01 203 10.82.255.36 200 TCP_ACCELERATED 39 348 CONNECT tcp pixel.quantserve.com 443 / - aeinstein - - - - - OBSERVED "FastwebRes_CallCntr;Web Ads/Analytics" - 142.182.19.21 192.184.67.40 "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36" -
2024-03-22 16:16:01 535 10.82.255.36 302 TCP_NC_MISS 298 809 GET https x.bidswitch.net 443 /sync ?ssp=outbrain&user_id=4j-4C6xkdi0fxtAr1zi3vjj5CQOVs0JfyrDcmeLF9_8pS5pxS1tF-2qSfks1f0jg&us_privacy=1---&gdpr=0&gdpr_pd=1&gdpr_consent=&initiator=ob aeinstein - - x.bidswitch.net - https://widgets.outbrain.com/ OBSERVED "Web Ads/Analytics" - 142.182.19.21 35.211.178.172 "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36" sha256WithRSAEncryption
2024-03-22 16:16:01 33 10.82.255.36 200 TCP_NC_MISS 947 893 GET https token.rubiconproject.com 443 /khaos.json ?khaos=LERH3RQR-10-H6DO aeinstein - - token.rubiconproject.com application/json;%20charset=UTF-8 https://eus.rubiconproject.com/ OBSERVED "FastwebRes_CallCntr;Web Ads/Analytics" - 142.182.19.21 8.43.72.98 "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36" sha256WithRSAEncryption
5 changes: 5 additions & 0 deletions packages/proxysg/changelog.yml
View file
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
# newer versions go on top
- version: 0.3.0
changes:
- description: Do not do syslog parsing by default in TCP and UCP inputs
type: enhancement
link: https://github.com/elastic/integrations/pull/11679
- version: 0.2.0
changes:
- description: Add support for 'bcreportermain_v1', 'bcreporterssl_v1' and 'ssl' formats
Expand Down
3 changes: 0 additions & 3 deletions packages/proxysg/data_stream/log/agent/stream/tcp.yml.hbs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,9 +22,6 @@ processors:
fields:
_conf: "{{config}}"
- add_locale: ~
- syslog:
field: message
format: rfc5424
{{#if processors}}
{{processors}}
{{/if}}
3 changes: 0 additions & 3 deletions packages/proxysg/data_stream/log/agent/stream/udp.yml.hbs
View file
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,6 @@ processors:
fields:
_conf: "{{config}}"
- add_locale: ~
- syslog:
field: message
format: rfc5424
{{#if processors}}
{{processors}}
{{/if}}
Expand Down
3 changes: 1 addition & 2 deletions packages/proxysg/data_stream/log/manifest.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -167,8 +167,7 @@ streams:
multi: false
required: false
show_user: false
default: |
framing: rfc6587
default: ""
description: Specify custom configuration options for the TCP input.
- name: ssl
type: yaml
Expand Down
2 changes: 1 addition & 1 deletion packages/proxysg/manifest.yml
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
format_version: 3.1.1
name: proxysg
title: "Broadcom ProxySG"
version: 0.2.0
version: 0.3.0
source:
license: "Elastic-2.0"
description: "Collect access logs from Broadcom ProxySG with Elastic Agent."
Expand Down

0 comments on commit 6097db7

Please sign in to comment.