Remove homebrew app casks #24593

mostlikelee · 2024-12-10T16:25:54Z

Removing homebrew casks that install app bundles from the macos software detail query as they are already reported in the apps table. apps table provides bundle_identifer which is used in vulnerability (CPE) matching to grab the correct vendor.

Changes file added for user-visible changes in changes/, orbit/changes/ or ee/fleetd-chrome/changes.
See Changes files for more information.
Added/updated tests
Manual QA for all new/changed functionality

lucasmrod · 2024-12-10T16:45:20Z

server/service/osquery_utils/queries.go

+  path AS installed_path
+FROM homebrew_packages
+WHERE type = 'cask'
+AND NOT EXISTS (SELECT 1 FROM file WHERE file.path LIKE CONCAT(homebrew_packages.path, '/%%%%', '/%%.app%%') LIMIT 1);


Double checking: LIKE with double %% (recursive search) on the file table can only be used at the end.

Double wildcards can NEVER be used mid-string (infix)

https://blog.1password.com/the-file-table-osquerys-secret-weapon/

interesting...i'll have to figure out why this is currently working

I updated the WHERE clause to be safe, but it's interesting that this works in osqueryi:

SELECT path FROM file WHERE file.path LIKE CONCAT('/opt/homebrew/Caskroom/firefox', '/%%%%', '/%%.app%%') LIMIT 1;

IIRC a query that uses %% not in the end "works" (returns results) but is it actually recursive?

lucasmrod · 2024-12-10T16:46:09Z

server/service/osquery_utils/queries.go

+  path AS installed_path
+FROM homebrew_packages
+WHERE type = 'cask'
+AND NOT EXISTS (SELECT 1 FROM file WHERE file.path LIKE CONCAT(homebrew_packages.path, '/%%%%', '/%%.app%%') LIMIT 1);


Am probably missing something, why not do CONCAT(homebrew_packages.path, '/%%%%/%%.app%%').

server/service/osquery_utils/queries.go

lucasmrod

Left some questions.

codecov · 2024-12-10T16:47:23Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 63.80%. Comparing base (669900c) to head (4e5cc48).
Report is 227 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #24593      +/-   ##
==========================================
+ Coverage   63.51%   63.80%   +0.28%     
==========================================
  Files        1592     1605      +13     
  Lines      151140   152819    +1679     
  Branches     3885     3885              
==========================================
+ Hits        95994    97500    +1506     
- Misses      47492    47538      +46     
- Partials     7654     7781     +127

Flag	Coverage Δ
backend	`64.60% <ø> (+0.30%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

lucasmrod · 2024-12-17T22:01:27Z

docs/Contributing/Understanding-host-vitals.md

+  path AS installed_path
+FROM homebrew_packages
+WHERE type = 'cask'
+AND NOT EXISTS (SELECT 1 FROM file WHERE file.path LIKE CONCAT(homebrew_packages.path, '/%%') AND file.path LIKE '/%.app%' LIMIT 1);


Isn't this file.path LIKE '/%.app%' just matching something like /Google Chrome.app? (on the root path / only)

I believe file.path is looking for a string match, which is why this is working:

osquery> SELECT path FROM file WHERE file.path LIKE CONCAT('/opt/homebrew/Caskroom/firefox', '/%%') AND file.path LIKE '/%.app%' LIMIT 1; +-----------------------------------------------------+ | path | +-----------------------------------------------------+ | /opt/homebrew/Caskroom/firefox/126.0.1/Firefox.app/ | +-----------------------------------------------------+ osquery> SELECT path FROM file WHERE file.path LIKE CONCAT('/opt/homebrew/Caskroom/ngrok', '/%%') AND file.path LIKE '/%.app%' LIMIT 1;

I think either should work, it seems clearer to use %.app% so i'll update

lucasmrod

LGTM.

Left one nit comment and the following question to discuss:
On the following sample file query:

SELECT * FROM file WHERE file.path LIKE '/usr/local/Caskroom/jd-gui/%%' AND file.path LIKE '/%.app';

The first LIKE is used to walk the directory and the second LIKE uses string matching?

server/service/osquery_utils/queries.go

lucasmrod · 2024-12-23T13:20:00Z

server/service/osquery_utils/queries.go

+  path AS installed_path
+FROM homebrew_packages
+WHERE type = 'cask'
+AND NOT EXISTS (SELECT 1 FROM file WHERE file.path LIKE CONCAT(homebrew_packages.path, '/%%%%') AND file.path LIKE '%%.app%%' LIMIT 1);


After some testing I found out what's happening behind the scenes:

SELECT 1 FROM file WHERE file.path LIKE CONCAT(homebrew_packages.path, '/%%') AND file.path LIKE '%.app' LIMIT 1

osquery will first walk the first directory in the LIKE: CONCAT(homebrew_packages.path, '/%%') and generate the list of paths.

osquery will then walk the other directory in the LIKE: '%.app', which given it's not an absolute path will try to look for .apps in the current working directory (which for osqueryd usually run as root will be /) and generate the list of paths, which on macOS will be an empty list.

So, only (1) will generate paths to process.

After (1) and (2), the sqlite engine will perform its own filtering with LIKE (string matching) over the returned paths in (1).

So, basically LIKE is used first by osquery code to generate the paths AND then by the sqlite engine which will do string matching as usual.

The trick here is that we can use this because we are expecting / to contain no apps and no large number of files or directories (I was not able to create files or .apps in /).

Rather than relying on "no large number of files or directories" to make this query efficient, should we instead use filename LIKE '%.app'?

lucasmrod

LGTM, pending this comment: https://github.com/fleetdm/fleet/pull/24593/files#r1894214609

docs/Contributing/Understanding-host-vitals.md

This reverts commit c9746df.

iansltx

Approving for docs

iansltx · 2024-12-23T20:50:52Z

...oh wait, this is host vitals rather than APIs so @marko-lisica has to approve this

mostlikelee added 2 commits December 10, 2024 09:06

remove app casks

f9be254

changelog

a528149

mostlikelee marked this pull request as ready for review December 10, 2024 16:25

mostlikelee requested review from rachaelshaw, noahtalerman and a team as code owners December 10, 2024 16:25

mostlikelee temporarily deployed to Docker Hub December 10, 2024 16:26 — with GitHub Actions Inactive

lucasmrod reviewed Dec 10, 2024

View reviewed changes

server/service/osquery_utils/queries.go Show resolved Hide resolved

lucasmrod reviewed Dec 10, 2024

View reviewed changes

lucasmrod self-assigned this Dec 10, 2024

mostlikelee marked this pull request as draft December 10, 2024 18:01

noahtalerman removed their request for review December 13, 2024 14:07

update WHERE clause

c0d9f84

mostlikelee had a problem deploying to Docker Hub December 17, 2024 13:17 — with GitHub Actions Error

mostlikelee marked this pull request as ready for review December 17, 2024 13:18

generate doc

dc187c5

mostlikelee temporarily deployed to Docker Hub December 17, 2024 13:20 — with GitHub Actions Inactive

lucasmrod reviewed Dec 17, 2024

View reviewed changes

rachaelshaw previously approved these changes Dec 17, 2024

View reviewed changes

remove prefix

29b26df

mostlikelee dismissed rachaelshaw’s stale review via 29b26df December 19, 2024 16:23

mostlikelee temporarily deployed to Docker Hub December 19, 2024 16:23 — with GitHub Actions Inactive

iansltx mentioned this pull request Dec 19, 2024

CVE-2023-48795 false positives #23556

Open

lucasmrod reviewed Dec 20, 2024

View reviewed changes

server/service/osquery_utils/queries.go Show resolved Hide resolved

lucasmrod reviewed Dec 23, 2024

View reviewed changes

use directory

c9746df

mostlikelee requested a review from marko-lisica as a code owner December 23, 2024 18:23

mostlikelee temporarily deployed to Docker Hub December 23, 2024 18:23 — with GitHub Actions Inactive

lucasmrod reviewed Dec 23, 2024

View reviewed changes

docs/Contributing/Understanding-host-vitals.md Outdated Show resolved Hide resolved

Revert "use directory"

4e5cc48

This reverts commit c9746df.

mostlikelee temporarily deployed to Docker Hub December 23, 2024 19:45 — with GitHub Actions Inactive

lucasmrod approved these changes Dec 23, 2024

View reviewed changes

iansltx approved these changes Dec 23, 2024

View reviewed changes

noahtalerman approved these changes Dec 23, 2024

View reviewed changes

marko-lisica approved these changes Dec 24, 2024

View reviewed changes

mostlikelee merged commit f6f35be into main Dec 24, 2024
27 checks passed

mostlikelee deleted the 22944-hombrew-casks branch December 24, 2024 20:25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Remove homebrew app casks #24593

Remove homebrew app casks #24593

mostlikelee commented Dec 10, 2024

lucasmrod Dec 10, 2024

lucasmrod Dec 10, 2024

mostlikelee Dec 10, 2024

mostlikelee Dec 17, 2024

lucasmrod Dec 17, 2024

lucasmrod Dec 10, 2024

lucasmrod left a comment

codecov bot commented Dec 10, 2024 •

edited

Loading

lucasmrod Dec 17, 2024

mostlikelee Dec 19, 2024

lucasmrod left a comment

lucasmrod Dec 23, 2024

zwass Dec 24, 2024

lucasmrod left a comment

iansltx left a comment

iansltx commented Dec 23, 2024

Remove homebrew app casks #24593

Remove homebrew app casks #24593

Conversation

mostlikelee commented Dec 10, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lucasmrod left a comment

Choose a reason for hiding this comment

codecov bot commented Dec 10, 2024 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lucasmrod left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lucasmrod left a comment

Choose a reason for hiding this comment

iansltx left a comment

Choose a reason for hiding this comment

iansltx commented Dec 23, 2024

codecov bot commented Dec 10, 2024 •

edited

Loading