Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize memory usage on Mac by pooling memory passed to sysctl #41

Open
cmacknz opened this issue Jul 13, 2022 · 1 comment
Open

Optimize memory usage on Mac by pooling memory passed to sysctl #41

cmacknz opened this issue Jul 13, 2022 · 1 comment
Labels
Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team

Comments

@cmacknz
Copy link
Member

cmacknz commented Jul 13, 2022

The sysctl implementation allocates 256 KiB for every invocation.

In go-sysinfo a sync.Pool is used to minimize the amount of memory that is allocated for each call. It uses a buffer from the pool to make the call then allocates only the size required rather than ARG_MAX.

Port this same optimization to the system metrics package. Here is one example where it would apply:

func getProcArgs(pid int, filter func(string) bool) ([]string, string, mapstr.M, error) {
mib := []C.int{C.CTL_KERN, C.KERN_PROCARGS2, C.int(pid)}
argmax := uintptr(C.ARG_MAX)
buf := make([]byte, argmax)
err := sysctl(mib, &buf[0], &argmax, nil, 0)
if err != nil {
return nil, "", nil, fmt.Errorf("error in sysctl: %w", err)
}

We could avoid allocating a new slice for every process with this optimization.

@cmacknz cmacknz added the Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team label Jul 13, 2022
@fearful-symmetry
Copy link
Contributor

fearful-symmetry commented Jul 26, 2022

Something else to consider: Instead of allocating a single, large chunk of memory ahead of time, we can use sysctl's own memory allocation reporting to make sure we only allocate what memory we need. For example, on MacOS 12, we will allocate ARG_MAX bytes, which is 1048576 but the average process will actually need something closer to only 600.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team
Projects
None yet
Development

No branches or pull requests

2 participants