Port/Copy GPU Throttling from OpenBSD

[vermaden]

Please port/copy this - GPU Throttling - from OpenBSD into FreeBSD.

This would gladly help 'FreeBSD Laptop' setups battery time.

https://marc.info/?l=openbsd-tech&m=156137425703557&w=2

Regards,
vermaden

[valpackett]

On my X240 (Haswell), the GPU power usage (measured via powermon or intel pcm tools) is 0.0W when idle, and very low when not rendering anything intensive. I'm not sure custom frequency limiting is necessary, the driver already manages power pretty well.

[vermaden]

the driver already manages power pretty well.

Is there a place where one can read more about it or just source code?

[debdrup]

There's two ways of setting states on CPUs, and presumably the same is true of iGPUs - which means that the power-states described by @myfreeweb are the C-states which can be forced by toggling the compat.linuxkpi.enable_rc6 - C-states in general tend to turn off the CPU core(s) when it's not in-use, which would explain why the power draw is idle when the iGPU is inactive.
The diff from tech@openbsd looks to me to be more akin to P-states which are commonly called (Enhanced) SpeedStep on Intel CPUs. FreeBSD implements this for CPUs via powerd in base or powerdxx in ports, but it's useful to have for the iGPU too, as the iGPU doesn't need to be running at its full frequency at all times.
As for the documentation about C-states, the only documentation I know of is the source code.

EDIT: it's probably also worth mentioning that a userland utility to provide load-based throttling (either powerd in base, powerdxx in ports, or something new in base or ports) would need to be implemented, for this to really make sense - but I don't know if that's feasible, it depends on whether the iGPU load can be exposed to userland.

[valpackett]

Intel GPUs obviously have P-states, this post by @bwidawsk explains them: https://bwidawsk.net/blog/index.php/2015/05/a-bit-on-intel-gpu-frequency/

Changing the frequency of each P-state is possible on Linux via sysfs, so it should be possible via sysctl here, but I can't find the sysctls — maybe something is lacking in the sysctl code.

On amdgpu, P-states are adjustable via the powerplay sysctls #150 (comment)

userland utility to provide load-based throttling

No no no, that's never necessary (or possible), pretty much ALL GPUs ever change P-states based on load automatically in the driver/firmware/hardware.

[netzbasis]

On my X240 (Haswell), the GPU power usage (measured via powermon or intel pcm tools) is 0.0W when idle, and very low when not rendering anything intensive. I'm not sure custom frequency limiting is necessary, the driver already manages power pretty well.

The intend an use case is to limit max power consumption. i.e. while under heavy graphic load my laptop GPU/CPU maxes out at 96degC, and runs into thermal throttling anyway. Limiting to a lower freq decreases graphic performance (almost unnoticeable for my use case) but keeps the GPU/CPU 10 - 15degC cooler. Running at 80 - 85degC.

[valpackett]

Yeah that should be the gt_*_freq_mhz sysctls I think.

I guess OpenBSD needed to insert some custom things because they removed all sysfs stuff entirely? We provide access to sysfs controls through sysctl.

[vermaden]

Yes, that would be welcome - to limit/set maximum frequency/power for the GPU.

To 'need' another daemon like gpowerd for graphics scaling was not my intention :)

My intention was to do probably the same as OpenBSD guys, to get more battery time of one does not need 100% performance of its graphics card - but for example limit to max 60% of its speed/power is sufficient.

[johalun]

Did you test with enabling rc6? That will let the GPU power down when not working. I can't see how capping the frequency would reduce power noticeably. Correct me if I'm wrong but the same task would only take longer at lower frequency, the amount of work it has to be done remains constant (work = power x time). This might not be completely linear but probably close to it...

[vermaden]

I assume that these are depracated now?

  drm.i915.enable_rc6=7
  drm.i915.semaphores=1
  drm.i915.intel_iommu_enabled=1

I use FreeBSD 11.3-RC3 currently so I have the compat.linuxkpi 'tree' available.

I assume that these should work well in /boot/loader.conf file well?

# INTEL DRM WITH drm-kmod PACKAGE (NEW)
# TRY TO SKIP UNNECESSARY MODE SETS AT BOOT TIME 
  compat.linuxkpi.fastboot=1
# USE SEMAPHORES FOR INTER-RING SYNC
  compat.linuxkpi.semaphores=1
# ENABLE POWER-SAVING RENDER C-STATE 6
  compat.linuxkpi.enable_rc6=7
# ENABLE POWER-SAVING DISPLAY C-STATES 
  compat.linuxkpi.enable_dc=2
# ENABLE FRAME BUFFER COMPRESSION FOR POWER SAVINGS
  compat.linuxkpi.enable_fbc=1

[valpackett]

Might be compat.linuxkpi.i915_enable_fbc=1 or something, there was a prefix thing added recently. Just run sysctl compat.linuxkpi and see what exists.

BTW, most of these things (like rc6) are enabled by default

[johalun]

I don't think the prefix thing was backported to 11.3. Double check the output of sysctl compat.linuxkpi to make sure.
Edit: What ^ said 😃

[vermaden]

I can't see how capping the frequency would reduce power noticeably.

Temperature (and passive cooling).

Lets say we have 1300 MHz default max setting. The task would finish in time X with 80C temp (fan needed to cooldown).

With limit to 1000 MHz the task will take about 1.3 x X but with 55C temp (passive cooling).

Just an example of course.

I currently run with these in /boot/loader.conf file. Thats on a 11.3-RC3 system.

% grep linuxkpi /boot/loader.conf 
  compat.linuxkpi.fastboot=1
  compat.linuxkpi.semaphores=1
  compat.linuxkpi.enable_rc6=7
  compat.linuxkpi.enable_dc=2
  compat.linuxkpi.enable_fbc=1

... or something else/more can be enabled?

[valpackett]

No one did anything in the week since I mentioned gt_*_freq_mhz.. Since I've been updating my intel laptop today, went to take a look. i915_sysfs was completely disabled. #157 is a quick patch that enables it, at least the frequency stuff:

https://streamable.com/7x7l7 (most of the power consumption is from vaapi encoding the screencast)

[vermaden]

Its all I got from sysctl sys.class.drm command:

% sysctl sys.class.drm
sys.class.drm.version: drm 1.1.0 20060810

Its 11.3-RC3 system.

Do I need newer graphics/drm-kmod or newer FreeBSD version for it?

[valpackett]

You need to rebuild drm-kmod with the patch.

(And since you're on 11, probably backport the patch to whatever drm-kmod version you're using? I have no idea if 11 gets drm v5.0. The patch is against 5.0)

[vermaden]

This one:

% pkg info|grep drm
drm-fbsd11.2-kmod-4.11g20190424 DRM modules for the linuxkpi-based KMS components
drm-kmod-g20181126             Metaport of DRM modules for the linuxkpi-based KMS components
libdrm-2.4.98,1                Userspace interface to kernel Direct Rendering Module services