Datashader

[jkrumbiegel]

Copied the prototype from https://github.com/cjdoris/ShadeYourData.jl over and modified it into a normal recipe. It's pretty slow when trackpad zooming (too many updates at once), but ok with the rectangle zoom.

using GLMakie
using DelimitedFiles

GLMakie.activate!(float = true)

airports = Point2f.(eachrow(readdlm(assetpath("airportlocations.csv"))))

fig, ax, ds = datashader(airports,
    colormap = [:black; Makie.to_colormap(:batlow)],
    binfactor = 2,
    post = mat -> log10.(mat .+ 1f0),
    axis = (; aspect = DataAspect())
)

[MakieBot]

Compile Times benchmark

Note, that these numbers may fluctuate on the CI servers, so take them with a grain of salt. All benchmark results are based on the mean time and negative percent mean faster than the base branch. Note, that GLMakie + WGLMakie run on an emulated GPU, so the runtime benchmark is much slower. Results are from running:

using_time = @ctime using Backend
# Compile time
create_time = @ctime fig = scatter(1:4; color=1:4, colormap=:turbo, markersize=20, visible=true)
display_time = @ctime Makie.colorbuffer(display(fig))
# Runtime
create_time = @benchmark fig = scatter(1:4; color=1:4, colormap=:turbo, markersize=20, visible=true)
display_time = @benchmark Makie.colorbuffer(display(fig))

	using	create	display	create	display
GLMakie	11.04s (10.90, 11.26) 0.11+-	1.12s (1.10, 1.14) 0.01+-	801.28ms (776.45, 853.50) 24.42+-	9.63ms (9.51, 9.71) 0.08+-	139.93ms (137.84, 145.89) 2.86+-
master	10.99s (10.90, 11.12) 0.08+-	1.11s (1.10, 1.14) 0.01+-	786.18ms (778.87, 790.43) 3.61+-	9.73ms (9.65, 9.80) 0.05+-	139.23ms (137.40, 140.94) 1.17+-
evaluation	+0.42%, 0.05s invariant (0.46d, 0.40p, 0.10std)	+1.10%, 0.01s invariant (0.99d, 0.09p, 0.01std)	+1.88%, 15.1ms invariant (0.87d, 0.15p, 14.01std)	-1.09%, -0.1ms faster ✓ (-1.62d, 0.01p, 0.06std)	+0.50%, 0.7ms invariant (0.32d, 0.56p, 2.02std)
CairoMakie	11.27s (11.12, 11.68) 0.20+-	1.18s (1.14, 1.23) 0.03+-	209.59ms (204.18, 218.77) 5.31+-	11.08ms (10.87, 11.40) 0.16+-	6.66ms (6.56, 6.78) 0.08+-
master	11.29s (11.06, 11.53) 0.17+-	1.17s (1.13, 1.22) 0.03+-	221.36ms (215.45, 230.45) 4.88+-	11.15ms (11.03, 11.51) 0.17+-	6.66ms (6.45, 6.86) 0.14+-
evaluation	-0.19%, -0.02s invariant (-0.12d, 0.83p, 0.19std)	+0.85%, 0.01s invariant (0.32d, 0.55p, 0.03std)	-5.62%, -11.77ms faster✅ (-2.31d, 0.00p, 5.10std)	-0.64%, -0.07ms invariant (-0.44d, 0.43p, 0.16std)	-0.05%, -0.0ms invariant (-0.03d, 0.96p, 0.11std)
WGLMakie	14.26s (14.04, 14.44) 0.17+-	1.35s (1.30, 1.39) 0.03+-	12.69s (12.40, 13.29) 0.31+-	16.77ms (15.15, 17.82) 1.02+-	1.22s (1.12, 1.27) 0.05+-
master	14.33s (14.07, 14.54) 0.16+-	1.38s (1.33, 1.45) 0.04+-	12.62s (12.28, 12.99) 0.22+-	16.90ms (15.42, 18.28) 0.89+-	1.19s (1.17, 1.21) 0.01+-
evaluation	-0.48%, -0.07s invariant (-0.42d, 0.45p, 0.16std)	-1.97%, -0.03s invariant (-0.70d, 0.22p, 0.04std)	+0.54%, 0.07s invariant (0.26d, 0.64p, 0.26std)	-0.76%, -0.13ms invariant (-0.13d, 0.81p, 0.96std)	+2.69%, 0.03s invariant (0.89d, 0.14p, 0.03std)

[asinghvi17]

It would be interesting to have a "datashader axis", which has the capability to datashade any plot plotted into it. It could theoretically do this by just consuming the atomic plot elements (lines + linesegments + scatter) and rasterizing them. Surfacelikes could potentially be disallowed/ignored.

This might be useful to visualize densely connected graph clusters and similar strategies. We could then also

[jkrumbiegel]

this file is huge, should be mp4

[jkrumbiegel]

(and you can also embed videos such that they auto-play and auto-loop)

[jkrumbiegel]

same here

[asinghvi17]

IMO the colorrange calculation should be slightly changed here, so that we don't have to fiddle with colormaps. Let the colorrange be (1, max) instead of (0, max), then set lowclip = :transparent, or :black, or what have you. This should also make the colormap a lot more exact!

[github-actions]

Missing reference images

Found 1 new images without existing references.
Upload new reference images before merging this PR.

[github-actions]

Missing reference images

Found 1 new images without existing references.
Upload new reference images before merging this PR.

[github-actions]

Missing reference images

Found 1 new images without existing references.
Upload new reference images before merging this PR.

[SimonDanisch]

@jkrumbiegel I'm going to merge this when this goes green... It's still not perfect, but I cant invest much more time into this.
I added a warning, that its still experimental!

[jkrumbiegel]

Yeah I'm ok with that. Not much more movement on it, maybe once people can actually use it.

[jkrumbiegel]

Should this work together with the new CategoricalColormap?

[SimonDanisch]

Should this work together with the new CategoricalColormap?

For now, I only implemented proper legend support for categorical datashader:

And normal colorbar for non categorical datashader:

[jkrumbiegel]

What's the legend element with the gradient? Did you add a new legend element kind?

[SimonDanisch]

It's a poly marker with per vertex color ;)

[jkrumbiegel]

It's a poly marker with per vertex color ;)

While it's kind of cool, I think I'd still change those to normal solid colors. It's the color that tells you the category, not the alpha.