Remove adjoints for LowerTriangular and UpperTriangular

[devmotion]

There are two reasons for this PR:

• These are defined in ChainRules
• Tests are broken on nightly (and apparently also on Julia 1.8-rc1, as observed upstream) that point to a problem in the backward pass for UpperTriangular, so I'd like to check if it helps to remove the rules from Zygote

There are more definitions that probably could be removed in favour of ChainRules but I wanted to keep the PR simple.

[ToucheSir]

It seems the CR version is defined differently from the one here. Perhaps that's the source of the gradtest failures?

[devmotion]

I had another look at the logs, and actually the problem seems to start earlier: the pullback for cholesky in ChainRules is not picked up it seems (https://github.com/FluxML/Zygote.jl/runs/7124557589?check_suite_focus=true#step:6:1540). I suspect this would be fixed by JuliaDiff/ChainRules.jl#633.

[simsurace]

Can we re-run the CI in order to test the hypothesis that JuliaDiff/ChainRules.jl#633 fixed this? Locally, the cholesky tests pass on Julia 1.8.0-rc3, but there are plenty of other failures:

Stacktrace

(Zygote) pkg> test
    Updating registry at `~/.julia/registries/General.toml`
    Updating `~/.julia/dev/Zygote.jl/Project.toml`
  [621f4979] + AbstractFFTs v1.2.1
  [082447d4] + ChainRules v1.43.1
  [d360d2e6] + ChainRulesCore v1.15.3
  [b552c78f] + DiffRules v1.11.0
  [1a297f60] + FillArrays v0.13.2
  [f6369f11] + ForwardDiff v0.10.32
  [7869d1d1] + IRTools v0.4.6
  [2ab3a3ac] + LogExpFunctions v0.3.17
  [1914dd2f] + MacroTools v0.5.9
  [77ba4419] + NaNMath v1.0.1
  [ae029012] + Requires v1.3.0
  [276daf66] + SpecialFunctions v2.1.7
  [700de1a5] + ZygoteRules v0.2.2
    Updating `~/.julia/dev/Zygote.jl/Manifest.toml`
  [621f4979] + AbstractFFTs v1.2.1
  [79e6a3ab] + Adapt v3.4.0
  [082447d4] + ChainRules v1.43.1
  [d360d2e6] + ChainRulesCore v1.15.3
  [9e997f8a] + ChangesOfVariables v0.1.4
  [bbf7d656] + CommonSubexpressions v0.3.0
  [34da2185] + Compat v4.1.0
  [9a962f9c] + DataAPI v1.10.0
  [e2d170a0] + DataValueInterfaces v1.0.0
  [163ba53b] + DiffResults v1.0.3
  [b552c78f] + DiffRules v1.11.0
  [ffbed154] + DocStringExtensions v0.9.1
  [1a297f60] + FillArrays v0.13.2
  [f6369f11] + ForwardDiff v0.10.32
  [46192b85] + GPUArraysCore v0.1.1
  [7869d1d1] + IRTools v0.4.6
  [3587e190] + InverseFunctions v0.1.7
  [92d709cd] + IrrationalConstants v0.1.1
  [82899510] + IteratorInterfaceExtensions v1.0.0
  [692b3bcd] + JLLWrappers v1.4.1
  [2ab3a3ac] + LogExpFunctions v0.3.17
  [1914dd2f] + MacroTools v0.5.9
  [77ba4419] + NaNMath v1.0.1
  [bac558e1] + OrderedCollections v1.4.1
  [21216c6a] + Preferences v1.3.0
  [c1ae055f] + RealDot v0.1.0
  [ae029012] + Requires v1.3.0
  [276daf66] + SpecialFunctions v2.1.7
  [90137ffa] + StaticArrays v1.5.2
  [1e83bf80] + StaticArraysCore v1.0.1
  [09ab397b] + StructArrays v0.6.11
  [3783bdb8] + TableTraits v1.0.1
  [bd369af6] + Tables v1.7.0
  [700de1a5] + ZygoteRules v0.2.2
  [efe28fd5] + OpenSpecFun_jll v0.5.5+0
  [0dad84c5] + ArgTools v1.1.1
  [56f22d72] + Artifacts
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [8ba89e20] + Distributed
  [f43a241f] + Downloads v1.6.0
  [7b1f6079] + FileWatching
  [b77e0a4c] + InteractiveUtils
  [b27032c2] + LibCURL v0.6.3
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [37e2e46d] + LinearAlgebra
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [ca575930] + NetworkOptions v1.2.0
  [44cfe95a] + Pkg v1.8.0
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [ea8e919c] + SHA v0.7.0
  [9e88b42a] + Serialization
  [6462fe0b] + Sockets
  [2f01184e] + SparseArrays
  [10745b16] + Statistics
  [fa267f1f] + TOML v1.0.0
  [a4e569a6] + Tar v1.10.0
  [8dfed614] + Test
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode
  [e66e0078] + CompilerSupportLibraries_jll v0.5.2+0
  [deac9b47] + LibCURL_jll v7.83.1+1
  [29816b5a] + LibSSH2_jll v1.10.2+0
  [c8ffd9c3] + MbedTLS_jll v2.28.0+0
  [14a3606d] + MozillaCACerts_jll v2022.2.1
  [4536629a] + OpenBLAS_jll v0.3.20+0
  [05823500] + OpenLibm_jll v0.8.1+0
  [83775a58] + Zlib_jll v1.2.12+3
  [8e850b90] + libblastrampoline_jll v5.1.1+0
  [8e850ede] + nghttp2_jll v1.47.0+0
  [3f19e933] + p7zip_jll v17.4.0+0
     Testing Zygote
      Status `/private/var/folders/hg/dtffhh5n2nd8q2zdclrv7x580000gn/T/jl_QQaYC1/Project.toml`
  [621f4979] AbstractFFTs v1.2.1
  [052768ef] CUDA v3.12.0
  [082447d4] ChainRules v1.43.1
  [d360d2e6] ChainRulesCore v1.15.3
  [cdddcdb0] ChainRulesTestUtils v1.9.3
  [b552c78f] DiffRules v1.11.0
  [b4f34e82] Distances v0.10.7
  [7a1cc6ca] FFTW v1.5.0
  [1a297f60] FillArrays v0.13.2
  [26cc04aa] FiniteDifferences v0.12.24
  [f6369f11] ForwardDiff v0.10.32
  [7869d1d1] IRTools v0.4.6
  [2ab3a3ac] LogExpFunctions v0.3.17
  [1914dd2f] MacroTools v0.5.9
  [77ba4419] NaNMath v1.0.1
  [ae029012] Requires v1.3.0
  [276daf66] SpecialFunctions v2.1.7
  [e88e6eb3] Zygote v0.6.41 `~/.julia/dev/Zygote.jl`
  [700de1a5] ZygoteRules v0.2.2
  [8ba89e20] Distributed `@stdlib/Distributed`
  [b77e0a4c] InteractiveUtils `@stdlib/InteractiveUtils`
  [37e2e46d] LinearAlgebra `@stdlib/LinearAlgebra`
  [9a3f8284] Random `@stdlib/Random`
  [2f01184e] SparseArrays `@stdlib/SparseArrays`
  [10745b16] Statistics `@stdlib/Statistics`
  [8dfed614] Test `@stdlib/Test`
      Status `/private/var/folders/hg/dtffhh5n2nd8q2zdclrv7x580000gn/T/jl_QQaYC1/Manifest.toml`
  [621f4979] AbstractFFTs v1.2.1
  [79e6a3ab] Adapt v3.4.0
  [ab4f0b2a] BFloat16s v0.2.0
  [fa961155] CEnum v0.4.2
  [052768ef] CUDA v3.12.0
  [082447d4] ChainRules v1.43.1
  [d360d2e6] ChainRulesCore v1.15.3
  [cdddcdb0] ChainRulesTestUtils v1.9.3
  [9e997f8a] ChangesOfVariables v0.1.4
  [bbf7d656] CommonSubexpressions v0.3.0
  [34da2185] Compat v4.1.0
  [9a962f9c] DataAPI v1.10.0
  [e2d170a0] DataValueInterfaces v1.0.0
  [163ba53b] DiffResults v1.0.3
  [b552c78f] DiffRules v1.11.0
  [b4f34e82] Distances v0.10.7
  [ffbed154] DocStringExtensions v0.9.1
  [e2ba6199] ExprTools v0.1.8
  [7a1cc6ca] FFTW v1.5.0
  [1a297f60] FillArrays v0.13.2
  [26cc04aa] FiniteDifferences v0.12.24
  [f6369f11] ForwardDiff v0.10.32
  [0c68f7d7] GPUArrays v8.4.2
  [46192b85] GPUArraysCore v0.1.1
  [61eb1bfa] GPUCompiler v0.16.2
  [7869d1d1] IRTools v0.4.6
  [3587e190] InverseFunctions v0.1.7
  [92d709cd] IrrationalConstants v0.1.1
  [82899510] IteratorInterfaceExtensions v1.0.0
  [692b3bcd] JLLWrappers v1.4.1
  [929cbde3] LLVM v4.14.0
  [2ab3a3ac] LogExpFunctions v0.3.17
  [1914dd2f] MacroTools v0.5.9
  [77ba4419] NaNMath v1.0.1
  [bac558e1] OrderedCollections v1.4.1
  [21216c6a] Preferences v1.3.0
  [74087812] Random123 v1.6.0
  [e6cf234a] RandomNumbers v1.5.3
  [c1ae055f] RealDot v0.1.0
  [189a3867] Reexport v1.2.2
  [ae029012] Requires v1.3.0
  [708f8203] Richardson v1.4.0
  [276daf66] SpecialFunctions v2.1.7
  [90137ffa] StaticArrays v1.5.2
  [1e83bf80] StaticArraysCore v1.0.1
  [82ae8749] StatsAPI v1.5.0
  [09ab397b] StructArrays v0.6.11
  [3783bdb8] TableTraits v1.0.1
  [bd369af6] Tables v1.7.0
  [a759f4b9] TimerOutputs v0.5.20
  [e88e6eb3] Zygote v0.6.41 `~/.julia/dev/Zygote.jl`
  [700de1a5] ZygoteRules v0.2.2
  [f5851436] FFTW_jll v3.3.10+0
  [1d5cc7b8] IntelOpenMP_jll v2018.0.3+2
  [dad2f222] LLVMExtra_jll v0.0.16+0
  [856f044c] MKL_jll v2022.0.0+0
  [efe28fd5] OpenSpecFun_jll v0.5.5+0
  [0dad84c5] ArgTools v1.1.1 `@stdlib/ArgTools`
  [56f22d72] Artifacts `@stdlib/Artifacts`
  [2a0f44e3] Base64 `@stdlib/Base64`
  [ade2ca70] Dates `@stdlib/Dates`
  [8ba89e20] Distributed `@stdlib/Distributed`
  [f43a241f] Downloads v1.6.0 `@stdlib/Downloads`
  [7b1f6079] FileWatching `@stdlib/FileWatching`
  [b77e0a4c] InteractiveUtils `@stdlib/InteractiveUtils`
  [4af54fe1] LazyArtifacts `@stdlib/LazyArtifacts`
  [b27032c2] LibCURL v0.6.3 `@stdlib/LibCURL`
  [76f85450] LibGit2 `@stdlib/LibGit2`
  [8f399da3] Libdl `@stdlib/Libdl`
  [37e2e46d] LinearAlgebra `@stdlib/LinearAlgebra`
  [56ddb016] Logging `@stdlib/Logging`
  [d6f4376e] Markdown `@stdlib/Markdown`
  [ca575930] NetworkOptions v1.2.0 `@stdlib/NetworkOptions`
  [44cfe95a] Pkg v1.8.0 `@stdlib/Pkg`
  [de0858da] Printf `@stdlib/Printf`
  [3fa0cd96] REPL `@stdlib/REPL`
  [9a3f8284] Random `@stdlib/Random`
  [ea8e919c] SHA v0.7.0 `@stdlib/SHA`
  [9e88b42a] Serialization `@stdlib/Serialization`
  [6462fe0b] Sockets `@stdlib/Sockets`
  [2f01184e] SparseArrays `@stdlib/SparseArrays`
  [10745b16] Statistics `@stdlib/Statistics`
  [fa267f1f] TOML v1.0.0 `@stdlib/TOML`
  [a4e569a6] Tar v1.10.0 `@stdlib/Tar`
  [8dfed614] Test `@stdlib/Test`
  [cf7118a7] UUIDs `@stdlib/UUIDs`
  [4ec0a83e] Unicode `@stdlib/Unicode`
  [e66e0078] CompilerSupportLibraries_jll v0.5.2+0 `@stdlib/CompilerSupportLibraries_jll`
  [deac9b47] LibCURL_jll v7.83.1+1 `@stdlib/LibCURL_jll`
  [29816b5a] LibSSH2_jll v1.10.2+0 `@stdlib/LibSSH2_jll`
  [c8ffd9c3] MbedTLS_jll v2.28.0+0 `@stdlib/MbedTLS_jll`
  [14a3606d] MozillaCACerts_jll v2022.2.1 `@stdlib/MozillaCACerts_jll`
  [4536629a] OpenBLAS_jll v0.3.20+0 `@stdlib/OpenBLAS_jll`
  [05823500] OpenLibm_jll v0.8.1+0 `@stdlib/OpenLibm_jll`
  [83775a58] Zlib_jll v1.2.12+3 `@stdlib/Zlib_jll`
  [8e850b90] libblastrampoline_jll v5.1.1+0 `@stdlib/libblastrampoline_jll`
  [8e850ede] nghttp2_jll v1.47.0+0 `@stdlib/nghttp2_jll`
  [3f19e933] p7zip_jll v17.4.0+0 `@stdlib/p7zip_jll`
Precompiling project...
  1 dependency successfully precompiled in 4 seconds. 63 already precompiled.
     Testing Running tests...
┌ Warning: CUDA not found - Skipping CUDA Tests
└ @ Main ~/.julia/dev/Zygote.jl/test/runtests.jl:14
┌ Warning: `ignore(f)` is deprecated, use `ChainRulesCore.ignore_derivatives(f)` instead.
│   caller = adjoint at deprecated.jl:31 [inlined]
└ @ Core ~/.julia/dev/Zygote.jl/src/deprecated.jl:31
┌ Warning: `ignore(f)` is deprecated, use `ChainRulesCore.ignore_derivatives(f)` instead.
│   caller = adjoint at deprecated.jl:31 [inlined]
└ @ Core ~/.julia/dev/Zygote.jl/src/deprecated.jl:31
┌ Warning: `ignore(f)` is deprecated, use `ChainRulesCore.ignore_derivatives(f)` instead.
│   caller = adjoint at deprecated.jl:31 [inlined]
└ @ Core ~/.julia/dev/Zygote.jl/src/deprecated.jl:31
┌ Warning: `ForwardDiff.gradient(f, x)` within Zygote cannot track gradients with respect to `f`,
│ and `f` appears to be a closure, or a struct with fields (according to `issingletontype(typeof(f))`).
│ typeof(f) = var"#126#145"{Vector{Float64}, var"#f1218#143"}
└ @ Zygote ~/.julia/dev/Zygote.jl/src/lib/forward.jl:142
[ Info: features.jl done
nested AD hitting identity(::Tuple) pullback: Error During Test at /Users/simone/.julia/dev/Zygote.jl/test/chainrules.jl:167
  Test threw exception
  Expression: begin
    (a3, pb3) = Zygote.pullback(h, 1)
    ((1,),) == pb3(1)
end
  TypeError: in typeassert, expected Tuple{T, T, T} where T, got a value of type Tuple{Int64, NoTangent, NoTangent}
  Stacktrace:
    [1] _totuple
      @ ./tuple.jl:333 [inlined]
    [2] (NTuple{4, T} where T)(itr::Vector{Any})
      @ Base ./tuple.jl:317
    [3] unbroadcast(x::Tuple{Nothing, Int64, Nothing, Nothing}, dx::Vector{Any})
      @ ChainRules ~/.julia/packages/ChainRules/HByMw/src/rulesets/Base/broadcast.jl:337
    [4] map
      @ ./tuple.jl:247 [inlined]
    [5] map(f::typeof(ChainRules.unbroadcast), t::Tuple{Tuple{Nothing, Nothing, Nothing, Float64}, Tuple{Nothing, Nothing, Float64, Nothing}, Tuple{Nothing, Int64, Nothing, Nothing}}, s::Tuple{Vector{NoTangent}, Vector{NoTangent}, Vector{Any}})
      @ Base ./tuple.jl:250
    [6] (::ChainRules.var"#back_generic#1684"{typeof(Zygote.accum), Tuple{Zygote.var"#ad_pullback#50"{Tuple{typeof(Zygote.accum), Nothing, Nothing, Nothing}, typeof(∂(accum))}, Zygote.var"#ad_pullback#50"{Tuple{typeof(Zygote.accum), Nothing, Nothing, Int64}, typeof(∂(accum))}, Zygote.var"#ad_pullback#50"{Tuple{typeof(Zygote.accum), Nothing, Float64, Nothing}, typeof(∂(accum))}, Zygote.var"#ad_pullback#50"{Tuple{typeof(Zygote.accum), Float64, Nothing, Nothing}, typeof(∂(accum))}}, Tuple{Tuple{Nothing, Nothing, Nothing, Float64}, Tuple{Nothing, Nothing, Float64, Nothing}, Tuple{Nothing, Int64, Nothing, Nothing}}})(dys::Tangent{Any, Tuple{ZeroTangent, Int64, ZeroTangent, ZeroTangent}})
      @ ChainRules ~/.julia/packages/ChainRules/HByMw/src/rulesets/Base/broadcast.jl:134
    [7] ZBack
      @ ~/.julia/dev/Zygote.jl/src/compiler/chainrules.jl:205 [inlined]
    [8] (::Zygote.var"#216#217"{Tuple{NTuple{4, Nothing}, Tuple{Nothing}}, Zygote.ZBack{ChainRules.var"#back_generic#1684"{typeof(Zygote.accum), Tuple{Zygote.var"#ad_pullback#50"{Tuple{typeof(Zygote.accum), Nothing, Nothing, Nothing}, typeof(∂(accum))}, Zygote.var"#ad_pullback#50"{Tuple{typeof(Zygote.accum), Nothing, Nothing, Int64}, typeof(∂(accum))}, Zygote.var"#ad_pullback#50"{Tuple{typeof(Zygote.accum), Nothing, Float64, Nothing}, typeof(∂(accum))}, Zygote.var"#ad_pullback#50"{Tuple{typeof(Zygote.accum), Float64, Nothing, Nothing}, typeof(∂(accum))}}, Tuple{Tuple{Nothing, Nothing, Nothing, Float64}, Tuple{Nothing, Nothing, Float64, Nothing}, Tuple{Nothing, Int64, Nothing, Nothing}}}}})(Δ::Tuple{Nothing, Int64, Nothing, Nothing})
      @ Zygote ~/.julia/dev/Zygote.jl/src/lib/lib.jl:207
    [9] (::Zygote.var"#1909#back#218"{Zygote.var"#216#217"{Tuple{NTuple{4, Nothing}, Tuple{Nothing}}, Zygote.ZBack{ChainRules.var"#back_generic#1684"{typeof(Zygote.accum), Tuple{Zygote.var"#ad_pullback#50"{Tuple{typeof(Zygote.accum), Nothing, Nothing, Nothing}, typeof(∂(accum))}, Zygote.var"#ad_pullback#50"{Tuple{typeof(Zygote.accum), Nothing, Nothing, Int64}, typeof(∂(accum))}, Zygote.var"#ad_pullback#50"{Tuple{typeof(Zygote.accum), Nothing, Float64, Nothing}, typeof(∂(accum))}, Zygote.var"#ad_pullback#50"{Tuple{typeof(Zygote.accum), Float64, Nothing, Nothing}, typeof(∂(accum))}}, Tuple{Tuple{Nothing, Nothing, Nothing, Float64}, Tuple{Nothing, Nothing, Float64, Nothing}, Tuple{Nothing, Int64, Nothing, Nothing}}}}}})(Δ::Tuple{Nothing, Int64, Nothing, Nothing})
      @ Zygote ~/.julia/packages/ZygoteRules/AIbCs/src/adjoint.jl:67
   [10] Pullback
      @ ./broadcast.jl:1304 [inlined]
   [11] (::Zygote.var"#216#217"{Tuple{Tuple{Nothing, Nothing}, Tuple{Nothing, Nothing}}, typeof(∂(broadcasted))})(Δ::Tuple{Nothing, Int64, Nothing, Nothing})
      @ Zygote ~/.julia/dev/Zygote.jl/src/lib/lib.jl:207
   [12] #1909#back
      @ ~/.julia/packages/ZygoteRules/AIbCs/src/adjoint.jl:67 [inlined]
   [13] Pullback
      @ ~/.julia/dev/Zygote.jl/src/lib/lib.jl:24 [inlined]
   [14] (::typeof(∂(accum)))(Δ::Tuple{Nothing, Int64, Nothing, Nothing})
      @ Zygote ~/.julia/dev/Zygote.jl/src/compiler/interface2.jl:0
   [15] Pullback
      @ ~/.julia/dev/Zygote.jl/test/chainrules.jl:150 [inlined]
   [16] (::typeof(∂(λ)))(Δ::Tuple{Nothing, Tuple{Int64, Nothing, Nothing}})
      @ Zygote ~/.julia/dev/Zygote.jl/src/compiler/interface2.jl:0
   [17] Pullback
      @ ~/.julia/dev/Zygote.jl/src/compiler/interface.jl:41 [inlined]
   [18] (::typeof(∂(λ)))(Δ::Tuple{Nothing, Tuple{Int64, Nothing, Nothing}})
      @ Zygote ~/.julia/dev/Zygote.jl/src/compiler/interface2.jl:0
   [19] Pullback
      @ ~/.julia/dev/Zygote.jl/test/chainrules.jl:154 [inlined]
   [20] (::typeof(∂(λ)))(Δ::Tuple{Nothing, Int64})
      @ Zygote ~/.julia/dev/Zygote.jl/src/compiler/interface2.jl:0
   [21] Pullback
      @ ~/.julia/dev/Zygote.jl/src/compiler/interface.jl:41 [inlined]
   [22] (::typeof(∂(λ)))(Δ::Int64)
      @ Zygote ~/.julia/dev/Zygote.jl/src/compiler/interface2.jl:0
   [23] Pullback
      @ ~/.julia/dev/Zygote.jl/test/chainrules.jl:161 [inlined]
   [24] (::typeof(∂(h)))(Δ::Int64)
      @ Zygote ~/.julia/dev/Zygote.jl/src/compiler/interface2.jl:0
   [25] (::Zygote.var"#60#61"{typeof(∂(h))})(Δ::Int64)
      @ Zygote ~/.julia/dev/Zygote.jl/src/compiler/interface.jl:41
   [26] macro expansion
      @ ~/.julia/juliaup/julia-1.8.0-rc3+0.aarch64/share/julia/stdlib/v1.8/Test/src/Test.jl:464 [inlined]
   [27] macro expansion
      @ ~/.julia/dev/Zygote.jl/test/chainrules.jl:167 [inlined]
   [28] macro expansion
      @ ~/.julia/juliaup/julia-1.8.0-rc3+0.aarch64/share/julia/stdlib/v1.8/Test/src/Test.jl:1357 [inlined]
   [29] macro expansion
      @ ~/.julia/dev/Zygote.jl/test/chainrules.jl:150 [inlined]
   [30] macro expansion
      @ ~/.julia/juliaup/julia-1.8.0-rc3+0.aarch64/share/julia/stdlib/v1.8/Test/src/Test.jl:1357 [inlined]
   [31] top-level scope
      @ ~/.julia/dev/Zygote.jl/test/chainrules.jl:5
[ Info: chainrules.jl done
The following printout is from testing that `print` doesn't upset gradients:
[5.0][5.0][5.0][5.0]
[5.0]
Array{Float64, (1,)}[5]Array{Float64, (1,)}[5]Array{Float64, (1,)}[5]
[4.999999992549419][4.999999992549419][4.999999992549419][4.999999992549419]
[4.999999992549419]
Array{Float64, (1,)}[5]Array{Float64, (1,)}[5]Array{Float64, (1,)}[5]
[5.000000007450581][5.000000007450581][5.000000007450581][5.000000007450581]
[5.000000007450581]
Array{Float64, (1,)}[5]Array{Float64, (1,)}[5]Array{Float64, (1,)}[5]
backsolve: Test Failed at /Users/simone/.julia/dev/Zygote.jl/test/gradcheck.jl:603
  Expression: gradtest(((L, Y)->begin
            LowerTriangular(L) \ Y
        end), L, Y)
Stacktrace:
 [1] macro expansion
   @ ~/.julia/juliaup/julia-1.8.0-rc3+0.aarch64/share/julia/stdlib/v1.8/Test/src/Test.jl:464 [inlined]
 [2] macro expansion
   @ ~/.julia/dev/Zygote.jl/test/gradcheck.jl:603 [inlined]
 [3] macro expansion
   @ ~/.julia/juliaup/julia-1.8.0-rc3+0.aarch64/share/julia/stdlib/v1.8/Test/src/Test.jl:1357 [inlined]
 [4] top-level scope
   @ ~/.julia/dev/Zygote.jl/test/gradcheck.jl:575
backsolve: Test Failed at /Users/simone/.julia/dev/Zygote.jl/test/gradcheck.jl:604
  Expression: gradtest(((L, Y)->begin
            LowerTriangular(L) \ Y
        end), L, y)
Stacktrace:
 [1] macro expansion
   @ ~/.julia/juliaup/julia-1.8.0-rc3+0.aarch64/share/julia/stdlib/v1.8/Test/src/Test.jl:464 [inlined]
 [2] macro expansion
   @ ~/.julia/dev/Zygote.jl/test/gradcheck.jl:604 [inlined]
 [3] macro expansion
   @ ~/.julia/juliaup/julia-1.8.0-rc3+0.aarch64/share/julia/stdlib/v1.8/Test/src/Test.jl:1357 [inlined]
 [4] top-level scope
   @ ~/.julia/dev/Zygote.jl/test/gradcheck.jl:575
backsolve: Test Failed at /Users/simone/.julia/dev/Zygote.jl/test/gradcheck.jl:609
  Expression: gradtest(((U, Y)->begin
            UpperTriangular(U) \ Y
        end), U, Y)
Stacktrace:
 [1] macro expansion
   @ ~/.julia/juliaup/julia-1.8.0-rc3+0.aarch64/share/julia/stdlib/v1.8/Test/src/Test.jl:464 [inlined]
 [2] macro expansion
   @ ~/.julia/dev/Zygote.jl/test/gradcheck.jl:609 [inlined]
 [3] macro expansion
   @ ~/.julia/juliaup/julia-1.8.0-rc3+0.aarch64/share/julia/stdlib/v1.8/Test/src/Test.jl:1357 [inlined]
 [4] top-level scope
   @ ~/.julia/dev/Zygote.jl/test/gradcheck.jl:575
backsolve: Test Failed at /Users/simone/.julia/dev/Zygote.jl/test/gradcheck.jl:610
  Expression: gradtest(((U, Y)->begin
            UpperTriangular(U) \ Y
        end), U, y)
Stacktrace:
 [1] macro expansion
   @ ~/.julia/juliaup/julia-1.8.0-rc3+0.aarch64/share/julia/stdlib/v1.8/Test/src/Test.jl:464 [inlined]
 [2] macro expansion
   @ ~/.julia/dev/Zygote.jl/test/gradcheck.jl:610 [inlined]
 [3] macro expansion
   @ ~/.julia/juliaup/julia-1.8.0-rc3+0.aarch64/share/julia/stdlib/v1.8/Test/src/Test.jl:1357 [inlined]
 [4] top-level scope
   @ ~/.julia/dev/Zygote.jl/test/gradcheck.jl:575
backsolve: Test Failed at /Users/simone/.julia/dev/Zygote.jl/test/gradcheck.jl:631
  Expression: gradtest(((L, Y)->begin
            Y' / LowerTriangular(L)
        end), L, Y)
Stacktrace:
 [1] macro expansion
   @ ~/.julia/juliaup/julia-1.8.0-rc3+0.aarch64/share/julia/stdlib/v1.8/Test/src/Test.jl:464 [inlined]
 [2] macro expansion
   @ ~/.julia/dev/Zygote.jl/test/gradcheck.jl:631 [inlined]
 [3] macro expansion
   @ ~/.julia/juliaup/julia-1.8.0-rc3+0.aarch64/share/julia/stdlib/v1.8/Test/src/Test.jl:1357 [inlined]
 [4] top-level scope
   @ ~/.julia/dev/Zygote.jl/test/gradcheck.jl:575
backsolve: Test Failed at /Users/simone/.julia/dev/Zygote.jl/test/gradcheck.jl:632
  Expression: gradtest(((L, Y)->begin
            Y' / LowerTriangular(L)
        end), L, y)
Stacktrace:
 [1] macro expansion
   @ ~/.julia/juliaup/julia-1.8.0-rc3+0.aarch64/share/julia/stdlib/v1.8/Test/src/Test.jl:464 [inlined]
 [2] macro expansion
   @ ~/.julia/dev/Zygote.jl/test/gradcheck.jl:632 [inlined]
 [3] macro expansion
   @ ~/.julia/juliaup/julia-1.8.0-rc3+0.aarch64/share/julia/stdlib/v1.8/Test/src/Test.jl:1357 [inlined]
 [4] top-level scope
   @ ~/.julia/dev/Zygote.jl/test/gradcheck.jl:575
backsolve: Test Failed at /Users/simone/.julia/dev/Zygote.jl/test/gradcheck.jl:637
  Expression: gradtest(((U, Y)->begin
            Y' / UpperTriangular(U)
        end), U, Y)
Stacktrace:
 [1] macro expansion
   @ ~/.julia/juliaup/julia-1.8.0-rc3+0.aarch64/share/julia/stdlib/v1.8/Test/src/Test.jl:464 [inlined]
 [2] macro expansion
   @ ~/.julia/dev/Zygote.jl/test/gradcheck.jl:637 [inlined]
 [3] macro expansion
   @ ~/.julia/juliaup/julia-1.8.0-rc3+0.aarch64/share/julia/stdlib/v1.8/Test/src/Test.jl:1357 [inlined]
 [4] top-level scope
   @ ~/.julia/dev/Zygote.jl/test/gradcheck.jl:575
backsolve: Test Failed at /Users/simone/.julia/dev/Zygote.jl/test/gradcheck.jl:638
  Expression: gradtest(((U, Y)->begin
            Y' / UpperTriangular(U)
        end), U, y)
Stacktrace:
 [1] macro expansion
   @ ~/.julia/juliaup/julia-1.8.0-rc3+0.aarch64/share/julia/stdlib/v1.8/Test/src/Test.jl:464 [inlined]
 [2] macro expansion
   @ ~/.julia/dev/Zygote.jl/test/gradcheck.jl:638 [inlined]
 [3] macro expansion
   @ ~/.julia/juliaup/julia-1.8.0-rc3+0.aarch64/share/julia/stdlib/v1.8/Test/src/Test.jl:1357 [inlined]
 [4] top-level scope
   @ ~/.julia/dev/Zygote.jl/test/gradcheck.jl:575
Test Summary:                                       | Pass  Fail  Error  Broken  Total     Time
all                                                 | 8670     8      1      19   8698  8m37.1s
  deprecated.jl                                     |    6                           6     7.9s
  Interface                                         |   79                          79    24.0s
  Tools                                             |   12                          12     1.9s
  Utils                                             |   54                    4     58    35.2s
  lib                                               |   28                          28     8.3s
  Features                                          |  204                    8    212  1m41.9s
  Forward                                           |   10                          10     9.2s
  Data Structures                                   |   13                          13     2.5s
  ChainRules                                        |  218            1            219  1m40.0s
    ChainRules integration                          |   48            1             49  1m29.1s
      ChainRules basics                             |   14                          14     0.1s
      Multiple output single input                  |    3                           3     0.1s
      multiple input, Single output                 |    3                           3     0.1s
      multiple input multiple output                |    6                           6     0.1s
      all AbstractZero partials                     |    1                           1     0.0s
      nested AD hitting identity(::Tuple) pullback  |    2            1              3  1m27.5s
      kwargs                                        |    6                           6     0.2s
      kwarg, with all AbstractZero partials         |    2                           2     0.2s
      Type only rrule                               |    1                           1     0.0s
      @opt_out                                      |    6                           6     0.3s
      NotImplemented                                |    4                           4     0.4s
    ChainRulesCore.rrule_via_ad                     |  159                         159     9.6s
    FastMath support                                |    2                           2     0.2s
    zygote2differential inference                   |    9                           9     0.0s
  Gradients                                         | 7903     8              6   7917  3m33.7s
    println, show, string, etc                      |    2                           2     0.6s
    power                                           |    9                           9     0.5s
    sum, prod, cumsum                               |   22                          22     6.8s
    getindex                                        |   14                          14     2.6s
    view                                            |    4                           4     1.2s
    eachcol                                         |    6                    2      8     4.1s
    collect                                         |    4                           4     0.5s
    fill                                            |    5                           5     1.2s
    circshift                                       |   25                          25     1.3s
    dot                                             |    3                           3     0.4s
    map                                             |    4                           4     1.9s
    Tuple adjoint                                   |    1                           1     0.3s
    empty tuples                                    |    3                           3     0.0s
    Vector{Nothing} cotangent                       |    4                           4     0.5s
    pmap                                            |    4                           4     4.7s
    Tuple adjoint                                   |    1                           1     0.0s
    empty tuples                                    |    3                           3     0.0s
    Vector{Nothing} cotangent                       |    4                           4     0.0s
    map inference                                   |                             None     0.2s
    map and tuples                                  |    7                           7     0.8s
    Alternative Pmap Dispatch                       |    4                           4     3.6s
    Stateful Map                                    |    1                    1      2     1.0s
    vararg map                                      |    8                           8     0.9s
    sort                                            |   13                          13     0.5s
    filter                                          |    3                           3     0.5s
    mean                                            |    5                           5     1.1s
    var                                             |   20                          20     3.2s
    std                                             |   20                          20     2.5s
    maximum                                         |    7                           7     6.9s
    minimum                                         |    5                           5     0.8s
    dropdims                                        |    3                           3     0.8s
    real(::AbstractArray)                           |    6                           6     0.2s
    conj(::AbstractArray)                           |    6                           6     0.2s
    imag(::AbstractArray)                           |    6                           6     0.4s
    (p)inv                                          |    4                           4     0.7s
    multiplication                                  |    8                           8     1.0s
    backsolve                                       |   43     8                    51     9.1s
      Cholesky                                      |    7                           7     2.3s
    Symmetric                                       |   40                          40     3.8s
    Hermitian                                       |   30                          30     1.8s
    diag                                            |    1                           1     0.1s
    Diagonal                                        |    3                           3     0.6s
    dense + UniformScaling                          |    3                           3     0.5s
    cholesky                                        |   14                          14     3.6s
    lyap                                            |    6                           6     0.2s
    matrix exponential                              |   21                          21     4.8s
    eigen(::RealHermSymComplexHerm)                 |    9                    3     12     4.2s
    eigvals(::RealHermSymComplexHerm)               |    6                           6     0.9s
    Hermitian/Symmetric power series functions      |  549                         549    34.1s
    ^(::Union{Symmetric,Hermitian}, p::Integer)     |   84                          84     4.2s
    distances                                       |   40                          40    12.2s
    vcat                                            |   60                          60     1.8s
    hcat                                            |   51                          51     1.0s
    hvcat                                           |    5                           5     0.4s
    cat(..., dims = 1)                              |    3                           3     1.1s
    cat(..., dims = 2)                              |    3                           3     0.2s
    cat(..., dims = 3)                              |    3                           3     0.3s
    cat(..., dims = 4)                              |    3                           3     1.1s
    cat(..., dims = 5)                              |    3                           3     1.4s
    cat(..., dims = Val(1))                         |    3                           3     0.6s
    cat(..., dims = Val(2))                         |    3                           3     0.3s
    cat(..., dims = Val(3))                         |    3                           3     0.3s
    cat(..., dims = Val(4))                         |    3                           3     0.3s
    cat(..., dims = Val(5))                         |    3                           3     0.3s
    cat empty                                       |    1                           1     0.4s
    one(s) and zero(s)                              |    4                           4     0.1s
    fma and muladd                                  |    2                           2     0.1s
    xlogx                                           |    3                           3     0.6s
    xlogy                                           |    3                           3     0.5s
    logistic                                        |    7                           7     0.3s
    logit                                           |    5                           5     0.2s
    log1pexp                                        |   16                          16     0.5s
    log1psq                                         |    9                           9     0.0s
    logaddexp                                       |    5                           5     0.8s
    logsubexp                                       |    4                           4     0.5s
    logsumexp                                       |    5                           5     1.1s
    * sizing                                        |    2                           2     0.5s
    broadcast                                       |   13                          13     7.9s
    Buffer                                          |   19                          19     2.4s
    AbstractArray Addition / Subtraction / Negation |    3                           3     0.5s
    AbstractFFTs                                    | 6050                        6050     9.8s
    FillArrays                                      |  325                         325     5.3s
    @nograd                                         |    4                           4     0.5s
    fastmath                                        |    4                           4     0.5s
    UniformScaling to Matrix                        |    9                           9     0.5s
    random                                          |   25                          25     1.5s
    broadcasted(+, Array, Bool)                     |   24                          24     2.2s
    broadcasted(-, Array, Bool)                     |   24                          24     1.2s
    broadcasted(*, Array, Bool)                     |   24                          24     2.3s
    norm                                            |    4                           4     2.1s
    zip & Iterators.product                         |    7                           7     6.1s
    Unused comprehension                            |    9                           9     1.1s
    Fix1 and Fix2                                   |    2                           2     0.9s
    Zygote 660                                      |    1                           1     0.5s
    CRC issue 440                                   |    2                           2     2.1s
    CR issue 537                                    |    6                           6     3.7s
    Zygote #1162                                    |    1                           1     1.6s
    Zygote #1184                                    |    1                           1     0.6s
  Complex                                           |  101                         101     9.3s
  Compiler                                          |   42                    1     43     3.1s
ERROR: LoadError: Some tests did not pass: 8670 passed, 8 failed, 1 errored, 19 broken.
in expression starting at /Users/simone/.julia/dev/Zygote.jl/test/runtests.jl:6
ERROR: Package Zygote errored during testing

[devmotion]

Let's see what CI tests tell us. At least it seems there's a clear improvement locally, before 20 tests threw an error whereas it seems now only 8 fail and 1 errors.