diff --git a/dev/.documenter-siteinfo.json b/dev/.documenter-siteinfo.json index d7b1f65..167c368 100644 --- a/dev/.documenter-siteinfo.json +++ b/dev/.documenter-siteinfo.json @@ -1 +1 @@ -{"documenter":{"julia_version":"1.10.0","generation_timestamp":"2024-02-13T15:11:34","documenter_version":"1.2.1"}} \ No newline at end of file +{"documenter":{"julia_version":"1.10.0","generation_timestamp":"2024-02-13T15:13:26","documenter_version":"1.2.1"}} \ No newline at end of file diff --git a/dev/alternatives/index.html b/dev/alternatives/index.html index 456174c..7f4c013 100644 --- a/dev/alternatives/index.html +++ b/dev/alternatives/index.html @@ -1,2 +1,2 @@ -Alternatives · EinExprs.jl

Alternatives

EinExprs is deeply inspired by opt_einsum and cotengra. Actually most of the contraction path search and slicing algorithms have been rewritten from there. If you happen to be working in Python, you should definetely check out these libraries.

Differences with `opt_einsum`

Although the differences are minimal, any user coming from opt_einsum should be aware that:

  • The "optimal" contraction path solver in opt_einsum is known as Exhaustive in EinExprs.
  • The "random-greedy" contraction path solver in opt_einsum is the Greedy optimizer in EinExprs but with a random choose function.
  • When counting FLOPs, opt_einsum gives a value $\times 2$ higher than the EinExprs.flops counter.[1]

Although we believe there is no similar project in the Julia world, there are some overlapping libraries that may suit you if EinExprs doesn't fit your case.

  • 1We are not sure of the reason behind this mismatch or which package gives the correct answer, but since the factor remains constant, it should not affect when comparing contraction paths during the minimization step.
+Alternatives · EinExprs.jl

Alternatives

EinExprs is deeply inspired by opt_einsum and cotengra. Actually most of the contraction path search and slicing algorithms have been rewritten from there. If you happen to be working in Python, you should definetely check out these libraries.

Differences with `opt_einsum`

Although the differences are minimal, any user coming from opt_einsum should be aware that:

  • The "optimal" contraction path solver in opt_einsum is known as Exhaustive in EinExprs.
  • The "random-greedy" contraction path solver in opt_einsum is the Greedy optimizer in EinExprs but with a random choose function.
  • When counting FLOPs, opt_einsum gives a value $\times 2$ higher than the EinExprs.flops counter.[1]

Although we believe there is no similar project in the Julia world, there are some overlapping libraries that may suit you if EinExprs doesn't fit your case.

  • 1We are not sure of the reason behind this mismatch or which package gives the correct answer, but since the factor remains constant, it should not affect when comparing contraction paths during the minimization step.
diff --git a/dev/assets/favicon.ico b/dev/assets/favicon.ico index 0d1c49c..d5b1abd 100644 Binary files a/dev/assets/favicon.ico and b/dev/assets/favicon.ico differ diff --git a/dev/counters/index.html b/dev/counters/index.html index 41be029..746a097 100644 --- a/dev/counters/index.html +++ b/dev/counters/index.html @@ -1,2 +1,2 @@ -Resource counting · EinExprs.jl

Resource counting

As explained before, EinExprs are symbolic expressions representing Einstein summation equations (i.e. tensor summation, permutation, contraction, ...) so no tensor operation is actually performed on them. Many times, information about the execution cost is needed to optimize the contraction path. In general, this is a hard task but thanks to Einsteam summation notation only representing linear algebra operations, and these operations are easy to estimate, we can count the resource requirements of any contraction path.

Currently there are 3 resource counters:

EinExprs.flopsFunction
flops(path::EinExpr)

Count the number of mathematical operations will be performed by the contraction of the root of the path tree.

source
EinExprs.removedsizeFunction
removedsize(path::EinExpr)

Count the amount of memory that will be freed after performing the contraction of the root of the path tree.

source
EinExprs.removedrankFunction
removedrank(path::EinExpr)

Count the rank reduction after performing the contraction of the root of the path tree.

source
Tip

These methods only count the resources spent of the contraction on the root of the tree. In order to count the resources of the whole tree, use mapreduce with Branches:

mapreduce(flops, +, Branches(path))
+Resource counting · EinExprs.jl

Resource counting

As explained before, EinExprs are symbolic expressions representing Einstein summation equations (i.e. tensor summation, permutation, contraction, ...) so no tensor operation is actually performed on them. Many times, information about the execution cost is needed to optimize the contraction path. In general, this is a hard task but thanks to Einsteam summation notation only representing linear algebra operations, and these operations are easy to estimate, we can count the resource requirements of any contraction path.

Currently there are 3 resource counters:

EinExprs.flopsFunction
flops(path::EinExpr)

Count the number of mathematical operations will be performed by the contraction of the root of the path tree.

source
EinExprs.removedsizeFunction
removedsize(path::EinExpr)

Count the amount of memory that will be freed after performing the contraction of the root of the path tree.

source
EinExprs.removedrankFunction
removedrank(path::EinExpr)

Count the rank reduction after performing the contraction of the root of the path tree.

source
Tip

These methods only count the resources spent of the contraction on the root of the tree. In order to count the resources of the whole tree, use mapreduce with Branches:

mapreduce(flops, +, Branches(path))
diff --git a/dev/einexpr/index.html b/dev/einexpr/index.html index a85576e..0dd4952 100644 --- a/dev/einexpr/index.html +++ b/dev/einexpr/index.html @@ -2,4 +2,4 @@ Einsum Expressions · EinExprs.jl

Einsum Expressions

Let's take an arbitrary tensor network like the following.

An arbitrary tensor network

This graph is a graphical notation equivalent to the following equation.

\[\sum_{i j k l m n o p} A_{mi} B_{ijp} C_{jkn} D_{pkl} E_{mno} F_{ol}\]

A naïve implementation of this equation is easy to implement.

result = zero(reduce(promote_type, eltype.([A,B,C,D,E,F])))
 for (i,j,k,l,m,n,o,p) in Iterators.product(1:I, 1:J, 1:K, 1:L, 1:M, 1:N, 1:O, 1:P)
     result += A[m,i] * B[i,j,p] * C[j,k,n] * D[p,k,l] * E[m,n,o] * F[o,l]
-end

But it has a cost of $\prod_\alpha \dim(\alpha)$ where $\alpha \in \{i,j,k,l,m,n,o,p\}$ which is of $\mathcal{O}(\exp(n))$ time complexity.

Basic utilities

The EinExpr type has a stable definition: it's secure to access its fields using the dot-syntax (e.g. path.head), but we recommend to use the following methods instead. They provide a much more stable API and can be used in a functional manner.

EinExprs.headFunction
head(path::EinExpr)

Return the indices of the resulting tensor from contracting path.

See also: inds, args.

source
EinExprs.argsFunction
args(path::EinExpr)

Return the children of the path, which correspond to input tensors for the contraction step in the top of the path.

See also: head.

source
args(sexpr::SizedEinExpr)

Note

Unlike args(::EinExpr), this function returns SizedEinExpr objects.

source
Missing docstring.

Missing docstring for Base.size(::EinExpr). Check Documenter's build log for details.

Some other useful methods are:

Base.ndimsMethod
ndims(path::EinExpr)

Return the number of indices of the resulting tensor from contracting path.

source
EinExprs.indsFunction
inds(path)

Return all the involved indices in path. If a tensor is passed, then it is equivalent to calling head.

See also: head.

source
EinExprs.sumindsFunction
suminds(path)

Indices of summation of an EinExpr.

\[\mathtt{path} \equiv \sum_{j k l m n o p} A_{mi} B_{ijp} C_{jkn} D_{pkl} E_{mno} F_{ol}\]

suminds(path) == [:j, :k, :l, :m, :n, :o, :p]
source
EinExprs.parsumindsFunction
parsuminds(path)

Indices of summation of possible pairwise tensors contractions between children of path.

source
EinExprs.selectFunction
select(path::EinExpr, i)

Return the child elements that contain i indices.

source

Construction by summation

One option for constructing EinExprs manually is to use the sum methods. For example, imagine that we have the following tensor equation and we want to contract first tensors $E_{mno}$ and $F_{ol}$. The resulting equation would be equivalent to adding a summatory to $E$ and $F$ as written in the right-hand side.

\[\sum_{i j k l m n o p} A_{mi} B_{ijp} C_{jkn} D_{pkl} E_{mno} F_{ol} = \sum_{i j k l m n p} A_{mi} B_{ijp} C_{jkn} D_{pkl} \sum_o E_{mno} F_{ol}\]

In EinExprs, we advocate for code that it's almost as easy as writing math. As such, one can write sum([E, F]) to create a new EinExpr where common indices are contracted or sum!(path, :o) for the in-place version where $E$ and $F$ are children of path.

Missing docstring.

Missing docstring for Base.sum(::EinExpr, ::Union{Symbol, Tuple{Vararg{Symbol}}, AbstractVector{<:Symbol}}). Check Documenter's build log for details.

In order to reverse the operation and unfix the contraction, the user may call the collapse! function.

EinExprs.collapse!Function
collapse!(path::EinExpr)

Collapses all sub-branches, merging all tensor leaves into the args field.

source

AbstractTrees integration

EinExpr type integrates with the AbstractTrees package in order to implement some of the tree-traversing algorithms. The interface is public and thus any user can use it to implement their own methods.

For example, the AbstractTrees.Leaves function returns an iterator through the leaves of any tree; i.e. the initial tensors in our case. We implement the Branches function in order to walk through the non-terminal nodes; i.e. the intermediate tensors.

EinExprs exports a variant of these methods which return collections.

EinExprs.leavesFunction
leaves(path::EinExpr[, i])

Return the terminal leaves of the path, which correspond to the initial input tensors. If i is specified, then only return the $i$-th tensor.

See also: branches.

source
EinExprs.branchesFunction
branches(path::EinExpr[, i])

Return the non-terminal branches of the path, which correspond to intermediate tensors result of contraction steps. If i is specified, then only return the $i$-th EinExpr.

See also: leaves, Branches.

source
+end

But it has a cost of $\prod_\alpha \dim(\alpha)$ where $\alpha \in \{i,j,k,l,m,n,o,p\}$ which is of $\mathcal{O}(\exp(n))$ time complexity.

Basic utilities

The EinExpr type has a stable definition: it's secure to access its fields using the dot-syntax (e.g. path.head), but we recommend to use the following methods instead. They provide a much more stable API and can be used in a functional manner.

EinExprs.headFunction
head(path::EinExpr)

Return the indices of the resulting tensor from contracting path.

See also: inds, args.

source
EinExprs.argsFunction
args(path::EinExpr)

Return the children of the path, which correspond to input tensors for the contraction step in the top of the path.

See also: head.

source
args(sexpr::SizedEinExpr)

Note

Unlike args(::EinExpr), this function returns SizedEinExpr objects.

source
Missing docstring.

Missing docstring for Base.size(::EinExpr). Check Documenter's build log for details.

Some other useful methods are:

Base.ndimsMethod
ndims(path::EinExpr)

Return the number of indices of the resulting tensor from contracting path.

source
EinExprs.indsFunction
inds(path)

Return all the involved indices in path. If a tensor is passed, then it is equivalent to calling head.

See also: head.

source
EinExprs.sumindsFunction
suminds(path)

Indices of summation of an EinExpr.

\[\mathtt{path} \equiv \sum_{j k l m n o p} A_{mi} B_{ijp} C_{jkn} D_{pkl} E_{mno} F_{ol}\]

suminds(path) == [:j, :k, :l, :m, :n, :o, :p]
source
EinExprs.parsumindsFunction
parsuminds(path)

Indices of summation of possible pairwise tensors contractions between children of path.

source
EinExprs.contractorderFunction
contractorder(path::EinExpr)

Transform path into a contraction order.

source
EinExprs.selectFunction
select(path::EinExpr, i)

Return the child elements that contain i indices.

source
EinExprs.neighboursFunction
neighbours(path::EinExpr, i)

Return the indices neighbouring to i.

source

Construction by summation

One option for constructing EinExprs manually is to use the sum methods. For example, imagine that we have the following tensor equation and we want to contract first tensors $E_{mno}$ and $F_{ol}$. The resulting equation would be equivalent to adding a summatory to $E$ and $F$ as written in the right-hand side.

\[\sum_{i j k l m n o p} A_{mi} B_{ijp} C_{jkn} D_{pkl} E_{mno} F_{ol} = \sum_{i j k l m n p} A_{mi} B_{ijp} C_{jkn} D_{pkl} \sum_o E_{mno} F_{ol}\]

In EinExprs, we advocate for code that it's almost as easy as writing math. As such, one can write sum([E, F]) to create a new EinExpr where common indices are contracted or sum!(path, :o) for the in-place version where $E$ and $F$ are children of path.

Base.sum!Method
sum!(path, indices)

Explicit, in-place sum over indices.

See also: sum, suminds.

source
Missing docstring.

Missing docstring for Base.sum(::EinExpr, ::Union{Symbol, Tuple{Vararg{Symbol}}, AbstractVector{<:Symbol}}). Check Documenter's build log for details.

In order to reverse the operation and unfix the contraction, the user may call the collapse! function.

EinExprs.collapse!Function
collapse!(path::EinExpr)

Collapses all sub-branches, merging all tensor leaves into the args field.

source

AbstractTrees integration

EinExpr type integrates with the AbstractTrees package in order to implement some of the tree-traversing algorithms. The interface is public and thus any user can use it to implement their own methods.

For example, the AbstractTrees.Leaves function returns an iterator through the leaves of any tree; i.e. the initial tensors in our case. We implement the Branches function in order to walk through the non-terminal nodes; i.e. the intermediate tensors.

EinExprs.BranchesFunction
Branches(path::EinExpr)

Iterator that walks through the non-terminal nodes of the path tree.

See also: branches.

source

EinExprs exports a variant of these methods which return collections.

EinExprs.leavesFunction
leaves(path::EinExpr[, i])

Return the terminal leaves of the path, which correspond to the initial input tensors. If i is specified, then only return the $i$-th tensor.

See also: branches.

source
EinExprs.branchesFunction
branches(path::EinExpr[, i])

Return the non-terminal branches of the path, which correspond to intermediate tensors result of contraction steps. If i is specified, then only return the $i$-th EinExpr.

See also: leaves, Branches.

source
diff --git a/dev/index.html b/dev/index.html index deb93dc..a2868d3 100644 --- a/dev/index.html +++ b/dev/index.html @@ -1,3 +1,3 @@ Home · EinExprs.jl
-

EinExprs is a Julia package that provides EinExprs: symbolic expressions representing a Einstein summation. These summations may be used to represent contraction paths of large tensor networks.

It is a complete redesign of OptimizedEinsum, which indeed was a Julia fork of opt_einsum. It powers Tenet but can easily be adapted to work with other packages.

  1. Einsum Expressions
  2. Optimizers
  3. Resource counting
  4. Slicing
  5. Alternatives

Planned features

  • Optimizers
    • Hypergraph Partitioning
    • Branch & Bound
    • Dynamic Programming
  • Subtree reconfiguration
  • Resource counting
    • Array structure-aware analysis
  • Execution planning
  • Permutation order optimization
  • Compilation
+

EinExprs is a Julia package that provides EinExprs: symbolic expressions representing a Einstein summation. These summations may be used to represent contraction paths of large tensor networks.

It is a complete redesign of OptimizedEinsum, which indeed was a Julia fork of opt_einsum. It powers Tenet but can easily be adapted to work with other packages.

  1. Einsum Expressions
  2. Optimizers
  3. Resource counting
  4. Slicing
  5. Alternatives

Planned features

diff --git a/dev/optimizers/exhaustive/index.html b/dev/optimizers/exhaustive/index.html index a9aa8be..04b6abd 100644 --- a/dev/optimizers/exhaustive/index.html +++ b/dev/optimizers/exhaustive/index.html @@ -1,2 +1,2 @@ -Exhaustive · EinExprs.jl

Exhaustive optimizer

EinExprs.ExhaustiveType
Exhaustive(; outer = false)

Exhaustive contraction path optimizers. It guarantees to find the optimal contraction path but at a large cost.

Keywords

  • outer instructs to consider outer products (aka tensor products) on the search for the optimal contraction path. It rarely provides an advantage over only considering inner products and thus, it is false by default.
Warning

The functionality of outer = true has not been yet implemented.

Implementation

The algorithm has a $\mathcal{O}(n!)$ time complexity if outer = true and $\mathcal{O}(\exp(n))$ if outer = false.

source
+Exhaustive · EinExprs.jl

Exhaustive optimizer

EinExprs.ExhaustiveType
Exhaustive(; outer = false)

Exhaustive contraction path optimizers. It guarantees to find the optimal contraction path but at a large cost.

Keywords

  • outer instructs to consider outer products (aka tensor products) on the search for the optimal contraction path. It rarely provides an advantage over only considering inner products and thus, it is false by default.
Warning

The functionality of outer = true has not been yet implemented.

Implementation

The algorithm has a $\mathcal{O}(n!)$ time complexity if outer = true and $\mathcal{O}(\exp(n))$ if outer = false.

source
diff --git a/dev/optimizers/greedy/index.html b/dev/optimizers/greedy/index.html index 1786bf8..2cb5aac 100644 --- a/dev/optimizers/greedy/index.html +++ b/dev/optimizers/greedy/index.html @@ -1,2 +1,2 @@ -Greedy · EinExprs.jl

Greedy optimizer

EinExprs.GreedyType
Greedy(; metric = removedsize, choose = pop!)

Greedy contraction path solver. Greedily selects contractions that maximize a metric.

Keywords

  • metric is a function that evaluates candidate pairwise tensor contractions. Defaults to removedsize.
  • choose is a function that extracts a pairwise tensor contraction between candidates. Defaults to candidate that maximize metric using pop!.
  • outer If true, consider outer products as candidates. Defaults to false.

Implementation

The implementation uses a binary heaptree to sort candidate pairwise tensor contractions. Then recursively,

  1. Selects and extracts a candidate from the heaptree using the choose function.
  2. Updates the metric of the candidates which contain neighbouring indices to the one selected.
  3. Append the selected index to the path and go back to step 1.
source
+Greedy · EinExprs.jl

Greedy optimizer

EinExprs.GreedyType
Greedy(; metric = removedsize, choose = pop!)

Greedy contraction path solver. Greedily selects contractions that maximize a metric.

Keywords

  • metric is a function that evaluates candidate pairwise tensor contractions. Defaults to removedsize.
  • choose is a function that extracts a pairwise tensor contraction between candidates. Defaults to candidate that maximize metric using pop!.
  • outer If true, consider outer products as candidates. Defaults to false.

Implementation

The implementation uses a binary heaptree to sort candidate pairwise tensor contractions. Then recursively,

  1. Selects and extracts a candidate from the heaptree using the choose function.
  2. Updates the metric of the candidates which contain neighbouring indices to the one selected.
  3. Append the selected index to the path and go back to step 1.
source
diff --git a/dev/slicing/index.html b/dev/slicing/index.html index 92c3c0e..569b653 100644 --- a/dev/slicing/index.html +++ b/dev/slicing/index.html @@ -1,4 +1,4 @@ -Slicing · EinExprs.jl

Slicing

Base.selectdimFunction
selectdim(path::EinExpr, index, i)

Project index to dimension i in a EinExpr. This is equivalent to tensor cutting aka slicing.

Arguments

  • path Contraction path.
  • index Index to cut.
  • i Dimension of index to select.

See also: view.

source
Base.viewFunction
view(path::EinExpr, cuttings...)

Project indices in contraction path to some of its dimensions. This is equivalent to:

reduce(cuttings) do path, (index, i)
+Slicing · EinExprs.jl

Slicing

Base.selectdimFunction
selectdim(path::EinExpr, index, i)

Project index to dimension i in a EinExpr. This is equivalent to tensor cutting aka slicing.

Arguments

  • path Contraction path.
  • index Index to cut.
  • i Dimension of index to select.

See also: view.

source
Base.viewFunction
view(path::EinExpr, cuttings...)

Project indices in contraction path to some of its dimensions. This is equivalent to:

reduce(cuttings) do path, (index, i)
     selectdim(path, index, i)
-end

Arguments

  • path Target contraction path.
  • cuttings List of Pair{Symbol,Int} representing the tensor cuttings aka slices.

See also: selectdim.

source
EinExprs.findslicesFunction
findslices(scorer, path::EinExpr; size, slices, overhead, temperature = 0.01, skip = head(path))

Search for indices to be cut/sliced such that the conditions given by size, overhead and slices are fulfilled. Reimplementation based on contengra's SliceFinder algorithm.

Arguments

  • scorer Heuristic function (or functor) that accepts a path and a candidate index for cutting, and returns a score.
  • path The contraction path target for tensor cutting aka slicing.

Keyword Arguments

  • size If specified, the largest intermediate tensor of the slice won't surpass this size (in number of elements).
  • slices If specified, there will be at least slices different slices when cutting all returnt indices.
  • overhead If specified, the amount of redundant operations between a slice and the original contraction won't supass this ratio.
  • temperature Temperature of the Boltzmann-like noise added for diffusing results.
  • skip Indices not to be considered for slicing.
source
+end

Arguments

  • path Target contraction path.
  • cuttings List of Pair{Symbol,Int} representing the tensor cuttings aka slices.

See also: selectdim.

source
EinExprs.findslicesFunction
findslices(scorer, path::EinExpr; size, slices, overhead, temperature = 0.01, skip = head(path))

Search for indices to be cut/sliced such that the conditions given by size, overhead and slices are fulfilled. Reimplementation based on contengra's SliceFinder algorithm.

Arguments

  • scorer Heuristic function (or functor) that accepts a path and a candidate index for cutting, and returns a score.
  • path The contraction path target for tensor cutting aka slicing.

Keyword Arguments

  • size If specified, the largest intermediate tensor of the slice won't surpass this size (in number of elements).
  • slices If specified, there will be at least slices different slices when cutting all returnt indices.
  • overhead If specified, the amount of redundant operations between a slice and the original contraction won't supass this ratio.
  • temperature Temperature of the Boltzmann-like noise added for diffusing results.
  • skip Indices not to be considered for slicing.
source