# What are some good syntaxes for tensor initialisation?

Inspired by this question - What are the disadvantages of being able to define a matrix without typing any words?

There is a more general question of what are some syntaxes for initialising tensors and where are their strengths and weaknesses.

There are some obvious cases to cover. Illustrated by pseudo-code below:

An array:

array[int] = {1,2,3};


A matrix:

matrix[int][int] = {{1,2,3},
{4,5,6},
{7,8,9}};


A rank3 tensor:

tensor3d[int][int][int] = {
{{1,2,3},
{4,5,6},
{7,8,9}},
{{10,11,12},
{13,14,15},
{16,17,18}},
{{19,20,21},
{22,23,24},
{23,25,27}},
};


You need to be able to specify:

• The rank - number of dimnensions
• The shape - the number of elements in each dimension
• The values - values for each item
• The types of values contained

You might want to specify the shape in advance or it might be implied by your initialisation. You might have a syntax allowing the shape to be inferred or requiring it to agree with the initialisation.

Things get hairier still if we think about sparse matrices (assume all elements are zero except those specified), triangular matrices or matrices where there is some symmetry.

You might also want to support populating them alogrithmically by defining a function e.g.

tensor[1:int, 2:int, 3:int] = f(x,y,z) -> int { //calc value for pos x,yz...  };


for a 1 by 2 by 3 tensor.

There is quite a lot of scope for variation and verbosity here.

What is the prior art here and where do existing approaches succeed or fail?

• Surely, you don't need to be able to specify the rank if you can specify the shape, since the rank is the length of the shape.
May 30, 2023 at 11:45
• Funny you should ask this. I've asked for permission (but not gotten an answer) to ask for feedback on my proposal for such a notation.
May 30, 2023 at 11:48
• Additional things to consider is how you'll represent a 0 by 2 tensor and other empty tensors.
May 30, 2023 at 11:49
• I've voted yes to your meta. Also you don't need to ask permission to post. The community will vote to close if your question is bad (which I suspect won't be the case). May 30, 2023 at 12:06
• Re the shape. You might specify it in advance or it might be implied by your initialisation. You might have a syntax allowing you to omit one or requiring them to agree. May 30, 2023 at 12:09

I think you actually need a compile-time DSL, and a general-purpose programming language will never be more concise than a domain specific language.

A directional problem is that it is very bad to write tensor data in the code, and the code should focus on logic rather than data.

The best approach is something like:

let t333 = Tensor::<(3, 3, 3), f64>::parse!(include_str!("data.tensor"))


Roughly equivalent to rust that supports method macros.

I will design the DSL into two types: dense and sparse.

• dense and static
let magic3 = Tensor::<(3, 3, 3), f64>::parse!("
// 9 elements per section, if the section elements are not enough, fill with 0
@dense(3, 3, 3, default: 0)

@view(1, _, _)
8  15 19
24 1  17
10 26 6

@view(2, _, _)
12 25 5
7  14 21
23 3  16

@view(3, _, _)
22 2  18
11 27 4
9  13 20
")

• It's stupid to write it twice inside and outside, maybe can omit outside one:
let magic2 = Tensor::<_, f64>::parse!("
@dense(3, 3)
// Read nine numbers directly, with or without line breaks
4 9 2 3 5 7 8 1 6
")

• sparse and dynamic
let mut magic3 = Tensor::<(3, 3, 3), f64>::parse!("
@sparse(3, 3, 3, default: 0)
// Also can support imaginary numbers, so avoid cut off using -
> 1, 1, 1: 8
> 1, 1, 2: 15
> 1, 1, 3: 19
.............
")
magic3[1, 2, 1] = ...


This DSL syntax can be simply compiled into a instruction machine.

For higher-dimensional data, read binary directly.

let t333 = Tensor::<(3, 3, 3), f64>::deserialize!(include_bytes!("data.tensor"))


At this time, textualization does not actually make any sense.

• A GPL is arguably partly a collection of DSLs. It is a good point that its generally more useful to parse of external data. Typically this is done at run time so a library solution is sufficient. As an aside, surprisingly, #embed has only very recently made it into C and doesn't include any parsing. Jun 30, 2023 at 12:12
• @BruceAdams Someone told me that the one that borrows the original language is called EDSL, and the one that parses strings is called DSL. I wonder if you assume that the tensor is a hypercube? If the support variable-length list in list, the starting point of the design will be different. Jun 30, 2023 at 12:27