Skip to content

Highlight tensor types and literals in the tensor reference #4941

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
sebastiannberg merged 1 commit into from
Oct 9, 2026
Merged

Highlight tensor types and literals in the tensor reference #4941

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
108 changes: 58 additions & 50 deletions en/reference/ranking/tensor.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,9 @@ <h2 id="tensor-type-spec">Tensor type spec</h2>
The dimensions of a tensor and the cell type defines its type.
A tensor type contains a list of dimensions on the format:
</p>
<pre>
tensor&lt;value-type&gt;(dimension-1,dimension-2,...,dimension-N)
</pre>
<pre>{% highlight vespa-ranking-expression %}
tensor<value-type>(dimension-1,dimension-2,...,dimension-N)
{% endhighlight %}</pre>
<p>The value-type is one of:</p>
<table class="table">
<thead>
Expand Down Expand Up @@ -65,39 +65,45 @@ <h2 id="tensor-type-spec">Tensor type spec</h2>
<li><code>dimension-name[size]</code> - an indexed dimension</li>
</ul>
<p>The tensor type for a tensor&lt;float&gt; with two mapped dimensions <em>x</em> and <em>y</em> looks like:</p>
<pre>
tensor&lt;float&gt;(x{},y{})
</pre>
<pre>{% highlight vespa-ranking-expression %}
tensor<float>(x{},y{})
{% endhighlight %}</pre>
<p>Example tensor with this type:</p>
<pre>{% raw %}
<pre>{% highlight vespa-ranking-expression %}
{% raw %}
{{x:a,y:b}:10.0, {x:c,y:d}:20.1}
{% endraw %}</pre>
{% endraw %}
{% endhighlight %}</pre>
<p>
The tensor type for a tensor&lt;float&gt; with two indexed dimensions <em>x</em> and <em>y</em>
with sizes 3 and 2 respectively looks like:
</p>
<pre>
tensor&lt;float&gt;(x[3],y[2])
</pre>
<pre>{% highlight vespa-ranking-expression %}
tensor<float>(x[3],y[2])
{% endhighlight %}</pre>
<p>Example tensor with this type (representing a matrix):</p>
<pre>{% raw %}
<pre>{% highlight vespa-ranking-expression %}
{% raw %}
{{x:0,y:0}:1, {x:0,y:1}:2.1,
{x:1,y:0}:3, {x:1,y:1}:5,
{x:2,y:0}:7, {x:2,y:1}:11}
{% endraw %}</pre>
{% endraw %}
{% endhighlight %}</pre>
<p>
Note that the labels are indexes in the range <em>[0,dimension-size&gt;</em>
</p><p>
A tensor&lt;double&gt; with both mapped and indexed dimensions is <em>mixed</em>:
</p>
<pre>
tensor&lt;double&gt;(key{},x[2])
</pre>
<pre>{% highlight vespa-ranking-expression %}
tensor<double>(key{},x[2])
{% endhighlight %}</pre>
<p>Example:</p>
<pre>{% raw %}
<pre>{% highlight vespa-ranking-expression %}
{% raw %}
{{key:a,x:0}:10, {key:b,x:0}:2.7,
{key:a,x:1}:5.3, {key:b,x:1}:-7 }
{% endraw %}</pre>
{% endraw %}
{% endhighlight %}</pre>



Expand Down Expand Up @@ -139,25 +145,25 @@ <h3 id="general-literal-form">General literal form</h3>

<h4 id="general-literal-form-examples">General literal form examples:</h4>
<p>An empty tensor:</p>
<pre>
<pre>{% highlight vespa-ranking-expression %}
{}
</pre>
{% endhighlight %}</pre>
<p>A single value tensor with a single mapped dimension <em>x</em>:</p>
<pre>
<pre>{% highlight vespa-ranking-expression %}
{ {x:foo}:5.0 }
</pre>
{% endhighlight %}</pre>
<p>A tensor with multiple values and mapped dimensions <em>x</em> and <em>y</em>:</p>
<pre>
<pre>{% highlight vespa-ranking-expression %}
{ {x:foo, y:bar}:5.0, {x:foo, y:baz}:7.0 }
</pre>
{% endhighlight %}</pre>
<p>A tensor where type is specified explicitly with a single indexed dimension <em>x</em> representing a vector:</p>
<pre>
tensor&lt;float&gt;(x[3]):{ {x:0}:3.0, {x:1}:5.0, {x:2}:7.0 }
</pre>
<pre>{% highlight vespa-ranking-expression %}
tensor<float>(x[3]):{ {x:0}:3.0, {x:1}:5.0, {x:2}:7.0 }
{% endhighlight %}</pre>
<p>A tensor with a type using the default value type (double) and quoted labels:</p>
<pre>
<pre>{% highlight vespa-ranking-expression %}
tensor(key{}):{ {key:'key.1'}:3.0, {key:'key 2'}:5.0, {key:"key's"}:7.0 }
</pre>
{% endhighlight %}</pre>


<h3 id="indexed-short-form">Indexed short form</h3>
Expand All @@ -176,23 +182,23 @@ <h3 id="indexed-short-form">Indexed short form</h3>

<h4 id="Indexed-short-form-examples">Indexed short form examples:</h4>
<p>A float 1d tensor in indexed form:</p>
<pre>
tensor&lt;float&gt;(x[3]):[3.0, 5.0, 7.0]
</pre>
<pre>{% highlight vespa-ranking-expression %}
tensor<float>(x[3]):[3.0, 5.0, 7.0]
{% endhighlight %}</pre>
<p>
A matrix in indexed form. Since the values for the right-most dimension (y) are adjacent,
the value 3 is here assigned to the cell {x:0,y:2}:
</p>
<pre>
tensor&lt;float&gt;(x[2],y[3]):[[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]]
</pre>
<pre>{% highlight vespa-ranking-expression %}
tensor<float>(x[2],y[3]):[[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]]
{% endhighlight %}</pre>
<p>
Deprecated:
Since inner brackets can be omitted, the above is equivalent to
</p>
<pre>
tensor&lt;float&gt;(x[2],y[3]):[1.0, 2.0, 3.0, 4.0, 5.0, 6.0]
</pre>
<pre>{% highlight vespa-ranking-expression %}
tensor<float>(x[2],y[3]):[1.0, 2.0, 3.0, 4.0, 5.0, 6.0]
{% endhighlight %}</pre>

<h3 id="indexed-hex-form">Binary hex format</h3>
<p>
Expand All @@ -207,9 +213,9 @@ <h3 id="indexed-hex-form">Binary hex format</h3>
0B22038405FF
</pre>
<p>which would be equivalent to:</p>
<pre>
<pre>{% highlight vespa-ranking-expression %}
[[11, 34, 3], [-124, 5, -1]]
</pre>
{% endhighlight %}</pre>


<h3 id="mapped-short-form">Mapped short form</h3>
Expand All @@ -220,9 +226,9 @@ <h3 id="mapped-short-form">Mapped short form</h3>
</p>

<h4 id="map-short-form-example">Map short form example:</h4>
<pre>
tensor&lt;float&gt;(key{}):{ key1:1.0, key2:2.0 }
</pre>
<pre>{% highlight vespa-ranking-expression %}
tensor<float>(key{}):{ key1:1.0, key2:2.0 }
{% endhighlight %}</pre>

<h3 id="mixed-short-form">Mixed short form</h3>
<p>
Expand All @@ -234,10 +240,10 @@ <h3 id="mixed-short-form">Mixed short form</h3>

<h4 id="mixed-short-form-example">Mixed short form example:</h4>
<p>A map of matrices:</p>
<pre>
tensor&lt;float&gt;(key{},x[2],y[3]):{ key1:[[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]],
<pre>{% highlight vespa-ranking-expression %}
tensor<float>(key{},x[2],y[3]):{ key1:[[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]],
key2:[[1.1, 2.1, 3.1], [4.1, 5.1, 6.1]] }
</pre>
{% endhighlight %}</pre>
<p>This may even be combined with a hexadecimal format for the dense subspace:</p>
<pre>
tensor&lt;int8&gt;(key{},x[5]):{ key1: 0102030405, key2: fffefdfcfb }
Expand All @@ -250,16 +256,17 @@ <h4 id="mixed-short-form-example">Mixed short form example:</h4>
in alphabetic order, so the nesting will follow a consistent
ordering.
</p>
<pre>
<pre>{% highlight vespa-ranking-expression %}
tensor(category{},key{},x[2],y[3]):{
cat1:{key1:[[1.0, 2.0, 3.0], [4.0, 5.0, 6.0]],
key2:[[1.1, 2.1, 3.1], [4.1, 5.1, 6.1]]},
cat2:{key1:[[7.3, 8.3, 9.3], [7.0, 8.0, 9.0]],
key3:[[7.5, 8.5, 9.5], [7.9, 8.9, 9.9]]}
}
</pre>
{% endhighlight %}</pre>
the equivalent fully-specified literal form would be
<pre>{% raw %}
<pre>{% highlight vespa-ranking-expression %}
{% raw %}
tensor(category{},key{},x[2],y[3]):{{category:cat1, key:key1, x:0, y:0}: 1.0,
{category:cat1, key:key1, x:0, y:1}: 2.0,
{category:cat1, key:key1, x:0, y:2}: 3.0,
Expand All @@ -284,7 +291,8 @@ <h4 id="mixed-short-form-example">Mixed short form example:</h4>
{category:cat2, key:key3, x:1, y:0}: 7.9,
{category:cat2, key:key3, x:1, y:1}: 8.9,
{category:cat2, key:key3, x:1, y:2}: 9.9}
{% endraw %}</pre>
{% endraw %}
{% endhighlight %}</pre>

<h2 id="tensor-functions">Tensor functions</h2>
<p>
Expand Down
Loading