Skip to content

Commit 7f33d68

Browse files
jue-yuanjue-yuan
committed
[GLE-8861] feat(vector): built-in TG function for pairwise vector embedding;
1 parent abd566c commit 7f33d68

File tree

8 files changed

+472
-0
lines changed

8 files changed

+472
-0
lines changed

‎gds/vector/cosine_distance.gsql

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
CREATE FUNCTION gds.vector.cosine_distance(list<double> list1, list<double> list2) RETURNS(float) {
2+
3+
/*
4+
First Author: Jue Yuan
5+
First Commit Date: Nov 27, 2024
6+
7+
Recent Author: Jue Yuan
8+
Recent Commit Date: Nov 27, 2024
9+
10+
Maturity:
11+
alpha
12+
13+
Description:
14+
Calculates the cosine distance between two vectors represented as lists of doubles.
15+
The cosine distance is derived from the cosine similarity and provides a measure of the angle
16+
between two non-zero vectors in a multi-dimensional space. A distance of 0 indicates identical
17+
vectors, while a distance of 1 indicates orthogonal (maximally dissimilar) vectors.
18+
19+
Parameters:
20+
list<double> list1:
21+
The first vector as a list of double values.
22+
list<double> list2:
23+
The second vector as a list of double values.
24+
25+
Returns:
26+
float:
27+
The cosine distance between the two input vectors.
28+
Exceptions:
29+
list_size_mismatch (90000):
30+
Raised when the input lists are not of equal size.
31+
32+
Logic Overview:
33+
Validates that both input vectors have the same length.
34+
Computes the inner (dot) product of the two vectors.
35+
Calculates the magnitudes (Euclidean norms) of both vectors.
36+
Returns the cosine distance as 1 - (inner product) / (product of magnitudes).
37+
38+
Use Case:
39+
This function is commonly used in machine learning, natural language processing,
40+
and information retrieval tasks to quantify the similarity between vector representations,
41+
such as word embeddings or document feature vectors.
42+
*/
43+
44+
EXCEPTION list_size_mismatch (90000);
45+
ListAccum<double> @@myList1 = list1;
46+
ListAccum<double> @@myList2 = list2;
47+
48+
IF (@@myList1.size() != @@myList2.size()) THEN
49+
RAISE list_size_mismatch ("Two lists provided for gds.vector.cosine_distance have different sizes.");
50+
END;
51+
52+
double innerP = inner_product(@@myList1, @@myList2);
53+
double v1_magn = sqrt(inner_product(@@myList1, @@myList1));
54+
double v2_magn = sqrt(inner_product(@@myList2, @@myList2));
55+
RETURN (1 - innerP / (v1_magn * v2_magn));
56+
}

‎gds/vector/dimension_count.gsql

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
CREATE FUNCTION gds.vector.dimension_count(list<double> list1) RETURNS(int) {
2+
3+
/*
4+
First Author: Jue Yuan
5+
First Commit Date: Nov 27, 2024
6+
7+
Recent Author: Jue Yuan
8+
Recent Commit Date: Nov 27, 2024
9+
10+
Maturity:
11+
alpha
12+
13+
Description:
14+
Returns the number of dimensions (elements) in a given vector, represented as a list of double values.
15+
This function is useful for determining the size or dimensionality of input vectors in mathematical
16+
and data processing operations.
17+
18+
Parameters:
19+
list<double> list1:
20+
The input vector as a list of double values.
21+
22+
Returns:
23+
int:
24+
The number of elements (dimensions) in the input vector.
25+
26+
Logic Overview:
27+
Accepts a list of double values as input.
28+
Calculates the size of the list, which corresponds to the number of dimensions.
29+
Returns the size as an integer.
30+
Use Case:
31+
This function is valuable in vector-based computations, such as machine learning or data analysis tasks,
32+
where understanding the dimensionality of vectors is crucial for validation, preprocessing, or compatibility checks.
33+
*/
34+
35+
ListAccum<double> @@myList1 = list1;
36+
RETURN @@myList1.size();
37+
}

‎gds/vector/distance.gsql

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
CREATE FUNCTION gds.vector.distance(list<double> list1, list<double> list2, string metric) RETURNS(float) {
2+
3+
/*
4+
First Author: Jue Yuan
5+
First Commit Date: Nov 27, 2024
6+
7+
Recent Author: Jue Yuan
8+
Recent Commit Date: Nov 27, 2024
9+
10+
Maturity:
11+
alpha
12+
13+
Description:
14+
Calculates the distance between two vectors represented as lists of double values,
15+
based on a specified distance metric. This function supports multiple metrics,
16+
allowing for flexible similarity or dissimilarity measurements in various computational tasks.
17+
18+
Parameters:
19+
list<double> list1:
20+
The first vector as a list of double values.
21+
list<double> list2:
22+
The second vector as a list of double values.
23+
string metric:
24+
The distance metric to use. Supported metrics are:
25+
"cosine": Cosine distance
26+
"euclidean": Euclidean distance
27+
"ip": Inner product (dot product)
28+
Returns:
29+
float:
30+
The computed distance between the two input vectors based on the specified metric.
31+
32+
Exceptions:
33+
list_size_mismatch (90000):
34+
Raised when the input vectors are not of equal size.
35+
invalid_metric_type (90001):
36+
Raised when an unsupported distance metric is provided.
37+
38+
Logic Overview:
39+
Input Validation:
40+
Ensures both vectors have the same size.
41+
Metric Handling:
42+
Cosine Distance:
43+
Calculated as 1 - (inner product of vectors) / (product of magnitudes).
44+
Euclidean Distance:
45+
Computes the square root of the sum of squared differences between corresponding elements.
46+
Inner Product:
47+
Directly computes the dot product of the two vectors.
48+
49+
Error Handling:
50+
Raises an exception if the provided metric is invalid.
51+
52+
Use Case:
53+
This function is essential for machine learning, data science, and information retrieval applications,
54+
where distance or similarity calculations between vector representations (such as embeddings or feature vectors) are required.
55+
*/
56+
57+
EXCEPTION list_size_mismatch (90000);
58+
EXCEPTION invalid_metric_type (90001);
59+
ListAccum<double> @@myList1 = list1;
60+
ListAccum<double> @@myList2 = list2;
61+
62+
IF (@@myList1.size() != @@myList2.size()) THEN
63+
RAISE list_size_mismatch ("Two lists provided for gds.vector.distance have different sizes.");
64+
END;
65+
66+
SumAccum<float> @@myResult;
67+
SumAccum<float> @@sqrSum;
68+
69+
CASE lower(metric)
70+
WHEN "cosine" THEN
71+
@@myResult = 1 - inner_product(@@myList1, @@myList2) / (sqrt(inner_product(@@myList1, @@myList1)) * sqrt(inner_product(@@myList2, @@myList2)));
72+
WHEN "euclidean" THEN
73+
FOREACH i IN [0, @@myList1.size() - 1 ] DO
74+
@@sqrSum += (@@myList1.get(i) - @@myList2.get(i)) * (@@myList1.get(i) - @@myList2.get(i));
75+
END;
76+
@@myResult = sqrt(@@sqrSum);
77+
WHEN "ip" THEN
78+
@@myResult = inner_product(@@myList1, @@myList2);
79+
ELSE
80+
RAISE invalid_metric_type ("Invalid metric algorithm provided, currently supported: cosine, euclidean and ip.");
81+
END
82+
;
83+
84+
RETURN @@myResult;
85+
}

‎gds/vector/elements_sum.gsql

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
CREATE FUNCTION gds.vector.elements_sum(list<double> list1) RETURNS(float) {
2+
3+
/*
4+
First Author: Jue Yuan
5+
First Commit Date: Nov 27, 2024
6+
7+
Recent Author: Jue Yuan
8+
Recent Commit Date: Nov 27, 2024
9+
10+
Maturity:
11+
alpha
12+
13+
Description:
14+
Calculates the sum of all elements in a vector, represented as a list of double values.
15+
This function is useful for aggregating vector components in mathematical and statistical operations.
16+
17+
Parameters:
18+
list<double> list1:
19+
The input vector as a list of double values.
20+
21+
Returns:
22+
float:
23+
The sum of all elements in the input vector.
24+
25+
Logic Overview:
26+
Iterates through each element in the input list.
27+
Accumulates the sum of all elements.
28+
Returns the final sum as a floating-point value.
29+
30+
Use Case:
31+
This function is valuable in various data processing tasks, such as computing vector norms,
32+
validating data integrity, or performing aggregations in machine learning and statistical analysis.
33+
*/
34+
35+
SumAccum<float> @@mySum;
36+
37+
FOREACH i IN list1 DO
38+
@@mySum += i;
39+
END;
40+
RETURN @@mySum;
41+
}

‎gds/vector/euclidean_distance.gsql

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
CREATE FUNCTION gds.vector.euclidean_distance(list<double> list1, list<double> list2) RETURNS(float) {
2+
3+
/*
4+
First Author: Jue Yuan
5+
First Commit Date: Nov 27, 2024
6+
7+
Recent Author: Jue Yuan
8+
Recent Commit Date: Nov 27, 2024
9+
10+
Maturity:
11+
alpha
12+
13+
Description:
14+
Calculates the Euclidean distance between two vectors represented as lists of double values.
15+
Euclidean distance measures the straight-line distance between two points in multi-dimensional space,
16+
making it a fundamental metric in various computational and analytical applications.
17+
18+
Parameters:
19+
list<double> list1:
20+
The first vector as a list of double values.
21+
list<double> list2:
22+
The second vector as a list of double values.
23+
24+
Returns:
25+
float:
26+
The Euclidean distance between the two input vectors.
27+
28+
Exceptions:
29+
list_size_mismatch (90000): Raised when the input vectors are not of equal size.
30+
31+
Logic Overview:
32+
Input Validation:
33+
Ensures both vectors have the same length.
34+
Distance Calculation:
35+
Iterates through corresponding elements of both vectors.
36+
Computes the sum of the squared differences between each pair of elements.
37+
Returns the square root of the accumulated sum, representing the Euclidean distance.
38+
39+
Formula:
40+
Distance = sqrt((x1 - y1)^2 + (x2 - y2)^2 + ... + (xn - yn)^2)
41+
Where xi and yi are elements of list1 and list2, respectively.
42+
43+
Use Case:
44+
This function is widely used in machine learning (e.g., k-nearest neighbors), data science,
45+
and pattern recognition tasks to measure the similarity or dissimilarity between data points.
46+
*/
47+
48+
EXCEPTION list_size_mismatch (90000);
49+
ListAccum<double> @@myList1 = list1;
50+
ListAccum<double> @@myList2 = list2;
51+
52+
IF (@@myList1.size() != @@myList2.size()) THEN
53+
RAISE list_size_mismatch ("Two lists provided for gds.vector.euclidean_distance have different sizes.");
54+
END;
55+
56+
SumAccum<float> @@sqrSum;
57+
FOREACH i IN [0, @@myList1.size() - 1 ] DO
58+
@@sqrSum += (@@myList1.get(i) - @@myList2.get(i)) * (@@myList1.get(i) - @@myList2.get(i));
59+
END;
60+
61+
RETURN sqrt(@@sqrSum);
62+
}

‎gds/vector/ip_distance.gsql

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
CREATE FUNCTION gds.vector.ip_distance(list<double> list1, list<double> list2) RETURNS(float) {
2+
3+
/*
4+
First Author: Jue Yuan
5+
First Commit Date: Nov 27, 2024
6+
7+
Recent Author: Jue Yuan
8+
Recent Commit Date: Nov 27, 2024
9+
10+
Maturity:
11+
alpha
12+
13+
Description:
14+
Calculates the inner product (dot product) between two vectors represented as lists of double values.
15+
The inner product is a key measure in linear algebra, indicating the magnitude of the projection of one vector onto another.
16+
This function provides a similarity measure commonly used in machine learning and data analysis.
17+
18+
Parameters:
19+
list<double> list1:
20+
The first vector as a list of double values.
21+
list<double> list2:
22+
The second vector as a list of double values.
23+
24+
Returns:
25+
float:
26+
The inner product (dot product) of the two input vectors.
27+
28+
Exceptions:
29+
list_size_mismatch (90000):
30+
Raised when the input vectors are not of equal size.
31+
32+
Logic Overview:
33+
Input Validation:
34+
Ensures both vectors have the same length.
35+
Inner Product Calculation:
36+
Computes the sum of the element-wise products of the two vectors.
37+
38+
Formula:
39+
Inner Product = (x1 x y1) + (x2 x y2) + ... + (xn x yn)
40+
Where xi and yi are elements of list1 and list2, respectively.
41+
42+
Use Case:
43+
This function is widely used in:
44+
Calculating similarity in machine learning models (e.g., recommendation systems).
45+
Performing vector projections in linear algebra.
46+
Evaluating similarity between embeddings in natural language processing (NLP).
47+
*/
48+
49+
EXCEPTION list_size_mismatch (90000);
50+
ListAccum<double> @@myList1 = list1;
51+
ListAccum<double> @@myList2 = list2;
52+
53+
IF (@@myList1.size() != @@myList2.size()) THEN
54+
RAISE list_size_mismatch ("Two lists provided for gds.vector.euclidean_distance have different sizes.");
55+
END;
56+
57+
RETURN inner_product(@@myList1, @@myList2);
58+
}

0 commit comments

Comments
 (0)