Skip to content

Commit f93c15a

Browse files
committed
Add BLAKE3CONCAT: a version of BLAKE3 that supports chunking
In PR bazelbuild#233 I proposed the addition of two new ContentAddressableStorage methods (ConcatenateBlobs and SplitBlobs) that allow one to gain random access it large CAS objects, while still providing a way to very data integrity. As part of that change, I added a new digest function to help with that, named BLAKE3CONCAT. This PR adds just this digest function, without bringing in any support for chunking. This will be done separately, as it was requested that both these features landed independently. I have also included test vectors for the BLAKE3CONCAT digest function. I have derived these by modifying the BLAKE3 reference implementation written in Rust, and rerunning the tool that emits the official test vectors: https://github.com/BLAKE3-team/BLAKE3/blob/master/test_vectors/test_vectors.json Furthermore, I have been able to validate the newly obtained test vectors using a custom BLAKE3CONCAT implementation that I have written in Go, which will become part of Buildbarn.
1 parent 3a21dee commit f93c15a

File tree

3 files changed

+453
-252
lines changed

3 files changed

+453
-252
lines changed
Lines changed: 145 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,145 @@
1+
{
2+
"_comment": "Each test is an input length and one hash output. The input in each case is filled with a repeating sequence of 251 bytes: 0, 1, 2, ..., 249, 250, 0, 1, ..., and so on.",
3+
"cases": [
4+
{
5+
"input_len": 0,
6+
"hash": "b4686b3c8d953f4d185d51bc9cd7bce6d9782d7659f8c06068d4c3ffa6e56841"
7+
},
8+
{
9+
"input_len": 1,
10+
"hash": "0df1a7881d71d287d0afc2fcd7d23d28f12e401ab858ca267f11c5f1710df315"
11+
},
12+
{
13+
"input_len": 2,
14+
"hash": "b2530fd8d3fa7ca531d3e03d3b44f434354af89b59c14123d1d1ecc664b243a6"
15+
},
16+
{
17+
"input_len": 3,
18+
"hash": "8e2451ec9f2ae6282e10a52c09d3bf7856236cfa990b5f1aaf0b3943d5b72d18"
19+
},
20+
{
21+
"input_len": 4,
22+
"hash": "826e2530317a718add9a012ad6cf1c2cd298c6f7fa09022dd7f1847cfd03ddb8"
23+
},
24+
{
25+
"input_len": 5,
26+
"hash": "c6e98dc8c0e86be0a25d8e4a4ac951f5bf4efa111d8a332843aabe723fab3452"
27+
},
28+
{
29+
"input_len": 6,
30+
"hash": "9532a581f853cd63594625ddd0bdcac2edaf9cf436b0b42af77938ed5cfc5181"
31+
},
32+
{
33+
"input_len": 7,
34+
"hash": "1d2cfb6af10502c1b7067f6e4515dac354cb4f6c9def44704e9a79e32f41688b"
35+
},
36+
{
37+
"input_len": 8,
38+
"hash": "ffa72fdba626d7aa0d1ad38bb7f289c91386ae4d2929fa093ccc5d188433e906"
39+
},
40+
{
41+
"input_len": 63,
42+
"hash": "eefdb8f62d0cb234d94b16a2837eb72665018861396a89ef9f28d1fb190fca24"
43+
},
44+
{
45+
"input_len": 64,
46+
"hash": "6dde41c9d05a39b09b486620f2c3cf7652fde7f3eb412353d95734294c5d348e"
47+
},
48+
{
49+
"input_len": 65,
50+
"hash": "5bf4b8a5dba62b01ebd11264f3a724638e3f0ef274f253dfa41c0a657db3ab3e"
51+
},
52+
{
53+
"input_len": 127,
54+
"hash": "cdfdc645bd59bb2415df258b0d97a1f07e68710f67e6e61e78aa15f4703d53a2"
55+
},
56+
{
57+
"input_len": 128,
58+
"hash": "d41a4a8d07d439dc3892f45a296b934f2fbbd966f29f8640e88f15dde20015c7"
59+
},
60+
{
61+
"input_len": 129,
62+
"hash": "7a0235d0b622ed10c03a5c9f7fcfb36b2fc8aefb50e3a95ded8add3e06126214"
63+
},
64+
{
65+
"input_len": 1023,
66+
"hash": "ad61f1f5c9f2a65c7b96152b47321c7d4320e2c2719e14bee6640815e4182ddb"
67+
},
68+
{
69+
"input_len": 1024,
70+
"hash": "5c9e654411e393d1f4bec710ccd5bc5669ab177d610a0eb691fcfee92fb4e8b1"
71+
},
72+
{
73+
"input_len": 1025,
74+
"hash": "ee286548954cbdc4bf9b5fef0555f1b7d813edbe2bc7f3693126d5825999139d"
75+
},
76+
{
77+
"input_len": 2048,
78+
"hash": "80c9b515da8b7fcabfefcc4b049b7cbcf16983c2cd1448e7717168f279b4d7ad"
79+
},
80+
{
81+
"input_len": 2049,
82+
"hash": "051bad3acf63a2d27429524232b86b8230234e0091f38e0686faf79df77e1b30"
83+
},
84+
{
85+
"input_len": 3072,
86+
"hash": "cd28dcf373e40cc36aa4a7349687d3805babb92229c0cf28d903ee2d08e1194c"
87+
},
88+
{
89+
"input_len": 3073,
90+
"hash": "fe07c25f6c1bc98d104ed1ca7971177e8515b28f6287e35569098b448f31f39a"
91+
},
92+
{
93+
"input_len": 4096,
94+
"hash": "65e4ec2f94073d6eb00a9bf5c085cde958ec73ed3baaf2b80d5e6f90dc868e94"
95+
},
96+
{
97+
"input_len": 4097,
98+
"hash": "7f59a96c5b78c65db13b981c327ebc170bcdd155123857dfb8720349b0ae7699"
99+
},
100+
{
101+
"input_len": 5120,
102+
"hash": "88cf814865f6b6f1488e3c8dd14ebab858659459e8c1552a26bcfd81f75465c7"
103+
},
104+
{
105+
"input_len": 5121,
106+
"hash": "01f181c74b1a8284101fa35c8082734f167fcc7f6ac75c2547ecfb0c765a1073"
107+
},
108+
{
109+
"input_len": 6144,
110+
"hash": "b739390236994f666fd8c0a6d57e3a6192034a43ca990906f2abbad1509c719f"
111+
},
112+
{
113+
"input_len": 6145,
114+
"hash": "a0e15c6796ed7da911d75d545ee14c960ff1e9dd37ecfbd9101c25f9e4d073f9"
115+
},
116+
{
117+
"input_len": 7168,
118+
"hash": "b55ab1c0b44dab6c0e25839c33198f3f76a688f31ec4a8d109f190befb3634f2"
119+
},
120+
{
121+
"input_len": 7169,
122+
"hash": "23cb0c89436728d339dae6771e9078109df0fdbdee617d6e49d615c5d5212ef2"
123+
},
124+
{
125+
"input_len": 8192,
126+
"hash": "fda9453f03d5f5080d3c41126fddc294c0d8f48aa3fdbcdd63f9cda4d33fc211"
127+
},
128+
{
129+
"input_len": 8193,
130+
"hash": "5da258b0a71add844c953fd1b3c74a99f3af9011baeaedae7962f4dde67e5cf9"
131+
},
132+
{
133+
"input_len": 16384,
134+
"hash": "222b977bbd1d5dc6529f424379ac4e987f927a10411f52ffce160e7e1f4c06b8"
135+
},
136+
{
137+
"input_len": 31744,
138+
"hash": "f1ae00b44cb183227d4296ad578af3c2b27132dcf3ee94e7731acc198fdeb114"
139+
},
140+
{
141+
"input_len": 102400,
142+
"hash": "9dd1fb18c727e9bfc58c9ff5487ac5b08e6b1f32d65348a68dc4131e967cd2ce"
143+
}
144+
]
145+
}

0 commit comments

Comments
 (0)