ccache.github.io/storage-helpers.html at master · ccache/ccache.github.io · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Ccache &mdash; Storage helpers</title>
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="stylesheet" href="ccache.css">
</head>
<body>
<header>
<div class="container">
<h1>
<a href="/">
<span style="font-size: 150%; ">Ccache</span> &mdash; a fast C/C++ compiler cache
</a>
</h1>
<button class="sidebar-toggle" aria-label="Toggle menu">☰</button>
</div>
</header>
<div class="layout">
<div class="overlay"></div>
<aside class="sidebar">
<nav>
<ul>
<li>
<a href="/">Overview</a>
</li>
<li>
<a href="/download.html">Download</a>
</li>
<li>
<a href="/documentation.html">Documentation</a>
<ul>
<li>
<a href="/platform-compiler-language-support.html">Supported platforms and compilers</a>
</li>
<li>
<a class="current" href="/storage-helpers.html">Storage helpers</a>
</li>
<li>
<a href="/performance.html">Performance</a>
</li>
</ul>
</li>
<li>
<a href="/contribution.html">Contribution</a>
<ul>
<li>
<a href="/bugs.html">Bug report</a>
</li>
<li>
<a href="/repo.html">Source repository</a>
</li>
<li>
<a href="https://github.com/ccache/ccache/discussions">Discussions</a>
</li>
<li>
<a href="https://lists.samba.org/mailman/listinfo/ccache/">Mailing list</a>
</li>
</ul>
</li>
<li>
<a href="/news.html">News</a>
</li>
<li>
<a href="/credits.html">Credits</a>
</li>
<li>
<a href="/license.html">License</a>
</li>
</ul>
</nav>
</aside>
<main class="content">
<h1>Storage helpers</h1>
<img src="storage-helpers.svg" size="600" alt="ccache <-> storage helper <-> remote storage server" />
<p>
To communicate with remote storage servers in an efficient way, ccache spawns
a long-lived local helper process that can keep connections alive, thus
amortizing the session setup cost. The helper process is shared by all ccache
processes that use the same
<a href="https://ccache.dev/manual/latest.html#config_remote_storage">remote
storage settings</a>.
</p>
<p>
<b>NOTE:</b> Support for storage helpers is available in ccache 4.13+.
</p>
<h2>Available storage helpers</h2>
<table>
<tr>
<th>Protocol(s)</th>
<th>Project</th>
<th>Notes</th>
</tr>
<tr>
<td><code>http</code>, <code>https</code></td>
<td><a href="https://github.com/ccache/ccache-storage-http-cpp">ccache/ccache-storage-http-cpp</a></td>
<td>Reference implementation (C++)</td>
</tr>
<tr>
<td><code>http</code>, <code>https</code></td>
<td><a href="https://github.com/ccache/ccache-storage-http-go">ccache/ccache-storage-http-go</a></td>
<td>Reference implementation (Go), prebuilt binaries available</td>
</tr>
</table>
<p>
Have you implemented a storage helper and want it to be listed above? Great!
<a href="https://github.com/ccache/ccache.github.io/issues/new">Open an
issue</a>, or maybe even create a pull request yourself for
the <a href="https://github.com/ccache/ccache.github.io">web site
repository</a>.
</p>
<h2>Wanted storage helpers</h2>
<table>
<tr>
<th>Description</th>
<th>Notes</th>
</tr>
<tr>
<td>AWS S3 Blob Storage</td>
<td>See <a href="https://github.com/ccache/ccache/issues/1201">#1201</a></td>
</tr>
<tr>
<td>Azure Blob Storage</td>
<td>See <a href="https://github.com/ccache/ccache/issues/1152">#1152</a></td>
</tr>
<tr>
<td>Google Cloud Storage</td>
<td></td>
</tr>
<tr>
<td>Redis (incl. encrypted)</td>
<td>
Not yet implemented, but will be before the Redis backend is removed from
ccache
</td>
</tr>
<tr>
<td>Redis Cluster</td>
<td>See <a href="https://github.com/ccache/ccache/issues/1101">#1101</a></td>
</tr>
</table>
<h2>Basic usage</h2>
<ol>
<li>
Install a helper for the protocol you want (e.g. <code>http</code>) in a
<a href="https://ccache.dev/manual/latest.html#_storage_helper_process">location
where ccache looks for helper programs</a>. Ccache will start the helper
automatically when needed.
</li>
<li>
Set <code>CCACHE_REMOTE_STORAGE</code> (or
<a href="https://ccache.dev/manual/latest.html#config_remote_storage"><code>remote_storage</code></a>
in ccache's configuration file) to the URL of the remote storage server,
e.g. <code>http://example.com/cache</code>.
</li>
</ol>
<h2>Documentation</h2>
<ul>
<li><a href="https://ccache.dev/manual/latest.html#config_remote_storage">Remote storage configuration</a></li>
<li><a href="https://github.com/ccache/ccache/blob/master/doc/remote_storage_helper_spec.md">Remote storage helper specification</a></li>
</ul>
<h2>Starting a helper explicitly</h2>
<p>
A storage helper is usually started automatically by ccache when needed.
However, it is possible to start the helper explicitly, for instance when
developing or debugging a storage helper, or in custom deployment scenarios
where the helper service is managed externally. Example:
</p>
<pre>
export CRSH_IPC_ENDPOINT=/tmp/ccache-example.sock   # On Linux/POSIX
export CRSH_IPC_ENDPOINT=ccache-example-pipe        # On Windows
export CRSH_URL=http://example.com/cache
ccache-storage-http
</pre>
<p>
Ccache can then be told to connect to the IPC endpoint using the
<a href="https://ccache.dev/manual/latest.html##crsh-storage-backend">CRSH
backend</a>:
</p>
<pre>
export CCACHE_REMOTE_STORAGE=crsh:/tmp/ccache-example.sock   # On Linux/POSIX
export CCACHE_REMOTE_STORAGE=crsh:ccache-example-pipe        # On Windows
</pre>
</main>
</div> <!-- layout -->
<script>
const sidebar = document.querySelector(".sidebar");
const overlay = document.querySelector(".overlay");
const toggle = document.querySelector(".sidebar-toggle");
toggle.addEventListener("click", () => {
sidebar.classList.toggle("show");
overlay.classList.toggle("show");
});
overlay.addEventListener("click", () => {
sidebar.classList.remove("show");
overlay.classList.remove("show");
});
</script>
</body>
</html>