Skip to content

Commit 8dad50b

Browse files
author
Documenter.jl
committed
build based on 7e8b418 
1 parent 9bf13d9 commit 8dad50b

File tree

14 files changed

+1225
-0
lines changed

14 files changed

+1225
-0
lines changed

‎previews/PR314/advanced/index.html‎

Lines changed: 70 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,70 @@
1+
<!DOCTYPE html>
2+
<html lang="en"><head><meta charset="UTF-8"/><meta name="viewport" content="width=device-width, initial-scale=1.0"/><title>Advanced techniques · StructArrays</title><script data-outdated-warner src="../assets/warner.js"></script><link href="https://cdnjs.cloudflare.com/ajax/libs/lato-font/3.0.0/css/lato-font.min.css" rel="stylesheet" type="text/css"/><link href="https://cdnjs.cloudflare.com/ajax/libs/juliamono/0.045/juliamono.min.css" rel="stylesheet" type="text/css"/><link href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.15.4/css/fontawesome.min.css" rel="stylesheet" type="text/css"/><link href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.15.4/css/solid.min.css" rel="stylesheet" type="text/css"/><link href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/5.15.4/css/brands.min.css" rel="stylesheet" type="text/css"/><link href="https://cdnjs.cloudflare.com/ajax/libs/KaTeX/0.13.24/katex.min.css" rel="stylesheet" type="text/css"/><script>documenterBaseURL=".."</script><script src="https://cdnjs.cloudflare.com/ajax/libs/require.js/2.3.6/require.min.js" data-main="../assets/documenter.js"></script><script src="../siteinfo.js"></script><script src="../../versions.js"></script><link class="docs-theme-link" rel="stylesheet" type="text/css" href="../assets/themes/documenter-dark.css" data-theme-name="documenter-dark" data-theme-primary-dark/><link class="docs-theme-link" rel="stylesheet" type="text/css" href="../assets/themes/documenter-light.css" data-theme-name="documenter-light" data-theme-primary/><script src="../assets/themeswap.js"></script></head><body><div id="documenter"><nav class="docs-sidebar"><div class="docs-package-name"><span class="docs-autofit"><a href="../">StructArrays</a></span></div><form class="docs-search" action="../search/"><input class="docs-search-query" id="documenter-search-query" name="q" type="text" placeholder="Search docs"/></form><ul class="docs-menu"><li><a class="tocitem" href="../">Overview</a></li><li><a class="tocitem" href="../examples/">Example usage</a></li><li><a class="tocitem" href="../counterintuitive/">Some counterintuitive behaviors</a></li><li class="is-active"><a class="tocitem" href>Advanced techniques</a><ul class="internal"><li><a class="tocitem" href="#Structures-with-non-standard-data-layout"><span>Structures with non-standard data layout</span></a></li><li><a class="tocitem" href="#Mutate-or-widen-style-accumulation"><span>Mutate-or-widen style accumulation</span></a></li><li><a class="tocitem" href="#Using-StructArrays-in-CUDA-kernels"><span>Using StructArrays in CUDA kernels</span></a></li></ul></li><li><a class="tocitem" href="../reference/">Index</a></li></ul><div class="docs-version-selector field has-addons"><div class="control"><span class="docs-label button is-static is-size-7">Version</span></div><div class="docs-selector control is-expanded"><div class="select is-fullwidth is-size-7"><select id="documenter-version-selector"></select></div></div></div></nav><div class="docs-main"><header class="docs-navbar"><nav class="breadcrumb"><ul class="is-hidden-mobile"><li class="is-active"><a href>Advanced techniques</a></li></ul><ul class="is-hidden-tablet"><li class="is-active"><a href>Advanced techniques</a></li></ul></nav><div class="docs-right"><a class="docs-edit-link" href="https://github.com/JuliaArrays/StructArrays.jl/blob/master/docs/src/advanced.md" title="Edit on GitHub"><span class="docs-icon fab"></span><span class="docs-label is-hidden-touch">Edit on GitHub</span></a><a class="docs-settings-button fas fa-cog" id="documenter-settings-button" href="#" title="Settings"></a><a class="docs-sidebar-button fa fa-bars is-hidden-desktop" id="documenter-sidebar-button" href="#"></a></div></header><article class="content" id="documenter-page"><h1 id="Advanced-techniques"><a class="docs-heading-anchor" href="#Advanced-techniques">Advanced techniques</a><a id="Advanced-techniques-1"></a><a class="docs-heading-anchor-permalink" href="#Advanced-techniques" title="Permalink"></a></h1><h2 id="Structures-with-non-standard-data-layout"><a class="docs-heading-anchor" href="#Structures-with-non-standard-data-layout">Structures with non-standard data layout</a><a id="Structures-with-non-standard-data-layout-1"></a><a class="docs-heading-anchor-permalink" href="#Structures-with-non-standard-data-layout" title="Permalink"></a></h2><p>StructArrays support structures with custom data layout. The user is required to overload <code>staticschema</code> in order to define the custom layout, <code>component</code> to access fields of the custom layout, and <code>createinstance(T, fields...)</code> to create an instance of type <code>T</code> from its custom fields <code>fields</code>. In other word, given <code>x::T</code>, <code>createinstance(T, (component(x, f) for f in fieldnames(staticschema(T)))...)</code> should successfully return an instance of type <code>T</code>.</p><p>Here is an example of a type <code>MyType</code> that has as custom fields either its field <code>data</code> or fields of its field <code>rest</code> (which is a named tuple):</p><pre><code class="language-julia-repl hljs" style="display:block;">julia&gt; using StructArrays</code><code class="nohighlight hljs ansi" style="display:block;"></code><br/><code class="language-julia-repl hljs" style="display:block;">julia&gt; struct MyType{T, NT&lt;:NamedTuple}
3+
data::T
4+
rest::NT
5+
end</code><code class="nohighlight hljs ansi" style="display:block;"></code><br/><code class="language-julia-repl hljs" style="display:block;">julia&gt; MyType(x; kwargs...) = MyType(x, values(kwargs))</code><code class="nohighlight hljs ansi" style="display:block;">Main.MyType</code></pre><p>Let&#39;s create a small array of these objects:</p><pre><code class="language-julia-repl hljs" style="display:block;">julia&gt; s = [MyType(i/5, a=6-i, b=2) for i in 1:5]</code><code class="nohighlight hljs ansi" style="display:block;">5-element Vector{Main.MyType{Float64, @NamedTuple{a::Int64, b::Int64}}}:
6+
Main.MyType{Float64, @NamedTuple{a::Int64, b::Int64}}(0.2, (a = 5, b = 2))
7+
Main.MyType{Float64, @NamedTuple{a::Int64, b::Int64}}(0.4, (a = 4, b = 2))
8+
Main.MyType{Float64, @NamedTuple{a::Int64, b::Int64}}(0.6, (a = 3, b = 2))
9+
Main.MyType{Float64, @NamedTuple{a::Int64, b::Int64}}(0.8, (a = 2, b = 2))
10+
Main.MyType{Float64, @NamedTuple{a::Int64, b::Int64}}(1.0, (a = 1, b = 2))</code></pre><p>The default <code>StructArray</code> does not unpack the <code>NamedTuple</code>:</p><pre><code class="language-julia-repl hljs" style="display:block;">julia&gt; sa = StructArray(s);</code><code class="nohighlight hljs ansi" style="display:block;"></code><br/><code class="language-julia-repl hljs" style="display:block;">julia&gt; sa.rest</code><code class="nohighlight hljs ansi" style="display:block;">5-element Vector{@NamedTuple{a::Int64, b::Int64}}:
11+
(a = 5, b = 2)
12+
(a = 4, b = 2)
13+
(a = 3, b = 2)
14+
(a = 2, b = 2)
15+
(a = 1, b = 2)</code><br/><code class="language-julia-repl hljs" style="display:block;">julia&gt; sa.a</code><code class="nohighlight hljs ansi" style="display:block;">ERROR: type NamedTuple has no field a</code></pre><p>Suppose we wish to give the keywords their own fields. We can define custom <code>staticschema</code>, <code>component</code>, and <code>createinstance</code> methods for <code>MyType</code>:</p><pre><code class="language-julia-repl hljs" style="display:block;">julia&gt; function StructArrays.staticschema(::Type{MyType{T, NamedTuple{names, types}}}) where {T, names, types}
16+
# Define the desired names and eltypes of the &quot;fields&quot;
17+
return NamedTuple{(:data, names...), Base.tuple_type_cons(T, types)}
18+
end;</code><code class="nohighlight hljs ansi" style="display:block;"></code><br/><code class="language-julia-repl hljs" style="display:block;">julia&gt; function StructArrays.component(m::MyType, key::Symbol)
19+
# Define a component-extractor
20+
return key === :data ? getfield(m, 1) : getfield(getfield(m, 2), key)
21+
end;</code><code class="nohighlight hljs ansi" style="display:block;"></code><br/><code class="language-julia-repl hljs" style="display:block;">julia&gt; function StructArrays.createinstance(::Type{MyType{T, NT}}, x, args...) where {T, NT}
22+
# Generate an instance of MyType from components
23+
return MyType(x, NT(args))
24+
end;</code><code class="nohighlight hljs ansi" style="display:block;"></code></pre><p>and now:</p><pre><code class="language-julia-repl hljs" style="display:block;">julia&gt; sa = StructArray(s);</code><code class="nohighlight hljs ansi" style="display:block;"></code><br/><code class="language-julia-repl hljs" style="display:block;">julia&gt; sa.a</code><code class="nohighlight hljs ansi" style="display:block;">5-element Vector{Int64}:
25+
5
26+
4
27+
3
28+
2
29+
1</code><br/><code class="language-julia-repl hljs" style="display:block;">julia&gt; sa.b</code><code class="nohighlight hljs ansi" style="display:block;">5-element Vector{Int64}:
30+
2
31+
2
32+
2
33+
2
34+
2</code></pre><p>The above strategy has been tested and implemented in <a href="https://github.com/JuliaGeometry/GeometryBasics.jl">GeometryBasics.jl</a>.</p><h2 id="Mutate-or-widen-style-accumulation"><a class="docs-heading-anchor" href="#Mutate-or-widen-style-accumulation">Mutate-or-widen style accumulation</a><a id="Mutate-or-widen-style-accumulation-1"></a><a class="docs-heading-anchor-permalink" href="#Mutate-or-widen-style-accumulation" title="Permalink"></a></h2><p>StructArrays provides a function <code>StructArrays.append!!(dest, src)</code> (unexported) for &quot;mutate-or-widen&quot; style accumulation. This function can be used via <a href="https://juliafolds.github.io/BangBang.jl/dev/#BangBang.append!!"><code>BangBang.append!!</code></a> and <a href="https://juliafolds.github.io/BangBang.jl/dev/#BangBang.push!!"><code>BangBang.push!!</code></a> as well.</p><p><code>StructArrays.append!!</code> works like <code>append!(dest, src)</code> if <code>dest</code> can contain all element types in <code>src</code> iterator; i.e., it <em>mutates</em> <code>dest</code> in-place:</p><pre><code class="language-julia hljs">julia&gt; dest = StructVector((a=[1], b=[2]))
35+
1-element StructArray(::Array{Int64,1}, ::Array{Int64,1}) with eltype NamedTuple{(:a, :b),Tuple{Int64,Int64}}:
36+
(a = 1, b = 2)
37+
38+
julia&gt; StructArrays.append!!(dest, [(a = 3, b = 4)])
39+
2-element StructArray(::Array{Int64,1}, ::Array{Int64,1}) with eltype NamedTuple{(:a, :b),Tuple{Int64,Int64}}:
40+
(a = 1, b = 2)
41+
(a = 3, b = 4)
42+
43+
julia&gt; ans === dest
44+
true</code></pre><p>Unlike <code>append!</code>, <code>append!!</code> can also <em>widen</em> element type of <code>dest</code> array:</p><pre><code class="language-julia hljs">julia&gt; StructArrays.append!!(dest, [(a = missing, b = 6)])
45+
3-element StructArray(::Array{Union{Missing, Int64},1}, ::Array{Int64,1}) with eltype NamedTuple{(:a, :b),Tuple{Union{Missing, Int64},Int64}}:
46+
NamedTuple{(:a, :b),Tuple{Union{Missing, Int64},Int64}}((1, 2))
47+
NamedTuple{(:a, :b),Tuple{Union{Missing, Int64},Int64}}((3, 4))
48+
NamedTuple{(:a, :b),Tuple{Union{Missing, Int64},Int64}}((missing, 6))
49+
50+
julia&gt; ans === dest
51+
false</code></pre><p>Since the original array <code>dest</code> cannot hold the input, a new array is created (<code>ans !== dest</code>).</p><p>Combined with <a href="https://docs.julialang.org/en/latest/manual/performance-tips/#kernel-functions-1">function barriers</a>, <code>append!!</code> is a useful building block for implementing <code>collect</code>-like functions.</p><h2 id="Using-StructArrays-in-CUDA-kernels"><a class="docs-heading-anchor" href="#Using-StructArrays-in-CUDA-kernels">Using StructArrays in CUDA kernels</a><a id="Using-StructArrays-in-CUDA-kernels-1"></a><a class="docs-heading-anchor-permalink" href="#Using-StructArrays-in-CUDA-kernels" title="Permalink"></a></h2><p>It is possible to combine StructArrays with <a href="https://github.com/JuliaGPU/CUDAnative.jl">CUDAnative</a>, in order to create CUDA kernels that work on StructArrays directly on the GPU. Make sure you are familiar with the CUDAnative documentation (esp. kernels with plain <code>CuArray</code>s) before experimenting with kernels based on <code>StructArray</code>s.</p><pre><code class="language-julia hljs">using CUDAnative, CuArrays, StructArrays
52+
d = StructArray(a = rand(100), b = rand(100))
53+
54+
# move to GPU
55+
dd = replace_storage(CuArray, d)
56+
de = similar(dd)
57+
58+
# a simple kernel, to copy the content of `dd` onto `de`
59+
function kernel!(dest, src)
60+
i = (blockIdx().x-1)*blockDim().x + threadIdx().x
61+
if i &lt;= length(dest)
62+
dest[i] = src[i]
63+
end
64+
return nothing
65+
end
66+
67+
threads = 1024
68+
blocks = cld(length(dd),threads)
69+
70+
@cuda threads=threads blocks=blocks kernel!(de, dd)</code></pre></article><nav class="docs-footer"><a class="docs-footer-prevpage" href="../counterintuitive/">« Some counterintuitive behaviors</a><a class="docs-footer-nextpage" href="../reference/">Index »</a><div class="flexbox-break"></div><p class="footer-message">Powered by <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> and the <a href="https://julialang.org/">Julia Programming Language</a>.</p></nav></div><div class="modal" id="documenter-settings"><div class="modal-background"></div><div class="modal-card"><header class="modal-card-head"><p class="modal-card-title">Settings</p><button class="delete"></button></header><section class="modal-card-body"><p><label class="label">Theme</label><div class="select"><select id="documenter-themepicker"><option value="documenter-light">documenter-light</option><option value="documenter-dark">documenter-dark</option></select></div></p><hr/><p>This document was generated with <a href="https://github.com/JuliaDocs/Documenter.jl">Documenter.jl</a> version 0.27.25 on <span class="colophon-date" title="Friday 1 November 2024 08:47">Friday 1 November 2024</span>. Using Julia version 1.11.1.</p></section><footer class="modal-card-foot"></footer></div></div></div></body></html>

0 commit comments

Comments
 (0)