Skip to content

Add non-scalar raw get key support#57

Open
vearutop wants to merge 4 commits intomasterfrom
extract-json
Open

Add non-scalar raw get key support#57
vearutop wants to merge 4 commits intomasterfrom
extract-json

Conversation

@vearutop
Copy link
Copy Markdown
Owner

@vearutop vearutop commented Mar 14, 2023

This PR enables extraction of non-scalar JSON values, e.g.

./flatjsonl -get-key ".foo.bar" -input events.log -raw foobar.log

would be equivalent to

cat events.log | jq -c ".foo.bar" > foobar.log

if .foo.bar is an object or array.

@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 14, 2023

Lines Of Code

Language Files Lines Code Comments Blanks Complexity Bytes
Go 16 3794 (+31) 2883 (+26) 110 801 (+5) 777 (+5) 77.1K (+816B)
Markdown 4 304 244 0 60 0 10.8K (+5B)

@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 14, 2023

Go API Changes

# summary
Inferred base version: v0.5.14
Suggested version: v0.5.15

@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 14, 2023

Unit Test Coverage

total: (statements) 72.3%
changed lines: (statements) 0.0%, coverage is less than 90.0%, consider testing the changes more thoroughly

Coverage of changed lines
File Function Coverage
Total 0.0%
flatjsonl/flattener.go 0.0%
flatjsonl/flattener.go:32 GetKey 0.0%
Coverage diff with base branch
File Function Base Coverage Current Coverage
Total 73.0% 72.3% (-0.7%)
github.com/vearutop/flatjsonl/flatjsonl/keys.go initKey 95.7% 100.0% (+4.3%)

@github-actions
Copy link
Copy Markdown

github-actions bot commented Mar 14, 2023

Benchmark Result

Benchmark diff with base branch
Module github.com/bool64/dev not found, downloading.
name                                     old time/op    new time/op    delta
NewProcessor/test_scanKeys-2               29.0ns ± 1%    29.3ns ± 2%       ~     (p=0.247 n=5+6)
NewProcessor/test_writeOutput-2            634ns ±121%    644ns ±121%       ~     (p=0.818 n=6+6)
NewProcessor/test_get_key_scanKeys-2       0.00ns ±20%    0.00ns ±22%    -19.23%  (p=0.024 n=6+6)
NewProcessor/test_get_key_writeOutput-2    1.21ns ±42%  41.60ns ±131%  +3335.85%  (p=0.002 n=6+6)
NewProcessor/transpose_scanKeys-2          36.1ns ± 2%    34.9ns ± 3%     -3.15%  (p=0.009 n=6+6)
NewProcessor/transpose_writeOutput-2        933ns ±98%     945ns ±96%       ~     (p=0.818 n=6+6)
NewProcessor/coalesce_scanKeys-2           76.3ns ± 2%    77.6ns ± 7%       ~     (p=0.247 n=5+6)
NewProcessor/coalesce_writeOutput-2       1.05µs ±113%   1.09µs ±110%       ~     (p=0.818 n=6+6)

name                                     old alloc/op   new alloc/op   delta
NewProcessor/test_scanKeys-2                4.00B ± 0%     4.00B ± 0%       ~     (all equal)
NewProcessor/test_writeOutput-2             11.8B ±41%     12.0B ±42%       ~     (p=0.957 n=6+6)
NewProcessor/test_get_key_scanKeys-2        0.00B          0.00B            ~     (all equal)
NewProcessor/test_get_key_writeOutput-2     0.00B       290.00B ±134%      +Inf%  (p=0.002 n=6+6)
NewProcessor/transpose_scanKeys-2           1.00B ± 0%     1.00B ± 0%       ~     (all equal)
NewProcessor/transpose_writeOutput-2         100B ±65%      100B ±65%       ~     (p=1.000 n=6+6)
NewProcessor/coalesce_scanKeys-2            7.00B ± 0%     7.00B ± 0%       ~     (all equal)
NewProcessor/coalesce_writeOutput-2         27.7B ±49%     31.0B ±13%       ~     (p=0.506 n=6+5)

name                                     old allocs/op  new allocs/op  delta
NewProcessor/test_scanKeys-2                 0.00           0.00            ~     (all equal)
NewProcessor/test_writeOutput-2              0.00           0.00            ~     (all equal)
NewProcessor/test_get_key_scanKeys-2         0.00           0.00            ~     (all equal)
NewProcessor/test_get_key_writeOutput-2      0.00           0.00            ~     (all equal)
NewProcessor/transpose_scanKeys-2            0.00           0.00            ~     (all equal)
NewProcessor/transpose_writeOutput-2        2.00 ±100%     2.00 ±100%       ~     (p=1.000 n=6+6)
NewProcessor/coalesce_scanKeys-2             0.00           0.00            ~     (all equal)
NewProcessor/coalesce_writeOutput-2          0.00           0.00            ~     (all equal)
Benchmark result
name                                     time/op
NewProcessor/coalesce_scanKeys-2          77.6ns ± 7%
NewProcessor/coalesce_writeOutput-2      1.09µs ±110%
NewProcessor/test_scanKeys-2              29.3ns ± 2%
NewProcessor/test_writeOutput-2           644ns ±121%
NewProcessor/test_get_key_scanKeys-2      0.00ns ±22%
NewProcessor/test_get_key_writeOutput-2  41.6ns ±131%
NewProcessor/transpose_scanKeys-2         34.9ns ± 3%
NewProcessor/transpose_writeOutput-2       945ns ±96%

name                                     alloc/op
NewProcessor/coalesce_scanKeys-2           7.00B ± 0%
NewProcessor/coalesce_writeOutput-2        31.0B ±13%
NewProcessor/test_scanKeys-2               4.00B ± 0%
NewProcessor/test_writeOutput-2            12.0B ±42%
NewProcessor/test_get_key_scanKeys-2       0.00B     
NewProcessor/test_get_key_writeOutput-2    290B ±134%
NewProcessor/transpose_scanKeys-2          1.00B ± 0%
NewProcessor/transpose_writeOutput-2        100B ±65%

name                                     allocs/op
NewProcessor/coalesce_scanKeys-2            0.00     
NewProcessor/coalesce_writeOutput-2         0.00     
NewProcessor/test_scanKeys-2                0.00     
NewProcessor/test_writeOutput-2             0.00     
NewProcessor/test_get_key_scanKeys-2        0.00     
NewProcessor/test_get_key_writeOutput-2     0.00     
NewProcessor/transpose_scanKeys-2           0.00     
NewProcessor/transpose_writeOutput-2       2.00 ±100%

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Mar 19, 2023

Codecov Report

Patch coverage has no change and project coverage change: -0.93 ⚠️

Comparison is base (154f30c) 63.42% compared to head (c028394) 62.50%.

📣 This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

Additional details and impacted files
@@            Coverage Diff             @@
##           master      #57      +/-   ##
==========================================
- Coverage   63.42%   62.50%   -0.93%     
==========================================
  Files          13       13              
  Lines        1553     1576      +23     
==========================================
  Hits          985      985              
- Misses        423      446      +23     
  Partials      145      145              
Flag Coverage Δ
unittests 62.50% <0.00%> (-0.93%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
flatjsonl/flattener.go 55.96% <0.00%> (-14.97%) ⬇️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants