fix(issues): Process new group_first_seen field on ingest #7355

thetruecpaul · 2025-08-14T17:53:59Z

This is part of an effort to fix issue search's sort-by-age. To do so, we're adding a new group_first_seen field.

This PR processes the new field on ingestion of both errors & search_issues events.

Note that I couldn't get the snapshot tests to run locally due to devenv problems, so the snapshot changes in this PR are merely taken from CI.

codecov · 2025-08-14T18:09:13Z

❌ 1 Tests Failed:

Tests completed	Failed	Passed	Skipped
503	1	502	6

View the top 1 failed test(s) by shortest run time

tests.datasets.test_errors_processor.TestErrorsProcessor::test_errors_basic

Stack Traces | 0.014s run time

Traceback (most recent call last):
  File ".../local/lib/python3.11........./site-packages/_pytest/runner.py", line 341, in from_call
    result: TResult | None = func()
                             ^^^^^^
  File ".../local/lib/python3.11........./site-packages/_pytest/runner.py", line 242, in <lambda>
    lambda: runtest_hook(item=item, **kwds), when=when, reraise=reraise
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../local/lib/python3.11....../site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../local/lib/python3.11....../site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../local/lib/python3.11.........................../site-packages/pluggy/_callers.py", line 182, in _multicall
    return outcome.get_result()
           ^^^^^^^^^^^^^^^^^^^^
  File ".../local/lib/python3.11.../site-packages/pluggy/_result.py", line 100, in get_result
    raise exc.with_traceback(exc.__traceback__)
  File ".../local/lib/python3.11.........................../site-packages/pluggy/_callers.py", line 167, in _multicall
    teardown.throw(outcome._exception)
  File ".../local/lib/python3.11....../site-packages/_pytest/threadexception.py", line 92, in pytest_runtest_call
    yield from thread_exception_runtest_hook()
  File ".../local/lib/python3.11....../site-packages/_pytest/threadexception.py", line 68, in thread_exception_runtest_hook
    yield
  File ".../local/lib/python3.11.........................../site-packages/pluggy/_callers.py", line 167, in _multicall
    teardown.throw(outcome._exception)
  File ".../local/lib/python3.11....../site-packages/_pytest/unraisableexception.py", line 95, in pytest_runtest_call
    yield from unraisable_exception_runtest_hook()
  File ".../local/lib/python3.11....../site-packages/_pytest/unraisableexception.py", line 70, in unraisable_exception_runtest_hook
    yield
  File ".../local/lib/python3.11.........................../site-packages/pluggy/_callers.py", line 167, in _multicall
    teardown.throw(outcome._exception)
  File ".../local/lib/python3.11....../site-packages/_pytest/logging.py", line 846, in pytest_runtest_call
    yield from self._runtest_for(item, "call")
  File ".../local/lib/python3.11....../site-packages/_pytest/logging.py", line 829, in _runtest_for
    yield
  File ".../local/lib/python3.11.........................../site-packages/pluggy/_callers.py", line 167, in _multicall
    teardown.throw(outcome._exception)
  File ".../local/lib/python3.11.../site-packages/_pytest/capture.py", line 880, in pytest_runtest_call
    return (yield)
            ^^^^^
  File ".../local/lib/python3.11.........................../site-packages/pluggy/_callers.py", line 167, in _multicall
    teardown.throw(outcome._exception)
  File ".../local/lib/python3.11.../site-packages/_pytest/skipping.py", line 257, in pytest_runtest_call
    return (yield)
            ^^^^^
  File ".../local/lib/python3.11.........................../site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../local/lib/python3.11........./site-packages/_pytest/runner.py", line 174, in pytest_runtest_call
    item.runtest()
  File ".../local/lib/python3.11....../site-packages/_pytest/python.py", line 1627, in runtest
    self.ihook.pytest_pyfunc_call(pyfuncitem=self)
  File ".../local/lib/python3.11....../site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../local/lib/python3.11....../site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../local/lib/python3.11.........................../site-packages/pluggy/_callers.py", line 139, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File ".../local/lib/python3.11.........................../site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../local/lib/python3.11....../site-packages/_pytest/python.py", line 159, in pytest_pyfunc_call
    result = testfunction(**testargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../tests/datasets/test_errors_processor.py", line 459, in test_errors_basic
    assert (
AssertionError: assert [{'contexts.key': ['geo.city', 'geo.country_code', 'geo.region', 'geo.subdivision', 'runtime.build', 'runtime.name', 'runtime.version', 'trace.span_id', 'trace.trace_id'], 'contexts.value': ['fake_city', 'XY', 'fake_region', 'fake_subdivision', '3.7.6', 'CPython', '3.7.6', 'deadbeef', 'a40daa1fbf9c4407818b564c581dd80a'], 'culprit': 'snuba.clickhouse.http in write', 'deleted': 0, 'dist': 'dist', 'environment': 'prod', 'event_id': 'dcb9d002-cac5-48c7-95d1-c9adbfc68040', 'exception_frames.abs_path': ['.../local/bin/snuba'], 'exception_frames.colno': [None], 'exception_frames.filename': ['snuba'], 'exception_frames.function': ['<module>'], 'exception_frames.in_app': [False], 'exception_frames.lineno': [11], 'exception_frames.module': ['__main__'], 'exception_frames.package': [None], 'exception_frames.stack_level': [0], 'exception_stacks.mechanism_handled': [0], 'exception_stacks.mechanism_type': ['excepthook'], 'exception_stacks.type': ['ClickHouseError'], 'exception_stacks.value': ['[171] DB::Exception: Block structure mismatch'], 'group_id': 100, 'group_first_seen': 1755979033508, 'http_method': 'POST', 'http_referer': 'tagstore.something', 'ip_address_v4': '127.0.0.1', 'level': 'error', 'location': 'snuba/clickhouse/http.py', 'message_timestamp': 1756151833, 'message': '', 'modules.name': ['cffi', 'ipython-genutils', 'isodate'], 'modules.version': ['1.13.2', '0.2.0', '0.6.0'], 'num_processing_errors': 0, 'offset': 2, 'partition': 2, 'platform': 'python', 'primary_hash': '04233d08-ac90-cf6f-c015-b1be5932e7e2', 'project_id': 2, 'received': 1756151828, 'release': '1.0.0', 'retention_days': 90, 'sdk_integrations': ['argv', 'atexit', 'dedupe', 'excepthook', 'logging', 'modules', 'stdlib', 'threading'], 'sdk_name': 'sentry.python', 'sdk_version': '0.0.0.0.1', 'span_id': 3735928559, 'tags.key': ['environment', 'handled', 'level', 'mechanism', 'runtime', 'runtime.name', 'sentry:dist', 'sentry:release', 'sentry:user', 'server_name'], 'tags.value': ['prod', 'no', 'error', 'excepthook', 'CPython 3.7.6', 'CPython', 'dist', '1.0.0', 'this_is_me', 'snuba'], 'flags.key': [], 'flags.value': [], 'timestamp': 1756151833, 'timestamp_ms': 1756151833508, 'title': 'ClickHouseError: [171] DB::Exception: Block structure mismatch', 'trace_id': 'a40daa1f-bf9c-4407-818b-564c581dd80a', 'transaction_name': '', 'type': 'error', 'user_email': 'me@myself.org', 'user_id': 'still_me', 'user_name': 'me', 'user': 'this_is_me', 'version': '7', 'sample_weight': None}] == [{'project_id': 2, 'timestamp': 1756151833, 'event_id': 'dcb9d002-cac5-48c7-95d1-c9adbfc68040', 'platform': 'python', 'dist': 'dist', 'environment': 'prod', 'release': '1.0.0', 'ip_address_v4': '127.0.0.1', 'user': 'this_is_me', 'user_name': 'me', 'user_id': 'still_me', 'user_email': 'me@myself.org', 'sdk_name': 'sentry.python', 'sdk_version': '0.0.0.0.1', 'http_method': 'POST', 'http_referer': 'tagstore.something', 'trace_id': 'a40daa1f-bf9c-4407-818b-564c581dd80a', 'span_id': 3735928559, 'tags.key': ['environment', 'handled', 'level', 'mechanism', 'runtime', 'runtime.name', 'sentry:dist', 'sentry:release', 'sentry:user', 'server_name'], 'tags.value': ['prod', 'no', 'error', 'excepthook', 'CPython 3.7.6', 'CPython', 'dist', '1.0.0', 'this_is_me', 'snuba'], 'contexts.key': ['geo.city', 'geo.country_code', 'geo.region', 'geo.subdivision', 'runtime.build', 'runtime.name', 'runtime.version', 'trace.span_id', 'trace.trace_id'], 'contexts.value': ['fake_city', 'XY', 'fake_region', 'fake_subdivision', '3.7.6', 'CPython', '3.7.6', 'deadbeef', 'a40daa1fbf9c4407818b564c581dd80a'], 'partition': 2, 'offset': 2, 'message_timestamp': 1756151833, 'timestamp_ms': 1756151833508, 'retention_days': 90, 'deleted': 0, 'group_id': 100, 'group_first_seen': 1755979033000, 'primary_hash': '04233d08-ac90-cf6f-c015-b1be5932e7e2', 'received': 1756151828, 'message': '', 'title': 'ClickHouseError: [171] DB::Exception: Block structure mismatch', 'culprit': 'snuba.clickhouse.http in write', 'level': 'error', 'location': 'snuba/clickhouse/http.py', 'version': '7', 'type': 'error', 'exception_stacks.type': ['ClickHouseError'], 'exception_stacks.value': ['[171] DB::Exception: Block structure mismatch'], 'exception_stacks.mechanism_type': ['excepthook'], 'exception_stacks.mechanism_handled': [0], 'exception_frames.abs_path': ['.../local/bin/snuba'], 'exception_frames.colno': [None], 'exception_frames.filename': ['snuba'], 'exception_frames.lineno': [11], 'exception_frames.in_app': [False], 'exception_frames.package': [None], 'exception_frames.module': ['__main__'], 'exception_frames.function': ['<module>'], 'exception_frames.stack_level': [0], 'sample_weight': None, 'sdk_integrations': ['argv', 'atexit', 'dedupe', 'excepthook', 'logging', 'modules', 'stdlib', 'threading'], 'modules.name': ['cffi', 'ipython-genutils', 'isodate'], 'modules.version': ['1.13.2', '0.2.0', '0.6.0'], 'transaction_name': '', 'num_processing_errors': 0, 'flags.key': [], 'flags.value': []}]
  
  At index 0 diff: {'contexts.key': ['geo.city', 'geo.country_code', 'geo.region', 'geo.subdivision', 'runtime.build', 'runtime.name', 'runtime.version', 'trace.span_id', 'trace.trace_id'], 'contexts.value': ['fake_city', 'XY', 'fake_region', 'fake_subdivision', '3.7.6', 'CPython', '3.7.6', 'deadbeef', 'a40daa1fbf9c4407818b564c581dd80a'], 'culprit': 'snuba.clickhouse.http in write', 'deleted': 0, 'dist': 'dist', 'environment': 'prod', 'event_id': 'dcb9d002-cac5-48c7-95d1-c9adbfc68040', 'exception_frames.abs_path': ['.../local/bin/snuba'], 'exception_frames.colno': [None], 'exception_frames.filename': ['snuba'], 'exception_frames.function': ['<module>'], 'exception_frames.in_app': [False], 'exception_frames.lineno': [11], 'exception_frames.module': ['__main__'], 'exception_frames.package': [None], 'exception_frames.stack_level': [0], 'exception_stacks.mechanism_handled': [0], 'exception_stacks.mechanism_type': ['excepthook'], 'exception_stacks.type': ['ClickHouseError'], 'exception_stacks.value': ['[171] DB::Exception: Block structure mismatch'], 'group_id': 100, 'group_first_seen': 1755979033508, 'http_method': 'POST', 'http_referer': 'tagstore.something', 'ip_address_v4': '127.0.0.1', 'level': 'error', 'location': 'snuba/clickhouse/http.py', 'message_timestamp': 1756151833, 'message': '', 'modules.name': ['cffi', 'ipython-genutils', 'isodate'], 'modules.version': ['1.13.2', '0.2.0', '0.6.0'], 'num_processing_errors': 0, 'offset': 2, 'partition': 2, 'platform': 'python', 'primary_hash': '04233d08-ac90-cf6f-c015-b1be5932e7e2', 'project_id': 2, 'received': 1756151828, 'release': '1.0.0', 'retention_days': 90, 'sdk_integrations': ['argv', 'atexit', 'dedupe', 'excepthook', 'logging', 'modules', 'stdlib', 'threading'], 'sdk_name': 'sentry.python', 'sdk_version': '0.0.0.0.1', 'span_id': 3735928559, 'tags.key': ['environment', 'handled', 'level', 'mechanism', 'runtime', 'runtime.name', 'sentry:dist', 'sentry:release', 'sentry:user', 'server_name'], 'tags.value': ['prod', 'no', 'error', 'excepthook', 'CPython 3.7.6', 'CPython', 'dist', '1.0.0', 'this_is_me', 'snuba'], 'flags.key': [], 'flags.value': [], 'timestamp': 1756151833, 'timestamp_ms': 1756151833508, 'title': 'ClickHouseError: [171] DB::Exception: Block structure mismatch', 'trace_id': 'a40daa1f-bf9c-4407-818b-564c581dd80a', 'transaction_name': '', 'type': 'error', 'user_email': 'me@myself.org', 'user_id': 'still_me', 'user_name': 'me', 'user': 'this_is_me', 'version': '7', 'sample_weight': None} != {'project_id': 2, 'timestamp': 1756151833, 'event_id': 'dcb9d002-cac5-48c7-95d1-c9adbfc68040', 'platform': 'python', 'dist': 'dist', 'environment': 'prod', 'release': '1.0.0', 'ip_address_v4': '127.0.0.1', 'user': 'this_is_me', 'user_name': 'me', 'user_id': 'still_me', 'user_email': 'me@myself.org', 'sdk_name': 'sentry.python', 'sdk_version': '0.0.0.0.1', 'http_method': 'POST', 'http_referer': 'tagstore.something', 'trace_id': 'a40daa1f-bf9c-4407-818b-564c581dd80a', 'span_id': 3735928559, 'tags.key': ['environment', 'handled', 'level', 'mechanism', 'runtime', 'runtime.name', 'sentry:dist', 'sentry:release', 'sentry:user', 'server_name'], 'tags.value': ['prod', 'no', 'error', 'excepthook', 'CPython 3.7.6', 'CPython', 'dist', '1.0.0', 'this_is_me', 'snuba'], 'contexts.key': ['geo.city', 'geo.country_code', 'geo.region', 'geo.subdivision', 'runtime.build', 'runtime.name', 'runtime.version', 'trace.span_id', 'trace.trace_id'], 'contexts.value': ['fake_city', 'XY', 'fake_region', 'fake_subdivision', '3.7.6', 'CPython', '3.7.6', 'deadbeef', 'a40daa1fbf9c4407818b564c581dd80a'], 'partition': 2, 'offset': 2, 'message_timestamp': 1756151833, 'timestamp_ms': 1756151833508, 'retention_days': 90, 'deleted': 0, 'group_id': 100, 'group_first_seen': 1755979033000, 'primary_hash': '04233d08-ac90-cf6f-c015-b1be5932e7e2', 'received': 1756151828, 'message': '', 'title': 'ClickHouseError: [171] DB::Exception: Block structure mismatch', 'culprit': 'snuba.clickhouse.http in write', 'level': 'error', 'location': 'snuba/clickhouse/http.py', 'version': '7', 'type': 'error', 'exception_stacks.type': ['ClickHouseError'], 'exception_stacks.value': ['[171] DB::Exception: Block structure mismatch'], 'exception_stacks.mechanism_type': ['excepthook'], 'exception_stacks.mechanism_handled': [0], 'exception_frames.abs_path': ['.../local/bin/snuba'], 'exception_frames.colno': [None], 'exception_frames.filename': ['snuba'], 'exception_frames.lineno': [11], 'exception_frames.in_app': [False], 'exception_frames.package': [None], 'exception_frames.module': ['__main__'], 'exception_frames.function': ['<module>'], 'exception_frames.stack_level': [0], 'sample_weight': None, 'sdk_integrations': ['argv', 'atexit', 'dedupe', 'excepthook', 'logging', 'modules', 'stdlib', 'threading'], 'modules.name': ['cffi', 'ipython-genutils', 'isodate'], 'modules.version': ['1.13.2', '0.2.0', '0.6.0'], 'transaction_name': '', 'num_processing_errors': 0, 'flags.key': [], 'flags.value': []}
  
  Full diff:
    [
        {
            'contexts.key': [
                'geo.city',
                'geo.country_code',
                'geo.region',
                'geo.subdivision',
                'runtime.build',
                'runtime.name',
                'runtime.version',
                'trace.span_id',
                'trace.trace_id',
            ],
            'contexts.value': [
                'fake_city',
                'XY',
                'fake_region',
                'fake_subdivision',
                '3.7.6',
                'CPython',
                '3.7.6',
                'deadbeef',
                'a40daa1fbf9c4407818b564c581dd80a',
            ],
            'culprit': 'snuba.clickhouse.http in write',
            'deleted': 0,
            'dist': 'dist',
            'environment': 'prod',
            'event_id': 'dcb9d002-cac5-48c7-95d1-c9adbfc68040',
            'exception_frames.abs_path': [
                '.../local/bin/snuba',
            ],
            'exception_frames.colno': [
                None,
            ],
            'exception_frames.filename': [
                'snuba',
            ],
            'exception_frames.function': [
                '<module>',
            ],
            'exception_frames.in_app': [
                False,
            ],
            'exception_frames.lineno': [
                11,
            ],
            'exception_frames.module': [
                '__main__',
            ],
            'exception_frames.package': [
                None,
            ],
            'exception_frames.stack_level': [
                0,
            ],
            'exception_stacks.mechanism_handled': [
                0,
            ],
            'exception_stacks.mechanism_type': [
                'excepthook',
            ],
            'exception_stacks.type': [
                'ClickHouseError',
            ],
            'exception_stacks.value': [
                '[171] DB::Exception: Block structure mismatch',
            ],
            'flags.key': [],
            'flags.value': [],
  -         'group_first_seen': 1755979033000,
  ?                                        ^^
  +         'group_first_seen': 1755979033508,
  ?                                       + ^
            'group_id': 100,
            'http_method': 'POST',
            'http_referer': 'tagstore.something',
            'ip_address_v4': '127.0.0.1',
            'level': 'error',
            'location': 'snuba/clickhouse/http.py',
            'message': '',
            'message_timestamp': 1756151833,
            'modules.name': [
                'cffi',
                'ipython-genutils',
                'isodate',
            ],
            'modules.version': [
                '1.13.2',
                '0.2.0',
                '0.6.0',
            ],
            'num_processing_errors': 0,
            'offset': 2,
            'partition': 2,
            'platform': 'python',
            'primary_hash': '04233d08-ac90-cf6f-c015-b1be5932e7e2',
            'project_id': 2,
            'received': 1756151828,
            'release': '1.0.0',
            'retention_days': 90,
            'sample_weight': None,
            'sdk_integrations': [
                'argv',
                'atexit',
                'dedupe',
                'excepthook',
                'logging',
                'modules',
                'stdlib',
                'threading',
            ],
            'sdk_name': 'sentry.python',
            'sdk_version': '0.0.0.0.1',
            'span_id': 3735928559,
            'tags.key': [
                'environment',
                'handled',
                'level',
                'mechanism',
                'runtime',
                'runtime.name',
                'sentry:dist',
                'sentry:release',
                'sentry:user',
                'server_name',
            ],
            'tags.value': [
                'prod',
                'no',
                'error',
                'excepthook',
                'CPython 3.7.6',
                'CPython',
                'dist',
                '1.0.0',
                'this_is_me',
                'snuba',
            ],
            'timestamp': 1756151833,
            'timestamp_ms': 1756151833508,
            'title': 'ClickHouseError: [171] DB::Exception: Block structure mismatch',
            'trace_id': 'a40daa1f-bf9c-4407-818b-564c581dd80a',
            'transaction_name': '',
            'type': 'error',
            'user': 'this_is_me',
            'user_email': 'me@myself.org',
            'user_id': 'still_me',
            'user_name': 'me',
            'version': '7',
        },
    ]

To view more test analytics, go to the Test Analytics Dashboard
_{📋 Got 3 mins? Take this short survey to help us improve Test Analytics.}

getsentry-bot · 2025-09-04T23:05:09Z

revert failed (conflict? already reverted?) -- check the logs

thetruecpaul requested a review from a team August 14, 2025 17:54

thetruecpaul force-pushed the cpaul/issueagesorting/ingest branch 3 times, most recently from d91afef to 7df5339 Compare August 25, 2025 19:50

JoshFerge approved these changes Aug 25, 2025

View reviewed changes

thetruecpaul force-pushed the cpaul/issueagesorting/ingest branch 3 times, most recently from 6557196 to 60a7d7b Compare August 28, 2025 17:42

thetruecpaul force-pushed the cpaul/issueagesorting/ingest branch 8 times, most recently from dc4120d to e8697e3 Compare September 3, 2025 17:59

initial commit

49efe96

thetruecpaul force-pushed the cpaul/issueagesorting/ingest branch from e8697e3 to 49efe96 Compare September 3, 2025 18:51

thetruecpaul marked this pull request as ready for review September 4, 2025 15:51

thetruecpaul requested a review from a team as a code owner September 4, 2025 15:51

volokluev approved these changes Sep 4, 2025

View reviewed changes

thetruecpaul merged commit 4407d9b into master Sep 4, 2025
35 checks passed

thetruecpaul deleted the cpaul/issueagesorting/ingest branch September 4, 2025 21:41

thetruecpaul added the Trigger: Revert label Sep 4, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix(issues): Process new group_first_seen field on ingest #7355

fix(issues): Process new group_first_seen field on ingest #7355

Uh oh!

thetruecpaul commented Aug 14, 2025 •

edited

Loading

Uh oh!

codecov bot commented Aug 14, 2025 •

edited

Loading

Uh oh!

Uh oh!

getsentry-bot commented Sep 4, 2025

Uh oh!

Uh oh!

Uh oh!

fix(issues): Process new group_first_seen field on ingest #7355

fix(issues): Process new group_first_seen field on ingest #7355

Uh oh!

Conversation

thetruecpaul commented Aug 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Aug 14, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

❌ 1 Tests Failed:

Uh oh!

Uh oh!

getsentry-bot commented Sep 4, 2025

Uh oh!

Uh oh!

thetruecpaul commented Aug 14, 2025 •

edited

Loading

codecov bot commented Aug 14, 2025 •

edited

Loading