Skip to content

Commit e51b3dd

Browse files
rtosholdings-botrtosholdings-bot
authored
Merge pull request #376 from rtosholdings/latest-v1.17.1
v1.17.1
2 parents 032583b + 77a7623 commit e51b3dd

File tree

5 files changed

+271
-6
lines changed

5 files changed

+271
-6
lines changed

‎.github/workflows/python-package.yml

Lines changed: 18 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ jobs:
1919
shell: bash -l {0}
2020
strategy:
2121
matrix:
22-
python-version: ["3.10", "3.11"]
22+
python-version: ["3.10", "3.11", "3.12"]
2323
steps:
2424
- name: Checkout repo
2525
uses: actions/checkout@v4
@@ -60,7 +60,7 @@ jobs:
6060
run:
6161
shell: bash -l {0}
6262
env:
63-
python_version: 3.11
63+
python_version: 3.12
6464
ANACONDA_USER: rtosholdings
6565
steps:
6666
- name: Checkout repo
@@ -147,7 +147,14 @@ jobs:
147147
matrix:
148148
os: ["ubuntu-latest", "windows-2022"]
149149
python-version: ["3.10", "3.11"]
150-
numpy-version: [1.23, 1.24]
150+
numpy-version: [1.23, 1.24, 1.25, 1.26]
151+
include:
152+
- os: "ubuntu-latest"
153+
python-version: "3.12"
154+
numpy-version: "1.26"
155+
- os: "windows-2022"
156+
python-version: "3.12"
157+
numpy-version: "1.26"
151158
env:
152159
ANACONDA_USER: rtosholdings
153160
BUILD_VERSION: ${{needs.conda_build.outputs.build_version}}
@@ -211,7 +218,14 @@ jobs:
211218
matrix:
212219
os: [ubuntu-latest, windows-2022]
213220
python-version: ["3.10", "3.11"]
214-
numpy-version: [1.23, 1.24]
221+
numpy-version: [1.23, 1.24, 1.25, 1.26]
222+
include:
223+
- os: "ubuntu-latest"
224+
python-version: "3.12"
225+
numpy-version: "1.26"
226+
- os: "windows-2022"
227+
python-version: "3.12"
228+
numpy-version: "1.26"
215229
steps:
216230
- name: Checkout repo (sparse)
217231
uses: actions/checkout@v4

‎docs/source/tutorial/tutorial.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,7 @@ Appendix
3434
tutorial_numpy_rt
3535
tutorial_cat_reduce
3636
tutorial_cat_adv_instantiation
37+
tutorial_rt_logging
3738

3839
More In-Depth Topics
3940
--------------------

‎docs/source/tutorial/tutorial_rt_logging.ipynb

Lines changed: 248 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,248 @@
1+
{
2+
"cells": [
3+
{
4+
"cell_type": "markdown",
5+
"metadata": {},
6+
"source": [
7+
"# Riptide Logging\n",
8+
"\n",
9+
"Riptable supports logging from riptide_cpp, a C++ library that Riptable relies on for multithreaded processing of arrays and the SDS file format. This logging capability enables you to examine and understand the native functions called by Riptable."
10+
]
11+
},
12+
{
13+
"cell_type": "markdown",
14+
"metadata": {},
15+
"source": [
16+
"The following examples require you to import the following modules and functions:"
17+
]
18+
},
19+
{
20+
"cell_type": "code",
21+
"execution_count": 1,
22+
"metadata": {},
23+
"outputs": [],
24+
"source": [
25+
"import riptable as rt\n",
26+
"import riptide_cpp as rc\n",
27+
"import logging\n",
28+
"import sys\n",
29+
"from riptable.rt_logging import enable_riptide_logs, disable_riptide_logs"
30+
]
31+
},
32+
{
33+
"cell_type": "markdown",
34+
"metadata": {},
35+
"source": [
36+
"View a list of available riptide_cpp loggers using `rc.GetRiptideLoggers()`:"
37+
]
38+
},
39+
{
40+
"cell_type": "code",
41+
"execution_count": 2,
42+
"metadata": {},
43+
"outputs": [
44+
{
45+
"data": {
46+
"text/plain": [
47+
"['riptable.riptide_cpp.SDSFile',\n",
48+
" 'riptable.riptide_cpp.Recycler',\n",
49+
" 'riptable.riptide_cpp.GroupBy']"
50+
]
51+
},
52+
"execution_count": 2,
53+
"metadata": {},
54+
"output_type": "execute_result"
55+
}
56+
],
57+
"source": [
58+
"rc.GetRiptideLoggers()"
59+
]
60+
},
61+
{
62+
"cell_type": "markdown",
63+
"metadata": {},
64+
"source": [
65+
"Logging in riptide_cpp is disabled by default. To enable logging capabilities, you must call the `enable_riptide_logs()` function from the `rt_logging` Riptable module:"
66+
]
67+
},
68+
{
69+
"cell_type": "code",
70+
"execution_count": 3,
71+
"metadata": {},
72+
"outputs": [],
73+
"source": [
74+
"enable_riptide_logs()"
75+
]
76+
},
77+
{
78+
"cell_type": "markdown",
79+
"metadata": {},
80+
"source": [
81+
"After enabling the riptide_cpp loggers, use the `logging` module from the Python standard library to configure them.\n",
82+
"\n",
83+
"\n"
84+
]
85+
},
86+
{
87+
"cell_type": "markdown",
88+
"metadata": {},
89+
"source": [
90+
"Setup an stdout handler with a custom formatter:"
91+
]
92+
},
93+
{
94+
"cell_type": "code",
95+
"execution_count": 4,
96+
"metadata": {},
97+
"outputs": [],
98+
"source": [
99+
"handler = logging.StreamHandler(sys.stdout)\n",
100+
"handler.setFormatter(logging.Formatter('%(name)s | %(levelname)s | %(message)s'))"
101+
]
102+
},
103+
{
104+
"cell_type": "markdown",
105+
"metadata": {},
106+
"source": [
107+
"Configure the SDSFile and GroupBy loggers. The Riptable and riptide_cpp loggers are in the following hierarchy:\n",
108+
"\n",
109+
"- riptable: root logger\n",
110+
" - riptable [riptable.riptable]: Logs for riptable side\n",
111+
" - rt_dataset [riptable.riptable.rt_dataset]\n",
112+
" - ...\n",
113+
" - riptide_cpp [riptable.riptide_cpp]: Logs for riptide_cpp\n",
114+
" - SDSFile [riptable.riptide_cpp.SDSFile]\n",
115+
" - GroupBy [riptable.riptide_cpp.GroupBy]\n",
116+
" - ..."
117+
]
118+
},
119+
{
120+
"cell_type": "code",
121+
"execution_count": 5,
122+
"metadata": {},
123+
"outputs": [],
124+
"source": [
125+
"sdsfile = logging.getLogger(\"riptable.riptide_cpp.SDSFile\")\n",
126+
"sdsfile.setLevel(logging.DEBUG)\n",
127+
"sdsfile.addHandler(handler)\n",
128+
"groupby = logging.getLogger(\"riptable.riptide_cpp.GroupBy\")\n",
129+
"groupby.setLevel(logging.DEBUG)\n",
130+
"groupby.addHandler(handler)"
131+
]
132+
},
133+
{
134+
"cell_type": "markdown",
135+
"metadata": {},
136+
"source": [
137+
"Create an example Dataset, and call functions that record events to the SDSFile and GroupBy\n",
138+
"loggers:"
139+
]
140+
},
141+
{
142+
"cell_type": "code",
143+
"execution_count": 6,
144+
"metadata": {},
145+
"outputs": [
146+
{
147+
"data": {
148+
"text/html": [
149+
"\n",
150+
"<html>\n",
151+
"<head>\n",
152+
"<style>\n",
153+
" table.tbl-5286 tbody td{white-space: nowrap;}\n",
154+
" table.tbl-5286 .lc{font-weight:bold;background-color: var( --jp-rendermime-table-row-hover-background);}\n",
155+
" table.tbl-5286 .lg{background-color: #66da9940;}\n",
156+
" table.tbl-5286 .lp{font-weight:bold;background-color: #ac66da40;}\n",
157+
" table.tbl-5286 .msc{font-weight:normal;background-color:#00000011;}\n",
158+
" table.tbl-5286 .al{text-align:left;}\n",
159+
" table.tbl-5286 .ar{text-align:right;}\n",
160+
" table.tbl-5286 .ac{text-align:center;}\n",
161+
" table.tbl-5286 .bld{font-weight:bold;}\n",
162+
" table.tbl-5286 .it{font-style:italic;}\n",
163+
" table.tbl-5286 .ul{text-decoration:underline;}\n",
164+
" table.tbl-5286 .st{text-decoration:line-through;}\n",
165+
"\n",
166+
"</style>\n",
167+
"</head>\n",
168+
" <body><table class='tbl-5286'><thead><tr><td class='lc bld ar'>#</td><td class='lc bld ar'>a</td><td class='lc bld ar'>b</td><td class='lc bld ar'>c</td></tr></thead><tbody><tr><td class='lc ar '>0</td><td class=' ar '>1</td><td class=' ar '>4</td><td class=' ar '>9</td></tr><tr><td class='lc ar '>1</td><td class=' ar '>1</td><td class=' ar '>5</td><td class=' ar '>9</td></tr><tr><td class='lc ar '>2</td><td class=' ar '>2</td><td class=' ar '>6</td><td class=' ar '>6</td></tr></tbody></table></body></html>\n",
169+
"\n",
170+
"[3 rows x 3 columns] total bytes: 72.0 B"
171+
],
172+
"text/plain": [
173+
"\u001b[1;36m#\u001b[0m \u001b[1;36ma\u001b[0m \u001b[1;36mb\u001b[0m \u001b[1;36mc\u001b[0m\n",
174+
"- - - -\n",
175+
"\u001b[1;36m0\u001b[0m 1\u001b[0m 4\u001b[0m 9\u001b[0m\n",
176+
"\u001b[1;36m1\u001b[0m 1\u001b[0m 5\u001b[0m 9\u001b[0m\n",
177+
"\u001b[1;36m2\u001b[0m 2\u001b[0m 6\u001b[0m 6\u001b[0m\n",
178+
"\n",
179+
"[3 rows x 3 columns] total bytes: 72.0 B"
180+
]
181+
},
182+
"metadata": {},
183+
"output_type": "display_data"
184+
},
185+
{
186+
"name": "stdout",
187+
"output_type": "stream",
188+
"text": [
189+
"riptable.riptide_cpp.GroupBy | DEBUG | GroupByCall: col 0 ==> outsize 3 len: 3 numpy types 7 --> 7 8 8\n",
190+
"riptable.riptide_cpp.GroupBy | DEBUG | GroupByCall: 0 typeCall 4 numpyOutType 7\n",
191+
"riptable.riptide_cpp.GroupBy | DEBUG | !!groupby done 1\n",
192+
"riptable.riptide_cpp.GroupBy | DEBUG | !!groupby returning\n",
193+
"\n",
194+
"riptable.riptide_cpp.SDSFile | DEBUG | Clearing errors\n",
195+
"riptable.riptide_cpp.SDSFile | DEBUG | linux handle open\n",
196+
"riptable.riptide_cpp.SDSFile | DEBUG | Using fileOffset 0\n",
197+
"riptable.riptide_cpp.SDSFile | DEBUG | main offsets 1024 1536 2048\n",
198+
"riptable.riptide_cpp.SDSFile | DEBUG | meta compressed to 148 vs 187 512\n",
199+
"riptable.riptide_cpp.SDSFile | DEBUG | Current fileoffset is 0\n",
200+
"riptable.riptide_cpp.SDSFile | DEBUG | -\n",
201+
"riptable.riptide_cpp.SDSFile | DEBUG | -\n",
202+
"riptable.riptide_cpp.SDSFile | DEBUG | -\n",
203+
"riptable.riptide_cpp.SDSFile | DEBUG | [0] started 24 0x7fc21c000b70\n",
204+
"riptable.riptide_cpp.SDSFile | DEBUG | [1] started 24 0x7fc214000b70\n",
205+
"riptable.riptide_cpp.SDSFile | DEBUG | [2] started 24 0x7fc218000b70\n",
206+
"riptable.riptide_cpp.SDSFile | DEBUG | [1][1] seek to fileOffset 2048 sz: 24\n",
207+
"riptable.riptide_cpp.SDSFile | DEBUG | [0][0] seek to fileOffset 2560 sz: 24\n",
208+
"riptable.riptide_cpp.SDSFile | DEBUG | [2][2] seek to fileOffset 3072 sz: 24\n",
209+
"riptable.riptide_cpp.SDSFile | DEBUG | End of compressing\n",
210+
"riptable.riptide_cpp.SDSFile | DEBUG | SDS: Array first offset --- 2048 Total comp size 1536\n",
211+
"riptable.riptide_cpp.SDSFile | DEBUG | Total arrays written 3 --- 512 512\n",
212+
"riptable.riptide_cpp.SDSFile | DEBUG | array block offset --- 1536 384 2560\n"
213+
]
214+
}
215+
],
216+
"source": [
217+
"import tempfile\n",
218+
"\n",
219+
"with tempfile.TemporaryDirectory() as tmpdir:\n",
220+
" ds = rt.Dataset({'a': rt.FA([1, 1, 2]), 'b': rt.FA([4, 5, 6])})\n",
221+
" ds.c = rt.Cat(ds.a).sum(ds.b, transform=True)\n",
222+
" ds.save(f'{tmpdir}/a.sds')\n",
223+
" display(ds)"
224+
]
225+
}
226+
],
227+
"metadata": {
228+
"kernelspec": {
229+
"display_name": "riptide-dev",
230+
"language": "python",
231+
"name": "python3"
232+
},
233+
"language_info": {
234+
"codemirror_mode": {
235+
"name": "ipython",
236+
"version": 3
237+
},
238+
"file_extension": ".py",
239+
"mimetype": "text/x-python",
240+
"name": "python",
241+
"nbconvert_exporter": "python",
242+
"pygments_lexer": "ipython3",
243+
"version": "3.11.8"
244+
}
245+
},
246+
"nbformat": 4,
247+
"nbformat_minor": 2
248+
}

‎riptable/rt_categorical.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6347,7 +6347,8 @@ def __del__(self):
63476347
# python has trouble deleting objects with circular references
63486348
if hasattr(self, "_categories_wrap"):
63496349
del self._categories_wrap
6350-
del self._grouping
6350+
if hasattr(self, "_grouping"):
6351+
del self._grouping
63516352

63526353
# ------------------------------------------------------------
63536354
@classmethod

‎riptable/rt_sharedmemory.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -114,7 +114,8 @@ def check_shared_memory_limit(cls):
114114
if size > cls.SM_LIMIT:
115115
readable = cls._convert_bytes(size)
116116
warnings.warn(
117-
f"!!!Shared memory is using {readable}. Consider using sm.clear() to remove.", stacklevel=2
117+
f"!!!Shared memory is using {readable}. Consider using riptable.SharedMemory.clear() to remove.",
118+
stacklevel=2,
118119
)
119120

120121
# ------------------------------------------------------------------------------------

0 commit comments

Comments
 (0)