Skip to content

Refactor documentation #499

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 36 commits into from
Jun 5, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
30bef1d
Refactor to get started.
JoeZiminski Mar 21, 2025
30f8a2c
Add getting started section.
JoeZiminski Mar 21, 2025
02f3b0c
Add datashuttle examples.
JoeZiminski Mar 21, 2025
594ea13
Add example scripts.
JoeZiminski Mar 21, 2025
323b250
More edits to documentation.
JoeZiminski Mar 22, 2025
81f1e33
More tidying up.
JoeZiminski Mar 22, 2025
46109a2
Add TUI to valiate configs.
JoeZiminski Mar 22, 2025
313eec4
Add examples.
JoeZiminski Mar 22, 2025
935daf7
Fix error in formatting.
JoeZiminski Mar 22, 2025
8930c11
Fix wrong filenames in walkthrough.
JoeZiminski Mar 23, 2025
cd2d7bc
Some tidy ups, update roadmap.
JoeZiminski May 19, 2025
ce662cb
More tidying up.
JoeZiminski May 19, 2025
86cf5a6
Fix small errors.
JoeZiminski May 19, 2025
2237152
More small tidy ups.
JoeZiminski May 19, 2025
c829cf6
Final tidy ups for documentation.
JoeZiminski May 21, 2025
12eaf0a
Tidy up roadmap.
JoeZiminski May 22, 2025
4373bd6
Remove full stops from cards.
JoeZiminski May 23, 2025
ff3ec9c
Use only python part of example script.
JoeZiminski May 27, 2025
a17d646
Move 'choose a terminal' to user guides.
JoeZiminski May 27, 2025
e5df2b9
First draft on clearer get-started page.
JoeZiminski May 27, 2025
0542d40
Link to a python install tutorial.
JoeZiminski May 27, 2025
e9ac4d5
Link to a python install tutorial.
JoeZiminski May 27, 2025
f817aea
Fix grammer on lab projects page.
JoeZiminski May 27, 2025
06a8082
Don't capitalise all words in a title.
JoeZiminski May 27, 2025
3eea681
Update text align.
JoeZiminski May 27, 2025
51edce2
Add more details to index page.
JoeZiminski May 27, 2025
a2eacff
Revert changes to .css
JoeZiminski May 27, 2025
fb42c13
rework card slightly
JoeZiminski May 27, 2025
95244d3
Fix center alignment, for header only.
JoeZiminski May 27, 2025
25195a1
world file -> filepath.
JoeZiminski May 27, 2025
f594606
Add fixed title.
JoeZiminski May 27, 2025
766379f
Fix title levels/
JoeZiminski May 27, 2025
229a875
Fix name typo.
JoeZiminski Jun 2, 2025
362d25e
Update install python link.
JoeZiminski Jun 3, 2025
5ac9727
Make validate from path page orphan to supress warning.
JoeZiminski Jun 3, 2025
2a8ed0f
Fix broken link.
JoeZiminski Jun 3, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 0 additions & 4 deletions docs/source/_static/css/custom.css
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,3 @@ html[data-theme=light] {
height: auto;
margin: 5px auto;
}

.center {
text-align: center;
}
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
98 changes: 72 additions & 26 deletions docs/source/index.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,20 @@
:html_theme.sidebar_secondary.remove:

{.center}
# **datashuttle**
<!-- We want to have a centered title, which is difficult in sphinx without centering
the entire page. We need the title here otherwise the tab title defaults to <no-title>.
Therefore, add the title and hide it, then add a custom centered title.
-->

<p style="text-align: center; font-size: 22px;">The tool to automate neuroscience project folder creation, validation and transfer.</p>
```{raw} html
<div style="height: 0; visibility: hidden;">
```
# datashuttle
```{raw} html
</div>
```

<p style="text-align: center; font-size: 48px;"><b>datashuttle</b></p>
<p style="text-align: center; font-size: 22px;">Automate the creation, validation and transfer of neuroscience project folders.</p>

```{image} _static/datashuttle-overview-light.png
:alt: My Logo
Expand All @@ -19,53 +30,88 @@
```
<br>

::::{grid} 1 2 2 3

::::{grid} 1 2 2 4
:gutter: 4

:::{grid-item-card} Tutorials
:link: pages/tutorials

:::{grid-item-card} Get started
:link: pages/get_started/index
:link-type: doc

Get started with ``datashuttle``
:::


:::{grid-item-card} User guides
:link: pages/user_guides/index
:link-type: doc

Walkthrough **datashuttle**.
Explore ``datashuttle``'s features
:::

:::{grid-item-card} How To
:link: pages/how_tos
:::{grid-item-card} Examples
:link: pages/examples/index
:link-type: doc

Short guides on specific actions.
``datashuttle`` in the real world
:::

:::{grid-item-card} Python API
:link: pages/api_index
:link-type: doc

Full Python reference.
Full Python reference
:::
::::

**datashuttle** creates and validates projects standardised to the
[NeuroBlueprint](https://neuroblueprint.neuroinformatics.dev)
specification.
::::

Dive right into **datashuttle** with our
[Getting Started Tutorial](tutorial-getting-started)
or targeted [How-To Guides](how-tos). \
It can be used through a graphical interface or Python API.
A lack of project standardization in systems neuroscience
[hinders data sharing and collaboration](https://neuroinformatics.dev/blog/neuroblueprint.html),
creating barriers to reproducibility and scientific progress.

``datashuttle`` helps standardise experimental
projects by automating folder creation and transfer
during acquisition and analysis. Its graphical interface or Python API builds
folder trees according to the [NeuroBlueprint](https://neuroblueprint.neuroinformatics.dev)
specification. Automation and validation ensures that no errors, such as duplicate session
names or incorrect dates, slip into the project.

Data can be transferred between acquisition, storage and analysis
machines with a single function call or button click. Standardisation makes
folder names predictable, meaning it is easy to transfer specific combinations
of subjects, sessions or data-types with ``datashuttle``.

Folders are standardised to the
[NeuroBlueprint](https://neuroblueprint.neuroinformatics.dev) specification:

```{image} /_static/NeuroBlueprint_project_tree_dark.png
:align: center
:class: only-dark
:width: 550px
```
```{image} /_static/NeuroBlueprint_project_tree_light.png
:align: center
:class: only-light
:width: 550px
```

Don't hesitate to get in contact through our
[GitHub Issues](https://github.com/neuroinformatics-unit/datashuttle/issues)
or
[Zulip Chat.](https://neuroinformatics.zulipchat.com/#narrow/stream/405999-DataShuttle)
Dive in with our [Getting Started page](pages/get_started/index)
or targeted [User Guides](pages/user_guides/index).

Have questions, issues or feedback? Get in contact through
[GitHub issues](https://github.com/neuroinformatics-unit/datashuttle/issues)
or our
[Zulip chat.](https://neuroinformatics.zulipchat.com/#narrow/stream/405999-DataShuttle)

```{toctree}
:maxdepth: 2
:caption: index
:hidden:

pages/tutorials
pages/how_tos
pages/community/index.md
pages/get_started/index
pages/user_guides/index
pages/examples/index
pages/community/index
pages/api_index
```
12 changes: 6 additions & 6 deletions docs/source/pages/community/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,17 +12,17 @@ The core development team will support you in contributing code, regardless of y

## Creating a development environment

For detailed instructions on installing **datashuttle** along with
For detailed instructions on installing ``datashuttle`` along with
all necessary packages for development, refer to the `Developers` tab on the
[How to install **datashuttle**](how-to-install) page.
[How to install ``datashuttle``](how-to-install) page.

Briefly, after installing Rclone, the **datashuttle** repository can be cloned:
Briefly, after installing Rclone, the ``datashuttle`` repository can be cloned:

```sh
git clone git@github.com:neuroinformatics-unit/datashuttle.git
```

and **datashuttle** pip-installed with the developer tag:
and ``datashuttle`` pip-installed with the developer tag:


```sh
Expand Down Expand Up @@ -117,8 +117,8 @@ Make sure to provide docstrings for all public functions, classes, and methods.

## Contributing documentation

It is very important that the documentation for **datashuttle** is clear,
accurate and concise. A key principle of **datashuttle** is it should
It is very important that the documentation for ``datashuttle`` is clear,
accurate and concise. A key principle of ``datashuttle`` is it should
make data acquisition simple and easy—therefore it should be
well-documented.

Expand Down
16 changes: 8 additions & 8 deletions docs/source/pages/community/guiding-principles.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,16 @@

In this document we lay out our vision for
[**NeuroBlueprint**](https://neuroblueprint.neuroinformatics.dev/)
and **datashuttle** through three guiding principles.
and ``datashuttle`` through three guiding principles.

For additional detail, please refer to our blog posts motivating
[**NeuroBlueprint**](https://neuroinformatics.dev/blog/neuroblueprint.html)
and
[**datashuttle**](https://neuroinformatics.dev/blog/datashuttle.html).
[``datashuttle``](https://neuroinformatics.dev/blog/datashuttle.html).

The three guiding principles driving the development of
**NeuroBlueprint**
and **datashuttle** are:
and ``datashuttle`` are:

## Align with existing community initiatives

Expand All @@ -30,15 +30,15 @@ The guiding principle is that some standardisation is better than no standardisa
**NeuroBlueprint** is not intended as a rival or replacement for more established
specifications, but rather as a stepping stone towards full standardisation.
We aim to align as closely as possible to the BIDS specification and in future
to provide conversion to NWB through **datashuttle**.
to provide conversion to NWB through ``datashuttle``.

## Strive to be lightweight

**NeuroBlueprint** aims to keep the specification as lightweight
as possible. There is no benefit in the specification proliferating
as it develops such that it ends up duplicating BIDS in scope.

In the initial versions (**datashuttle** v0.4, **NeuroBlueprint** v0.2),
In the initial versions (``datashuttle`` v0.4, **NeuroBlueprint** v0.2),
the goal is to have a simple organisational
system in which the raw data for different datatypes (e.g. ephys, behaviour)
can be automatically discovered in any given project.
Expand All @@ -49,17 +49,17 @@ New standards will always be goal-orientated and customisable, only required
to achieve a particular research goal. It is anticipated that as these 'rules'
are adopted, projects will become progressively easier to convert to BIDS or NWB.

The role of **datashuttle** will be to enforce these rules in a flexible manner.
The role of ``datashuttle`` will be to enforce these rules in a flexible manner.

## Keep releases versioned and modular

**NeuroBlueprint** and **datashuttle** will be
**NeuroBlueprint** and ``datashuttle`` will be
properly versioned and modular, enabling smooth development over time.
New versions will prioritize backward compatibility, with changes that break
compatibility occurring infrequently and only aimed at enhancing alignment with existing standards.

Typically, new versions will include new
sets of standardisation rules,
which users may choose to adopt or not. **datashuttle** will
which users may choose to adopt or not. ``datashuttle`` will
enforce these in a flexible way, with efforts being
made to minimise changes to its API as much as possible.
12 changes: 6 additions & 6 deletions docs/source/pages/community/index.md
Original file line number Diff line number Diff line change
@@ -1,21 +1,21 @@
# Community


Contributions to **datashuttle** are strongly encouraged. If you would like to submit
a bug fix or implement a new feature, please feel free to do so! We are a
Contributions to ``datashuttle`` are strongly encouraged. If you would like to submit
a bug fix or implement a new feature, please feel free to open an issue or pull request. We are a
friendly bunch and happy to answer any questions before,
during or after a contribution. See the [Contributing Guide](contributing.md) for more details.

We are generally very keen to hear feedback on what works and doesn't work
in **datashuttle** and
in ``datashuttle`` and
[**NeuroBlueprint**](https://neuroblueprint.neuroinformatics.dev).
Please get in contact with any thoughts on how these tools can be
improved by opening a
[GitHub Issue](https://github.com/neuroinformatics-unit/datashuttle/issues)
[GitHub issue](https://github.com/neuroinformatics-unit/datashuttle/issues)
or by joining our
[Zulip Chat](https://neuroinformatics.zulipchat.com/#narrow/stream/405999-DataShuttle).
[Zulip chat](https://neuroinformatics.zulipchat.com/#narrow/stream/405999-DataShuttle).

To see what we have planned for the future of **NeuroBlueprint** and **datashuttle**, visit
To see what we have planned for the future of **NeuroBlueprint** and ``datashuttle``, visit
our [Guiding Principles](guiding-principles.md) and [Roadmap](roadmap.md).


Expand Down
18 changes: 10 additions & 8 deletions docs/source/pages/community/roadmap.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Roadmap

The roadmap for **datashuttle**
The roadmap for ``datashuttle``
contains our development plans for the package.
The roadmap is responsive to the requirements
of the community and we encourage people to get in
Expand All @@ -12,14 +12,16 @@ creation, transfer and logging of projects organised to
support implementation of metadata standardisation planned for the next
NeuroBlueprint version and facilitate conversion to NWB and customisation.

**Q4 2024**
**Q3 2025**
- Support Google Drive and Amazon Web Service and central storage drives.

Standardised metadata support.

**Q2 2025**
**Q4 2025**
- Export of NeuroBlueprint projects to [Neurodata Without Borders](https://www.nwb.org/) format.

Export of NeuroBlueprint projects to [Neurodata Without Borders](https://www.nwb.org/) format.
**2026**
- Support for customising data structure (e.g. specific mandatory elements or new file types).

**Q4 2025**

Support for customising data structure (e.g. specific mandatory elements or new file types).
## Notes on previous goals
Q4 2024, Standardised metadata support. After [discussion](https://github.com/neuroinformatics-unit/NeuroBlueprint/issues/30),
this was put on hold to await further consultation with researchers.
53 changes: 53 additions & 0 deletions docs/source/pages/examples/acquisition-script.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
:html_theme.sidebar_secondary.remove:

# Acquisition script

This script shows how Laura Schwarz (O'Keefe Lab, [Sainsbury Wellcome Centre](https://www.sainsburywellcome.org/web/))
uses ``datashuttle`` to create project folders during the acquisition of a behavioural
task in mice.

```python
def get_file_path():

# get your project
project = DataShuttle("social_sleaping")

# create a prompt to enter the ID number
# (which we will use to get the subject number)
id_number = input("Enter ID number: ")
sub = ID_DICT.get(id_number)

# get your session number and create a new folder
# for the session you are about to record.
# the function get_next_ses() normally checks for the next session
# if you are recording for a new subject you can use it as well to create
# the first session folder for this subject.
session = project.get_next_ses(top_level_folder="rawdata",
sub=f"sub-{sub}_id-{id_number}")

# create the folders
created_folders = project.create_folders(
top_level_folder="rawdata",
sub_names=f"sub-{sub}_id-{id_number}",
ses_names=f"{session}_@DATETIME@",
datatype=["behav"]
)
# create a prompt to enter the experiment information and
# conspecific ID for social experiments.
# (this is only important for the video file name and might not be
# relevant for you.)
exp_number = input("Enter Experiment condition: ")
comsp_id = input("Enter Conspecific ID: ")

# print the start of your acquisition
print(datetime.now())

# create the video file name
file_name_video_1 = f"{exp_number}_{comsp_id}.avi"

# create the path to the video file
file_path1 = created_folders['behav'][0] / file_name_video_1
file_path1.touch()

return file_path1
```
41 changes: 41 additions & 0 deletions docs/source/pages/examples/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
:html_theme.sidebar_secondary.remove:

(examples)=
# Examples


::::{grid} 1 1 2 2
:gutter: 4


:::{grid-item-card} {fas}`desktop;sd-text-primary` Acquisition script
:link: acquisition-script
:link-type: doc
:class-img-top: tutorial-link-image

Using ``datashuttle`` to create project folders in a behavioural-data acquisition script

:::


:::{grid-item-card} {fas}`desktop;sd-text-primary` Validate all projects in a lab
:link: lab-project-checker
:link-type: doc
:class-img-top: tutorial-link-image

A script used to validate every project in a lab at once

:::

::::


```{toctree}
:maxdepth: 2
:caption: Examples
:hidden:

acquisition-script
lab-project-checker

```
Loading
Loading