Refactor documentation (#499)

* Refactor to get started.

* Add getting started section.

* Add datashuttle examples.

* Add example scripts.

* More edits to documentation.

* More tidying up.

* Add TUI to valiate configs.

* Add examples.

* Fix error in formatting.

* Fix wrong filenames in walkthrough.

* Some tidy ups, update roadmap.

* More tidying up.

* Fix small errors.

* More small tidy ups.

* Final tidy ups for documentation.

* Tidy up roadmap.

* Remove full stops from cards.

* Use only python part of example script.

* Move 'choose a terminal' to user guides.

* First draft on clearer get-started page.

* Link to a python install tutorial.

* Link to a python install tutorial.

* Fix grammer on lab projects page.

* Don't capitalise all words in a title.

* Update text align.

* Add more details to index page.

* Revert changes to .css

* rework card slightly

* Fix center alignment, for header only.

* world file -> filepath.

* Add fixed title.

* Fix title levels/

* Fix name typo.

* Update install python link.

* Make validate from path page orphan to supress warning.

* Fix broken link.

Author: JoeZiminski

docs/source/_static/css/custom.css

@@ -46,7 +46,3 @@ html[data-theme=light] {
   height: auto;
   margin: 5px auto;
 }
-
-.center {
-    text-align: center;
-}

docs/source/_static/screenshots/how-to-choose-a-terminal-bad-dark.png renamed to docs/source/_static/screenshots/choose-a-terminal-bad-dark.png

@@ -1,9 +1,20 @@
 :html_theme.sidebar_secondary.remove:
 
-{.center}
-# **datashuttle**
+<!-- We want to have a centered title, which is difficult in sphinx without centering
+the entire page. We need the title here otherwise the tab title defaults to <no-title>.
+Therefore, add the title and hide it, then add a custom centered title.
+-->
 
-<p style="text-align: center; font-size: 22px;">The tool to automate neuroscience project folder creation, validation and transfer.</p>
+```{raw} html
+<div style="height: 0; visibility: hidden;">
+```
+# datashuttle
+```{raw} html
+</div>
+```
+
+<p style="text-align: center; font-size: 48px;"><b>datashuttle</b></p>
+<p style="text-align: center; font-size: 22px;">Automate the creation, validation and transfer of neuroscience project folders.</p>
 
 ```{image} _static/datashuttle-overview-light.png
 :alt: My Logo
@@ -19,53 +30,88 @@
 ```
 <br>
 
-::::{grid} 1 2 2 3
+
+::::{grid} 1 2 2 4
 :gutter: 4
 
-:::{grid-item-card} Tutorials
-:link: pages/tutorials
+
+:::{grid-item-card} Get started
+:link: pages/get_started/index
+:link-type: doc
+
+Get started with ``datashuttle``
+:::
+
+
+:::{grid-item-card} User guides
+:link: pages/user_guides/index
 :link-type: doc
 
-Walkthrough **datashuttle**.
+Explore ``datashuttle``'s features
 :::
 
-:::{grid-item-card} How To
-:link: pages/how_tos
+:::{grid-item-card} Examples
+:link: pages/examples/index
 :link-type: doc
 
-Short guides on specific actions.
+``datashuttle`` in the real world
 :::
 
 :::{grid-item-card} Python API
 :link: pages/api_index
 :link-type: doc
 
-Full Python reference.
+Full Python reference
 :::
-::::
 
-**datashuttle** creates and validates projects standardised to the
-[NeuroBlueprint](https://neuroblueprint.neuroinformatics.dev)
-specification.
+::::
 
-Dive right into **datashuttle** with our
-[Getting Started Tutorial](tutorial-getting-started)
-or targeted [How-To Guides](how-tos).  \
-It can be used through a graphical interface or Python API.
+A lack of project standardization in systems neuroscience
+[hinders data sharing and collaboration](https://neuroinformatics.dev/blog/neuroblueprint.html),
+creating barriers to reproducibility and scientific progress.
+
+``datashuttle`` helps standardise experimental
+projects by automating folder creation and transfer
+during acquisition and analysis. Its graphical interface or Python API builds
+folder trees according to the [NeuroBlueprint](https://neuroblueprint.neuroinformatics.dev)
+specification. Automation and validation ensures that no errors, such as duplicate session
+names or incorrect dates, slip into the project.
+
+Data can be transferred between acquisition, storage and analysis
+machines with a single function call or button click. Standardisation makes
+folder names predictable, meaning it is easy to transfer specific combinations
+of subjects, sessions or data-types with ``datashuttle``.
+
+Folders are standardised to the
+[NeuroBlueprint](https://neuroblueprint.neuroinformatics.dev) specification:
+
+```{image} /_static/NeuroBlueprint_project_tree_dark.png
+   :align: center
+   :class: only-dark
+   :width: 550px
+```
+```{image} /_static/NeuroBlueprint_project_tree_light.png
+   :align: center
+   :class: only-light
+   :width: 550px
+```
 
-Don't hesitate to get in contact through our
-[GitHub Issues](https://github.com/neuroinformatics-unit/datashuttle/issues)
-or
-[Zulip Chat.](https://neuroinformatics.zulipchat.com/#narrow/stream/405999-DataShuttle)
+Dive in with our [Getting Started page](pages/get_started/index)
+or targeted [User Guides](pages/user_guides/index).
 
+Have questions, issues or feedback? Get in contact through
+[GitHub issues](https://github.com/neuroinformatics-unit/datashuttle/issues)
+or our
+[Zulip chat.](https://neuroinformatics.zulipchat.com/#narrow/stream/405999-DataShuttle)
 
 ```{toctree}
 :maxdepth: 2
 :caption: index
 :hidden:
 
-pages/tutorials
-pages/how_tos
-pages/community/index.md
+pages/get_started/index
+pages/user_guides/index
+pages/examples/index
+pages/community/index
 pages/api_index
 ```

docs/source/_static/screenshots/how-to-choose-a-terminal-bad-light.png renamed to docs/source/_static/screenshots/choose-a-terminal-bad-light.png

@@ -12,17 +12,17 @@ The core development team will support you in contributing code, regardless of y
 
 ## Creating a development environment
 
-For detailed instructions on installing **datashuttle** along with
+For detailed instructions on installing ``datashuttle`` along with
 all necessary packages for development, refer to the `Developers` tab on the
-[How to install **datashuttle**](how-to-install) page.
+[How to install ``datashuttle``](how-to-install) page.
 
-Briefly, after installing Rclone, the **datashuttle** repository can be cloned:
+Briefly, after installing Rclone, the ``datashuttle`` repository can be cloned:
 
 ```sh
 git clone git@github.com:neuroinformatics-unit/datashuttle.git
 ```
 
-and **datashuttle** pip-installed with the developer tag:
+and ``datashuttle`` pip-installed with the developer tag:
 
 
 ```sh
@@ -117,8 +117,8 @@ Make sure to provide docstrings for all public functions, classes, and methods.
 
 ## Contributing documentation
 
-It is very important that the documentation for **datashuttle** is clear,
-accurate and concise. A key principle of **datashuttle** is it should
+It is very important that the documentation for ``datashuttle`` is clear,
+accurate and concise. A key principle of ``datashuttle`` is it should
 make data acquisition simple and easy—therefore it should be
 well-documented.
 

docs/source/_static/screenshots/how-to-choose-a-terminal-good-dark.png renamed to docs/source/_static/screenshots/choose-a-terminal-good-dark.png

@@ -2,16 +2,16 @@
 
 In this document we lay out our vision for
 [**NeuroBlueprint**](https://neuroblueprint.neuroinformatics.dev/)
-and **datashuttle** through three guiding principles.
+and ``datashuttle`` through three guiding principles.
 
 For additional detail, please refer to our blog posts motivating
 [**NeuroBlueprint**](https://neuroinformatics.dev/blog/neuroblueprint.html)
 and
-[**datashuttle**](https://neuroinformatics.dev/blog/datashuttle.html).
+[``datashuttle``](https://neuroinformatics.dev/blog/datashuttle.html).
 
 The three guiding principles driving the development of
 **NeuroBlueprint**
-and **datashuttle** are:
+and ``datashuttle`` are:
 
 ## Align with existing community initiatives
 
@@ -30,15 +30,15 @@ The guiding principle is that some standardisation is better than no standardisa
 **NeuroBlueprint** is not intended as a rival or replacement for more established
 specifications, but rather as a stepping stone towards full standardisation.
 We aim to align as closely as possible to the BIDS specification and in future
-to provide conversion to NWB through **datashuttle**.
+to provide conversion to NWB through ``datashuttle``.
 
 ## Strive to be lightweight
 
 **NeuroBlueprint** aims to keep the specification as lightweight
 as possible. There is no benefit in the specification proliferating
 as it develops such that it ends up duplicating BIDS in scope.
 
-In the initial versions (**datashuttle** v0.4, **NeuroBlueprint** v0.2),
+In the initial versions (``datashuttle`` v0.4, **NeuroBlueprint** v0.2),
 the goal is to have a simple organisational
 system in which the raw data for different datatypes (e.g. ephys, behaviour)
 can be automatically discovered in any given project.
@@ -49,17 +49,17 @@ New standards will always be goal-orientated and customisable, only required
 to achieve a particular research goal. It is anticipated that as these 'rules'
 are adopted, projects will become progressively easier to convert to BIDS or NWB.
 
-The role of **datashuttle** will be to enforce these rules in a flexible manner.
+The role of ``datashuttle`` will be to enforce these rules in a flexible manner.
 
 ## Keep releases versioned and modular
 
-**NeuroBlueprint** and **datashuttle** will be
+**NeuroBlueprint** and ``datashuttle`` will be
 properly versioned and modular, enabling smooth development over time.
 New versions will prioritize backward compatibility, with changes that break
 compatibility occurring infrequently and only aimed at enhancing alignment with existing standards.
 
 Typically, new versions will include new
 sets of standardisation rules,
-which users may choose to adopt or not. **datashuttle** will
+which users may choose to adopt or not. ``datashuttle`` will
 enforce these in a flexible way, with efforts being
 made to minimise changes to its API as much as possible.

docs/source/_static/screenshots/how-to-choose-a-terminal-good-light.png renamed to docs/source/_static/screenshots/choose-a-terminal-good-light.png

@@ -1,21 +1,21 @@
 # Community
 
 
-Contributions to **datashuttle** are strongly encouraged. If you would like to submit
-a bug fix or implement a new feature, please feel free to do so! We are a
+Contributions to ``datashuttle`` are strongly encouraged. If you would like to submit
+a bug fix or implement a new feature, please feel free to open an issue or pull request. We are a
 friendly bunch and happy to answer any questions before,
 during or after a contribution. See the [Contributing Guide](contributing.md) for more details.
 
 We are generally very keen to hear feedback on what works and doesn't work
-in **datashuttle** and
+in ``datashuttle`` and
 [**NeuroBlueprint**](https://neuroblueprint.neuroinformatics.dev).
 Please get in contact with any thoughts on how these tools can be
 improved by opening a
-[GitHub Issue](https://github.com/neuroinformatics-unit/datashuttle/issues)
+[GitHub issue](https://github.com/neuroinformatics-unit/datashuttle/issues)
 or by joining our
-[Zulip Chat](https://neuroinformatics.zulipchat.com/#narrow/stream/405999-DataShuttle).
+[Zulip chat](https://neuroinformatics.zulipchat.com/#narrow/stream/405999-DataShuttle).
 
-To see what we have planned for the future of **NeuroBlueprint** and **datashuttle**, visit
+To see what we have planned for the future of **NeuroBlueprint** and ``datashuttle``, visit
 our [Guiding Principles](guiding-principles.md) and [Roadmap](roadmap.md).
 
 

docs/source/_static/screenshots/tutorial-1-transfer-screen-upload-dark.png

@@ -1,6 +1,6 @@
 # Roadmap
 
-The roadmap for **datashuttle**
+The roadmap for ``datashuttle``
 contains our development plans for the package.
 The roadmap is responsive to the requirements
 of the community and we encourage people to get in
@@ -12,14 +12,16 @@ creation, transfer and logging of projects organised to
 support implementation of metadata standardisation planned for the next
 NeuroBlueprint version and facilitate conversion to NWB and customisation.
 
-**Q4 2024**
+**Q3 2025**
+- Support Google Drive and Amazon Web Service and central storage drives.
 
-Standardised metadata support.
-
-**Q2 2025**
+**Q4 2025**
+- Export of NeuroBlueprint projects to [Neurodata Without Borders](https://www.nwb.org/) format.
 
-Export of NeuroBlueprint projects to [Neurodata Without Borders](https://www.nwb.org/) format.
+**2026**
+- Support for customising data structure (e.g. specific mandatory elements or new file types).
 
-**Q4 2025**
 
-Support for customising data structure (e.g. specific mandatory elements or new file types).
+## Notes on previous goals
+Q4 2024, Standardised metadata support. After [discussion](https://github.com/neuroinformatics-unit/NeuroBlueprint/issues/30),
+this was put on hold to await further consultation with researchers.

docs/source/_static/screenshots/tutorial-1-transfer-screen-upload-light.png

@@ -0,0 +1,53 @@
+:html_theme.sidebar_secondary.remove:
+
+# Acquisition script
+
+This script shows how Laura Schwarz (O'Keefe Lab, [Sainsbury Wellcome Centre](https://www.sainsburywellcome.org/web/))
+uses ``datashuttle`` to create project folders during the acquisition of a behavioural
+task in mice.
+
+```python
+def get_file_path():
+
+    # get your project
+    project = DataShuttle("social_sleaping")
+
+    # create a prompt to enter the ID number
+    # (which we will use to get the subject number)
+    id_number = input("Enter ID number: ")
+    sub = ID_DICT.get(id_number)
+
+    # get your session number and create a new folder
+    # for the session you are about to record.
+    # the function get_next_ses() normally checks for the next session
+    # if you are recording for a new subject you can use it as well to create
+    # the first session folder for this subject.
+    session = project.get_next_ses(top_level_folder="rawdata",
+                                   sub=f"sub-{sub}_id-{id_number}")
+
+    # create the folders
+    created_folders = project.create_folders(
+        top_level_folder="rawdata",
+        sub_names=f"sub-{sub}_id-{id_number}",
+        ses_names=f"{session}_@DATETIME@",
+        datatype=["behav"]
+    )
+    # create a prompt to enter the experiment information and
+    # conspecific ID for social experiments.
+    # (this is only important for the video file name and might not be
+    # relevant for you.)
+    exp_number = input("Enter Experiment condition: ")
+    comsp_id = input("Enter Conspecific ID: ")
+
+    # print the start of your acquisition
+    print(datetime.now())
+
+    # create the video file name
+    file_name_video_1 = f"{exp_number}_{comsp_id}.avi"
+
+    # create the path to the video file
+    file_path1 = created_folders['behav'][0] / file_name_video_1
+    file_path1.touch()
+
+    return file_path1
+```

docs/source/_static/screenshots/tutorial-validation-dark.png

@@ -0,0 +1,41 @@
+:html_theme.sidebar_secondary.remove:
+
+(examples)=
+# Examples
+
+
+::::{grid} 1 1 2 2
+:gutter: 4
+
+
+:::{grid-item-card} {fas}`desktop;sd-text-primary` Acquisition script
+:link: acquisition-script
+:link-type: doc
+:class-img-top: tutorial-link-image
+
+Using ``datashuttle`` to create project folders in a behavioural-data acquisition script
+
+:::
+
+
+:::{grid-item-card} {fas}`desktop;sd-text-primary` Validate all projects in a lab
+:link: lab-project-checker
+:link-type: doc
+:class-img-top: tutorial-link-image
+
+A script used to validate every project in a lab at once
+
+:::
+
+::::
+
+
+```{toctree}
+:maxdepth: 2
+:caption: Examples
+:hidden:
+
+acquisition-script
+lab-project-checker
+
+```