docs: refactor tonbo docs

.github/workflows/doc.yml

@@ -21,6 +21,7 @@ jobs:
           cd guide
           curl -L https://github.com/rust-lang/mdBook/releases/download/v0.4.28/mdbook-v0.4.28-x86_64-unknown-linux-gnu.tar.gz | tar xzf -
           echo $PWD >> $GITHUB_PATH
+          cargo install mdbook-toc
           ./mdbook build
       - name: Deploy
         uses: JamesIves/github-pages-deploy-action@v4

.gitignore

@@ -1,6 +1,8 @@
-/target
+target
 /.idea
 Cargo.lock
 db_path
-bindings/python/target
-guide/book
+guide/book
+__pycache__
+
+.DS_Store

guide/book.toml

@@ -7,5 +7,10 @@ title = "The Tonbo Guide"
 
 [output.html]
 git-repository-url = "https://github.com/tonbo-io/tonbo"
+mathjax-support = true
 [output.html.playground]
 runnable = false
+
+[preprocessor.toc]
+command = "mdbook-toc"
+renderer = ["html"]

guide/src/SUMMARY.md

@@ -2,10 +2,22 @@
 
 [Introduction](./introduction.md)
 
-- [Getting started](./start.md)
+- [Getting Started](./start.md)
+- [Usage]()
+  - [Tonbo](./usage/tonbo.md)
+  - [Python Binding](./usage/python.md)
+  - [JavaScript Binding](./usage/wasm.md)
+  - [Configuration](./usage/conf.md)
+  - [Advance](./usage/advance.md)
+  - [FAQ](./usage/faq.md)
 - [Examples](./examples/index.md)
   - [Using Tonbo](./examples/declare.md)
   - [Integrate with Datafusio](./examples/datafusion.md)
   - [Using under Wasm](./examples/wasm.md)
-- [Contribution](./contribution/index.md)
+- [Contribution]()
   - [Building](./contribution/build.md)
+  - [Submitting PR](./contribution/pr.md)
+- [TonboLite](./tonbolite/index.md)
+  - [Getting Started](./tonbolite/start.md)
+  - [Building and Testing](./tonbolite/build.md)
+  - [Usage](./tonbolite/usage.md)

guide/src/contribution/build.md

@@ -1,3 +1,93 @@
-# Building Tonbo
+# Building and Testing
+<!-- toc -->
 
-TODO
+To get started using tonbo you should make sure you have [Rust](https://www.rust-lang.org/tools/install) installed on your system. If you haven't alreadly done yet, try following the instructions [here](https://www.rust-lang.org/tools/install).
+
+## Building and Testing for Rust
+
+### Building and Testing with Non-WASM
+
+To use local disk as storage backend, you should import [tokio](https://github.com/tokio-rs/tokio) crate and enable "tokio" feature (enabled by default)
+
+```bash
+cargo build
+```
+
+If you build Tonbo successfully, you can run the tests with:
+
+```bash
+cargo test
+```
+
+### Building and Testing with WASM
+
+If you want to build tonbo under wasm, you should add wasm32-unknown-unknown target first.
+
+```bash
+# add wasm32-unknown-unknown target
+rustup target add wasm32-unknown-unknown
+# build under wasm
+cargo build --target wasm32-unknown-unknown --no-default-features --features wasm
+```
+
+Before running the tests, make sure you have installed [wasm-pack](https://github.com/rustwasm/wasm-pack) and run `wasm-pack build` to build the wasm module. If you build successfully, you can run the tests with:
+
+```bash
+wasm-pack test --chrome --headless --test wasm --no-default-features --features aws,bytes,opfs
+```
+
+
+## Building and Testing for Python
+
+### Building
+We use the [pyo3](https://github.com/PyO3/pyo3) to generate a native Python module and use [maturin](https://github.com/PyO3/maturin) to build Rust-based Python packages.
+
+First, follow the commands below to build a new Python virtualenv, and install maturin into the virtualenv using Python's package manager, pip:
+
+```bash
+# setup virtualenv
+python -m venv .env
+# activate venv
+source .env/bin/activate
+
+# install maturin
+pip install maturin
+# build bindings
+maturin develop
+
+```
+
+Whenever Rust code changes run:
+
+```bash
+maturin develop
+```
+
+### Testing
+
+If you want to run tests, you need to build with "test" options:
+
+```base
+maturin develop -E test
+```
+
+After building successfully, you can run the tests with:
+
+```bash
+# run tests except benchmarks(This need duckdb to be installed)
+pytest --ignore=tests/bench -v .
+
+# run all tests
+pip install duckdb
+python -m pytest
+```
+
+## Building and Testing for JavaScript
+To build tonbo for JavaScript, you should install [wasm-pack](https://github.com/rustwasm/wasm-pack). If you haven't already done so, try following the instructions [here](https://rustwasm.github.io/wasm-pack/installer/).
+
+```bash
+# add wasm32-unknown-unknown target
+rustup target add wasm32-unknown-unknown
+# build under wasm
+wasm-pack build --target web
+```

guide/src/contribution/pr.md

@@ -0,0 +1,36 @@
+# Submitting a Pull Request
+
+Thanks for your contribution! The Tonbo project welcomes contribution of various types -- new features, bug fixes and reports, typo fixes, etc. If you want to contribute to the Tonbo project, you will need to pass necessary checks. If you have any question, feel free to start a new discussion or issue, or ask in the Tonbo [Discord](https://discord.gg/j27XVFVmJM).
+
+## Running Tests and Checks
+This is a Rust project, so [rustup](https://rustup.rs/) and [cargo](https://doc.rust-lang.org/cargo/) are the best place to start.
+
+- `cargo check` to analyze the current package and report errors.
+- `cargo +nightly fmt` to format the current code.
+- `cargo build` to compile the current package.
+- `cargo clippy` to catch common mistakes and improve code.
+- `cargo test` to run unit tests.
+- `cargo bench` to run benchmark tests.
+
+
+> **Note**: If you have any changes to *bindings/python*, please make sure to run checks and tests before submitting your PR. If you don not know how to build and  run tests, please refer to the [Building Tonbo for Python](./build.md#building-tonbo-for-python) section.
+
+## Pull Request title
+As described in [here](https://gist.github.com/joshbuchea/6f47e86d2510bce28f8e7f42ae84c716), a valid PR title should begin with one of the following prefixes:
+- feat: new feature for the user, not a new feature for build script
+- fix: bug fix for the user, not a fix to a build script
+- doc: changes to the documentation
+- style: formatting, missing semi colons, etc; no production code change
+- refactor: refactoring production code, eg. renaming a variable
+- test: adding missing tests, refactoring tests; no production code change
+- chore: updating grunt tasks etc; no production code change
+
+Here is an example of a valid PR title:
+```
+feat: add float type
+^--^  ^------------^
+|     |
+|     +-> Summary in present tense.
+|
++-------> Type: chore, docs, feat, fix, refactor, style, or test.
+```

guide/src/contribution/testing.md

@@ -0,0 +1,6 @@
+# Testing
+
+
+## Testing Tonbo in Rust
+
+## Testing Tonbo in WASM