You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
JZLab is the self-hosted version of GitData.AI that allows you to deploy and manage your own DataHub and Workflow on-prem.
5
+
JZHub is an open-source data hub designed to empower researchers, data scientists, and open science enthusiasts to manage, version, and publish scientific data with ease. Built on JZFS (a Git-like version control file system) and integrated with Resource Hub for models, workflows,storage and comptation, JZHub provides a flexible platform for collaborative data workflows. Whether you're creating citable data papers, sharing datasets, or tracking experimental changes, JZHub streamlines the process with modern tools and automation.
6
6
7
7
----
8
-
## JZLab include but not limited to：
9
-
10
-
1. Visualize and interact with JZFS repositories
11
-
2. Visualize and interact JZFlow pipelines and Directed Acyclic Graphs (DAGs).
12
-
3. One-click to start the fully functional JupyterLab environment.
13
-
4. Examine the performance of versioned/registered models.
14
-
5. Monitor the status of model services including health and resource usage stats, and view deployment history and related logs.
15
-
6. Easily upload and manage datasets and shared resources.
16
-
7. Browse and share files with other group members in a collaborative, group-centric, environment.
17
-
8. Submit and schedule jobs to run automatically in the background. Easily monitor job progress from the job stats panel.
18
-
9. The easiest way to do the infrastructure orchestration for setting up 10+ different tools to build your infrastructure.
19
-
20
-
21
-
## How different roles use JZLab
22
-
- Data Scientist
23
-
- Data scientists can stay informed and focused on training and running their models.
24
-
- In the past, there was a lot of manual work of setting the environment, which is a fragmented and time consuming analysis process. And it is hard to collaborate with a team on the same project.
25
-
- Now, you can carry out data analytics and optimizations with ML easily, and contribute your time on what really matters.
26
-
- IT Leader
27
-
- Gives IT leaders flexibility and administration authority to configure resources.
28
-
- In the past, it’s hard to keep track of each teams' needs and environment settings. Also, the hardware, resources, and GPU usage are unsure.
29
-
- Now, You can equip and enable data teams with the tools and resources they need as easy as pie. And easily deploy the model within an hour.
8
+
## Features
9
+
10
+
JZHub offers a range of capabilities to meet the needs of researchers and data scientists, inspired by public data hubs like Dataset Hubs, LLM hubs, and AI hubs. Here’s what JZHub provides:
11
+
- Data Versioning and Collaboration: Track changes to datasets, models, and documents using JZFS, enabling Git-like versioning for collaborative research workflows, supporting team-based data management. This mirrors data hubs’ focus on versioning and collaboration, as seen in platforms like GitHub.
12
+
13
+
14
+
- Hosting and Publishing: Store and share data and models in a hub, ensuring persistent, accessible, and citable outputs for open science, with cross-domain interoperability.
15
+
16
+
17
+
- Metadata Management and Discovery: Automatically generate and manage metadata for datasets and models, with search and discovery features to enhance data reuse and accessibility.
18
+
19
+
20
+
- LLM Integration: Leverage large language models (e.g., Deepseek) for automated content generation (e.g., data papers, blogs, documentation) and retrieval-augmented generation (RAG) for querying data.
21
+
22
+
23
+
- Model Hosting and Fine-Tuning: Host and fine-tune LLMs or AI models, with integration for on-device deployment via Cloud or decentralized storage, supporting scalable AI research。
24
+
25
+
26
+
- Security, Compliance, and Governance: Offer access controls, compliance monitoring, and data protection features for sensitive research data, ensuring trust in multi-institutional collaborations.
27
+
28
+
30
29
----
31
-
## Basic Build And Usage
30
+
## Use Cases
31
+
JZHub’s use cases are designed to support researchers in open science and AI-driven research.Here’s how JZHub can be used:
32
+
- Data Paper Creation and Publication: Generate citable data papers from datasets using LLMs, publish with datasets as IPLD products or other formats for open science and reproducibility.
32
33
33
-
#### Requirement
34
34
35
-
1. todo
36
-
2. todo
35
+
- Collaborative Research Workflows: Track experiment data or model versions across distributed teams, share securely via decentralized storage, enhancing multi-institutional research.
37
36
38
-
#### Build And Running
39
37
38
+
- AI-Driven Insights and Reporting: Use RAG to query datasets/models for insights, with LLM-generated summaries for reports, supporting data-driven decision-making in research.
39
+
40
+
41
+
- Decentralized Model Deployment: Host fine-tuned LLMs or AI models for on-device use in research applications, enabling innovation in resource-constrained environments.
42
+
43
+
44
+
- Compliance and Governance for Sensitive Data: Manage sensitive data with access controls and compliance monitoring, ensuring ethical use in collaborative open science projects.
45
+
46
+
47
+
## Getting Started
48
+
49
+
#### Prerequisites
50
+
51
+
#### Build And Running
40
52
deploy the system to your server,you can get help from this repository:
41
53
```bash
42
-
https://github.com/GitDataAI/jzfs
54
+
https://github.com/GitDataAI/jzhub
43
55
```
44
56
45
-
clone JZLab repository to your server:
57
+
clone JZHub repository to your server:
46
58
47
59
```bash
48
-
git clone git@github.com:GitDataAI/jzlab.git
60
+
git clone git@github.com:GitDataAI/jzhub.git
49
61
```
50
62
51
63
Before you run the project for the first time, run the following script to install packages from `package.json`:
@@ -57,32 +69,32 @@ After waiting for the installation to complete,run the following script to start
57
69
npm run dev
58
70
```
59
71
72
+
#### Cloud
60
73
74
+
[Try without installing](https://jzhub.io)
61
75
62
-
----
63
-
### Cloud
64
76
65
-
[Try without installing](https://gitdata.ai)
66
-
67
-
----
68
-
### Contributors
77
+
## Contributing
69
78
70
79
<!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section -->
Dual-licensed under [MIT](https://github.com/GitDataAI/jiaozifs/blob/main/LICENSE-MIT) + [Apache 2.0](https://github.com/GitDataAI/jiaozifs/blob/main/LICENSE-APACHE)
88
+
We welcome contributions! Please read our for guidelines. Here’s how to get started:
83
89
90
+
1. Fork the repo.
91
+
2. Create a feature branch (`git checkout -b feature/my-feature`).
92
+
3. Submit a pull request.
84
93
85
-
## Contributors ✨
94
+
#### Current Needs
95
+
1. Actions support.
96
+
2. UI enhancements.
97
+
3. Integration with LLM.
86
98
87
99
Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/docs/en/emoji-key)):
88
100
@@ -93,4 +105,11 @@ Thanks goes to these wonderful people ([emoji key](https://allcontributors.org/d
93
105
<!-- prettier-ignore-end -->
94
106
<!-- ALL-CONTRIBUTORS-LIST:END -->
95
107
96
-
This project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind welcome!
108
+
This project follows the [all-contributors](https://github.com/all-contributors/all-contributors) specification. Contributions of any kind welcome!
109
+
110
+
111
+
----
112
+
### License
113
+
114
+
Dual-licensed under [MIT](https://github.com/GitDataAI/jiaozifs/blob/main/LICENSE-MIT) + [Apache 2.0](https://github.com/GitDataAI/jiaozifs/blob/main/LICENSE-APACHE)
0 commit comments