Skip to content

@crawlab-team Crawlab Team

Change the repository type filter

All

    Repositories list

    57 repositories

    • crawlab

      Public
      Distributed web crawler admin platform for spiders management regardless of languages and frameworks. 分布式爬虫管理平台，支持任何语言和框架
      godockerplatformcrawlerspiderweb-crawlerscrapywebcrawlerscrapyd-uiwebspider
      Go
      •
      BSD 3-Clause "New" or "Revised" License
      •1.9k•12k•147•2•Updated Aug 16, 2025Aug 16, 2025

    • crawlab-docs

      Public
      Documentation for Crawlab
      documentationgitbook-clicrawlab
      JavaScript
      •53•39•2•2•Updated Jul 16, 2025Jul 16, 2025

    • crawleval

      Public archive
      Resources and tools for evaluating the performance and behavior of web crawling systems
      Python
      •
      MIT License
      •1•7•0•0•Updated May 9, 2025May 9, 2025

    • crawlab-ui

      Public
      🎉 A Vue.js 3.0 UI Library made by Crawlab team
      vueelement-uivue3crawlab
      Vue
      •
      BSD 3-Clause "New" or "Revised" License
      •19k•23•0•0•Updated Apr 14, 2025Apr 14, 2025

    • bm25

      Public
      This is a golang implementation of various BM25 algorithms. It is a port of `dorianbrown/rank_bm25`
      Go
      •
      MIT License
      •3•0•0•0•Updated Mar 28, 2025Mar 28, 2025

    • crawlab-mcp

      Public
      Python
      •4•4•1•2•Updated Mar 18, 2025Mar 18, 2025

    • fizz

      Public
      🍋 Gin wrapper with OpenAPI 3 spec generation
      Go
      •
      MIT License
      •56•0•0•0•Updated Mar 17, 2025Mar 17, 2025

    • mcp-go

      Public
      A Go implementation of the Model Context Protocol (MCP), enabling seamless integration between LLM applications and external data sources and tools.
      Go
      •
      MIT License
      •710•0•0•0•Updated Mar 12, 2025Mar 12, 2025

    • crawlab-openapi

      Public
      JavaScript
      •0•0•0•0•Updated Mar 6, 2025Mar 6, 2025

    • crawlab-python-sdk

      Public
      Python SDK for Crawlab
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •1•1•0•0•Updated Feb 14, 2025Feb 14, 2025

    • crawlab-node-sdk

      Public
      Node.js SDK for Crawlab
      TypeScript
      •
      BSD 3-Clause "New" or "Revised" License
      •0•0•0•0•Updated Jan 6, 2025Jan 6, 2025

    • crawlab-go-sdk

      Public
      Crawlab Go SDK
      Go
      •
      BSD 3-Clause "New" or "Revised" License
      •9•10•0•0•Updated Jan 6, 2025Jan 6, 2025

    • crawlab-java-sdk

      Public
      Java SDK for Crawlab
      Java
      •
      BSD 3-Clause "New" or "Revised" License
      •0•0•0•0•Updated Jan 3, 2025Jan 3, 2025

    • e2e-tests

      Public
      TypeScript
      •0•1•0•0•Updated Dec 20, 2024Dec 20, 2024

    • crawlab-test-report

      Public
      0•0•0•0•Updated Dec 19, 2024Dec 19, 2024

    • crawlab-core

      Public archive
      Backend core modules for Crawlab
      Go
      •
      Apache License 2.0
      •56•50•0•0•Updated Jun 14, 2024Jun 14, 2024

    • crawlab-grpc

      Public archive
      gRPC for Crawlab
      Shell
      •
      BSD 3-Clause "New" or "Revised" License
      •12•4•0•0•Updated Jun 14, 2024Jun 14, 2024

    • docker-base-images

      Public archive
      Shell
      •
      BSD 3-Clause "New" or "Revised" License
      •7•2•0•0•Updated Jun 11, 2024Jun 11, 2024

    • crawlab-demo

      Public archive
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •6•2•0•0•Updated Jun 4, 2024Jun 4, 2024

    • crawlab-ai-sdk

      Public
      SDK for Crawlab AI
      Python
      •
      MIT License
      •3•8•2•0•Updated Jun 4, 2024Jun 4, 2024

    • crawlab-sdk

      Public
      SDK for Crawlab, including SDK for different programming languages such as Python, Node.js and Java, and a CLI Tool written in Python.
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •52•56•2•5•Updated Jun 3, 2024Jun 3, 2024

    • crawlab-fs

      Public
      Backend file system module for Crawlab
      Go
      •
      BSD 3-Clause "New" or "Revised" License
      •12•4•0•0•Updated Mar 29, 2024Mar 29, 2024

    • goseaweedfs

      Public
      A complete Golang client for SeaweedFS
      Go
      •
      Apache License 2.0
      •51•4•0•0•Updated Mar 29, 2024Mar 29, 2024

    • scrapy-ai

      Public
      AI-powered scrapy plugin
      Python
      •
      MIT License
      •0•1•0•0•Updated Feb 7, 2024Feb 7, 2024

    • crawlab-vcs

      Public
      Version Control System (VCS) for Crawlab
      Go
      •
      BSD 3-Clause "New" or "Revised" License
      •11•4•0•1•Updated Aug 8, 2023Aug 8, 2023

    • webspot

      Public archive
      An intelligent web service to automatically detect web content and extract information from it.
      crawlerwebspidercrawlab
      Python
      •
      BSD 3-Clause "New" or "Revised" License
      •12•86•1•0•Updated Jul 13, 2023Jul 13, 2023

    • crawlab-next

      Public
      C#
      •
      Apache License 2.0
      •1•0•0•0•Updated May 31, 2023May 31, 2023

    • html-to-json-enhanced

      Public
      Convert HTML to JSON. Can also (intelligently) convert HTML tables to JSON (using table headers (if available) as keys in the resulting JSON).
      HTML
      •
      MIT License
      •8•4•0•0•Updated Apr 1, 2023Apr 1, 2023

    • artipub

      Public
      Article publishing platform that automatically distributes your articles to various media channels
      nodejstypescriptarticlemediaarticle-publisherdockermongodbcsdnsegmentfault
      TypeScript
      •
      BSD 3-Clause "New" or "Revised" License
      •533•3.1k•29•11•Updated Mar 5, 2023Mar 5, 2023

    • crawlab-lite

      Public
      Lite version of Crawlab. 轻量版 Crawlab 爬虫管理平台
      platformcrawlerspiderweb-crawlerscrapyscrapydscrapy-uiscrapyd-uicrawling-taskscrawlab
      Vue
      •
      BSD 3-Clause "New" or "Revised" License
      •77•228•1•8•Updated Feb 9, 2023Feb 9, 2023