Skip to content

Commit cc3f27a

Browse files
author
John Waller
committed
sleeping 5 seconds on 429 response
1 parent b324b61 commit cc3f27a

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

41 files changed

+22048
-20118
lines changed

R/zzz.r

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -340,6 +340,17 @@ rgbif_ual <- list(`User-Agent` = rgbif_ua(), `X-USER-AGENT` = rgbif_ua())
340340
gbif_GET <- function(url, args, parse=FALSE, curlopts = list(), mssg = NULL) {
341341
cli <- crul::HttpClient$new(url = url, headers = rgbif_ual, opts = curlopts)
342342
temp <- cli$get(query = args)
343+
if(temp$status_code == 429) {
344+
message("Too many requests! To download GBIF occurrence data in bulk, please use occ_download().")
345+
seconds <- 5
346+
for (i in 1:seconds) {
347+
cat(sprintf("\rWaiting [%s%s] %d/%d sec",
348+
strrep("=", i), strrep(" ", seconds - i), i, seconds))
349+
Sys.sleep(1)
350+
flush.console()
351+
}
352+
return(gbif_GET(url, args, parse, curlopts, mssg))
353+
}
343354
if (temp$status_code == 204)
344355
stop("Status: 204 - not found ", mssg, call. = FALSE)
345356
if (temp$status_code > 200) {

README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,8 @@ occ_search(taxonKey = taxonKey)
5151

5252
GBIF **strongly recommends** the use of `occ_download()` rather than `occ_search()` for serious research projects. See article [Getting Occurrence Data From GBIF](https://docs.ropensci.org/rgbif/articles/getting_occurrence_data.html).
5353

54+
> If repeated requests are made to the GBIF search API via `occ_search()`, users will be temporatily paused for 5 seconds. Please use `occ_download()` for bulk downloads.
55+
5456
It is required to set up your [GBIF credentials](https://docs.ropensci.org/rgbif/articles/gbif_credentials.html) to make downloads from GBIF.
5557

5658
```r

tests/fixtures/gbif_get_wait.yml

Lines changed: 1729 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

tests/fixtures/occ_search.yml

Lines changed: 386 additions & 384 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

tests/fixtures/occ_search_catalognumber.yml

Lines changed: 3 additions & 1 deletion
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

tests/fixtures/occ_search_coordinateUncertaintyInMeters.yml

Lines changed: 314 additions & 308 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

tests/fixtures/occ_search_datasetkey.yml

Lines changed: 920 additions & 918 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

tests/fixtures/occ_search_degreeOfEstablishment.yml

Lines changed: 402 additions & 394 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

tests/fixtures/occ_search_depth.yml

Lines changed: 50 additions & 44 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

tests/fixtures/occ_search_distanceFromCentroidInMeters.yml

Lines changed: 296 additions & 290 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)