Skip to contents

Fetch results via pagination

Source: R/fetch_paged.R
fetch_paged.Rd

This function performs a request for data from the UniProt REST API, fetches the results using pagination, and saves them to a file or into memory.

You likely won't use this function directly, but rather one of the wrapper functions: uniprot_map(), uniprot_search(), or uniprot_single().

Things to note:

  1. The pagination endpoint is less expensive for the API infrastructure to deal with versus fetch_stream(), as the memory demand is distributed over a longer period of time.

  2. If the connection is interrupted while fetching results with pagination, only the request for the current page needs to be reattempted. With the stream endpoint, the entire request needs to be completely restarted.

Usage

fetch_paged(req, n_pages, format = "tsv", path = NULL, verbosity = NULL)

Arguments

req

httr2_request object, generated by e.g. httr2::request() or uniprot_request().

n_pages

integer, the number of pages to be fetched. This can be calculated by dividing the number of total results by the page size e.g. resp$headers$x-total-results / page_size.

format

string, data format to fetch. Can one of "tsv", or "fasta".

path

string (optional), file path to save the results, e.g. "path/to/results.tsv". The file must not already exist, otherwise an error is thrown.

verbosity

integer (optional), how much information to print?

  • 0: no output

  • NULL (default): minimal output

  • 1: show request headers

  • 2: show request headers and bodies

  • 3: show request headers, bodies, and curl status messages

Value

By default, returns an object whose type depends on format:

If parse = FALSE, returns an httr2_response. If path is specified, saves the parsed results to the file path indicated, and returns NULL

invisibly.

Examples

if (FALSE) {
  req <- uniprot_request(
    "https://rest.uniprot.org/uniref/search",
    query = "P99999",
    format = "tsv",
    fields = "id,name,count",
    size = 1
  )

  fetch_paged(req, n_pages = 3)
}