Web Scraper API

The Web Scraper API provides web scraper services to retrieve HTML and plaintext data from any website URL.

To retrieve web page metadata and URL information, use the Webpage API instead.

(4.3)

From 2,209 users

500+

Applications using this API

API Status

Online - All Systems Operational

API Response Times

Average
P50
P75
P90
P95
391ms403ms490ms876ms1069ms

Didn't find what you were looking for? Suggest an improvement

/v1/webscraper GET

https://api.api-ninjas.com/v1/webscraper

Returns the HTML or plaintext data scraped from a given URL. Maximum size of data returned is 2MB.

Parameters

  • url  required

    URL to scrape.

  • text_only  optional

    Whether to only extract visible text (ignores HTML tags and metadata). Must be either true or false. Default is false.

  • user_agent  optional

    User-Agent string to use in the request header.

Headers

  • X-Api-Key  required

    API Key associated with your account.

Response

A JSON object with the following fields or an error if the request is unsuccessful.

  • data

    The scraped HTML or extracted text, depending on the text_only parameter.

The API may also include additional metadata fields in the future; refer to the sample response for the exact structure.

Sample Request Live Demo!

url
text_only
user_agent

Try this API endpoint with all available parameters in our API playground

https://api.api-ninjas.com/v1/webscraper?url=https%3A%2F%2Fexample.com

Headers

X-Api-KeyLog in or sign up to get your API Key

Sample Response

JSON

Code Examples

1 2 curl -X GET "https://api.api-ninjas.com/v1/webscraper?url=https://example.com" \ -H "X-Api-Key: YOUR_API_KEY"

If your programming language is not listed in the Code Example above, you can still make API calls by using a HTTP request library written in your programming language and following the above documentation.