Direct HTTP examples

WebScrapingAPI integration examples for server-side REST calls.

Build the same documented Scraper API request with the standard HTTP client used by eight backend languages. Every example keeps the key server-side, encodes the target URL, sets a timeout, and checks the response status.

Choose a language Open request documentation

Language entry points

Use the HTTP client already in your backend.

Each example calls the REST endpoint directly, so you can start with familiar language tooling. For released product-specific packages, review the SDK directory.

子 shell · curl

curl --get --fail-with-body --max-time 120 \
  "https://api.webscrapingapi.com/v2" \
  --data-urlencode "api_key=$WSA_API_KEY" \
  --data-urlencode "url=https://example.com/public-page" \
  --data-urlencode "json_response=1"

curl --get --fail-with-body --max-time 120 \
  "https://api.webscrapingapi.com/v2" \
  --data-urlencode "api_key=$WSA_API_KEY" \
  --data-urlencode "url=https://example.com/public-page" \
  --data-urlencode "json_response=1"

字符串 requests

import os
import requests

response = requests.get(
    "https://api.webscrapingapi.com/v2",
    params={
        "api_key": os.environ["WSA_API_KEY"],
        "url": "https://example.com/public-page",
        "json_response": 1,
    },
    timeout=120,
)
response.raise_for_status()
print(response.json())

import os
import requests

response = requests.get(
    "https://api.webscrapingapi.com/v2",
    params={
        "api_key": os.environ["WSA_API_KEY"],
        "url": "https://example.com/public-page",
        "json_response": 1,
    },
    timeout=120,
)
response.raise_for_status()
print(response.json())

Node.js fetch · Node 18+

const apiKey = process.env.WSA_API_KEY;
if (!apiKey) {
  throw new Error("WSA_API_KEY is required");
}
const params = new URLSearchParams({
  api_key: apiKey,
  url: "https://example.com/public-page",
  json_response: "1",
});

const response = await fetch(
  "https://api.webscrapingapi.com/v2?" + params,
  { signal: AbortSignal.timeout(120_000) }
);
if (!response.ok) {
  throw new Error("API status " + response.status);
}
console.log(await response.json());

const apiKey = process.env.WSA_API_KEY;
if (!apiKey) {
  throw new Error("WSA_API_KEY is required");
}
const params = new URLSearchParams({
  api_key: apiKey,
  url: "https://example.com/public-page",
  json_response: "1",
});

const response = await fetch(
  "https://api.webscrapingapi.com/v2?" + params,
  { signal: AbortSignal.timeout(120_000) }
);
if (!response.ok) {
  throw new Error("API status " + response.status);
}
console.log(await response.json());

语 cURL · PHP 8+

<?php
$apiKey = getenv("WSA_API_KEY");
if ($apiKey === false || $apiKey === "") {
    throw new RuntimeException("WSA_API_KEY is required");
}
$query = http_build_query([
    "api_key" => $apiKey,
    "url" => "https://example.com/public-page",
    "json_response" => 1,
]);
$client = curl_init(
    "https://api.webscrapingapi.com/v2?" . $query
);
curl_setopt_array($client, [
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_TIMEOUT => 120,
]);
$body = curl_exec($client);
$status = curl_getinfo($client, CURLINFO_RESPONSE_CODE);
$error = curl_error($client);
curl_close($client);
if ($body === false || $status >= 400) {
    throw new RuntimeException(
        $error ?: "API request failed with status " . $status
    );
}
echo $body;

<?php
$apiKey = getenv("WSA_API_KEY");
if ($apiKey === false || $apiKey === "") {
    throw new RuntimeException("WSA_API_KEY is required");
}
$query = http_build_query([
    "api_key" => $apiKey,
    "url" => "https://example.com/public-page",
    "json_response" => 1,
]);
$client = curl_init(
    "https://api.webscrapingapi.com/v2?" . $query
);
curl_setopt_array($client, [
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_TIMEOUT => 120,
]);
$body = curl_exec($client);
$status = curl_getinfo($client, CURLINFO_RESPONSE_CODE);
$error = curl_error($client);
curl_close($client);
if ($body === false || $status >= 400) {
    throw new RuntimeException(
        $error ?: "API request failed with status " . $status
    );
}
echo $body;

雅瓦 HttpClient · Java 11+

import java.net.URI;
import java.net.URLEncoder;
import java.net.http.*;
import java.nio.charset.StandardCharsets;
import java.time.Duration;

String apiKey = System.getenv("WSA_API_KEY");
if (apiKey == null || apiKey.isBlank()) {
    throw new IllegalStateException("WSA_API_KEY is required");
}
String key = URLEncoder.encode(
    apiKey, StandardCharsets.UTF_8
);
String target = URLEncoder.encode(
    "https://example.com/public-page",
    StandardCharsets.UTF_8
);
var request = HttpRequest.newBuilder()
    .uri(URI.create("https://api.webscrapingapi.com/v2"
        + "?api_key=" + key + "&url=" + target
        + "&json_response=1"))
    .timeout(Duration.ofSeconds(120)).GET().build();
var response = HttpClient.newHttpClient().send(
    request, HttpResponse.BodyHandlers.ofString()
);
if (response.statusCode() >= 400) {
    throw new RuntimeException("API status " + response.statusCode());
}
System.out.println(response.body());

import java.net.URI;
import java.net.URLEncoder;
import java.net.http.*;
import java.nio.charset.StandardCharsets;
import java.time.Duration;

String apiKey = System.getenv("WSA_API_KEY");
if (apiKey == null || apiKey.isBlank()) {
    throw new IllegalStateException("WSA_API_KEY is required");
}
String key = URLEncoder.encode(
    apiKey, StandardCharsets.UTF_8
);
String target = URLEncoder.encode(
    "https://example.com/public-page",
    StandardCharsets.UTF_8
);
var request = HttpRequest.newBuilder()
    .uri(URI.create("https://api.webscrapingapi.com/v2"
        + "?api_key=" + key + "&url=" + target
        + "&json_response=1"))
    .timeout(Duration.ofSeconds(120)).GET().build();
var response = HttpClient.newHttpClient().send(
    request, HttpResponse.BodyHandlers.ofString()
);
if (response.statusCode() >= 400) {
    throw new RuntimeException("API status " + response.statusCode());
}
System.out.println(response.body());

Go net/http · standard library

package main

import (
    "fmt"
    "io"
    "net/http"
    "net/url"
    "os"
    "time"
)

func main() {
    key := os.Getenv("WSA_API_KEY")
    if key == "" { panic("WSA_API_KEY is required") }
    params := url.Values{}
    params.Add("api_key", key)
    params.Add("url", "https://example.com/public-page")
    params.Add("json_response", "1")
    client := &http.Client{Timeout: 120 * time.Second}
    response, err := client.Get(
        "https://api.webscrapingapi.com/v2?" + params.Encode(),
    )
    if err != nil { panic(err) }
    defer response.Body.Close()
    if response.StatusCode >= 400 { panic(response.Status) }
    body, err := io.ReadAll(response.Body)
    if err != nil { panic(err) }
    fmt.Println(string(body))
}

package main

import (
    "fmt"
    "io"
    "net/http"
    "net/url"
    "os"
    "time"
)

func main() {
    key := os.Getenv("WSA_API_KEY")
    if key == "" { panic("WSA_API_KEY is required") }
    params := url.Values{}
    params.Add("api_key", key)
    params.Add("url", "https://example.com/public-page")
    params.Add("json_response", "1")
    client := &http.Client{Timeout: 120 * time.Second}
    response, err := client.Get(
        "https://api.webscrapingapi.com/v2?" + params.Encode(),
    )
    if err != nil { panic(err) }
    defer response.Body.Close()
    if response.StatusCode >= 400 { panic(response.Status) }
    body, err := io.ReadAll(response.Body)
    if err != nil { panic(err) }
    fmt.Println(string(body))
}

子 Net::HTTP · standard library

require "net/http"
require "uri"

uri = URI("https://api.webscrapingapi.com/v2")
uri.query = URI.encode_www_form(
  api_key: ENV.fetch("WSA_API_KEY"),
  url: "https://example.com/public-page",
  json_response: 1
)
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
http.open_timeout = 10
http.read_timeout = 120
response = http.get(uri.request_uri)
unless response.is_a?(Net::HTTPSuccess)
  raise "API status #{response.code}"
end
puts response.body

require "net/http"
require "uri"

uri = URI("https://api.webscrapingapi.com/v2")
uri.query = URI.encode_www_form(
  api_key: ENV.fetch("WSA_API_KEY"),
  url: "https://example.com/public-page",
  json_response: 1
)
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
http.open_timeout = 10
http.read_timeout = 120
response = http.get(uri.request_uri)
unless response.is_a?(Net::HTTPSuccess)
  raise "API status #{response.code}"
end
puts response.body

C# HttpClient · .NET 6+

using System.Net;

var apiKey = Environment.GetEnvironmentVariable("WSA_API_KEY");
if (string.IsNullOrWhiteSpace(apiKey)) {
    throw new InvalidOperationException("WSA_API_KEY is required");
}
var key = WebUtility.UrlEncode(apiKey);
var target = WebUtility.UrlEncode(
    "https://example.com/public-page"
);
var requestUrl =
    "https://api.webscrapingapi.com/v2?api_key=" + key
    + "&url=" + target + "&json_response=1";
using var client = new HttpClient {
    Timeout = TimeSpan.FromSeconds(120)
};
var response = await client.GetAsync(requestUrl);
response.EnsureSuccessStatusCode();
Console.WriteLine(
    await response.Content.ReadAsStringAsync()
);

using System.Net;

var apiKey = Environment.GetEnvironmentVariable("WSA_API_KEY");
if (string.IsNullOrWhiteSpace(apiKey)) {
    throw new InvalidOperationException("WSA_API_KEY is required");
}
var key = WebUtility.UrlEncode(apiKey);
var target = WebUtility.UrlEncode(
    "https://example.com/public-page"
);
var requestUrl =
    "https://api.webscrapingapi.com/v2?api_key=" + key
    + "&url=" + target + "&json_response=1";
using var client = new HttpClient {
    Timeout = TimeSpan.FromSeconds(120)
};
var response = await client.GetAsync(requestUrl);
response.EnsureSuccessStatusCode();
Console.WriteLine(
    await response.Content.ReadAsStringAsync()
);

Tooling and handoff review

Delivery destinations are scoped during evaluation.

Direct API responses, files, and destination-based deliveries require different handoffs. Review the output, authentication, cadence, retention, and operating owner for your selected product.

Evaluation category

Postman request review

Model the documented HTTP request in your workspace and review secret handling, variables, expected responses, and collection conventions with your team.

Evaluation category

Direct HTTP response

Confirm the response format, status handling, parsing, bounded retries, and storage for products that return data to the request.

Evaluation category

File and transfer delivery

For recurring Data Feeds or a Managed Data delivery, review file format, packaging, transfer method, cadence, naming, encryption, and acceptance checks.

Evaluation category

Object storage or warehouse delivery

For prepared data from the 数据市场 or scheduled Data Feeds, evaluate the destination, credentials, partitioning, schema evolution, refresh behavior, and operating ownership.

开发人员资源

Continue with the resource built for your next step.

Use documentation for request behavior, the SDK directory for released packages, status for service condition, and the account dashboard for your access.

Browse all developer product paths

文档Published setup, parameters, examples, and responsesOpen SDK目录Released product-specific packages and repositoriesReview StatusCurrent condition of listed API servicesCheck AccountYour account access and dashboardSign in Technical guidanceProduct selection, request design, and delivery planningDiscuss 代理网页访问Choose the right API path for agent-driven web research and retrievalExplore

Run your first request

Connect your backend to Scraper API.

Choose a language example, replace the target, and verify the returned response. The Scraper API guide explains the access model and available request controls.

Start building free Explore Scraper API

探索产品

代理基础设施

Web 访问 API

数据集和数据源

托管数据

业务目标

工业

数据类型

训练和地面

多型式培训

数据基础

代理人

Web 访问 API

数据交付

建立

评估

运行

WebScrapingAPI integration examples for server-side REST calls.

Use the HTTP client already in your backend.

Delivery destinations are scoped during evaluation.

Postman request review

Direct HTTP response

File and transfer delivery

Object storage or warehouse delivery

Continue with the resource built for your next step.

Connect your backend to Scraper API.