Get data byc url that downloads javascpirt as xhr from visited url
I'm trying to get data from (https)://example.com/page/ with javascript which downloads additional data from (https)://example.com/page/details. So, by curl I get a data from the first url but without a data from the second url. Also if I try to get data from the 2-nd url, it returns a simiral data of the 1-st url.
Headers of requests (2 different):
$headers[] = array('Host: https://example.com/page/details',
'User-Agent: установлен',
'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'Accept-Language: ru-RU,ru;q=0.8,en-US;q=0.5,en;q=0.3',
'Accept-Encoding: gzip, deflate, br',
'DNT: 1',
'Upgrade-Insecure-Requests: 1');
// $headers[] = array('Host: https://example.com/page/details',
// 'Accept: */*',
// 'Accept-Language: ru-RU,ru;q=0.8,en-US;q=0.5,en;q=0.3',
// 'X-CSRFToken: установлен',
// 'X-Requested-With: XMLHttpRequest',
// 'Referer: https://example.com/page/',
// 'DNT: 1',
// 'If-None-Match: "установлен"',
// 'Connection: keep-alive',
// 'Pragma: no-cache',
// 'Cache-Control: no-cache');
My code:
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_TIMEOUT, 60 );
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers[$i]);
Headers of response when browser gets a right data though the 2-nd url (there is not Connection: keep-alive). Connection is break down and the 2-nd url reloaded with disabled js.
Content-Length: 10813
Content-Type: text/html; charset=utf-8
Date:
Etag: "[a-z0-9]+"
Public-Key-Pins: max-age=5184000; includeSubDomains; pin-sha256="страшная абракадабра"; pin-sha256="и тут тоже"
Server: nginx
Strict-Transport-Security: max-age=31536000
Vary: User-Agent
X-Content-Type-Options: nosniff
content-security-policy: script-src https://ssl.google-analytics.com/ga.js https://www.google.com/recaptcha/ https://www.gstatic.com/recaptcha/ https://addons.cdn.mozilla.net; style-src 'self' 'unsafe-inline' https://addons.cdn.mozilla.net; default-src 'self'; frame-src 'self' https://www.google.com/recaptcha/; child-src 'self' https://www.google.com/recaptcha/; img-src 'self' data: blob: https://ssl.google-analytics.com https://addons.cdn.mozilla.net https://static.addons.mozilla.net https://sentry.prod.mozaws.net; media-src https://videos.cdn.mozilla.net; object-src 'none'; connect-src 'self' https://sentry.prod.mozaws.net; font-src 'self' https://addons.cdn.mozilla.net; form-action 'self' https://developer.mozilla.org; base-uri 'self' https://addons.mozilla.org; report-uri /__cspreport__
x-frame-options: DENY
x-xss-protection: 1; mode=block
What are needed to fix to get a correct data?
链接地址: http://www.djcxy.com/p/87308.html