You are here

Home » API reference » Feeds 8.3 » HttpFetcher.php

class HttpFetcher in Feeds 8.3

Defines an HTTP fetcher.

Plugin annotation


@FeedsFetcher(
  id = "http",
  title = @Translation("Download from url"),
  description = @Translation("Downloads data from a URL using Drupal's HTTP request handler."),
  form = {
    "configuration" = "Drupal\feeds\Feeds\Fetcher\Form\HttpFetcherForm",
    "feed" = "Drupal\feeds\Feeds\Fetcher\Form\HttpFetcherFeedForm",
  }
)

Hierarchy

Expanded class hierarchy of HttpFetcher

1 file declares its use of HttpFetcher
HttpFetcherTest.php in tests/src/Unit/Feeds/Fetcher/HttpFetcherTest.php

File

src/Feeds/Fetcher/HttpFetcher.php, line 35

Namespace

Drupal\feeds\Feeds\Fetcher
View source 
class HttpFetcher extends PluginBase implements ClearableInterface, FetcherInterface, ContainerFactoryPluginInterface {

  /**
   * The Guzzle client.
   *
   * @var \GuzzleHttp\ClientInterface
   */
  protected $client;

  /**
   * The cache backend.
   *
   * @var \Drupal\Core\Cache\CacheBackendInterface
   */
  protected $cache;

  /**
   * Drupal file system helper.
   *
   * @var \Drupal\Core\File\FileSystemInterface
   */
  protected $fileSystem;

  /**
   * Constructs an UploadFetcher object.
   *
   * @param array $configuration
   *   The plugin configuration.
   * @param string $plugin_id
   *   The plugin id.
   * @param array $plugin_definition
   *   The plugin definition.
   * @param \GuzzleHttp\ClientInterface $client
   *   The Guzzle client.
   * @param \Drupal\Core\Cache\CacheBackendInterface $cache
   *   The cache backend.
   * @param \Drupal\Core\File\FileSystemInterface $file_system
   *   The Drupal file system helper.
   */
  public function __construct(array $configuration, $plugin_id, array $plugin_definition, ClientInterface $client, CacheBackendInterface $cache, FileSystemInterface $file_system) {
    $this->client = $client;
    $this->cache = $cache;
    $this->fileSystem = $file_system;
    parent::__construct($configuration, $plugin_id, $plugin_definition);
  }

  /**
   * {@inheritdoc}
   */
  public static function create(ContainerInterface $container, array $configuration, $plugin_id, $plugin_definition) {
    return new static($configuration, $plugin_id, $plugin_definition, $container
      ->get('http_client'), $container
      ->get('cache.feeds_download'), $container
      ->get('file_system'));
  }

  /**
   * {@inheritdoc}
   */
  public function fetch(FeedInterface $feed, StateInterface $state) {
    $sink = $this->fileSystem
      ->tempnam('temporary://', 'feeds_http_fetcher');
    $sink = $this->fileSystem
      ->realpath($sink);

    // Get cache key if caching is enabled.
    $cache_key = $this
      ->useCache() ? $this
      ->getCacheKey($feed) : FALSE;
    $response = $this
      ->get($feed
      ->getSource(), $sink, $cache_key);

    // @todo Handle redirects.
    // @codingStandardsIgnoreStart
    // $feed->setSource($response->getEffectiveUrl());
    // @codingStandardsIgnoreEnd
    // 304, nothing to see here.
    if ($response
      ->getStatusCode() == Response::HTTP_NOT_MODIFIED) {
      $state
        ->setMessage($this
        ->t('The feed has not been updated.'));
      throw new EmptyFeedException();
    }
    return new HttpFetcherResult($sink, $response
      ->getHeaders());
  }

  /**
   * Performs a GET request.
   *
   * @param string $url
   *   The URL to GET.
   * @param string $sink
   *   The location where the downloaded content will be saved. This can be a
   *   resource, path or a StreamInterface object.
   * @param string $cache_key
   *   (optional) The cache key to find cached headers. Defaults to false.
   *
   * @return \Guzzle\Http\Message\Response
   *   A Guzzle response.
   *
   * @throws \RuntimeException
   *   Thrown if the GET request failed.
   *
   * @see \GuzzleHttp\RequestOptions
   */
  protected function get($url, $sink, $cache_key = FALSE) {
    $url = Feed::translateSchemes($url);
    $options = [
      RequestOptions::SINK => $sink,
    ];

    // Adding User-Agent header from the default guzzle client config for feeds
    // that require that.
    if (isset($this->client
      ->getConfig('headers')['User-Agent'])) {
      $options[RequestOptions::HEADERS]['User-Agent'] = $this->client
        ->getConfig('headers')['User-Agent'];
    }

    // Add cached headers if requested.
    if ($cache_key && ($cache = $this->cache
      ->get($cache_key))) {
      if (isset($cache->data['etag'])) {
        $options[RequestOptions::HEADERS]['If-None-Match'] = $cache->data['etag'];
      }
      if (isset($cache->data['last-modified'])) {
        $options[RequestOptions::HEADERS]['If-Modified-Since'] = $cache->data['last-modified'];
      }
    }
    try {
      $response = $this->client
        ->get($url, $options);
    } catch (RequestException $e) {
      $args = [
        '%site' => $url,
        '%error' => $e
          ->getMessage(),
      ];
      throw new \RuntimeException($this
        ->t('The feed from %site seems to be broken because of error "%error".', $args));
    }
    if ($cache_key) {
      $this->cache
        ->set($cache_key, array_change_key_case($response
        ->getHeaders()));
    }
    return $response;
  }

  /**
   * Returns if the cache should be used.
   *
   * @return bool
   *   True if results should be cached. False otherwise.
   */
  protected function useCache() {
    return !$this->configuration['always_download'];
  }

  /**
   * Returns the download cache key for a given feed.
   *
   * @param \Drupal\feeds\FeedInterface $feed
   *   The feed to find the cache key for.
   *
   * @return string
   *   The cache key for the feed.
   */
  protected function getCacheKey(FeedInterface $feed) {
    return $feed
      ->id() . ':' . hash('sha256', $feed
      ->getSource());
  }

  /**
   * {@inheritdoc}
   */
  public function clear(FeedInterface $feed, StateInterface $state) {
    $this
      ->onFeedDeleteMultiple([
      $feed,
    ]);
  }

  /**
   * {@inheritdoc}
   */
  public function defaultConfiguration() {
    return [
      // @todo auto_detect_feeds causes issues with downloading files that are
      // not a RSS feed. Set the default to TRUE as soon as that issue is
      // resolved.
      'auto_detect_feeds' => FALSE,
      'use_pubsubhubbub' => FALSE,
      'always_download' => FALSE,
      'fallback_hub' => '',
      'request_timeout' => 30,
    ];
  }

  /**
   * {@inheritdoc}
   */
  public function onFeedDeleteMultiple(array $feeds) {
    foreach ($feeds as $feed) {
      $this->cache
        ->delete($this
        ->getCacheKey($feed));
    }
  }

}

Members

Namesort descending Modifiers Type Description Overrides
DependencySerializationTrait::$_entityStorages protected property An array of entity type IDs keyed by the property name of their storages.
DependencySerializationTrait::$_serviceIds protected property An array of service IDs keyed by property name used for serialization.
DependencySerializationTrait::__sleep public function 1
DependencySerializationTrait::__wakeup public function 2
DependencyTrait::$dependencies protected property The object's dependencies.
DependencyTrait::addDependencies protected function Adds multiple dependencies.
DependencyTrait::addDependency protected function Adds a dependency.
HttpFetcher::$cache protected property The cache backend.
HttpFetcher::$client protected property The Guzzle client.
HttpFetcher::$fileSystem protected property Drupal file system helper.
HttpFetcher::clear public function Removes all stored results for a feed. Overrides ClearableInterface::clear
HttpFetcher::create public static function Creates an instance of the plugin. Overrides ContainerFactoryPluginInterface::create
HttpFetcher::defaultConfiguration public function Gets default configuration for this plugin. Overrides PluginBase::defaultConfiguration
HttpFetcher::fetch public function Fetch content from a feed and return it. Overrides FetcherInterface::fetch
HttpFetcher::get protected function Performs a GET request.
HttpFetcher::getCacheKey protected function Returns the download cache key for a given feed.
HttpFetcher::onFeedDeleteMultiple public function A feed is being deleted. Overrides PluginBase::onFeedDeleteMultiple
HttpFetcher::useCache protected function Returns if the cache should be used.
HttpFetcher::__construct public function Constructs an UploadFetcher object. Overrides PluginBase::__construct
MessengerTrait::$messenger protected property The messenger. 29
MessengerTrait::messenger public function Gets the messenger. 29
MessengerTrait::setMessenger public function Sets the messenger.
PluginBase::$configuration protected property Configuration information passed into the plugin. 1
PluginBase::$feedType protected property The importer this plugin is working for.
PluginBase::$linkGenerator protected property The link generator.
PluginBase::$pluginDefinition protected property The plugin implementation definition. 1
PluginBase::$pluginId protected property The plugin_id.
PluginBase::$urlGenerator protected property The url generator.
PluginBase::calculateDependencies public function Calculates dependencies for the configured plugin. Overrides DependentPluginInterface::calculateDependencies 2
PluginBase::container private function Returns the service container.
PluginBase::defaultFeedConfiguration public function Returns default feed configuration. Overrides FeedsPluginInterface::defaultFeedConfiguration 3
PluginBase::DERIVATIVE_SEPARATOR constant A string which is used to separate base plugin IDs from the derivative ID.
PluginBase::getBaseId public function Gets the base_plugin_id of the plugin instance. Overrides DerivativeInspectionInterface::getBaseId
PluginBase::getConfiguration public function Gets this plugin's configuration. Overrides ConfigurableInterface::getConfiguration
PluginBase::getDerivativeId public function Gets the derivative_id of the plugin instance. Overrides DerivativeInspectionInterface::getDerivativeId
PluginBase::getPluginDefinition public function Gets the definition of the plugin implementation. Overrides PluginInspectionInterface::getPluginDefinition 3
PluginBase::getPluginId public function Gets the plugin_id of the plugin instance. Overrides PluginInspectionInterface::getPluginId
PluginBase::isConfigurable public function Determines if the plugin is configurable.
PluginBase::l protected function Renders a link to a route given a route name and its parameters.
PluginBase::linkGenerator protected function Returns the link generator service.
PluginBase::onFeedSave public function A feed is being saved.
PluginBase::onFeedTypeDelete public function The feed type is being deleted. 1
PluginBase::onFeedTypeSave public function The feed type is being saved. 1
PluginBase::pluginType public function Returns the type of plugin. Overrides FeedsPluginInterface::pluginType
PluginBase::setConfiguration public function Sets the configuration for this plugin instance. Overrides ConfigurableInterface::setConfiguration 1
PluginBase::url protected function Generates a URL or path for a specific route based on the given parameters.
PluginBase::urlGenerator protected function Returns the URL generator service.
StringTranslationTrait::$stringTranslation protected property The string translation service. 1
StringTranslationTrait::formatPlural protected function Formats a string containing a count of items.
StringTranslationTrait::getNumberOfPlurals protected function Returns the number of plurals supported by a given language.
StringTranslationTrait::getStringTranslation protected function Gets the string translation service.
StringTranslationTrait::setStringTranslation public function Sets the string translation service to use. 2
StringTranslationTrait::t protected function Translates a string to the current language or to a given language.