Skip to main content

HTTP Headers - Comprehensive List, Classification, Usage.

HTTP headers are integral to the infrastructure of web communication, serving as a mechanism for the exchange of information between client and server machines. These headers dictate numerous aspects of how requests and responses are processed, encompassing content management, caching strategies, and security protocols. Head to HTTP Headers - Introduction for getting started to about about them. This article is a second in the series to provide a comprehensive list of HTTP headers, providing application developers and integration specialists with a resource to deepen their understanding.

Content Negotiation​

By leveraging client information like language, device type, and supported formats, content negotiation facilitates optimal content presentation, enhancing user engagement and website performance. Here is a list of some important HTTP headers.

  • Accept: This header informs the server about the media types (e.g., text/html, image/png) the client is willing to accept. It can include multiple types, each with an optional quality factor (q) to express preference. The web-browsers send Accept headers to indicate supported formats, allowing servers to choose the most suitable one.
  • Accept-Charset: This header specifies the character encodings (e.g., UTF-8, ISO-8859-1) the client understands. It can include multiple encodings with quality factors. It is useful for serving content in the appropriate character encoding based on the client's capabilities.
  • Accept-Language: Using this the server learns about the languages (e.g., en-US, fr-FR) the client or browser can understand. It can include multiple languages with quality factors. The server then delivers content in the user's preferred language, eventually enhancing user experience.
  • Accept-Encoding: This header specifies the compression algorithms (e.g., gzip, deflate) the client can handle. It can include multiple algorithms with quality factors.
  • Vary: This header informs clients about response headers that are dependent on the values in specific request headers like Accept and Accept-Language. It helps clients cache responses based on negotiated content for future requests.
  • Content-Type: While not directly for negotiation, this header in the server response confirms the media type of the delivered content. It validates the server's choice and helps clients process the content correctly.
  • Content-Language: Similar to Accept-Language, this header in the server response indicates the actual language of the delivered content.

Caching​

Understanding web caching is crucial, and HTTP headers significantly influence the behavior of web browsers and your application. Here's a list:

  • Cache-Control: This is the most important header for caching strategies. It allows you to specify how long a resource can be cached on the client side. It also suggests if the resource can be cached by public or private caches, and how it should be revalidated.
  • Expires: This header is deprecated, but it can still be used to specify an absolute expiration date for a resource.
  • Last-Modified: This header indicates the date and time that a resource was last modified. This is optional, yet it can be used by caches to validate whether a cached resource is still up-to-date.
  • ETag: This header is a unique identifier for a resource. It can be used by caches to validate whether a cached resource is still up-to-date, even if the Last-Modified header has not changed.
  • If-Modified-Since: This header can be used by a client to ask a server if a cached resource is still up-to-date. The client sends the Last-Modified header from the cached resource, and the server responds with a 304 Not Modified if the resource is still up-to-date, or with the new resource if it has been modified.
  • If-None-Match: This header can be used by a client to ask a server if a cached resource is still up-to-date. The client sends the ETag header from the cached resource, and the server responds with a 304 Not Modified if the resource is still up-to-date, or with the new resource if it has been modified.

Web Security​

Web security standards have seen significant evolution over the past decade. Below is a comprehensive list of HTTP headers related to web security:

  • X-Frame-Options: This header helps prevent clickjacking attacks by stopping a page from loading within a frame or iframe.
  • X-XSS-Protection: This header helps prevent reflected cross-site scripting (XSS) attacks by stopping pages from loading when they detect them. However, it is not recommended to use this header, as it can create XSS vulnerabilities in otherwise safe websites.
  • X-Content-Type-Options: This header helps prevent MIME type sniffing attacks by stopping browsers from guessing the MIME type of a resource.
  • Referrer-Policy: This header controls how much referrer information is sent with requests.
  • Content-Type: This header specifies the original media type of a resource. Setting it correctly helps prevent XSS vulnerabilities.
  • Set-Cookie: This header is used to send cookies from the server to the user agent.
  • Strict-Transport-Security (HSTS): This header tells a website to only be accessed using HTTPS.
  • Expect-CT: This header is used to report Certificate Transparency (CT) requirements. It is not recommended to use this header, as it is being phased out.
  • Content-Security-Policy (CSP): This header helps mitigate XSS and data injection attacks by specifying the origins of content that can be loaded on a website.
  • Access-Control-Allow-Origin (CORS): This header relaxes the Same Origin Policy (SOP) by allowing certain origins to access resources from a website.
  • Cross-Origin-Opener-Policy (COOP): This header helps prevent attacks like Spectre by stopping a top-level document from sharing a browsing context group with cross-origin documents.
  • Cross-Origin-Embedder-Policy (COEP): This header helps prevent attacks like Spectre by stopping a document from loading any cross-origin resources that don't explicitly grant the document permission.
  • Cross-Origin-Resource-Policy (CORP): This header helps prevent attacks like Spectre by controlling the set of origins that are allowed to load a resource.
  • Permissions-Policy: This header controls which origins can use which browser features.
  • FLoC (Federated Learning of Cohorts): This header allows a site to opt-out of being included in a user's list of sites for cohort calculation, which is used for interest-based advertising.
  • Server: This header describes the software used by the origin server. It is not a security header, but attackers can use the information in it to find vulnerabilities.
  • X-Powered-By: This header describes the technologies used by the web server. Attackers can use the information in it to find vulnerabilities.
  • X-AspNet-Version: This header provides information about the .NET version. It is recommended to disable sending this header.
  • X-AspNetMvc-Version: This header provides information about the .NET version. It is recommended to disable sending this header.
  • X-DNS-Prefetch-Control: This header controls DNS prefetching, which is a feature by which browsers proactively perform domain name resolution.
  • Public-Key-Pins (HPKP): This header is used to associate a specific cryptographic public key with a certain web server to decrease the risk of MITM attacks with forged certificates. It is deprecated and should not be used anymore.

Cross-Origin Resource Sharing​

For security reasons, websites typically restrict resources (data, images) from being directly accessed by other websites residing in different domains. This prevents unauthorized data access and malicious activities. CORS (Cross-Origin Resource Sharing) serves as a sophisticated mechanism to bridge these security boundaries. It empowers websites to specify which other websites (potentially belonging to different domains) are authorized to access their resources and under what conditions. All this is controlled using HTTP headers.

  • Origin (Request header): Identifies the origin of the request (originating website). This should be sent in the request from the client to the remote site.
  • Vary (Response header): Indicates headers that influence the response, potentially including Origin (relevant for CORS).
  • Access-Control-Allow-Origin (Server Response): Controls which website origins (e.g., domain, protocol) are permitted to access the requested resource. This prevents unauthorized cross-origin requests and enhances security.
  • Access-Control-Allow-Credentials (Server Response): Specifies whether the server allows credentials (cookies, authorization headers) to be sent in cross-origin requests. This is crucial for functionalities like authenticated API calls across different domains.
  • Access-Control-Expose-Headers (Server Response): Allows the server to explicitly list custom response headers that should be made accessible to the client-side JavaScript code, even if not included in the default CORS-accessible headers. This enables utilization of data within the client application.
  • Access-Control-Request-Method (Preflight Request): Used in preflight requests (OPTIONS method) sent by the browser before the actual request. It informs the server about the HTTP method intended for the actual request (e.g., POST, PUT).
  • Access-Control-Request-Headers (Preflight Request): Another preflight request header informing the server about any custom request headers the browser plans to send with the actual request. This allows the server to verify compatibility and handle custom headers appropriately.

During a CORS request,

  • the browser first sends a preflight request (OPTIONS method) with the Access-Control-Request-Method and Access-Control-Request-Headers (optional) to check if the server allows the actual request.
  • the server responds with the Access-Control-Allow-Origin, Access-Control-Allow-Credentials (optional), and Access-Control-Expose-Headers (optional) headers, indicating permissions and accessible data.
  • if the preflight is successful, the browser sends the actual request with the allowed method and headers.

The Origin header (in the request) and Vary header (in the response) plays a role in CORS as well, though they are not strictly CORS-specific headers. For details specs, features, and limitations refer to MDN Docs on CORS

HTTP Communication​

There are headers required for basic communication. HTTP offers a a bunch of headers catering to basic client/server communication. Here's an extended list:

  • Accept-Encoding: Sent in HTTP requests. It informs the server about what compression formats the client can understand (e.g., gzip, deflate, br), so that the server can potentially compress the response body for efficient transmission over TCP. The objective is to save bandwidth and improving performance of the websites.
  • Content-Encoding: Sent in the HTTP response. Indicates the compression method applied to the response body by the server (e.g., gzip, br). This helps the client decompress the received data accurately.
  • Location: This header is present in the response. Instructs the client to redirect to a different URL, often used for error handling (e.g., 301 Moved Permanently) or load balancing purposes.
  • Range: Used to request a specific portion of a resource, rather than the entire content. This is particularly beneficial for large files downloads, enabling the client to download only the desired section (e.g., Range: bytes=100-200).
  • Content-Range: When responding to a range request, the server includes this header to specify the range of bytes delivered in the partial response. This helps the client understand the extent of the received data and potentially make further requests if necessary.
  • Authorization: This is sent in HTTP requests and carries authentication credentials (e.g., username and password) to allow access to protected resources.
  • Accept-CH (Request): Signals the client's support for Client Hints, a mechanism allowing the server to request specific pieces of information from the client before responding. This can improve efficiency by preemptively fetching resources the client is likely to need.
  • DNT: This provides an optional signal from the user regarding their Do Not Track (DNT) preference. The header value can be either 0 (disabled) or 1 (enabled) and should be sent in the request. While not mandatory for websites to respect this header, it allows users to express their preference for limiting online tracking.