What a web developer needs to know about the HTTP protocol. HTTP protocol rules
.) It is through the ability to specify how a message is encoded that the client and server can exchange binary data, although this protocol is text.
Proxy servers
History of development
HTTP/0.9
In addition to the usual GET method, there is also a distinction. Conditional GET requests contain If-Modified-Since, If-Match, If-Range, and similar headers. Partial GETs contain Range in the request. Execution order similar requests defined separately by standards.
HEAD
Similar to the GET method, except that there is no body in the server response. The HEAD request is typically used to retrieve metadata, check for the existence of a resource (URL validation), and see if it has changed since it was last accessed.
Response headers may be cached. If a resource's metadata does not match the corresponding information in the cache, the copy of the resource is marked as out of date.
POST
Used to transfer user data to a specified resource. For example, on blogs, visitors can typically enter their comments on posts into an HTML form, after which they are POSTed to the server and placed on the page. In this case, the transmitted data (in the example with blogs, the text of the comment) is included in the body of the request. Similarly, using the POST method, files are usually uploaded to the server.
Unlike the GET method, the POST method is not considered idempotent, that is, repeating the same POST requests may return different results (for example, after each comment is submitted, one copy of that comment will appear).
If the execution result is 200 (Ok), a message about the completion of the request should be included in the response body. If a resource has been created, the server SHOULD return a 201 (Created) response with the URI of the new resource in the Location header.
The server response message to the POST method is not cached.
PUT
Used to load the request content to the URI specified in the request. If a resource does not exist at the given URI, the server creates it and returns status 201 (Created). If the resource has been changed, the server returns 200 (Ok) or 204 (No Content). The server MUST NOT ignore invalid Content-* headers sent by the client along with the message. If any of these headers cannot be recognized or are not valid under current conditions, then an error code of 501 (Not Implemented) must be returned.
The fundamental difference between the POST and PUT methods is the understanding of the purpose of the resource URI. The POST method assumes that the specified URI will be used to process the content sent by the client. By using PUT, the client assumes that the content being downloaded matches the resource located at the given URI.
Server response messages to the PUT method are not cached.
PATCH
Similar to PUT, but applies only to a fragment of the resource.
DELETE
Deletes the specified resource.
TRACE
Returns the received request so that the client can see what information intermediate servers add or change in the request.
LINK
Establishes a connection between the specified resource and others.
UNLINK
Removes the connection of the specified resource with others.
CONNECT
Converts a request connection into a transparent TCP/IP tunnel, typically to facilitate the establishment of a secure SSL connections via an unencrypted proxy.
Status Codes
The status code is part of the first line of the server response. It represents an integer of three Arabic numerals. The first digit indicates the class of the condition. The response code is usually followed by an explanatory phrase separated by a space. English, which explains to the person the reason for this particular answer. Examples:
201 Webpage Created 403 Access allowed only for registered users 507 Insufficient Storage
The client learns from the response code about the results of its request and determines what actions to take next. The set of status codes is a standard and they are described in the corresponding RFCs. The introduction of new codes should be made only after agreement with the IETF. The client may not know all status codes, but it must respond according to the class of the code.
There are currently five classes of status codes.
1xx Informational (Russian) Informational)This class contains codes that inform about the transfer process. In HTTP/1.0, messages with such codes should be ignored. In HTTP/1.1, the client must be prepared to accept this class of messages as a normal response, but does not need to send anything to the server. The messages themselves from the server contain only start line response and, if required, several response-specific header fields. Proxy servers must send such messages further from the server to the client.
2xx Success (Russian) Success)Messages of this class inform about cases of successful acceptance and processing of a client’s request. Depending on the status, the server may also transmit the headers and body of the message.
3xx Redirection (Russian) Redirection )Class 3xx codes tell the client that in order to successfully complete the operation, it is necessary to make another request (usually to a different URI). Of this class, five codes , , , and relate directly to redirections (redirect). The server specifies the address to which the client should make the request in the Location header. However, it is possible to use fragments in the target URI.
4xx Client Error (Russian) Client error)The 4xx code class is intended to indicate errors on the client side. When using all methods except HEAD, the server must return a hypertext explanation to the user in the body of the message.
To remember the values of codes from 400 to 417, there are illustrative mnemonics techniques
5xx Server Error (Russian) Server error)Codes 5xx are allocated for cases of unsuccessful operation due to the fault of the server. For all situations other than using the HEAD method, the server must include in the body of the message an explanation that the client will display to the user.
Headings
Message body
The HTTP message body (message-body), if present, is used to convey the body of the object associated with the request or response. The message-body differs from the entity-body only when transfer encoding is applied, as indicated by the Transfer-Encoding header field.
Message-body = entity-body |
The Transfer-Encoding field must be used to indicate any transfer encoding applied by the application to ensure that the message is transmitted securely and correctly. The Transfer-Encoding field is a property of the message, not the object, and thus can be added or removed by any application in the request/response chain.
The rules governing whether a message body is acceptable in a message are different for requests and responses.
The presence of a message body in a request is indicated by adding a Content-Length or Transfer-Encoding header field to the request headers. A message body (message-body) MAY only be added to a request when the request method allows an entity-body.
Whether or not a message-body is included in the response message depends on both the request method and the response status code. All responses to a request with the HEAD method must not include a message-body, even if entity-header fields are present to make one believe that the entity is present. No responses with status codes 1xx (Informational), 204 (No Content), and 304 (Not Modified) must contain a message-body. All other responses contain a message body, even if it has zero length.
HTTP Dialog Examples
Regular GET request
There are two main types of approvals:
- Server-managed Server-Driven).
- Client managed Agent-Driven).
Both types or each of them separately can be used simultaneously.
The main protocol specification (RFC 2616) also highlights the so-called transparent negotiation. Transparent Negotiation) as the preferred option for combining both types. The latter mechanism should not be confused with the independent technology Transparent Content Negotiation (TCN, Russian. Transparent content approval , see RFC 2295), which is not part of the HTTP protocol, but can be used with it. Both have significant differences in the principle of operation and the very meaning of the word “transparent”. In the HTTP specification, transparency means that the process is invisible to the client and server, and in TCN technology, transparency means accessibility full list resource options for all participants in the data delivery process.
Server Managed
If there are multiple versions of a resource, the server can analyze the client's request headers to produce what it believes is the most appropriate version. The main headers analyzed are Accept, Accept-Charset, Accept-Encoding, Accept-Languages and User-Agent. It is advisable for the server to include a Vary header in the response indicating the parameters by which the content of the requested URI differs.
The geographic location of the client can be determined by the remote IP address. This is possible due to the fact that IP addresses, like domain names, are registered on specific person or organization. When registering, you specify the region in which the desired address space will be used. This data is publicly available, and corresponding freely available databases and ready-made data can be found on the Internet. software modules to work with them (you should focus on keywords"Geo IP").
It should be remembered that this method is capable of determining the location to a maximum of a city (from here the country is determined). In this case, the information is relevant only at the time of registration of the address space. For example, if a Moscow provider registers a range of addresses indicating Moscow and begins to provide access to clients from the nearest Moscow region, then its subscribers may observe on some sites that they are from Moscow, and not from Krasnogorsk or Dzerzhinsky.
Server-driven negotiation has several disadvantages:
- The server only guesses which option is most preferable for end user, but cannot know exactly what is needed in at the moment(for example, version in Russian or English).
- There are a lot of Accept group headers sent, but few resources with multiple options. Because of this, the equipment experiences excessive load.
- The shared cache is limited in its ability to produce the same response to identical requests from different users.
- Passing Accept headers may also reveal some information about its preferences, such as languages used, browser, encoding.
Customer driven
IN in this case the content type is determined only on the client side. To do this, the server returns with a status code of 300 (Multiple Choices) or 406 (Not Acceptable) a list of options from which the user selects the appropriate one. Client-driven reconciliation is good when content varies in common ways (such as language and encoding) and a public cache is used.
The main disadvantage is extra load, since you have to make an additional request to get the desired content.
Transparent approval
This negotiation is completely transparent to the client and server. In this case, a shared cache is used that contains a list of options, similar to client-driven negotiation. If the cache understands all these options, then it makes the choice itself, as in server-driven negotiation. This reduces the load on the origin server and eliminates the additional request from the client.
The core HTTP specification does not describe the transparent negotiation mechanism in detail.
Multiple Contents
HTTP protocol supports the transfer of multiple entities within a single message. Moreover, entities can be transmitted not only in the form of a single-level sequence, but in the form of a hierarchy with nesting of elements into each other. The media types multipart/* are used to indicate multiple content. Working with such types is carried out using general rules as described in RFC 2046 (unless otherwise specified by a specific media type). If the recipient doesn't know how to handle the type, then it treats it the same way as multipart/mixed .
The boundary parameter means the separator between various types transmitted messages. For example, the DestAddress parameter passed from the form passes the value e-mail addresses, and the AttachedFile1 element that follows it sends the binary content of the image in .jpg format
On the server side, messages with multiple contents can be sent in response to requests for multiple resource fragments. In this case, the media type multipart/byteranges is used.
On the client side, when submitting an HTML form, the POST method is most often used. A typical example: email sending pages with file attachments. When sending such a letter, the browser generates a message of the multipart/form-data type, integrating into it as separate parts entered by the user, the subject of the letter, the recipient's address, the text itself and attached files:
POST /send-message.html HTTP/1.1 Host: mail.example.com Referer: http://mail.example.com/send-message.html User-Agent: BrowserForDummies/4.67b Content-Type: multipart/form- data; boundary="Asrf456BGe4h" Content-Length: (total volume including child headers) Connection: keep-alive Keep-Alive: 300 (empty line) (missing preamble) --Asrf456BGe4h Content-Disposition: form-data; name="DestAddress" (empty line) [email protected]--Asrf456BGe4h Content-Disposition: form-data; name="MessageTitle" (empty line) I'm indignant --Asrf456BGe4h Content-Disposition: form-data; name="MessageText" (empty line) Hello Vasily! Your pet lion, which you left with me last week, tore up my entire sofa. Please pick him up soon! Attached are two photos with the consequences. --Asrf456BGe4h Content-Disposition: form-data; name="AttachedFile1"; filename="horror-photo-1.jpg" Content-Type: image/jpeg (empty line) (binary content of the first photo) --Asrf456BGe4h Content-Disposition: form-data; name="AttachedFile2"; filename="horror-photo-2.jpg" Content-Type: image/jpeg (empty line) (binary content of second photo) --Asrf456BGe4h-- (missing epilogue)
In the example in the Content-Disposition headers, the name parameter corresponds to name attribute in HTML tags And
Protocol Features
Most protocols provide for the establishment of a TCP session, during which authorization occurs once, and further actions are executed in the context of this authorization. HTTP, on the other hand, establishes a separate TCP session for each request; Later versions of HTTP allowed multiple requests to be made during a single TCP session, but browsers typically request only the page and its included objects (images, cascading styles, etc.) and then immediately terminate the TCP session. To support authorized (non-anonymous) access, HTTP uses cookies; Moreover, this authorization method allows you to save the session even after rebooting the client and server.
When accessing data via FTP or file protocols, the file type (more precisely, the type of data it contains) is determined by the file name extension, which is not always convenient. HTTP, before transmitting the data itself, transmits the “Content-Type: type/subtype” header, which allows the client to uniquely determine how to process the sent data. This is especially important when working with CGI scripts, when the file name extension indicates not the type of data sent to the client, but the need to run this file on the server and sending the client the results of the program written in this file (in this case, the same file, depending on the request arguments and its own considerations, can generate responses different types- in the simplest case, pictures in different formats).
In addition, HTTP allows the client to send parameters to the server that will be passed to the CGI script being launched. For this purpose, forms were introduced into HTML.
The listed features of HTTP made it possible to create search engines (the first of which was AltaVista, created by DEC), forums and Internet stores. This commercialized the Internet, companies emerged whose main field of activity was providing Internet access (providers) and creating websites.
Notes
See also
Links
Allows you to receive various resources, such as HTML documents. The HTTP protocol underlies data exchange on the Internet. HTTP is a client-server communication protocol, which means requests to the server are initiated by the recipient itself, usually a web browser. The resulting final document will be reconstructed from various sub-documents, for example, from separately obtained text, a description of the document structure, images, video files, scripts and much more.
Clients and servers interact by exchanging individual messages(not a data stream). Messages sent by a client, usually a web browser, are called requests, and messages sent by the server are called answers.
Although HTTP was developed in the early 1990s, it has been continually improved due to its extensibility. HTTP is a protocol application level, which most often uses the capabilities of another protocol - TCP (or TLS - secure TCP) - to forward its messages, however, any other reliable transport protocol can theoretically be used to deliver such messages. Due to its extensibility, it is used not only for the client to receive hypertext documents or images and videos, but also for transmitting content to servers, for example, using HTML forms. HTTP can also be used to retrieve only parts of a document for the purpose of updating a web page on demand.
Components of HTTP-based systems
HTTP is a client-server protocol, that is, requests are sent by one party - the user-agent (or a proxy instead). Most often, a web browser acts as a user agent, but it can be anyone, for example, a robot traveling the Web to replenish and update web page indexing data for search engines.
Each individual request request) is sent to the server, which processes it and returns a response (eng. response). Between these requests and responses there are numerous intermediaries called proxies, which perform various operations and act as gateways or caches, for example.
In reality, between the browser and the server there are many more different intermediary devices that play some role in processing the request: routers, modems, and so on. Due to the fact that the Network is built on the basis of a system of interaction levels (layers), these intermediaries are “hidden” at the network and transport levels. In this level system, HTTP ranks highest top level which is called the "application" (or "application layer"). Knowledge of network layers such as presentation, session, transport, network, link and physical, essential for network understanding and diagnostics possible problems, are not required to describe and understand HTTP.
Client: user agent
A user agent is any tool or device that acts on behalf of a user. This role primarily belongs to the web browser; In some cases, user agents are programs that are used by engineers and web developers to debug their applications.
Browser Always is the entity that initiates the request. The server never does this (although over the many years of the network's existence, mechanisms have been created that can simulate requests from the server).
To display a web page, the browser sends an initial request to obtain the HTML document of that page. After this, the browser parses this document and requests additional files, necessary for displaying the content of the web page (executable scripts, information about the page layout - CSS tables styles, additional resources in the form of images and video files). Next, the browser connects all these resources to display them to the user in the form of a single document - a web page. Scripts executed by the browser itself can receive additional resources over the network at later stages of processing of the web page, and the browser updates the user's view of that page accordingly.
A web page is a hypertext document. This means that some parts of the displayed text are links that can be activated (usually by clicking a mouse button) to retrieve and therefore display a new web page. This allows the user to direct their user agent when navigating the Web. The browser translates these “traffic directions” into HTTP requests and subsequently interprets the HTTP responses in a user-readable form.
Web server
On the other side communication channel the server that serves (English) is located serve) user, providing him with documents upon request. From the end user's point of view, the server is always one virtual machine, completely or partially generating a document, although in fact it can be a group of servers between which the load is balanced, that is, requests from different users are redistributed, or a complex software polling other computers (such as caching servers, database servers, application servers e-commerce and others).
A server is not necessarily located on one machine, and vice versa - several servers can be located (hosted) on the same machine. According to HTTP/1.1 version and having a Host header, they can even share the same IP address.
Proxy
Between the web browser and the server are large number network nodes transmitting HTTP messages. Due to their layered structure, most of them also operate on the transport network or physical levels, becoming transparent to the HTTP layer and potentially reducing performance. These application-level operations are called proxy . They may or may not be transparent (modifying requests will not pass through them), and can perform many functions:
- caching (cache can be public or private, like browser cache)
- filtering (like antivirus scanning, parental controls, …)
- load balancing (allow multiple servers to serve different requests)
- authentication (control access to different resources)
- logging (permission to store transaction history)
Basic Aspects of HTTP
HTTP is simple
Even with the greater complexity introduced in HTTP/2 by encapsulating HTTP messages in frames, HTTP is generally simple and human-readable. HTTP messages can be read and understood by humans, providing easier testing for developers and reduced complexity for new users.
HTTP - extensible
The HTTP headers introduced in HTTP/1.0 made the protocol easy to extend and experiment with. New functionality can even be introduced by a simple agreement between client and server on the semantics of the new header.
HTTP is stateless but has a session
HTTP is stateless: there is no relationship between two requests that are executed sequentially over the same connection. This immediately implies the potential for problems for a user attempting to interact with a particular page sequentially, such as when using a shopping cart in an online store. But while HTTP core is stateless, cookies enable stateful sessions. Using header extensibility, cookies are added to the worker thread, allowing the session to share some context, or state, on each HTTP request.
HTTP and connections
The connection is managed at the transport layer, and therefore fundamentally goes beyond the boundaries of HTTP. Although HTTP does not require the underlying transport protocol to be connection-based, requiring only reliability, or no lost messages (i.e., at least an error representation). Among the two most common Internet transport protocols, TCP is reliable while UDP is not. HTTP subsequently relies on the TCP standard being connection-based, even though a connection is not always required.
HTTP/1.0 opened a TCP connection for each request/response exchange, with two important drawbacks: opening a connection requires multiple message exchanges and is therefore slow, although it becomes more efficient when sending multiple messages, or when sending messages regularly: warm connections are more effective than cold.
To mitigate these shortcomings, HTTP/1.1 introduced pipelining (which proved difficult to implement) and stable connections: lying in TCP based the connection can be partially controlled through the Connection header. HTTP/2 took the next step by adding multiplexing of messages across a simple connection, helping to keep the connection warm and more efficient.
Experiments are underway to develop better transport protocol, more suitable for HTTP. For example, Google is experimenting with QUIC, which is based on UDP, to provide a more reliable and efficient transport protocol.
What can be controlled via HTTP
The natural extensibility of HTTP has allowed greater control and functionality of the Web over time. Cache and authentication methods were early features in HTTP history. The ability to relax the original restrictions, on the other hand, was added in the 2010s.
Listed below general functions, managed with HTTP.
-
The server can instruct proxies and clients what to cache and for how long. The client can instruct intermediate cache proxies to ignore stored documents. - Relaxing Source Constraints
To prevent spyware and other privacy-violating intrusions, the web browser enforces strict separation between websites. Only pages from same source can access information on the web page. Although such restrictions are taxing on the server, HTTP headers can relax the strict separation on the server side, allowing the document to become part of information from different domains (for security reasons). - Authentication
Some pages are only available to special users. Basic authentication can be provided via HTTP, either through the use of the WWW-Authenticate and similar headers, or by setting up a special session using cookies. - Proxy and tunneling
Servers and/or clients are often located on an intranet, and hide their true IP addresses from others. HTTP requests go through a proxy to cross this network barrier. Not all proxies are HTTP proxies. The SOCKS protocol, for example, operates at a lower level. Others, such as ftp, can be handled by these proxies. - Sessions
Usage HTTP cookie allows you to associate a request with a state on the server. This creates a session, even though HTTP is a stateless protocol at its core. This is useful not only for shopping carts in online stores, but also for any sites that allow the user to customize the exit.
HTTP stream
When a client wants to communicate with a server, whether it is a final server or an intermediate proxy, it follows these steps:
- Opening a TCP connection: A TCP connection will be used to send a request or requests and receive a response. The client can open a new connection, reuse an existing one, or open multiple TCP connections to the server.
- Sending an HTTP message: HTTP messages (before HTTP/2) are human-readable. Since HTTP/2, simple messages are encapsulated in frames, making them impossible to read directly, but fundamentally remain the same. GET / HTTP/1.1 Host: site Accept-Language: fr
- Reads response from server: HTTP/1.1 200 OK Date: Sat, 09 Oct 2010 14:28:02 GMT Server: Apache Last-Modified: Tue, 01 Dec 2009 20:18:22 GMT ETag: "51142bc1-7449-479b075b2891b" Accept-Ranges: bytes Content-Length: 29769 Content-Type: text/html
- Closes or reuses the connection for further requests.
If the HTTP pipeline is enabled, multiple requests can be sent without waiting for the first response to be received in its entirety. The HTTP pipeline is difficult to integrate into existing networks, where old pieces of software coexist with modern versions. The HTTP pipeline was replaced in HTTP/2 with more reliable multiplexed requests in a frame.
HTTP messages
HTTP/1.1 and earlier HTTP messages are human-readable. In HTTP/2, these messages are embedded in a new binary structure, a frame, that allows optimizations such as header compression and multiplexing. Even if part of the original HTTP message is sent in this version of HTTP, the semantics of each message are not changed and the client recreates (virtually) the original HTTP request. It is also useful for understanding HTTP/2 messages in HTTP/1.1 format.
HTTP (HyperText Transfer Protocol) was developed as the basis of the World Wide Web.
The HTTP protocol works as follows: the client program establishes a TCP connection with the server (standard port number 80) and issues an HTTP request to it. The server processes this request and issues an HTTP response to the client.
HTTP request structure
An HTTP request consists of a request header and a request body, separated by an empty line. The request body may be missing.
The request header consists of the main (first) line of the request and subsequent lines that clarify the request in the main line. Subsequent lines may also be missing.
The main line query consists of three parts, separated by spaces:
Method(in other words, the HTTP command):
GET- document request. The most commonly used method; in HTTP/0.9, they say, he was the only one.
HEAD- document title request. It differs from GET in that only the request header with information about the document is returned. The document itself is not issued.
POST- this method is used to transfer data to CGI scripts. The data itself appears in subsequent lines of the request in the form of parameters.
PUT- place the document on the server. As far as I know, it is rarely used. A request with this method has a body in which the document itself is transmitted.
Resource- this is the path to a specific file on the server that the client wants to receive (or place - for the PUT method). If the resource is simply some file to be read, the server must return it in the response body for this request. If this is the path to a CGI script, then the server runs the script and returns the result of its execution. By the way, thanks to this unification of resources, the client is practically indifferent to what he represents on the server.
Protocol version-version of the HTTP protocol with which the client program works.
So a simple HTTP request might look like this:
This requests the root file from the web server's root directory.
The lines after the main query line have the following format:
Parameter: value.
This is how the request parameters are set. This is optional; all lines after the main query line may be missing; in this case, the server accepts their value by default or based on the results of the previous request (when working in Keep-Alive mode).
I will list some of the most commonly used HTTP request parameters:
Connection(connection) - can take the values Keep-Alive and close. Keep-Alive means that after issuing this document, the connection to the server is not broken, and more requests can be issued. Most browsers work in Keep-Alive mode, since it allows you to “download” an html page and images for it in one connection to the server. Once set, Keep-Alive mode is maintained until the first error or until the next Connection: close request is explicitly specified.
close ("close") - the connection is closed after responding to this request.
User-Agent- the value is the browser "code", for example:
Mozilla/4.0 (compatible; MSIE 5.0; Windows 95; DigExt)
Accept- a list of content types supported by the browser in order of their preference for a given browser, for example for my IE5:
Accept: image/gif, image/x-xbitmap, image/jpeg, image/pjpeg, application/vnd.ms-excel, application/msword, application/vnd.ms-powerpoint, */*
This is obviously necessary for the case when the server can output the same document in different formats.
The value of this parameter is used mainly by CGI scripts to generate a response adapted for a given browser.
Referrer- URL from which you came to this resource.
Host- the name of the host from which the resource is requested. Useful if the server has several virtual servers under the same IP address. In this case, the name of the virtual server is determined by this field.
Accept-Language- supported language. Significant for a server that may serve the same document in different language versions.
HTTP response format
The response format is very similar to the request format: it also has a header and body separated by an empty line.
The header also consists of a main line and parameter lines, but the format of the main line is different from that of the request header.
The main query string consists of 3 fields separated by spaces:
Protocol version- similar to the corresponding request parameter.
Error code- code designation of the “success” of the request. Code 200 means "everything is normal" (OK).
Verbal description of the error- “deciphering” the previous code. For example, for 200 it is OK, for 500 it is Internal Server Error.
The most common http response parameters:
Connection- similar to the corresponding request parameter.
If the server does not support Keep-Alive (there are some), then the Connection value in the response is always close.
Therefore, in my opinion, the correct browser tactic is the following:
1. issue Connection: Keep-Alive in the request;
2. The connection status can be judged by the Connection field in the response.
Content-Type(“content type”) - contains a designation of the content type of the response.
Depending on the Content-Type value, the browser interprets the response as an HTML page, a gif or jpeg image, a file to be saved to disk, or something else and takes appropriate action. The Content-Type value for the browser is the same as the file extension value for Windows.
Some content types:
text/html - text in HTML format (web page);
text/plain - plain text (similar to Notepad);
image/jpeg - picture in JPEG format;
image/gif - the same, in GIF format;
application/octet-stream - a stream of "octets" (i.e. just bytes) to write to disk.
There are actually many more types of content.
Content-Length("content length") - the length of the response content in bytes.
Last-Modified(“Last modified”) - the date the document was last modified.
The HTTP protocol (HyperText Transfer Protocol) is an application-level protocol that communicates applications within distributed, collaborative, or heterogeneous information systems. The protocol allows applications to exchange data presented in a human-readable form.
As its name suggests, HTTP was originally intended to transfer so-called hypertext between applications, which is a special type of data created in accordance with the HTML standard (HyperText Markup Language). A hypertext document consists of data marked up using HTML tags and is a combination of text, images, hyperlinks and other means of presenting data. Hyperlinks, one of the most important components of an HTML document, are interactive areas that, when acted upon, produce data associated with the hyperlink. This allows a user working with hypertext information to navigate within a set of documents or even the entire Internet, obtaining information of interest using contextual hyperlinks.
The HTTP protocol is an add-on to the TCP protocol and is a means of controlling the content of transmitted data. Unlike TCP, which did not take into account the structure of transmitted packets, HTTP embeds metainformation into the data, allowing the recipient to correctly interpret the received data. The global Internet (also called the World Wide Web or WWW) operates on the basis of HTTP. The first version of the protocol - HTTP/0.9 - had rather limited capabilities, but with the active development of the World Wide Web, new versions appeared: HTTP/1.0 and HTTP/1.1, which make it possible to control the transmission of not only hypertext information over computer networks, but also arbitrary binary files: sound, graphic, archival, etc.
Due to the fact that HTTP is a superstructure over the TCP protocol, there are also two sides to data transfer: client and server.
The client initiates the connection and requests some data or services from the server. The client, as a rule, is a program called a browser, which allows both displaying hypertext information and accepting files of other formats. To obtain some information of interest, the client sends a request to the server containing a description of the requested information.
The server when transmitting data via HTTP is called a web server. This program processes requests from clients, transmitting the requested data in the form of responses, containing, in addition to the required data, meta-information that describes them.
Obtaining by the user the data he is interested in consists of the following steps:
The user enters the address of the resource he is interested in in the browser line.
The browser, based on information received from the user, as well as taking into account its settings and operating system configuration, generates a request.
The browser connects to a server, possibly located on a remote computer, and sends a request to it.
The server, analyzing the request, performs the necessary actions: generates a response, including the body of the requested document. If it is a hypertext document, it is loaded from a file or generated dynamically through a script. The response also includes information about the data it contains.
The server sends the response to the browser.
The browser parses the response and either saves the resulting data to a file or, in the case of a hypertext document, parses the HTML tags and displays the document on the screen.
It should be noted that the client program can be not only a browser, however, all steps, with the possible exception of the first, are performed in any case.
It should be noted that here we consider a direct connection of the client to the server containing the information of interest, however, this is not always possible due to various circumstances. In this case, the connection can be made through one or more intermediate connection points. These intermediate points can be divided into three groups:
Proxy servers (proxy-server) are an intermediary program that performs the functions of both a client and a server for the purpose of creating requests on behalf of other clients. Requests are served by the proxy server, or forwarded to it with changes made to them (in which case the proxy server is called opaque) or without changes (in which case the proxy server is called transparent). A proxy server allows a group of computers to act as one client, which is often used when connecting local networks to the Internet.
Gateway - like a proxy server, it broadcasts requests, however, I do not change them. The gateway receives a request from the client as if it were a server containing the requested resource. Thus, the client cannot determine whether it is connecting through the gateway or directly to the server containing the resource.
Tunnel is an intermediary program that maintains a connection. Although the tunnel is not considered part of the HTTP transport once the connection is established, the connection is typically initiated by an HTTP request. The tunnel terminates its operation if at least one of the participants in the data exchange closes the connection.
To maintain the functionality of data transfer when connecting through intermediate points, no changes are required to requests and responses, except in the case of a proxy server: in this case, the client request must contain additional fields. However, from the server's point of view, data is exchanged directly with the client, therefore, no changes in requests occur. Therefore, when developing the program, the possibility of connecting through intermediate points was not taken into account.
The request sent by the client to the server serves to accurately identify the requested resource, and also contains information necessary for correct processing of the request.
The structure of the request consists of three parts:
Query string
Header block
The request line consists of three fields separated by space characters (ASCII code 20h, hereinafter SP), and ends with a combination of two characters: carriage return (ASCII code 0Dh, hereinafter CR) and line feed (ASCII code 0Ah, hereinafter LF) . Query string elements are represented by the following fields:
Method - defines the processing method applied to the requested resource. Depending on the specified method, the request format may be different. Valid methods:
During the development of the program, support for only the GET method was introduced, due to the fact that it is this method that the browser specifies in the request created by default.
URI (Universal Resource Identifier) resource (resource URI) - indicates the location of the requested resource in a standardized format, that is, it is the address of the resource. When using the GET method, this string may include a set of parameters passed to the server in the form of strings of the format "parameter_name = parameter_value", separated by ampersand characters `&". The parameter block is located at the end of the URI string and is separated from the address by the question mark character `?".
HTTP protocol version - during the development of the program, support for receiving requests corresponding to versions 1.0 and 1.1 was implemented, which correspond to the lines “HTTP/1.0” and “HTTP/1.1”, respectively.
The header block following the query line may consist of one or more headers:
Request header - contains fields that serve as request modifiers and contain information about the request and the configuration of the client machine.
Object header - if the request includes some object (an arbitrary set of data), the fields of this header describe the object, indicating its format, encoding and other parameters.
General header - contains service parameters necessary to ensure the correctness of the transfer and enable additional services, such as caching.
The header section ends with two pairs of characters CR and LF, which makes it easy to determine that the request has been received since the request itself cannot contain such a combination of characters.
The response sent by the server to the client can only be generated by processing the client's request. It contains a description of the results of the query and, if data was requested, includes the resource requested.
The structure of the answer consists of the following parts:
Status bar
Header block
The status line consists of three fields separated by SP characters and contains the sequence of CR, LF characters at the end. Status bar elements:
HTTP protocol version - the developed program always uses the string “HTTP/1.1”.
Status code is a three-character numeric code that identifies the result of a request. Although the standard defines a fairly large set of status codes, the following codes are used in the program:
- 200 - successful execution;
- 400 - invalid request;
- 401 - unauthorized access;
- 404 - resource not found;
- 405 - not applicable method;
- 505 - unsupported HTTP version.
Reason phrase - a short phrase that explains the request execution status code. The standard offers a standard set of phrases, but in the program this set has been slightly modified.
The header block following the status line may consist of one or more headers:
Request header
Object title
General title
Headings were discussed in detail in section 2.2.3.3.
The header section ends with two pairs of characters CR and LF, followed by an arbitrary set of characters - an object. When the program is running, such objects can only be hypertext documents in HTML format, dynamically generated by plug-ins.
We present to you a description of the main aspects of the HTTP protocol - a network protocol that, from the early 90s to this day, allows your browser to load web pages. This article was written for those who are just starting to work with computer networks and develop network applications, and who still find it difficult to read the official specifications on their own.
HTTP- a widely used data transfer protocol, originally intended for the transfer of hypertext documents (that is, documents that may contain links that allow navigation to other documents).
The abbreviation HTTP stands for HyperText Transfer Protocol, "hypertext transfer protocol". According to the OSI specification, HTTP is an application (upper, 7th) layer protocol. The current version of the protocol, HTTP 1.1, is described in the RFC 2616 specification.
The HTTP protocol involves the use of a client-server data transfer structure. The client application generates a request and sends it to the server, after which the server software processes the request, generates a response and sends it back to the client. The client application can then continue to send other requests, which will be processed in the same way.
A task that is traditionally solved using the HTTP protocol is the exchange of data between a user application that accesses web resources (usually a web browser) and a web server. At the moment, it is thanks to the HTTP protocol that the World Wide Web operates.
HTTP is also often used as a transport protocol for other application layer protocols such as SOAP, XML-RPC and WebDAV. In this case, the HTTP protocol is said to be used as a “transport”.
The API of many software products also implies the use of HTTP for data transfer - the data itself can be in any format, for example, XML or JSON.
Typically, HTTP data transfer is carried out over TCP/IP connections. In this case, server software usually uses TCP port 80 (and, if the port is not specified explicitly, then client software usually uses port 80 by default for opening HTTP connections), although it can use any other one.
How to send an HTTP request?
The easiest way to understand the HTTP protocol is to try to access some web resource manually. Imagine that you are a browser and you have a user who really wants to read articles by Anatoly Alizar.Let's say he entered the following in the address bar:
Http://alizar.site/
Accordingly, you, as a web browser, now need to connect to the web server at alizar.site.
To do this, you can use any suitable command line utility. For example, telnet:
Telnet alizar.site 80
Let me clarify right away that if you suddenly change your mind, press Ctrl + “]” and then enter - this will allow you to close the HTTP connection. In addition to telnet, you can try nc (or ncat) - depending on your taste.
After you connect to the server, you need to send an HTTP request. This, by the way, is very easy - HTTP requests can consist of just two lines.
In order to generate an HTTP request, you need to compose a starting line, and also set at least one header - this is the Host header, which is mandatory and must be present in every request. The fact is that the conversion of a domain name to an IP address is carried out on the client side, and, accordingly, when you open a TCP connection, the remote server does not have any information about which address was used for the connection: it could be, for example , address alizar..ru or m.. However, in fact, the network connection in all cases opens with node 212.24.43.44, and even if initially when opening the connection it was not this IP address, but some domain name that was specified, then the server reports this is not informed in any way - and that is why this address must be passed in the Host header.
The starting (initial) request line for HTTP 1.1 is composed according to the following scheme:
For example (such a starting line may indicate that the main page of the site is being requested):
And, of course, don’t forget that any technology becomes much simpler and clearer when you actually start using it.
Good luck and fruitful learning!
Tags:
- http
- alizar
- spdy