Remote procedures: calling remote procedures, definition and features. Remote procedure call What is remote procedure call rpc

The purpose of this article is to discuss terminology. The article is not about how and why, but only about the use of terminology. The article reflects the opinion of the author and does not pretend to be scientific.

Introduction

If you work in programming distributed systems or in systems integration, then most of what is presented here is not new to you.

The problem comes when people who use different technologies meet, and when those people start having technical conversations. In this case, mutual misunderstandings often arise due to terminology. Here I will try to bring together the terminologies used in different contexts.

Terminology

There is no clear terminology and classification in this area. The terminology used below is a reflection of the author’s model, that is, it is strictly subjective. Any criticism and any discussions are welcome.

I've divided the terminology into three areas: RPC (Remote Procedure Call), Messaging and REST. These areas have historical roots.

RPC

RPC technologies - the oldest technologies. The most prominent representatives of RPC are - CORBA And DCOM.

In those days, it was mainly necessary to connect systems over fast and relatively reliable local networks. The main idea behind RPC was to make calling remote systems much like calling functions within a program. The entire mechanics of remote calls were hidden from the programmer. At least they tried to hide it. Programmers in many cases were forced to work at a deeper level, where the terms marshaling appeared ( marshalling) And unmarshalling(how is that in Russian?), which essentially meant serialization. Normal function calls within processes were handled at the caller's end in Proxy, and on the side of the system performing the function, in Dispatcher. Ideally, neither the calling system nor the processing system would deal with the intricacies of transferring data between systems. All these subtleties were concentrated in the Proxy - Dispatcher bundle, the code of which was generated automatically.

So you won't notice, you shouldn't notice, any difference between a local function call and a remote function call.
Now there is a kind of RPC renaissance, the most prominent representatives of which are: Google ProtoBuf, Thrift, Avro.

Messaging

Over time, it turned out that the attempt to protect the programmer from the fact that the called function still differs from the local one did not lead to the desired result. The implementation details and fundamental differences between distributed systems were too great to be resolved using automatically generated Proxy code. Gradually, the understanding came that the fact that the systems are connected by an unreliable, slow, low-speed environment must be explicitly reflected in the program code.

Technologies have appeared web services. We started talking ABC: Address, Binding, Contract. It is not entirely clear why contracts appeared, which are essentially Envelopes for input arguments. Contracts often complicate the overall model rather than simplify it. But... it doesn't matter.

Now the programmer explicitly created service(Service) or client(Client) calling the service. The service consisted of a set operations (Operation), each of which took at the input request(Request) and issued answer(Response). Client explicitly sent(Sent) request, the service explicitly received ( Receive) and answered him (Sent), sending the answer. The client received a response and the call ended.

Just like in RPC, there was a Proxy and Dispatcher running somewhere. And as before, their code was generated automatically and the programmer did not need to understand it. Unless the client explicitly used classes from Proxy.

Requests and responses are explicitly converted to a format intended for transmission over the wire. Most often this is a byte array. The transformation is called Serialization And Deserialization and sometimes hides in the Proxy code.
The culmination of messaging manifested itself in the emergence of the paradigm ESB (Enterprise Service Bus). No one can really articulate what it is, but everyone agrees that data moves through the ESB in the form of messages.

REST

In the constant struggle with code complexity, programmers took the next step and created REST.

The main principle of REST is that function operations are sharply limited and left only a set of operations CRUD: Create - Read - Update - Delete. In this model, all operations are always applied to some data. The operations available in CRUD are sufficient for most applications. Since REST technologies in most cases imply the use of the HTTP protocol, the CRUD commands were reflected in the commands HTTP (Post - Get - Put - Delete) . It is constantly stated that REST is not necessarily tied to HTTP. But in practice, reflection of operation signatures onto the syntax of HTTP commands is widely used. For example, calling the function

EntityAddress ReadEntityAddress(string param1, string param2)

Expressed like this:

GET: entityAddress?param1=value1¶m2=value2

Conclusion

Before starting a discussion on distributed systems or integration, define the terminology. If Proxy will always mean the same thing in different contexts, then request, for example, will mean little in RPC terms, and marshalling will cause confusion when discussing REST technologies.

Programs communicating over a network need a communication mechanism. At the lower level, upon arrival of packets, a signal is sent and processed by a network signal processing program. At the top level, the rendezvous mechanism, adopted in the Ada language, operates. NFS uses a remote procedure call (RPC) mechanism in which the client communicates with the server (see Figure 1). According to this process, the client first calls a procedure that sends a request to the server. Upon arrival of a request packet, the server calls the packet opening procedure, performs the requested service, sends a response, and control returns to the client.

The RPC interface can be thought of as consisting of three layers:

1. The top level is completely transparent. A program at this level may, for example, contain a call to the rnusers() procedure, which returns the number of users on the remote machine. You don't need to know about using the RPC mechanism since you are making the call in the program.
2. The middle tier is intended for the most common applications. RPC calls at this level are handled by the registerrpc() and callrpc() subroutines: registerrpc() receives system-wide code, and callrpc() executes a remote procedure call. The rnusers() call is implemented using these two routines.
3. The lower level is used for more complex tasks that change the default values ​​of procedure parameters. At this level, you can explicitly manipulate the sockets used to transmit RPC messages.

As a general rule, you should use the top level and avoid using the lower levels unless absolutely necessary.

Although in this tutorial we only consider the C interface, calls to remote procedures can be made from any language. The operation of the RPC mechanism for organizing communication between processes on different machines is no different from its operation on one machine.

RPC (Remote Procedure Call) is an interface between remote users and certain host programs that are launched at the request of these users. The RPC service of any host, as a rule, provides clients with a set of programs. Each of these programs consists, in turn, of one or more remote procedures. For example, an NFS remote file system service that is built on RPC calls may consist of only two programs: for example, one program interacts with high-level user interfaces and the other with low-level I/O functions.

Each remote procedure call involves two parties: the active client, which sends the procedure call request to the server, and the server, which sends the response to the client.

Note. It should be kept in mind that the terms "client" and "server" in this case refer to a specific transaction. A particular host or software (process or program) can act as both a client and a server. For example, a program that provides a remote procedure service can at the same time be a client for working with a network file system.

The RPC protocol is built on a remote procedure call model, similar to the local procedure call mechanism. When you call a local procedure, you place arguments in a specific location in memory, on the stack, or in environment variables, and transfer control of the process to a specific address. Once the job is complete, you read the results at a specific address and continue with your process.

When working with a remote procedure, the main difference is that the remote function call is handled by two processes: a client process and a server process.

The client process sends a message to the server, which includes the parameters of the called procedure, and waits for a response message with the results of its work. When a response is received, the result is read and the process continues. On the server side, the call handler process is in a waiting state, and when a message arrives, it reads the parameters of the procedure, executes it, sends a response and becomes in a waiting state for the next call.

The RPC protocol does not impose any requirements on additional communications between processes and does not require synchronization of the functions performed, that is, calls can be asynchronous and mutually independent, so that the client can perform other procedures while waiting for a response. The RPC server can allocate a separate process or virtual machine for each function, therefore, without waiting for previous requests to complete, it can immediately accept the next ones.

However, there are several important differences between local and remote procedure calls:

1. Error handling. In any case, the client should receive notification of errors that occur when calling remote procedures on the server or network.
2. Global variables. Because the server does not have access to the client's address space, remote procedure calls cannot use hidden parameters in the form of global variables.
3. Performance. The speed of executing remote procedures is usually one or two orders of magnitude lower than the speed of executing similar local procedures.
4. Authentication. Because remote procedure calls occur over the network, client authentication mechanisms must be used.

Principles of protocol construction.

The RPC protocol can use several different transport protocols. The responsibilities of the RPC protocol are only to provide standards and interpret the transmission of messages. The reliability and reliability of message transmission is entirely ensured by the transport layer.

However, RPC can control the selection and some functions of the transport protocol. As an example of the interaction between RPC and the transport protocol, consider the procedure for assigning an RPC port for the operation of an application process through RPC - Portmapper.

This function dynamically (on demand) assigns an RPC connection to a specific port. The Portmapper function is used quite often because the set of transport ports reserved for RPC is limited, and the number of processes that can potentially run simultaneously is very high. Portmapper, for example, is called when selecting ports for interaction between the client and the NFS system server.

The Portmapper service uses a mechanism for broadcasting RPC messages to a specific port - III. The client sends a port request broadcast message for a specific RPC service to this port. The Portmapper service processes the tax message, determines the address of the local RPC service and sends a response to the client. The RPC Portmapper service can work with both TCP and UDP protocols.

RPC can work with various transport protocols, but never duplicates their functions, i.e. if RPC runs on top of TCP, all concerns about the reliability and validity of the RPC connection are assigned to TCP. However, if RPC is installed on top of UDP, it can provide additional native message delivery functionality.

Note.

Application tasks can consider the RPC protocol as a specific procedure for calling a function over the JSR (Jump Subroutine Instruction) network.

For the RPC protocol to work, the following conditions must be met:

1. Unique identification of all remotely called procedures on a given host. RPC requests contain three identifier fields - the number of the remote program (service), the version number of the remote program and the number of the remote procedure of the specified program. The program number is assigned by the service manufacturer, the procedure number indicates the specific function of this service
2. Identification of the RPC protocol version. RPC messages contain an RPC protocol version field. It is used to coordinate the formats of passed parameters when the client is working with different versions of RPC.
3. Providing client authentication mechanisms to the server. The RPC protocol provides a procedure for authenticating the client to the service, and, if necessary, each time a request is made or a response is sent to the client. In addition, RPC allows the use of various additional security mechanisms.

RPC can use four types of authentication mechanisms:

• AUTH_NULL - no authentication required
• AUTH_UNIX - authentication according to the UNIX standard
• AUTH_SHORT - UNIX standard authentication with its own encoding structure
• AUTH_DES - authentication according to the DES standard

1. Identification of response messages to corresponding requests. RPC response messages contain the ID of the request from which they were constructed. This identifier can be called the RPC call transaction identifier. This mechanism is especially necessary when working in asynchronous mode and when performing a sequence of several RPC calls.
2. Identification of protocol errors. All network or server errors have unique identifiers, by which each of the connection participants can determine the cause of the failure.

Protocol message structures

When sending RPC messages over a transport protocol, multiple RPC messages can be located within a single transport packet. In order to separate one message from another, a record marker (RM - Record Marker) is used. Each RPC message is "marked" with exactly one RM.

An RPC message can consist of several fragments. Each fragment consists of four bytes of header and (0 to 2**31-1) data. The first bit of the header indicates whether the fragment is the last one, and the remaining 31 bits indicate the length of the data packet.

The RPC structure is formally described in the language for describing and representing data formats - XDR with additions regarding the description of procedures. You could even say that the RPC description language is an extension of XDR, complemented by work with procedures.

The structure of the RPC packet looks like this:

The response structure (reply_body) can contain either an error structure (in which case it contains the error code) or a successful request processing structure (in which case it contains the return data).

High level software interface.

Using subroutines in a program is a traditional way to structure a task and make it clearer. The most frequently used routines are collected in libraries where they can be used by various programs. In this case, we are talking about a local (local) call, i.e. both the calling and the called objects work within the same program on the same computer.

In the case of a remote call, a process running on one computer starts a process on the remote computer (that is, it actually runs procedure code on the remote computer). Obviously, a remote procedure call differs significantly from a traditional local one, but from a programmer’s point of view, such differences are practically absent, i.e., the architecture of a remote procedure call allows you to simulate a local procedure call.

However, if in the case of a local call the program passes parameters to the called procedure and receives the result of the work through the stack or shared memory areas, then in the case of a remote call the transfer of parameters turns into the transmission of a request over the network, and the result of the work is in the received response.

This approach is a possible basis for creating distributed applications, and although many modern systems do not use this mechanism, the basic concepts and terms are retained in many cases. When describing the RPC mechanism, we will traditionally call the calling process a client, and the remote process that implements the procedure a server.

A remote procedure call involves the following steps:

1. The client program makes a local call to a procedure called a stub. In this case, the client “seems” that by calling the stub, he is actually calling the server procedure. Indeed, the client passes the necessary parameters to the stub, and it returns the result. However, things are not quite as the client imagines. The stub's job is to accept the arguments intended for the remote procedure, possibly convert them into some standard format, and formulate a network request. Packaging the arguments and creating the network request is called marshalling.
2. The network request is forwarded across the network to the remote system. To do this, the stub uses appropriate calls, for example, those discussed in the previous sections. Note that various transport protocols can be used, and not only the TCP/IP family.
3. On the remote host, everything happens in the reverse order. The server stub listens for a request and, upon receipt, retrieves the parameters - the arguments to the procedure call. Unmarshalling may involve necessary transformations (for example, byte order changes).
4. The stub makes a call to the real server procedure to which the client's request is addressed, passing it the arguments received over the network.
5. After the procedure is completed, control returns to the server stub, passing it the required parameters. Like the client stub; The server stub converts the values ​​returned by the procedure, generating a network response message that is transmitted over the network to the system from which the request came.
6. The operating system passes the received message to the client stub, which, after the necessary transformation, passes the values ​​(which are the values ​​returned by the remote procedure) to the client, which treats this as a normal return from the procedure.

Thus, from the client's point of view, it makes a call to a remote procedure as it would for a local one. The same can be said about the server: a procedure is called in a standard way, a certain object (server stub) calls a local procedure and receives the values ​​​​returned by it. The client treats the stub as a callable server procedure, and the server treats its own stub as a client.

Thus, stubs form the core of the RPC system, responsible for all aspects of message generation and transmission between the client and the remote server (procedure), although both client and server believe that the calls are occurring locally. This is the basic concept of RPC - to completely hide the distributed (network) nature of the interaction in the stub code. The advantages of this approach are obvious: both the client and the server are independent of the network implementation, they both operate within a distributed virtual machine, and procedure calls have a standard interface.

Passing parameters

Passing value parameters does not cause any particular difficulties. In this case, the client stub places the parameter value in the network request, possibly performing conversions to the standard form (for example, reversing the byte order). The situation is much more complicated with the passing of pointers, when the parameter represents the address of the data, and not its value. Passing the address in the request is meaningless, since the remote procedure is executed in a completely different address space. The simplest solution used in RPC is to prohibit clients from passing parameters other than by value, although this, of course, imposes serious restrictions.

Binding

Before a client can call a remote procedure, it must be associated with a remote system that hosts the required server. Thus, the binding task breaks down into two:

1. Finding a remote host with the required server
2. Finding the required server process on a given host

Various approaches can be used to find the host. A possible option is to create some kind of centralized directory in which hosts advertise their servers, and where the client, if desired, can select the host and procedure address that are suitable for him.

Each RPC procedure is uniquely identified by a program and procedure number. The program number identifies a group of remote procedures, each of which has its own number. Each program is also assigned a version number, so that if you make minor changes to the program (for example, adding a procedure), there is no need to change its number. Typically, several functionally similar procedures are implemented in one software module, which, when launched, becomes the server of these procedures, and which is identified by the program number.

Thus, when a client wants to call a remote procedure, it needs to know the program, version, and procedure numbers that provide the required service.

To transmit a request, the client also needs to know the host network address and port number associated with the server program providing the required procedures. The portmap(IM) daemon is used for this (called rpcbind(IM) on some systems). The daemon runs on the host that provides the remote procedure service and uses a well-known port number. When a server process is initialized, it registers its procedures and port numbers in portmap(IM). Now, when a client needs to know the port number to call a specific procedure, it sends a request to the portmap(IM) server, which in turn either returns the port number or forwards the request directly to the remote procedure server and, after executing it, returns a response to the client. In any case, if the required procedure exists, the client receives the procedure port number from the portmap(IM) server, and further requests can be made directly to this port.

Handling special situations (exception)

Handling exceptions when calling local procedures is not particularly problematic. UNIX provides processing for process errors such as division by zero, access to an invalid memory area, etc. In the case of a remote procedure call, the likelihood of error situations increases. In addition to server and stub errors, errors associated, for example, with receiving an erroneous network message are added.

For example, when using UDP as a transport protocol, messages are retransmitted after a certain timeout. An error is returned to the client if, after a certain number of attempts, a response from the server has not been received. In the case where the TCP protocol is used, an error is returned to the client if the server has closed the TCP connection.

Call semantics

Calling a local procedure unambiguously leads to its execution, after which control returns to the main program. The situation is different when calling a remote procedure. It is impossible to determine when exactly the procedure will be performed, whether it will be performed at all, and if so, how many times? For example, if the request is received by the remote system after the server program has crashed, the procedure will not be executed at all. If the client, when not receiving a response after a certain period of time (timeout), resends the request, then a situation may arise when the response is already transmitted over the network, and the repeated request is again accepted for processing by the remote procedure. In this case, the procedure will be performed several times.

Thus, the execution of a remote procedure can be characterized by the following semantics:

• Once and only once. This behavior (in some cases the most desirable) is difficult to require due to possible server crashes.
• Maximum times. This means that the procedure was either not performed at all or was performed only once. A similar statement can be made when receiving an error instead of a normal response.
• At least once. The procedure was probably performed once, but it is possible more. To work normally in such a situation, the remote procedure must have the idempotency property (from the English idemponent). This property is possessed by a procedure whose repeated execution does not cause cumulative changes. For example, reading a file is idempotent, but adding text to a file is not.

Data presentation

When the client and server are running on the same system on the same computer, there are no data incompatibility problems. For both the client and the server, data in binary form is represented in the same way. In the case of a remote call, the matter is complicated by the fact that the client and server may be running on systems with different architectures, having different data representations (for example, floating point representation, byte order, etc.)

Most RPC system implementations define some standard data representation to which all values ​​passed in requests and responses must be converted.

For example, the format for presenting data in RPC from Sun Microsystems is as follows:

1. Byte order - Most significant - last
2. Floating Point Representation - IEEE
3. Character representation - ASCII

Net

In terms of functionality, the RPC system occupies an intermediate place between the application layer and the transport layer. According to the OSI model, this position corresponds to the presentation and session layers. Thus, RPC is theoretically independent of the network implementation, in particular, of the transport layer network protocols.

Software implementations of the system, as a rule, support one or two protocols. For example, the RPC system developed by Sun Microsystems supports message transmission using the TCP and UDP protocols. The choice of one protocol or another depends on the application requirements. The choice of the UDP protocol is justified for applications that have the following characteristics:

• Called procedures are idempotent
• The size of the transmitted arguments and the returned result is less than the size of the UDP packet - 8 KB.
• The server provides work with several hundred clients. Since when working with TCP protocols, the server is forced to maintain a connection with each of the active clients, this takes up a significant part of its resources. The UDP protocol is less resource-intensive in this regard

On the other hand, TCP provides efficient operation of applications with the following characteristics:

• The application requires a reliable transfer protocol
• Called procedures are non-identical
• The size of the arguments or return result exceeds 8 KB

The choice of protocol is usually left to the client, and the system organizes the generation and transmission of messages in different ways. Thus, when using the TCP protocol, for which the transmitted data is a stream of bytes, it is necessary to separate messages from each other. For this, for example, the record marking protocol described in RFC1057 "RPC: Remote Procedure Call Protocol specification version 2" is used, in which a 32-bit integer is placed at the beginning of each message, defining the size of the message in bytes.

The situation is different with the semantics of the call. For example, if RPC is performed using an unreliable transport protocol (UDP), the system retransmits the message at short intervals (timeouts). If the client application does not receive a response, then it is safe to say that the procedure was executed zero or more times. If a response is received, the application can conclude that the procedure was executed at least once. When using a reliable transport protocol (TCP), if a response is received, the procedure can be said to have been performed once. If a response is not received, it is impossible to definitely say that the procedure was not completed3.

How does this work?

Essentially, the RPC system itself is built into the client program and the server program. It's nice that when developing distributed applications, you don't have to delve into the details of the RPC protocol or program message processing. The system assumes the existence of an appropriate development environment, which greatly simplifies the life of application software creators. One of the key points in RPC is that the development of a distributed application begins with the definition of an object interface - a formal description of the server's functions, written in a special language. Based on this interface, client and server stubs are then automatically generated. The only thing you need to do after this is write the actual code for the procedure.

As an example, consider RPC from Sun Microsystems. The system consists of three main parts:

• rpcgen(1) is an RPC compiler that, based on the description of the remote procedure interface, generates client and server stubs in the form of C programs.
• The XDR (eXternal Data Representation) library, which contains functions for converting various data types into a machine-independent form that allows information exchange between heterogeneous systems.
• A library of modules that ensure the operation of the system as a whole.

Let's look at an example of a simple distributed event logging application. When the client starts, it calls a remote procedure to write a message to the log file of the remote computer.

To do this, you will have to create at least three files: the specification of the interfaces of the remote procedures log.x (in the interface description language), the actual text of the remote procedures log.c and the text of the main client program main () - client.c (in C language).

The rpcgen(l) compiler creates three files based on the log.x specification: the text of the client and server stubs in C (log clnt.c and log svc.c) and the description file log.h, used by both stubs.

So, let's look at the source codes of the programs.

This file specifies the registration parameters of the remote procedure - program, version and procedure numbers, and also defines the call interface - input arguments and return values. Thus, an RLOG procedure is defined that takes a string as an argument (which will be written to the log), and the return value standardly indicates the success or failure of the ordered operation.

The rpcgen(l) compiler creates a header file log.h, where, in particular, the procedures are defined:

Let's look at this file carefully. The compiler translates the RLOG name defined in the interface description file into rlog_1, replacing uppercase characters with lowercase ones and adding the program version number with an underscore. The return type has changed from int to int*. This is the rule - RPC allows you to transmit and receive only the addresses of the parameters declared when describing the interface. The same rule applies to the string passed as an argument. Although it doesn't appear from print.h, the rlog_l() function actually passes the address of the string as an argument.

In addition to the header file, the rpcgen(l) compiler produces client stub and server stub modules. Essentially, the text of these files contains all the code for the remote call.

The server stub is the main program that handles all network interactions with the client (more precisely, with its stub). To perform the operation, the server stub makes a local function call, the text of which must be written:

The client stub accepts the argument passed to the remote procedure, does the necessary conversions, issues a request to the portmap(1M) server, communicates with the remote procedure server, and finally passes the return value to the client. For the client, a call to a remote procedure is reduced to a call to a stub and is no different from a regular local call.

client.c

The client stub log_clnt.c is compiled with the client.c module to produce an executable client program.

Now on some host server.nowhere.ru you need to start a server process:

Then, when you run the rlog client on another machine, the server will add a corresponding entry to the log file.

The RPC operation diagram in this case is shown in Fig. 1. Modules interact as follows:

1. When the server process starts, it creates a UDP socket and binds any local port to that socket. Next, the server calls the library function svc_register(3N) to register program numbers and versions. To do this, the function calls the portmap(IM) process and passes the required values. The portmap(IM) server is usually started when the system is initialized and binds to some well-known port. Now portmap(3N) knows the port number for our program and version. The server is waiting to receive the request. Note that all the described actions are performed by a server stub created by the rpcgen(IM) compiler.
2. When rlog runs, the first thing it does is call the library function clnt_create(3N), giving it the remote system address, program and version numbers, and transport protocol. The function sends a request to the portmap(IM) server of the remote system server.nowhere.m and obtains the remote port number for the log server.
3. The client calls the rlog_1() procedure defined in the client stub and transfers control to the stub. That, in turn, generates a request (converting the arguments to XDR format) in the form of a UDP packet and sends it to the remote port received from the portmap (IM) server. It then waits for a response for some time and, if not received, resends the request. Under favorable circumstances, the request is accepted by the logger server (server stub module). The stub determines which function was called (by procedure number) and calls the rlog_1() function of the log.c module. After control returns to the stub, the latter converts the value returned by the rlog_1() function into XDR format, and generates a response also in the form of a UDP packet. Upon receiving the response, the client stub extracts the returned value, transforms it, and returns it to the client's main program.

The idea of ​​a Remote Procedure Call (RPC) is to extend the well-known and understood mechanism for transferring control and data within a program running on a single machine to transfer control and data over a network. Remote procedure call tools are designed to facilitate the organization of distributed computing. The greatest efficiency of using RPC is achieved in those applications in which there is interactive communication between remote components with fast response times and a relatively small amount of data transferred. Such applications are called RPC-oriented.

The characteristic features of calling local procedures are:

Asymmetry, that is, one of the interacting parties is the initiator; Synchronicity, that is, execution of the calling procedure stops from the moment the request is issued and is resumed only after the called procedure returns.

Implementing remote calls is much more complicated than implementing local procedure calls. To begin with, since the calling and called procedures are executed on different machines, they have different address spaces, and this creates problems when passing parameters and results, especially if the machines are not identical. Since RPC cannot rely on shared memory, this means that RPC parameters must not contain pointers to non-stack memory locations and that parameter values ​​must be copied from one computer to another. The next difference between RPC and a local call is that it necessarily uses the underlying communication system, but this should not be explicitly visible either in the definition of the procedures or in the procedures themselves. Remoteness introduces additional problems. The execution of the calling program and the called local procedure on the same machine is implemented within a single process. But the implementation of RPC involves at least two processes - one on each machine. If one of them crashes, the following situations may arise: if the calling procedure crashes, the remotely called procedures will become “orphaned”, and if the remote procedures crash, the calling procedures will become “orphaned parents”, waiting in vain for a response from the remote procedures.

In addition, there are a number of problems associated with the heterogeneity of programming languages ​​and operating environments: the data structures and procedure call structures supported in any one programming language are not supported in the same way in all other languages.

These and some other problems are solved by the widespread RPC technology, which underlies many distributed operating systems.

Basic RPC Operations

To understand how RPC works, let's first consider making a local procedure call on a typical machine running offline. Let this be, for example, a system call

Count=read(fd,buf,nbytes);

where fd is an integer,
buf - array of characters,
nbytes is an integer.

To make the call, the calling procedure pushes the parameters onto the stack in reverse order (Figure 3.1). After the read call is executed, it places the return value into a register, moves the return address, and returns control to the calling procedure, which pops parameters from the stack, returning it to its original state. Note that in the C language, parameters can be called either by reference (by name) or by value (by value). In relation to the called procedure, value parameters are initialized local variables. The called procedure can change them without affecting the original values ​​of these variables in the calling procedure.

If a pointer to a variable is passed to the called procedure, then changing the value of this variable by the called procedure entails changing the value of this variable for the calling procedure. This fact is very significant for RPC.

There is also another mechanism for passing parameters that is not used in C. It is called call-by-copy/restore, which requires the caller to copy variables onto the stack as values, and then copy them back after the call is made over the original values ​​of the calling procedure.

The decision about which parameter passing mechanism to use is made by the language developers. Sometimes it depends on the type of data being transferred. In C, for example, integers and other scalar data are always passed by value, and arrays are always passed by reference.

Rice. 3.1. a) The stack before the read call is executed;
b) Stack during procedure execution;
c) Stack after returning to the calling program

The idea behind RPC is to make a remote procedure call look as similar as possible to a local procedure call. In other words, make RPC transparent: the calling procedure does not need to know that the called procedure is on another machine, and vice versa.

RPC achieves transparency in the following way. When the called procedure is actually remote, another version of the procedure, called a client stub, is placed in the library instead of the local procedure. Like the original procedure, the stub is called using a calling sequence (as in Figure 3.1), and an interrupt occurs when accessing the kernel. Only, unlike the original procedure, it does not place parameters in registers and does not request data from the kernel; instead, it generates a message to be sent to the kernel of the remote machine.

RPC Execution Stages

The interaction of software components when performing a remote procedure call is illustrated in Figure 3.2. After the client stub has been called by the client program, its first task is to fill the buffer with the message being sent. In some systems, the client stub has a single fixed-length buffer that is filled from the very beginning with each new request. In other systems, the message buffer is a pool of buffers for individual message fields, some of which are already full. This method is especially suitable for cases where the packet has a format consisting of a large number of fields, but the values ​​of many of these fields do not change from call to call.

The parameters must then be converted to the appropriate format and inserted into the message buffer. At this point, the message is ready to be sent, so the kernel call interrupt is executed.

Rice. 3.2. Remote Procedure Call

When the kernel gains control, it switches contexts, saves processor registers and memory map (page handles), and installs a new memory map that will be used to run in kernel mode. Because the kernel and user contexts are different, the kernel must copy the message exactly into its own address space so that it can access it, remember the destination address (and possibly other header fields), and it must pass it to the network interface. This completes the work on the client side. The transmission timer is turned on, and the kernel can either cyclically poll for a response or pass control to the scheduler, which will select some other process to run. In the first case, query execution is accelerated, but multiprogramming is absent.

On the server side, incoming bits are placed by the receiving hardware either in an on-chip buffer or in RAM. When all information has been received, an interrupt is generated. The interrupt handler checks the correctness of the packet data and determines which stub it should be sent to. If none of the stubs are expecting this packet, the handler must either buffer it or discard it altogether. If there is a waiting stub, the message is copied to it. Finally, a context switch is performed, as a result of which the registers and memory map are restored, taking the values ​​that they had at the moment when the stub made the receive call.

Now the server stub starts working. It unpacks the parameters and pushes them appropriately onto the stack. When everything is ready, a call to the server is made. After executing the procedure, the server transmits the results to the client. To do this, perform all the steps described above, only in reverse order.

Figure 3.3 shows the sequence of commands that must be executed for each RPC call, and Figure 3.4 shows what percentage of the total RPC execution time is spent on each of the 14 steps described. The tests were conducted on a DEC Firefly multi-processor workstation, and while the presence of five processors necessarily affected the results of the measurements, the histogram shown in the figure gives a general idea of ​​​​the RPC execution process.

Rice. 3.3. Steps to perform an RPC procedure

Rice. 3.4. Time distribution between 14 stages of RPC execution

1. Calling a stub

2. Prepare a buffer

3. Pack parameters

4. Fill in the title field

5. Calculate the checksum in the message

6. Interrupt to the kernel

7. Package queue for execution

8. Transmitting a message to the controller via the QBUS bus

9. Ethernet transmission time

10. Receive packet from controller

11. Interrupt handling procedure

12. Checksum calculation

13. Context switching to user space

14. Performing a server stub

Dynamic Linking

Let's consider how the client specifies the location of the server. One method to solve this problem is to directly use the server's network address in the client program. The disadvantage of this approach is its extreme inflexibility: when moving a server, or increasing the number of servers, or changing the interface in all these and many other cases, it is necessary to recompile all programs that used a hard-coded server address. To avoid all these problems, some distributed systems use what is called dynamic linking.

The starting point for dynamic binding is the formal definition (specification) of the server. The specification contains the file server name, version number and a list of service procedures provided by this server to clients (Figure 3.5). For each procedure, a description of its parameters is given, indicating whether this parameter is input or output relative to the server. Some parameters can be both input and output - for example, some array that is sent by the client to the server, modified there, and then returned back to the client (copy/restore operation).

Rice. 3.5. RPC Server Specification

The formal server specification is used as input to the stub generator program, which creates both client and server stubs. They are then placed in the appropriate libraries. When a user (client) program calls any procedure defined in the server specification, the corresponding stub procedure is associated with the program binary code. Likewise, when a server is compiled, server stubs are associated with it.

When a server starts up, the very first thing it does is pass its server interface to a special program called a binder. This process, known as the server registration process, involves the server passing its name, version number, unique identifier, and a handle to the location of the server. The handle is system independent and can be an IP, Ethernet, X.500 or some other address. In addition, it may contain other information, for example, related to authentication.

When a client calls one of the remote procedures for the first time, for example, read, the client stub sees that it is not yet connected to the server and sends a message to the binder program with a request to import the interface of the desired version of the desired server. If such a server exists, then binder passes the descriptor and unique identifier to the client stub.

When sending a message with a request, the client stub uses a descriptor as an address. The message contains parameters and a unique identifier that the server core uses to route the incoming message to the desired server if there are several of them on this machine.

This method of importing/exporting interfaces is highly flexible. For example, there may be multiple servers supporting the same interface, and clients are randomly distributed across the servers. Within the framework of this method, it becomes possible to periodically poll servers, analyze their performance and, in case of failure, automatically shut down, which increases the overall fault tolerance of the system. This method can also support client authentication. For example, the server may determine that it can only be used by clients from a specific list.

However, dynamic binding has disadvantages, such as additional overhead (time) for exporting and importing interfaces. The magnitude of these costs can be significant, since many client processes exist for a short time, and each time the process starts, the interface import procedure must be performed again. In addition, in large distributed systems, the binder program can become a bottleneck, and creating several programs with a similar purpose also increases the overhead of creating and synchronizing processes.

RPC semantics in case of failures

Ideally, RPC should function correctly even in the event of failures. Consider the following failure classes:

The client cannot locate the server, for example, if the desired server fails, or because the client program was compiled a long time ago and used an old version of the server interface. In this case, in response to the client's request, a message containing an error code is received. The request from the client to the server was lost. The simplest solution is to repeat the request after a certain time. The response message from the server to the client was lost. This option is more complicated than the previous one, since some procedures are not idempotent. An idempotent procedure is a procedure whose execution request can be repeated several times without changing the result. An example of such a procedure would be reading a file. But the procedure for withdrawing a certain amount from a bank account is not idempotent, and if the response is lost, a repeated request can significantly change the state of the client’s account. One possible solution is to make all procedures idempotent. However, in practice this is not always possible, so another method can be used - sequential numbering of all requests by the client kernel. The server core remembers the number of the most recent request from each client, and upon receiving each request, it analyzes whether this request is a primary or a repeated one. The server crashed after receiving the request. The property of idempotency is also important here, but unfortunately the approach with query numbering cannot be applied. In this case, it matters when the failure occurred - before or after the operation. But the client kernel cannot recognize these situations; it only knows that the response time has expired. There are three approaches to this problem: Wait until the server reboots and try the operation again. This approach ensures that the RPC completes at least once, and possibly more. Immediately report the error to the application. This approach ensures that the RPC was executed at most once. The third approach does not guarantee anything. When the server fails, no support is provided to the client. The RPC may either not be executed at all, or it may be executed many times. In any case, this method is very easy to implement.

Neither of these approaches is very attractive. And the ideal option, which would guarantee exactly one RPC execution, in the general case cannot be implemented for fundamental reasons. Let, for example, a remote operation be printing some text, which includes loading the printer buffer and setting one bit in some printer control register, as a result of which the printer starts. A server crash can occur either a microsecond before or a microsecond after the control bit is set. The moment of failure entirely determines the recovery procedure, but the client cannot find out about the moment of failure. In short, the possibility of a server crash radically changes the nature of RPC and clearly reflects the difference between a centralized and a distributed system. In the first case, a server crash leads to a client crash, and recovery is impossible. In the second case, it is both possible and necessary to perform system recovery actions.

The client crashed after sending the request. In this case, calculations are performed on results that no one expects. Such calculations are called "orphans". The presence of orphans can cause various problems: wasted CPU time, blocking of resources, substitution of a response to a current request with a response to a request that was issued by the client machine before the system was restarted.

How to deal with orphans? Let's look at 4 possible solutions.

Destruction. Before the client stub sends an RPC message, it makes a note in the log indicating what it will do now. The log is stored on disk or other fault-tolerant storage. After the accident, the system is rebooted, the log is analyzed and the orphans are eliminated. Disadvantages of this approach include, first, the increased overhead associated with writing each RPC to disk, and, second, possible inefficiency due to the appearance of second-generation orphans generated by RPC calls issued by first-generation orphans. Reincarnation. In this case, all problems are solved without using disk recording. The method consists of dividing time into sequentially numbered periods. When the client reboots, it broadcasts a message to all machines to announce the start of a new period. After receiving this message, all remote calculations are eliminated. Of course, if the network is segmented, then some orphans may survive. Soft re-incarnation is similar to the previous case, except that not all deleted calculations are found and destroyed, but only the calculations of the rebooting client. Expiration date. Each request is given a standard time period T during which it must be completed. If the request is not completed within the allotted time, then an additional quantum is allocated. Although this requires additional work, if after a client crash the server waits for an interval T before rebooting the client, then all orphans are necessarily destroyed.

In practice, neither of these approaches is desirable, and in fact, killing orphans may make the situation worse. For example, suppose an orphan has locked one or more database files. If the orphan is suddenly destroyed, then these locks will remain, in addition, the destroyed orphans may remain standing in various system queues, in the future they may cause the execution of new processes, etc.

Remote Procedure Call RPC The concept of Remote Procedure Call The idea behind Remote Procedure Call (RPC) is to extend the well-known and understood mechanism for transferring control and data within a program running on one machine to transfer control and data over a network. Remote procedure call tools are designed to facilitate the organization of distributed computing. The greatest efficiency of using RPC is achieved in those applications in which there is interactive communication between remote components with short response times and a relatively small amount of transferred data.

Such applications are called RPC-oriented. The characteristic features of calling local procedures are Asymmetry, that is, one of the interacting parties is the initiator Synchronicity, that is, the execution of the calling procedure stops from the moment the request is issued and resumes only after returning from the called procedure. The implementation of remote calls is much more complicated than the implementation of calls to local procedures.

To begin with, since the calling and called procedures are executed on different machines, they have different address spaces, and this creates problems when passing parameters and results, especially if the machines are not identical. Since RPC cannot rely on shared memory, this means that RPC parameters should not contain pointers to non-stack memory locations and that parameter values ​​should be copied from one computer to another.

The next difference between RPC and a local call is that it necessarily uses the underlying communication system, but this should not be explicitly visible either in the definition of the procedures or in the procedures themselves. Remoteness introduces additional problems. The execution of the calling program and the called local procedure on the same machine is implemented within a single process. But the implementation of RPC involves at least two processes - one in each machine.

In case one of them crashes, the following situations may arise: if the calling procedure crashes, the remotely called procedures will become orphaned, and if the remote procedures crash, the calling procedures will become orphaned parents, who will wait in vain for a response from the remote procedures. In addition, there are a number of problems associated with the heterogeneity of programming languages ​​and operating environments, the data structures and procedure call structures supported in any one programming language are not supported in the same way in all other languages.

These and some other problems are solved by the widespread RPC technology, which underlies many distributed operating systems. Basic RPC OperationsTo understand how RPC works, let's first consider making a local procedure call on a regular machine running autonomously. Let it be, for example, the system call count read fd,buf,nbytes where fd is an integer, buf is an array of characters, nbytes is an integer .

To make the call, the calling procedure pushes the parameters onto the stack in the reverse order of Figure 3.1. After the read call is executed, it places the return value into a register, moves the return address, and returns control to the calling procedure, which pops the parameters from the stack, returning it to its original state. Note that in C, parameters can be called by reference or by name , or by value. In relation to the called procedure, value parameters are initialized local variables.

The called procedure can change them without affecting the original values ​​of these variables in the calling procedure. If a pointer to a variable is passed to the called procedure, then changing the value of this variable by the called procedure entails changing the value of this variable for the calling procedure. This fact is very significant for RPC. There is also another mechanism for passing parameters that is not used in C. It is called call-by-copy restore and involves the calling program copying variables onto the stack as values, and then copying them back after the call is made over the original values ​​of the calling procedure.

The decision about which parameter passing mechanism to use is made by the language developers. Sometimes this depends on the type of data being passed. In C, for example, integers and other scalar data are always passed by value, and arrays are always passed by reference.

Rice. 3.1. a The stack before the read call is executed b The stack during the execution of the procedure c The stack after returning to the calling program The idea behind RPC is to make a call to a remote procedure look as similar as possible to a call to a local procedure. In other words, to make RPC transparent, the calling procedure does not need to know that the called procedure is on another machine, and vice versa. RPC achieves transparency in the following way.

When the called procedure is actually remote, instead of the local procedure, another version of the procedure, called the client stub, is placed in the library. Similar to the original procedure, the stub is called using the calling sequence as in Figure 3.1, and an interrupt occurs when accessing the kernel. Only, unlike the original procedure, it does not place parameters in registers and does not request data from the kernel; instead, it generates a message to be sent to the kernel of the remote machine. Stages of RPC executionThe interaction of software components when performing a remote procedure call is illustrated in Figure 3.2. After the client stub has been called by the client program, its first task is to fill the buffer with the message being sent.

In some systems, the client stub has a single fixed-length buffer that is filled from the very beginning with each new request. In other systems, the message buffer is a pool of buffers for individual message fields, some of which are already full.

This method is especially suitable for cases where the packet has a format consisting of a large number of fields, but the values ​​of many of these fields do not change from call to call. The parameters must then be converted to the appropriate format and inserted into the message buffer. At this point, the message is ready to be sent, so the kernel call interrupt is executed. Rice. 3.2. Remote Procedure Call When the kernel gains control, it switches contexts, saves processor registers and memory map page handles, and installs a new memory map that will be used to run in kernel mode. Because the kernel and user contexts are different, the kernel must copy the message exactly into its own address space so that it can access it, remember the destination address and possibly other header fields, and it must pass it to the network interface.

This completes the work on the client side.

The transmission timer is turned on, and the kernel can either cyclically poll for a response or pass control to the scheduler, which will select some other process to run. In the first case, query execution is accelerated, but multiprogramming is absent. On the server side, incoming bits are placed by the receiving hardware either in the built-in buffer or in RAM. When all the information has been received, an interrupt is generated.

The interrupt handler checks the packet's data for validity and determines which stub to pass it to. If no stub is expecting the packet, the interrupt handler must either buffer it or discard it altogether. If there is a waiting stub, the message is copied to it. Finally, a context switch is performed, as a result of which the registers and memory map are restored, taking the values ​​that they had at the moment when the stub made the receive call.

Now the server stub starts working. It unpacks the parameters and pushes them appropriately onto the stack. When everything is ready, a call to the server is made. After completing the procedure, the server transmits the results to the client. To do this, all the steps described above are performed, only in reverse order. Figure 3.3 shows the sequence of commands that must be executed for each RPC call, and Figure 3.4 shows what percentage of the total RPC execution time is spent on each of the 14 steps described.

The tests were conducted on a DEC Firefly multi-processor workstation, and while the presence of five processors necessarily affected the results of the measurements, the histogram shown in the figure gives a general idea of ​​​​the RPC execution process. Rice. 3.3. Stages of the RPC procedure Fig. 3.4. Time distribution between 14 stages of RPC execution 1. Call a stub 2. Prepare a buffer 3. Pack parameters 4. Fill in the header field 5. Calculate the checksum in the message 6. Interrupt to the kernel 7. Queue the packet for execution 8. Transfer the message to the controller via the QBUS bus 9. Transfer time over the Ethernet network 10. Receive a packet from the controller 11. Interrupt handling procedure 12. Checksum calculation 13. Context switching to user space 14. Performing a server stub Dynamic binding Let's consider the question of how the client specifies the location of the server.

One method to solve this problem is to directly use the server's network address in the client program.

The disadvantage of this approach is that it is extremely inflexible when moving the server, or increasing the number of servers, or changing the interface; in all these and many other cases, it is necessary to recompile all programs that used a hard-coded server address. In order to avoid all these problems, in Some distributed systems use what is called dynamic linking.

The starting point for dynamic binding is to formally define the server specification. The specification contains the file server name, version number and a list of service procedures provided by this server to clients (Figure 3.5). For each procedure, a description of its parameters is given, indicating whether this parameter is input or output relative to the server. Some parameters can be both input and output - for example, some array that is sent by the client to the server is modified there, and then returned back to the client operation copy restore . Rice. 3.5. RPC Server Specification The formal server specification is used as input to the stub generator program, which creates both client and server stubs.

They are then placed in the appropriate libraries. When a user client program calls any procedure defined in the server specification, the corresponding stub procedure is associated with the program binary code.

Likewise, when a server is compiled, server stubs are associated with it. When the server starts, its very first action is to transfer its server interface to a special program called binder. This process, known as the server registration process, involves the server transmitting its name, version number, unique identifier, and a descriptor of the server's location. The descriptor is system independent and can be an IP, Ethernet, X.500, or some other address.

In addition, it may contain other information, for example related to authentication. When a client calls one of the remote procedures for the first time, for example, read, the client stub sees that it is not yet connected to the server, and sends a message to the binder program with a request to import the interface of the desired version of the desired server. If such a server exists, then binder sends descriptor and unique identifier for the client stub.

When sending a message with a request, the client stub uses a descriptor as an address. The message contains parameters and a unique identifier that the server core uses to route the incoming message to the desired server if there are several of them on this machine. This method of importing and exporting interfaces is highly flexible. For example, there may be several servers supporting the same interface, and clients are randomly distributed among the servers.

Within the framework of this method, it becomes possible to periodically poll servers, analyze their performance and, in case of failure, automatically shut down, which increases the overall fault tolerance of the system. This method can also support client authentication. For example, the server may determine that it can only be used by clients from a specific list. However, dynamic binding has disadvantages, such as additional overhead and time spent exporting and importing interfaces.

The magnitude of these costs can be significant, since many client processes exist for a short time, and each time the process starts, the interface import procedure must be performed again. In addition, in large distributed systems, the binder program can become a bottleneck, and creating multiple programs with the same purpose also increases the overhead of creating and synchronizing processes. RPC semantics in the event of failures Ideally, RPC should function correctly in the event of failures.

Consider the following failure classes: 1. The client cannot locate the server, for example, if the desired server fails, or because the client program was compiled a long time ago and used an old version of the server interface. In this case, in response to the client's request, a message containing an error code is received. 2. The request from the client to the server was lost. The simplest solution is to repeat the request after a certain time. 3. The response message from the server to the client is lost.

This option is more complicated than the previous one, since some procedures are not idempotent. An idempotent procedure is a procedure whose execution request can be repeated several times without changing the result. An example of such a procedure is reading a file. But the procedure for withdrawing a certain amount from a bank account is not idempotent, and if the response is lost, a repeated request can significantly change the state of the client’s account.

One possible solution is to make all procedures idempotent. However, in practice this is not always possible, so another method can be used - sequential numbering of all requests by the client kernel. The server core remembers the number of the most recent request from each client, and upon receiving each request, it analyzes whether this request is a primary or a repeated one. 4. The server crashed after receiving the request. The idempotency property is also important here, but unfortunately the request numbering approach cannot be applied.

In this case, it matters when the failure occurred - before or after the operation. But the client kernel cannot recognize these situations; it only knows that the response time has expired. There are three approaches to this problem: Wait until the server reboots and try the operation again. This approach ensures that the RPC is completed at least once, and possibly more. Immediately report the error to the application.

This approach ensures that the RPC is executed at most once. The third approach does not guarantee anything. When the server fails, no support is provided to the client. The RPC may either not be executed at all, or it may be executed many times. In any case, this method is very easy to implement. Neither of these approaches is very attractive. And the ideal option, which would guarantee exactly one RPC execution, in the general case cannot be implemented for reasons of principle.

Let, for example, a remote operation be printing some text, which includes loading the printer buffer and setting one bit in some printer control register, as a result of which the printer starts. A server crash can occur either a microsecond before or a microsecond after the control bit is set. The moment of failure entirely determines the recovery procedure, but the client cannot find out about the moment of failure.

In short, the possibility of a server crash radically changes the nature of RPC and clearly reflects the difference between a centralized and a distributed system. In the first case, a server crash leads to a client crash, and recovery is impossible. In the second case, it is both possible and necessary to perform system recovery actions. 1. The client crashed after sending the request. In this case, calculations are performed on results that no one expects. Such calculations are called orphan calculations. The presence of orphans can cause various problems: overhead of CPU time, blocking of resources, replacement of the response to the current request with a response to a request that was issued by the client machine before the system was restarted.

How to deal with orphans? Let's look at 4 possible solutions. Destruction. Before the client stub sends an RPC message, it makes a note in the log indicating what it will do next. The log is stored on disk or other fault-tolerant memory.

After the accident, the system is rebooted, the log is analyzed and the orphans are eliminated. Disadvantages of this approach include, first, the increased overhead associated with writing each RPC to disk, and, second, possible inefficiency due to the appearance of second-generation orphans generated by RPC calls issued by first-generation orphans. Reincarnation. In this case, all problems are solved without using disk recording. The method consists of dividing time into sequentially numbered periods. When the client reboots, it broadcasts a message to all machines to announce the start of a new period.

After receiving this message, all remote calculations are eliminated. Of course, if the network is segmented, then some orphans may survive. Soft re-incarnation is similar to the previous case, except that not all deleted calculations are found and destroyed, but only the calculations of the rebooting client. Expiration. Each request is given a standard period of time T within which it must be completed.

If the request is not completed within the allotted time, then an additional quantum is allocated. Although this requires additional work, if after a client crash the server waits for an interval T before rebooting the client, then all orphans are necessarily destroyed. In practice, neither of these approaches is desirable; in fact, destroying orphans may make the situation worse. For example, suppose an orphan has locked one or more database files.

If the orphan is suddenly destroyed, then these locks will remain, in addition, the destroyed orphans may remain standing in various system queues, in the future they may cause the execution of new processes, etc.

What will we do with the received material:

If this material was useful to you, you can save it to your page on social networks: