# TAO Implementation Repository

Revision 3.05

## Table of Contents

- Recent Changes
- Overview
  - Persistent and Transient IORs
  - The Implementation Repository
- TAO's Implementation Repository
  - Virtual Servers
  - Ping Object
  - New IORs
    - What was wrong with the old IOR?
    - Why does the Implementation Repository profile contain an Object Key?
  - POA Extensions
  - Possible Future Goals
  - Server Restrictions
  - Preliminary Interface
- Alternate Implementations
- Accessing the Implementation Repository
  - Helper Application
  - Locating an instance of the Implementation Repository
    - Server Side
    - Client Side
- How It Works
  - How a server produces a Persistent IOR (in the default case)
  - How a server produces a Persistent IOR (in the complex case)
  - How a client uses a Persistent IOR

## Recent Changes

Since 3.04

- Moved it to the same directory as all the new
- After a long delay (too long, if you ask me) addressed the comments that John Mulhern <9107@mn3.lawson.lawson.com> sent me.

Since 3.03

- Added information on the new POA policy where the format of the persistent IOR can be changed from that of both the last-known-server-IOR and Implementation Repository to that just of the Implementation Repository
- TAO is now fork-safe, with the introduction of the CLOEXEC flag through ACE_CLOEXEC.

Since 3.02

- Added a section to give more detailed information on how Ping Objects work.
- Rewrote the Virtual Server section

## Overview

This document describes the proposed design of the TAO Implementation Repository, which was originally known as the reactivator/activation service. If you have any questions or comments on our design, please post them to the ACE mailing list <ace-users@cs.wustl.edu> or send email to Darrell Brunsch <brunsch@cs.wustl.edu>.

### Persistent and Transient IORs

CORBA defines two types of object references: persistent and transient. The difference between the two stems from the lifetime of the reference in relation to the lifetime of the server process that created it. The lifetime of a transient object reference is limited to the lifetime of its server process. Once the server process exits the transient object reference no longer exists. All references to this object should now be invalid, even if the server is restarted. In contrast, persistent object references can outlive their originating server process. Therefore, the server can exit and be restarted without invalidating its persistent object references. This enables the implementation of features like automatic server activation and object migration.

Note that both persistent and transient object references can refer to objects that reside in manually activated servers, i.e., the so-called ``persistent servers.'' A persistent server is a server that is launched manually, i.e., it is always running. A persistent server can generate transient references and/or persistent references.

Developers should be aware that persistence of the object reference does not imply any persistence on the object implementation state. It is certainly possible to provide persistent object references for objects whose state is not persistent. Therefore, servant implementors are responsible for preserving the state of their servants, e.g., using a database or file.

### The Implementation Repository

According to the CORBA specification, "The Implementation Repository contains information that allows the ORB to locate and activate implementations of objects" [CORBA Spec Rev. 2.2: 2.1.14] In earlier revisions of the specification, there was a method `get_implementation` in the CORBA Object interface. This has been deprecated as of the CORBA 2.2 specification, leaving both the interface and implementation of the Implementation Repository to the ORB vendor.

A good paper describing the functionality of the CORBA Implementation Repository is "Binding, Migration, and Scalability in CORBA" [Michi Henning]. This paper describes the following three functions of the Implementation Repository:

1. Maintain a registry of known servers.
2. Record which server is currently running, and which port and host it uses.
3. Starts servers on demand if they are registered with the Implementation Repository.

The TAO Implementation Repository is based on the design in this paper. He also wrote an earlier email that may be useful to see the issues involved. The next section details our goals and plans for the implementation.

## TAO's Implementation Repository

The following is an brief outline of TAO'S Implementation Repository.

- Use of TAO's Implementation Repository will be optional. Real-time applications can choose not to use the Implementation Repository according to their performance/predictability/footprint requirements.
- Use of TAO's Implementation Repository will be invisible to clients and servers for common use-case. For more complicated behavior, programs can use Implementation Repository extensions of the POA.
- TAO's Implementation Repository will work with any CORBA client that supports `LOCATION_FORWARD` IIOP messages and multiple profiles in IORs, even if the client is not implemented using TAO.
- TAO's Implementation Repository will know if one of the servers registered with it is running by the use of a "ping"-like service in the server. This service allows the Implementation Repository to know when it should restart the server.
- TAO will be fork-safe. Since there will be an open connection to the client while the server is restarted (via fork or CreateProcess) then care will be needed to make sure that the open sockets will be closed in the client process. The addition of CLOEXEC feature to TAO will cover this problem.
- TAO will exploit features of IIOP 1.1 to safely and efficiently verify if an IOR was generated by TAO itself on the client side. The server will still determine this through the object key, since that is all that is passed in a request.
- TAO will support multiple profiles in IORs. A profile contains the host/port and object key of a CORBA Object. An optimization that will be possible is to have a last known profile of the object as the first profile and an Implementation Repository as the second profile in an IOR. The client will first try the object to see if it still active at the host/port before it contacts the Implementation Repository. This strategy will reduce latency.

### Virtual Servers

TAO's Implementation Repository must keep track of whether an object's implementation is currently running or is stopped. To have a record for every object would require too much overhead, but having a record for every executable server would be inflexible and prevent the migration of objects. In the Henning paper, he mentions the use of a server name as the index for the table maintained by the Implementation Repository.

A virtual server does not refer to the executable but instead to a group of objects. An executable may have one or more virtual servers within it. This allows one virtual server to be moved off the executable to another executable (for instance, onto another machine) without affecting the objects in other virtual servers on the original executable.

Each virtual server will be indexed in the Implementation Repository by a name that is given to it by the user. It is the users responsibility to make sure that each virtual server name is unique. By default, this name is the name of the executable (since by default there is only one virtual server per executable). However, this default behavior can be overridden.

### Ping Object

Ping objects are simple objects that reside in the server, one for every virtual server. It is contacted by the Implementation Repository to determine if the virtual server is still running and responding. At certain intervals the Implementation Repository will invoke a one-way method on the ping object, and then will expect a "pong" message to be sent back. Different strategies for pinging (which will depend on a TAO option) will be used by the implementation repository. If a server is expected to be responsive, the Implementation Repository will not wait long for a response before considering the server to be gone. Other servers may be computationally-intensive and need to be held under less stringent expectations.

We chose the ping method to be a one-way (instead of two-way) because if the server became unresponsive, it would not return from the method invocation. The Implementation Repository needs some form of a timeout with the ping to be able to determine if the server is unresponsive or not.

### New IORs

Standard CORBA IORs contain the following two sections:

Type ID | Sequence of Tagged Profiles

The Type ID is an indicator for the most derived type known at the time of the reference creation. It is used as a hint for the client in determining what interfaces the object can support. The Sequence of Tagged Profiles consist of one or more profiles that encapsulate information used by the associated protocol in order to communicate with the object (host, port, object id, etc.).

Currently, TAO uses only one IIOP 1.0 Tagged Profile, which is defined as follows:

Version | Host | Port | Object Key

Object Key: |
Transient/Persistent Flag | TimeStamp | POA ID | OBJ ID

To accomodate the Implementation Repository and IIOP 1.1, the Profile was changed according to the CORBA specification as follows:

Version | Host | Port | Object Key | Components

Object Key: |
TAO | TAO version | TimeStamp/Server Name | POA ID | OBJ ID

The two main changes is the addition of the Components field (which is a IIOP 1.1 thing) and the addition of TAO with its version. Transient object references will still have a TimeStamp to ensure uniqueness, but persistent object references will have a server name to identify themselves to the Implementation Repository.

For servers that move around or need to be restarted often, the IOR will contain a reference to the Implementation Repository with the object key of the server and the server name imbedded. Once the client contacts the Implementation Repository, it will be forwarded to the correct object. This IOR will look like the following:

Version | Host | Port | Object Key | Components

Object Key: |
TAO | TAO version | Server Name | POA ID | OBJ ID (actually the OBJ Key of the Server)

For servers that expect to remain in the same host/port for a long time, the above IOR can be optimized by placing the server profile in the IOR before the Implementation Repository profile. TAO clients will first try the server, and if that fails, then try the Implementation Repository. Clients from other ORBs may behave the same way, but this isn't guaranteed since the handling of multiple profiles is not yet in the CORBA spec. There will be an option to only generate the IR-only IORs for ORBs that do not support multiple profiles in the needed manner.

There will be a POA policy to determine which type of Persistent IOR to use. By default, the Implementation Repository alone version will be used.

#### What was wrong with the old IOR?

We need a place to put a TAO marker in the IOR so TAO servers can differentiate TAO IORs from IORs of other vendors. In the original scheme used in TAO, Persistent IORs had a null timestamp. To support virtual servers, we will use that slot to store the server name so the Implementation Repository knows which server to launch.

#### Why does the Implementation Repository profile contain an Object Key?

It needs to know what the object key of the object when forwarding is used. A server may contain more than one object, so the object key is needed to forward to the correct object on the server.

### POA Extensions

TAO's POA will contain a new TAO-specific method called `create_reference_with_virtual_server[_and_id] (...)`. This method takes additional arguments for a virtual server name and a sequence of Implementation Repository IORs. The POA will register the virtual server name with each of the Implementation Repositories in the sequence passed in. Several Implementation Repositories can be specified to enhance availability through redundancy.

TAO's POA will also contain a policy for the type of IOR created with `create_reference`. It can either produce the standard type, with just a reference to the Implementation Repository, or it can produce one also containing a reference to the current server.

### Possible Future Goals

The following are features that may be added to support TAO's Implementation Repository:

- Optimization on TAO clients to recognize when a server is restarted, and change all other IORs that contain the server instead of going through the Implementation Repository
- Some sort of server security that checks the executable to make sure it is the correct executable (checksum, signatures, etc).
- Add the ability to put servers into DLLs or Shared Object files so the Implementation Repository can load it via those methods.
- GUI interface for such things as the helper application.
- Federations of Implementation Repositories.
- The ability to start a remote server (possibly with rsh, ssh, rexec, etc)

### Server Restrictions

Most often servers that have Persistent IORs will save their state to secondary storage. Databases are a good example of this, where the server can be stopped and restarted with all the information remaining on disk.

The server must also make sure it creates the POA and Object in a way that does not change the POA ID and Object ID. The Implementation Repository forwards requests based on the information in the IOR; if the POA ID or Object ID changes, then the Implementation Repository will be unable to sucessfully forward requests. If the server implements dynamic servants and dynamic POA activations, then this is not an issue since the necessary POAs and servants will be created on demand.

### Preliminary Interface

The following is a proposed IDL interface for the TAO Implementation Repository:

    module TAO
    {
      // ....

      exception Already_Registered {};
      // Object already bound in the Implementation Repository

      exception Cannot_Activate
      {
        string reason_;
      };

      exception Not_Found {};
      // Object not found in the Implementation Repository

      struct Environment_Variable
      {
        string name_;
        string value_;
      };
      // One environment variable

      struct INET_Addr
      {
        unsigned short port_;
        unsigned long host_;
      };
      // The location of a server

      typedef sequence<Environment_Variable> Environment;
      // Complete environment

      typedef sequence<string> Command_Line_Options;
      // Command line options

      struct Process_Options
      {
        string executable_name_;
        // Executable name

        Command_Line_Options command_line_options_;
        // Command line options

        Environment environment_;
        // Environment

        string working_directory_;
        // Working directory

        unsigned long creation_flags_;
        // Creation flags
      };

      interface Ping_Object
      {
        oneway void ping ();
        // Used for checking for liveness of a server.  When the server receives
        // this, it should send back a response indicating it is sill alive.
        // Depending on the policy specified, a timeout can be reached where the
        // Implementation Repository will restart the server.
      };

      interface Implementation_Repository
      {
        Object activate_object (in Object obj)
          raises (Not_Found,
                  Cannot_Activate);
          // Restart server that will contain this persistent object and return the
          // new Object reference.
          //
          // The <Not_Found> exception is raised when <obj> is not found
          // in the Implementation Repository.  The <Cannot_Activate> exception
          // is raised when <obj> is found in the Repository but could not be
          // activated.

        INET_Addr activate_server (in string server)
          raises (Not_Found,
                  Cannot_Activate);
          // Restart server that is named <server> and return the host/port
          //
          //
          // The <Not_Found> exception is raised when <server> is not found
          // in the Implementation Repository.  The <Cannot_Activate> exception
          // is raised when <server> is found in the Repository but could not be
          // activated.

        void register_server (in string server,
                              in Process_Options options)
          raises (Already_Registered);
          // Restart server process when client is looking for <server>.
          //
          // The <Already_Registered> exception is raised when <server> has
          // already been registered with the Implementation Repository.
          //
          // The <Object_Not_Persistent> exception is raised when <server> is
          // not a Persistent Object Reference.

        void reregister_server (in string server,
                                in Process_Options options)
          raises (Already_Registered);
          // Restart server process when client is looking for <server>.
          //
          // The <Already_Registered> exception is raised when <server> has
          // already been registered with the Implementation Repository.
          //
          // The <Object_Not_Persistent> exception is raised when <server> is
          // not a Persistent Object Reference.

        void remove_server (in string server)
          raises (Not_Found);
          // Remove <server> from the Implementation Repository.
          //
          // The <Not_Found> exception is raised when <server> is not found
          // in the Implementation Repository.

        Profile server_is_running (in string server,
                                   in INET_Addr addr,
                                   in Ping_Object ping);
          // Used to notify the Implementation Repository that <server> is alive and
          // well at <addr>.

        void server_is_shutting_down (in string server);
          // Used to tell the Implementation Repository that <server> is shutting
          // down.
        };
    };

## Alternate Implementations

Other ORB vendors use alternative techniques for their Implementation Repositories. These techniques usually require new naming techniques to access persistent object references and new client-side APIs to bind to persistent object references. TAO's Implementation Repository will not require such extensions.

A possible design alternative for the IR might use an Object Reference that points to the Implementation Repository instead of pointing directly to the persistent object. This extra level of indirection would be used by the Implementation Repository to start the server, if needed. The Location Forwarding mechanism would then be used to forward the client request to the server. The difference between this design and TAO's design is that the persistent IOR in TAO will contain a profile pointing to a location of the server (where it still might be running) to try first, and then only if that fails does the client contact the Implementation Repository. This is an optimization for case where the server does not shut down often, and most requests do not need to be forwarded to a new address.

In cases where most requests will require a forward, TAO can support a policy that is just like this alternative, where the Implmentation Repository will be contacted first.

## Accessing the Implementation Repository

The Implementation Repository will be transparent to the clients and the servers. Clients will only deal with IIOP 1.1 IORs, and in the default case all the Implementation Repository logic on the server side will be handled internally by the ORB and the POA.

### Helper Application

A helper application will be included with the Implementation Repository. It will be a command-line utility that will assist users with adding and removing server records (containing virtual server names and executable name/options) from the Implementation Repository.

### Locating an Instance of the Implementation Repository

#### Server-side

In the default case, the Implementation Repository will be found via the command-line, environment variables, and multicast (in that order). This location strategy is consistent with that used by TAO to local its default Naming Service instance. Using the POA extensions, other Implementation Repositories can be specified in the call to `POA::create_reference_with_virtual_server`. The default port of the Implementation Repository can be overridden through command-line options or environment variables.

#### Client-side

One or more Implementation Repositories will be stored in additional profiles in the IOR. Other Implementation Repositories can also be located by multicasting (on a default multicast group) the server name of the Persistent Object the client is interested in. The default multicast group and default port of the Implementation Repository can be overridden through command line options or environment variables.

In most cases, one Implementation Repository will be enough. For redundancy, several Implementation Repositories can be specified.

## How It Works

### How a server produces a Persistent IOR (in the default case)

Before a server starts, it must be registered (via a command-line utility) with an implementation repository. On platforms that don't support multicast, the Implementation Repository must be specified on the command line or in an environment variable.

1. When the server starts up it calls `ORB_init`. `ORB_init`, if not passed a server name, will take argv[0] and use that as a default server name (TAO expects this to be the executable name).
2. `ORB_init` will create a ping object.
3. `ORB_init` will look for Implementation Repositories on the command-line, environmental variables, and then through multicast (in that order). Once it finds one it registers itself and passes the ping object to the implementation repository with `server_is_running` operation.
4. The profile returned by registration will be stored for later use.
5. Client later can call the `POA::create_reference` operation.
6. The `create_reference` operation will create the local profile.
7. The stored Implementation Repository profile will have its object id changed to be the object key just created.
8. Both profiles will be joined together if the multiple profile IOR policy is set, and then returned.

### How a server produces a Persistent IOR (in complex cases)

As with the default case, the server must be registered with an Implementation Repository, although it does not need to be multicast aware since the IORs will be passed to the POA by the program.

1. `ORB_init` is called and does the default work (if it has Implementation Repositories to contact).
2. `POA::create_reference_with_virtual_server[_and_id]` will be called with a server name and list of Implementation Repositories.
3. The profile for the object is created.
4. The ping object created in `ORB_init` and the object key is passed to the Implementation Repositories, and their profiles are returned.
5. Both profiles will be joined together if the multiple profile IOR policy is set, and then returned.

### How a client uses a Persistent IOR

For all Clients:

- Client obtains a Persistent Object Reference, which contains multiple profiles to both regular objects and Implementation Repositories.
- It will now make a request on the first profile.
- If the first profile if the server profile, and the server is still there, then it will be successful. If the server has moved (or shut down), then the next profile will be tried.
- If the first profile is the Implementation Repository, or if the server profile failed, then it will be contacted. The Implemenation Repository will then return NOT_FOUND or will start up the server and return a Location Forward message.

If everything fails, then most clients will return failure for the request. TAO clients will attempt to contact other Implementation Repositories that are specified on the command-line, in environment variables, or found through multicast.

- If all of the profiles fail, then contact the other Implementation Repositories. First get those specified on the command line or in environment variables.
- Then, if multicast is available:
  - Multicast the Object Reference to a group of Implementation Repositories
  - Wait until response or a timeout. The response will contain the Object Reference of a Implementation Repository that knows about the Object Reference
- Connect to the Implementation Repository that responds first.
- Call activate passing the Persistent Object Reference.
- If a new Object Reference was sent back then retry the request using the it. If this request fails, then fail (no more retries).
- If a null reference was sent back, then fail.

TAO clients will have an optimization where if there are several IORs that have the same server name, and one of them gets forwarded, then the client will be able to change its other IORs without going through the overhead of contacting Implementation Repository.

Last update to this document: $Date$

Back to TAO Release Notes

Back to Implementation Repository