Connections, Sessions, Requests, Logging#

Connection processing methods#

Angie supports a variety of connection processing methods. The availability of a particular method depends on the platform used. On platforms that support several methods Angie will normally select the most efficient method automatically. However, if needed, a connection processing method can be selected explicitly with the use directive.

The following connection processing methods are supported:

`select`#

Standard method. The supporting module is built automatically on platforms that lack more efficient methods. The ‑‑with‑select_module and ‑‑without‑select_module configuration parameters can be used to forcibly enable or disable the build of this module.

`poll`#

Standard method. The supporting module is built automatically on platforms that lack more efficient methods. The ‑‑with‑poll_module and ‑‑without‑poll_module configuration parameters can be used to forcibly enable or disable the build of this module.

`kqueue`#

Efficient method used on FreeBSD 4.1+, OpenBSD 2.9+, NetBSD 2.0, and macOS.

`epoll`#

Efficient method used on Linux 2.6+.

`/dev/poll`#

Efficient method used on Solaris 7 11/99+, HP/UX 11.22+ (eventport), IRIX 6.5.15+, and Tru64 UNIX 5.1A+.

`eventport`#

event ports, method used on Solaris 10+ (due to known issues, it is recommended using the /dev/poll method instead).

HTTP session processing#

Each HTTP request passes through a sequence of phases. In each phase a distinct type of processing is performed on the request.

`Post-read`	First phase. The http_realip module is invoked at this phase.
`Server-rewrite`	Phase where directives of http_rewrite defined in a server block (but outside a location block) are processed.
`Find-config`	Special phase where a location is chosen based on the request URI.
`Rewrite`	Same as `Server-rewrite`, but for rewrite rules defined in the location, chosen in the previous phase.
`Post-rewrite`	Special phase where the request is redirected to a new location like at `Find-config` phase, if its URI changed during a `Rewrite`.
`Preaccess`	The standard Angie modules http_limit_conn and http_limit_req register their handlers at this phase.
`Access`	Phase where it is verified that the client is authorized to make the request by invoking standard Angie modules such as http_access and http_auth_basic.
`Post-access`	Special phase where the satisfy any directive is processed.
`Precontent`	Standard modules such as try_files and http_mirror register their handlers at this phase.
`Content`	Phase where the response is normally generated. Multiple Angie standard modules register their handlers at this phase, including http_index. They are called sequentially until one of them produces the output.
`Log`	The final phase where request logging is performed. Currently, only the http_log registers its handler at this stage for access logging.

TCP/UDP session processing#

A TCP/UDP session from a client is processed in successive phases:

`Post-accept`	The first phase after accepting a client connection. The stream_realip is invoked at this phase.
`Pre-access`	Preliminary check for access. The stream_limit_conn and stream_set modules are invoked at this phase.
`Access`	Client access limitation before actual data processing. At this phase, the stream_access is invoked.
`SSL`	TLS/SSL termination. The stream_ssl is invoked at this phase.
`Preread`	Reading initial bytes of data into the preread buffer to allow modules such as stream_ssl_preread analyze the data before its processing.
`Content`	Mandatory phase where data is actually processed, usually proxied to upstream servers, or a specified value is returned to a client.
`Log`	The final phase where the result of a client session processing is recorded. The stream_log is invoked at this phase.

Processing a request#

Virtual server selection#

First, a connection is created in a default server context. Then, the server name can be determined in the following request processing stages, each involved in server configuration selection:

during SSL handshake, in advance, according to SNI.
after processing the request line
after processing the “Host” header field

If the server name was not determined after processing the request line or from the “Host” header field, Angie will use the empty name as the server name.

At each of these stages, different server configurations can be applied. As such, certain directives should be specified with caution:

in case of the ssl_protocols directive, the protocol list is set by the OpenSSL library before the server configuration could be applied according to the name requested through SNI, thus, protocols should be specified only for a default server;
the client_header_buffer_size and merge_slashes directives are involved before reading the request line, thus, such directives use a default server configuration or the server configuration chosen by SNI;
in case of the ignore_invalid_headers, large_client_header_buffers and underscores_in_headers directives involved in processing request header fields, it additionally depends whether the server configuration was updated according to the request line or the “Host” header field;
an error response will be handled with the error_page directive in the server that currently fulfills the request.

Name-based virtual servers#

Angie first decides which server should process the request. Let’s start with a simple configuration where all three virtual servers listen on port *:80:

server {
  listen      80;
  server_name example.org www.example.org;
# ...
}

server {
  listen      80;
  server_name example.net www.example.net;
#  ...
}

server {
  listen      80;
  server_name example.com www.example.com;
#  ...
}

In this configuration Angie tests only the request’s header field “Host” to determine which server the request should be routed to. If its value does not match any server name, or the request does not contain this header field at all, then Angie will route the request to the default server for this port. In the configuration above, the default server is the first one — which is Angie’s standard default behavior. It can also be set explicitly which server should be default, with the default_server parameter in the listen directive:

server {
  listen      80 default_server;
  server_name example.net www.example.net;
#  ...
}

Note

Note that the default server is a property of the listen socket and not of the server name.

Internationalized names#

Internationalized domain names (IDNs) should be specified using an ASCII (Punycode) representation in the server_name directive:

server {
    listen       80;
    server_name  xn--e1afmkfd.xn--80akhbyknj4f;  # пример.испытание
#    ...
}

How to prevent processing requests with undefined server names#

If requests without the “Host” header field should not be allowed, a server that just drops the requests can be defined:

server {
  listen      80;
  server_name "";
  return      444;
}

Here, the server name is set to an empty string that will match requests without the “Host” header field, and a special non-standard code 444 is returned that closes the connection.

Mixed name-based and IP-based virtual servers#

Let’s look at a more complex configuration where some virtual servers listen on different addresses:

server {
  listen      192.168.1.1:80;
  server_name example.org www.example.org;
#  ...
}

server {
  listen      192.168.1.1:80;
  server_name example.net www.example.net;
#  ...
}

server {
  listen      192.168.1.2:80;
  server_name example.com www.example.com;
#  ...
}

In this configuration, Angue first tests the IP address and port of the request against the listen directives of the server blocks. It then tests the “Host” header field of the request against the server_name entries of the server blocks that matched the IP address and port. If the server name is not found, the request will be processed by the default server. For example, a request for www.example.com received on the 192.168.1.1:80 port will be handled by the default server of the 192.168.1.1:80 port, i.e., by the first server, since there is no www.example.com defined for this port.

As already stated, a default server is a property of the listen port, and different default servers may be defined for different ports:

server {
  listen      192.168.1.1:80;
  server_name example.org www.example.org;
#  ...
}

server {
  listen      192.168.1.1:80 default_server;
  server_name example.net www.example.net;
#  ...
}

server {
  listen      192.168.1.2:80 default_server;
  server_name example.com www.example.com;
#  ...
}

Choosing location#

Based on simple PHP site configuration:

server {
  listen      80;
  server_name example.org www.example.org;
  root        /data/www;

  location / {
     index   index.html index.php;
  }

  location ~* \.(gif|jpg|png)$ {
     expires 30d;
  }

  location ~ \.php$ {
     fastcgi_pass  localhost:9000;
     fastcgi_param SCRIPT_FILENAME
                   $document_root$fastcgi_script_name;
     include       fastcgi_params;
  }
}

Angie first searches for the most specific prefix location given by literal strings regardless of the listed order. In the configuration above the only prefix location is location / and since it matches any request it will be used as a last resort. Then Angie checks locations given by regular expression in the order listed in the configuration file. The first matching expression stops the search and Angie will use this location. If no regular expression matches a request, then Angie uses the most specific prefix location found earlier.

Note

Note that locations of all types test only a URI part of request line without arguments. This is done because arguments in the query string may be given in several ways, for example:

/index.php?user=john&page=1
/index.php?page=1&user=john

Besides, anyone may request anything in the query string:

/index.php?page=1&something+else&user=john

Now let’s look at how requests would be processed in the configuration above:

A request /logo.gif is matched by the prefix location / first and then by the regular expression .(gif|jpg|png)$, therefore, it is handled by the latter location. Using the directive root /data/www the request is mapped to the file /data/www/logo.gif, and the file is sent to the client.

A request /index.php is also matched by the prefix location / first and then by the regular expression .(php)$. Therefore, it is handled by the latter location and the request is passed to a FastCGI server listening on localhost:9000. The fastcgi_param directive sets the FastCGI parameter SCRIPT_FILENAME to /data/www/index.php, and the FastCGI server executes the file. The variable $document_root is equal to the value of the root directive and the variable $fastcgi_script_name is equal to the request URI, i.e. /index.php.

A request /about.html is matched by the prefix location / only, therefore, it is handled in this location. Using the directive root /data/www the request is mapped to the file /data/www/about.html, and the file is sent to the client.

Handling a request / is more complex. It is matched by the prefix location / only, therefore, it is handled by this location. Then the index directive tests for the existence of index files according to its parameters and the root /data/www directive. If the file /data/www/index.html does not exist, and the file /data/www/index.php exists, then the directive does an internal redirect to /index.php, and Angie searches the locations again as if the request had been sent by a client. As we saw before, the redirected request will eventually be handled by the FastCGI server.

Proxying and Load Balancing#

One of the frequent uses of Angie is setting it up as a proxy server, which means a server that receives requests, passes them to the proxied servers, retrieves responses from them, and sends them to the clients.

Simple Proxy Server:

server {
    location / {
        proxy_pass http://backend:8080;
    }

proxy_pass directive instructs Angie to pass client requests to the backend backend:8080 (proxied server). There are many more directives that may be used to further configure a proxy connection.

FastCGI Proxying#

Angie can be used to route requests to FastCGI servers which run applications built with various frameworks and programming languages such as PHP.

The most basic Angie configuration to work with a FastCGI server includes using the fastcgi_pass directive instead of the proxy_pass directive, and fastcgi_param directives to set parameters passed to a FastCGI server. Suppose the FastCGI server is accessible on localhost:9000. In PHP, the SCRIPT_FILENAME parameter is used for determining the script name, and the QUERY_STRING parameter is used to pass request parameters. The resulting configuration would be:

server {
    location / {
        fastcgi_pass  localhost:9000;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param QUERY_STRING    $query_string;
    }

    location ~ \.(gif|jpg|png)$ {
        root /data/images;
    }
}

This will set up a server that will route all requests except for requests for static images to the proxied server operating on localhost:9000 through the FastCGI protocol.

WebSocket proxying#

To turn a connection between a client and server from HTTP/1.1 into WebSocket, the protocol switch mechanism available in HTTP/1.1 is used.

There is one subtlety however: since the “Upgrade” is a hop-by-hop header, it is not passed from a client to proxied server. With forward proxying, clients may use the CONNECT method to circumvent this issue. This does not work with reverse proxying however, since clients are not aware of any proxy servers, and special processing on a proxy server is required.

Angie implements special mode of operation that allows setting up a tunnel between a client and proxied server if the proxied server returned a response with the code 101 (Switching Protocols), and the client asked for a protocol switch via the “Upgrade” header in a request.

As noted above, hop-by-hop headers including “Upgrade” and “Connection” are not passed from a client to proxied server, therefore in order for the proxied server to know about the client’s intention to switch a protocol to WebSocket, these headers have to be passed explicitly:

location /chat/ {
    proxy_pass http://backend;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "upgrade";
}

A more sophisticated example in which a value of the “Connection” header field in a request to the proxied server depends on the presence of the “Upgrade” field in the client request header:

http {
    map $http_upgrade $connection_upgrade {
        default upgrade;
        ''      close;
    }

    server {
        ...

        location /chat/ {
            proxy_pass http://backend;
            proxy_http_version 1.1;
            proxy_set_header Upgrade $http_upgrade;
            proxy_set_header Connection $connection_upgrade;
        }
    }

By default, the connection will be closed if the proxied server does not transmit any data within 60 seconds. This timeout can be increased with the proxy_read_timeout directive. Alternatively, the proxied server can be configured to periodically send WebSocket ping frames to reset the timeout and check if the connection is still alive.

Load balancing#

Load balancing across multiple application instances is a commonly used technique for optimizing resource utilization, maximizing throughput, reducing latency, and ensuring fault-tolerant configurations.

It is possible to use Angie as a very efficient HTTP load balancer to distribute traffic to several application servers and to improve performance, scalability and reliability of web applications with Angie.

The simplest configuration for load balancing with Angie may look like the following:

http {
    upstream myapp1 {
        server srv1.example.com;
        server srv2.example.com;
        server srv3.example.com;
    }

    server {
        listen 80;

        location / {
            proxy_pass http://myapp1;
        }
    }
}

In the example above, there are 3 instances of the same application running on srv1-srv3. When the load balancing method is not specifically configured, it defaults to round-robin. Other supported load balancing mechanisms: weight, least_conn, ip_hash. Reverse proxy implementation in Angie includes in-band (or passive) server health checks. It’s configured with max_fails and fail_timeout directives of server in upstream context.

Logging#

Logging to syslog#

The error_log and access_log directives support logging to syslog. The following parameters configure logging to syslog:

`server=`address	Defines the address of a syslog server. The address can be specified as a domain name or IP address, with an optional port, or as a UNIX domain socket path specified after the “unix:” prefix. If port is not specified, the UDP port 514 is used. If a domain name resolves to several IP addresses, the first resolved address is used.
`facility=`string	Sets facility of syslog messages, as defined in RFC 3164. Facility can be one of: “kern”, “user”, “mail”, “daemon”, “auth”, “intern”, “lpr”, “news”, “uucp”, “clock”, “authpriv”, “ftp”, “ntp”, “audit”, “alert”, “cron”, “local0”..”local7”. Default is “local7”.
`severity=`string	Sets severity of syslog messages for access_log as defined in RFC 3164. Possible values are the same as for the second parameter (level) of the error_log directive. Default is “info”. Severity of error messages is determined by Angie, thus the parameter is ignored in the error_log directive.
`tag=`string	Sets the tag of syslog messages. Default is “angie”.
`nohostname`	Disables adding the “hostname” field into the syslog message header

Example syslog configuration:

error_log syslog:server=192.168.1.1 debug;

access_log syslog:server=unix:/var/log/angie.sock,nohostname;
access_log syslog:server=[2001:db8::1]:12345,facility=local7,tag=angie,severity=info combined;

A debugging log#

To enable a debugging log, Angie needs to be configured to support debugging during the build:

./configure –with-debug …

To verify that Angie is configured to support debugging, run the angie -V command:

configure arguments: –with-debug …

Pre-built Linux packages provide out-of-the-box support for debugging log with the angie-debug binary. Installed from the package Angie provides two binaries in place:

$ ls -l /usr/sbin/ | grep angie
   lrwxrwxrwx 1 root root      13 Sep 21 18:58 angie -> angie-nodebug
   -rwxr-xr-x 1 root root 1561224 Sep 21 18:58 angie-debug
   -rwxr-xr-x 1 root root 1426056 Sep 21 18:58 angie-nodebug

systemd service Angie is configured as follows:

DAEMON=${DAEMON:-/usr/sbin/angie}

Thus, to run debug binary you need to:

set debug level with the error_log directive:

error_log /path/to/log debug;

reassign simlink:

$ sudo ln -fs angie-debug /usr/sbin/angie

and run Upgrade Executable on the Fly

$ sudo angie -t && sudo service angie upgrade

To disable debug log:

reassign simlink:

$ sudo ln -fs angie-nodebug /usr/sbin/angie

and run:

$ sudo angie -t && sudo service angie upgrade

If you keep here debug level in error_log directive Angie will store log on info level.

Note that redefining the log without also specifying the debug level will disable the debugging log. In the example below, redefining the log on the server level disables the debugging log for this server:

error_log /path/to/log debug;

http {
   server {
     error_log /path/to/log;
    # ...

To avoid this, either the line redefining the log should be commented out, or the debug level specification should also be added:

error_log /path/to/log debug;

http {
   server {
     error_log /path/to/log debug;
   #  ...

Debugging log for selected clients#

It is also possible to enable the debugging log for selected client addresses only:

error_log /path/to/log;

events {
  debug_connection 192.168.1.1;
  debug_connection 192.168.10.0/24;
}

Logging to a cyclic memory buffer#

Logging to a cyclic memory buffer:

error_log memory:32m debug;

Logging to the memory buffer on the debug level does not have significant impact on performance even under high load. In this case, the log can be extracted using a gdb script like the following one:

set $log = ngx_cycle->log

while $log->writer != ngx_log_memory_writer
  set $log = $log->next
end

set $buf = (ngx_log_memory_buf_t *) $log->wdata
dump binary memory debug_log.txt $buf->start $buf->end

Connections, Sessions, Requests, Logging

Contents

Connections, Sessions, Requests, Logging#

Connection processing methods#

select#

poll#

kqueue#

epoll#

/dev/poll#

eventport#

HTTP session processing#

TCP/UDP session processing#

Processing a request#

Virtual server selection#

Name-based virtual servers#

Internationalized names#

How to prevent processing requests with undefined server names#

Mixed name-based and IP-based virtual servers#

Choosing location#

Proxying and Load Balancing#

FastCGI Proxying#

WebSocket proxying#

Load balancing#

Logging#

Logging to syslog#

A debugging log#

Debugging log for selected clients#

Logging to a cyclic memory buffer#

`select`#

`poll`#

`kqueue`#

`epoll`#

`/dev/poll`#

`eventport`#