http_client

Olle E. Johansson

Edvina AB

Juha Heinanen

TutPro Inc.

Carsten Bock

ng-voice GmbH

Hugh Waite

Xura Inc

Table of Contents

1. Admin Guide
1. Overview
2. Dependencies
2.1. Kamailio Modules
2.2. External Libraries or Applications
3. Parameters
3.1. httpredirect (int)
3.2. httpproxy (string)
3.3. httpproxyport (string)
3.4. useragent (string)
3.5. maxdatasize (int)
3.6. connection_timeout (int)
3.7. client_cert (string)
3.8. client_key (string)
3.9. cacert (string)
3.10. cipher_suites (string)
3.11. verify_peer (int)
3.12. verify_host (int)
3.13. tlsversion (int)
3.14. authmethod (int)
3.15. keep_connections (int)
3.16. query_result (int)
3.17. query_maxdatasize (int)
3.18. httpcon (string)
3.19. config_file (string)
3.20. netinterface (string)
4. Functions
4.1. http_connect(connection, url, [content_type, data,] result)
4.2. http_connect_raw(connection, url, content_type, data, result)
4.3. http_get_redirect(connection, result)
4.4. http_client_query(url, [post-data], [hdrs], result)
5. Pseudovariables
5.1. $curlerror(error)
6. RPC Commands
6.1. httpclient.listcon
7. Counters
7.1. httpclient.connections
7.2. httpclient.connok
7.3. httpclient.connfail
8. Remarks
2. Developer Guide
1. Introduction
2. Available Functions
2.1. int http_connect(msg, connection, url, result, content_type, post)
2.2. int http_connection_exists(str *connection)
2.3. int http_query(msg, url, dest, post)
2.4. http_get_content_type(str connection)

List of Examples

1.1. Set httpredirect parameter
1.2. Set httpproxy parameter
1.3. Set httpproxyport parameter
1.4. Set useragent parameter
1.5. Set maxdatasize parameter
1.6. Set connection_timeout parameter
1.7. Set client_cert parameter
1.8. Set client_key parameter
1.9. Set cacert parameter
1.10. Set cipher_suites parameter
1.11. Set verify_peer parameter
1.12. Set verify_host parameter
1.13. Set tlsversion parameter
1.14. Set authmethod parameter
1.15. Set keep_connections parameter
1.16. Set query_result parameter
1.17. Set query_maxdatasize parameter
1.18. Set httpcon parameter
1.19. Set config_file parameter
1.20. Short http_client config file
1.21. Set netinterface parameter
1.22. http_connect() usage
1.23. http_connect_raw() usage
1.24. http_get_redirect() usage
1.25. http_client_query() usage

Chapter 1. Admin Guide

1. Overview

This module implements protocol functions that use the libcurl library to fetch data from external HTTP servers or post data to HTTP servers. The module is using a concept of "connections" to define properties of HTTP sessions in a simple way. A connection has one or multiple servers and a set of settings that apply to the specific connection.

The http_client module has multiple settings, some of them applies to a defined connection. You can set timeouts, max data sizes for download and much more either using modparam settings or parameters to the connection definition.

The connections can either be defined with the "httpcon" module parameter or in a separate configuration file, as specified by the "config_file" module parameter.

Like in SIP, the HTTP URL may need encoding to be transported safely over the network. Check the string encoding functions in the Transformation Cookbook (as used in the http_client_query() example below).

The function http_client_query() allows Kamailio to issue an HTTP GET request and get access to parts of the reply. This function has been ported from the utils module and now use the same libcurl functions. We recommend using the new functionality provided by this module.

The http_client module use the CURL library setting up connections. The CURL library by default use the system configured DNS resolvers, not the Kamailio resolver.

The module is limited to using HTTP and HTTPS protocols.

2. Dependencies

2.1. Kamailio Modules

The following modules must be loaded before this module:

  • TLS - if you use TLS connections (https) the tls module should be loaded first in order to initialize OpenSSL properly.

2.2. External Libraries or Applications

The following libraries or applications must be installed before running Kamailio with this module loaded:

3. Parameters

3.1. httpredirect (int)

If set to 1, enabled, http_client will follow HTTP 302 Redirects. If set to 0, http_client will not follow redirects. Default is 1, enabled.

The latest redirect URL will be stored in the $curlredirect pseudovariable.

Example 1.1. Set httpredirect parameter

...
modparam("http_client", "httpredirect", 0)
...

3.2. httpproxy (string)

URL for a HTTP proxy to use as a default proxy for all connections.

This setting is also available on a per connection basis in the http_client configuration file.

Example 1.2. Set httpproxy parameter

...
modparam("http_client", "httpproxy", "https://superproxy.example.com")
...

3.3. httpproxyport (string)

Port number for a HTTP proxy to use as a default proxy port for all connections.

This setting is also available on a per connection basis in the http_client configuration file.

Example 1.3. Set httpproxyport parameter

...
modparam("http_client", "httpproxyport", 8042)
...

3.4. useragent (string)

Useragent to use in the HTTP protocol for requests. Defaults to the Kamailio SIP useragent string - including software version and platform.

Example 1.4. Set useragent parameter

...
modparam("http_client", "useragent", "Secret HTTP REST grabber 0.42")
...

3.5. maxdatasize (int)

Defines the maximum size in bytes for a response. Note that this is allocated from pkg memory (process memory) dynamically.

Default value is zero, i.e., the limit on the datasize is disabled.

Example 1.5. Set maxdatasize parameter

...
modparam("http_client", "maxdatasize", 2000)
...

3.6. connection_timeout (int)

Defines in seconds how long Kamailio waits for response from servers.

Default value is zero, i.e., the timeout function is disabled.

Example 1.6. Set connection_timeout parameter

...
modparam("http_client", "connection_timeout", 2)
...

3.7. client_cert (string)

File name for a TLS client certificate. The certificate needs to be encoded in PEM format.

Default value is empty string, i.e. no client certificate used. Note that if you specify a client cert, you also need to specify the client_key.

Example 1.7. Set client_cert parameter

...
modparam("http_client", "client_cert", "/var/certs/sollentuna.example.com.cert")
...

3.8. client_key (string)

File name for a TLS client key. The key needs to be encoded in PEM format.

Default value is empty string, i.e. no client certificate or key is used. Note that if you specify a client key, you also need to specify the client_cert.

Example 1.8. Set client_key parameter

...
modparam("http_client", "client_key", "/var/certs/sollentuna.example.com.key")
...

3.9. cacert (string)

File name for the trusted TLS CA cert used to verify servers. The certificates need to be encoded in PEM format.

Default value is empty string, i.e. no CA certificate is used to verify the host. If tlsverifyhost is on, all TLS connections will fail without any CA certificate to validate with.

Example 1.9. Set cacert parameter

...
modparam("http_client", "cacert", "/var/certs/ca/edvina-sip-ca.pem")
...

3.10. cipher_suites (string)

List of allowed cipher suites. See http://curl.haxx.se/libcurl/c/CURLOPT_SSL_CIPHER_LIST.html for details of the cipher list curl option.

Default value is empty string, i.e. the default list of ciphers in libcurl will be used.

Example 1.10. Set cipher_suites parameter

...
modparam("http_client", "cipher_suites", "ecdhe_ecdsa_aes_128_gcm_sha_256,rsa_aes_128_gcm_sha_256")
...

3.11. verify_peer (int)

If set to 0, TLS verification of the server certificate is disabled. This means that the connection will get encrypted, but there's no authentication. There's no proof that the transmission of data is to the host that is meant to receive data.

If set to 1, default setting, and one or more CA certificates is configured, the server TLS certificate will be validated. If validation fails, the connection fails.

See the curl documentation for more details. http://curl.haxx.se/libcurl/c/CURLOPT_SSL_VERIFYPEER.html

Example 1.11. Set verify_peer parameter

...
modparam("http_client", "verify_peer", 1)
...

3.12. verify_host (int)

If set to 0, domain verification of the server certificate is disabled. This means that the connection will get encrypted but there is no check that data will be sent to the host that is meant to receive it. Disable with caution.

If set to 2, default setting, the hostname in the URL will be verified against the Common Name or Subject Alt Name in the certificate. If validation fails, the connection fails.

See the curl documentation for more details. http://curl.haxx.se/libcurl/c/CURLOPT_SSL_VERIFYHOST.html

Example 1.12. Set verify_host parameter

...
modparam("http_client", "verify_host", 2)
...

3.13. tlsversion (int)

Sets the preferred TLS/SSL version.

Valid values are:

  • 0 - Use libcurl default

  • 1 - "TLSv1"

  • 2 - "SSLv2"

  • 3 - "SSLv3"

  • 4 - "TLSv1.0"

  • 5 - "TLSv1.1"

  • 6 - "TLSv1.2"

SSL versions are now disabled by default. See the curl documentation for more details. http://curl.haxx.se/libcurl/c/CURLOPT_SSLVERSION.html

Example 1.13. Set tlsversion parameter

...
modparam("http_client", "tlsversion", 6)
...

3.14. authmethod (int)

Sets the preferred authentication mode for HTTP/HTTPS requests. The value is a bitmap and multiple methods can be used. Note that in this case, the CURL library will make an extra request to discover server-supported authentication methods. You may want to use a specific value.

Valid values are:

  • 1 - BASIC authentication

  • 2 - HTTP Digest authentication

  • 4 - GSS-Negotiate authentication

  • 8 - NTLM authentication

  • 16 - HTTP Digest with IE flavour

Default value is 3 - BASIC and Digest authentication.

This is also configurable per connection in the http_client configuration file.

Example 1.14. Set authmethod parameter

...
# Use the best of BASIC and Digest authentication.
modparam("http_client", "authmethod", 3)
...

3.15. keep_connections (int)

If an HTTP server is accessed multiple times keeping the connection open for reuse saves a significant amount of time, especially if TLS is used. If this function is enabled, the Curl library will try to reuse existing open connections. The HTTP server will have to support this feature and keep connections open for it to work properly.

Valid values are:

  • 0 - Close connections after request (default)

  • 1 - Reuse connections

This is also configurable per connection in the http_client configuration file.

Example 1.15. Set keep_connections parameter

...
modparam("http_client", "keep_connections", 1)
...

3.16. query_result (int)

Control what is returned by the http_client_query(...) in the result variable.

Valid values are:

  • 0 - Return the entire HTTP result body

  • 1 - Return the first line from HTTP result body

Default value: 1 (return first line).

Example 1.16. Set query_result parameter

...
modparam("http_client", "query_result", 0)
...

3.17. query_maxdatasize (int)

Control the size in bytes of the data to be returned by the http_client_query(...) in the result variable.

Default value: 0 (disabled, unlimited size).

Example 1.17. Set query_maxdatasize parameter

...
modparam("http_client", "query_maxdatasize", 2048)
...

3.18. httpcon (string)

Defines a connection and credentials for the connection for use in a connection-oriented function call in this module.

Syntax: <connection-name>=><schema>://[<username>:<password>@]<hostname/address>[;param=value]

The address in the URL is the base for the URL in the http_connect() call. The address given in the function call will be appended to the base URL in the connection definition.

The HTTP connection will be defined using default values in modparam's above the definition of the httpcon in the configuration file. Also note that connections can be defined in a separate text file if you have many parameters per connection, or want to use a per-connection setting that can be set in that file but not in the httpcon modparam, like authmethod.

By default, no connections are defined.

Parameters

  • useragent Useragent used for HTTP requests. Overrides useragent modparam.

  • verify_peer Set to 1 to enable or 0 to disable server certificate verification. Overrides verify_peer modparam.

  • verify_host Set to 2 to enable or 0 to disable server hostname verification. Overrides verify_host modparam.

  • client_cert Client certificate used for this connection. Overrides the default client_cert modparam.

  • client_key Client key used for this connection. Overrides the default client_key modparam.

  • cipher_suites Client certificate used for this connection. Overrides the default cipher_suite modparam.

  • timeout Timeout used for this connection. Overrides the default connection_timeout for the module.

  • tlsversion TLS version used for this connection. Overrides the default tlsversion for the module.

  • maxdatasize The maximum datasize for a response. Overrides the maxdatasize modparam setting.

  • httpredirect Set to 1 for following HTTP 302 redirect. 0 to disable. Overrides the default httpredirect modparam.

  • failover The name of another httpcon connection to use with the same arguments in case a connection with this http_con fails. Failure is either a connection failure or a response code of 500 or above.

Example 1.18. Set httpcon parameter

...
modparam("http_client", "httpcon", "apione=>http://atlanta.example.com")
modparam("http_client", "httpcon", "apitwo=>http://atlanta.example.com/api/12")
modparam("http_client", "httpcon", "apithree=>http://annabella:mysecret@atlanta.example.com/api/12")
modparam("http_client", "httpcon", "apifour=>http://stockholm.example.com/api/getstuff;timeout=12;failover=apione")
...

3.19. config_file (string)

The file name of a configuration file containing definitions of http connections. This is an alternative to the "httpcon" module parameter - especially when the number of options per line gets too big.

If the file or directory name starts with a '.' the path will be relative to the working directory (at runtime). If it starts with a '/' it will be an absolute path and if it starts with anything else the path will be relative to the main config file directory (e.g.: for kamailio -f /etc/kamailio/kamailio.cfg it will be relative to /etc/kamailio/).

The following parameters can be set in the config file, for each connection. If a parameter is not specified, the default values set by the modparams will be used.

  • url

  • username

  • password

  • authmethod

  • keep_connections

  • useragent

  • verify_peer

  • verify_host

  • client_cert

  • client_key

  • cipher_suites

  • tlsversion - Valid values are:

    • "DEFAULT"

    • "TLSv1"

    • "SSLv22

    • "SSLv3"

    • "TLSv1.0"

    • "TLSv1.1"

    • "TLSv1.2"

  • timeout

  • maxdatasize

  • http_follow_redirect

  • httpproxy

  • httpproxyport

  • failover

See the "httpcon" module parameter for explanation of these settings.

By default no config file is specified.

All the parameters that take filenames as values will be resolved using the same rules as for the tls config filename itself: starting with a '.' means relative to the working directory, a '/' means an absolute path and anything else a path relative to the directory of the current Kamailio main config file.

To set a string value to null, in order to override default settings, you can specify an value of "" - two quotation marks. In order to disable a http proxy setting you can set the port to zero.

Example 1.19. Set config_file parameter

...
modparam("http_client", "config_file", "httpconnections.cfg)
...

Example 1.20. Short http_client config file

[authapiserver]
url = https://api.runbo.example.com/v4.2/auth
timeout = 1
maxdatasize = 4
tlsversion = TLSv1.2
verify_peer = yes
client_key = default_key.pem
client_cert = default_cert.pem
http_follow_redirect = no

3.20. netinterface (string)

Set local network interface to be used for HTTP queries. It can be interface name or IP address. For more details see: https://curl.haxx.se/libcurl/c/CURLOPT_INTERFACE.html .

Default value not set.

Example 1.21. Set netinterface parameter

...
modparam("http_client", "netinterface", "eth0")
...

4. Functions

4.1.  http_connect(connection, url, [content_type, data,] result)

Sends HTTP GET or POST request to a given connection. For a POST request, content-type can be specified.

  • connection - the name of an existing HTTP connection, defined by a httpcon modparam.

    url - the part of the URL to add to the predefined URL in the connection definition.

    content_type - Used only when posting data with HTTP POST. An Internet Media type, like "application/json" or "text/plain". Will be added to the HTTP request as a header.

    data - Data or a pseudo variable holding data to be posted. (may contain pseudo variable)

    result - The name of a pseudo variable that will have the data of the response from the HTTP server.

The return value is the HTTP return code (if >=100) or the CURL error code if below 100. See the $curlerror pseudovariable below for more information about CURL error codes.

This function can be used from REQUEST_ROUTE, ONREPLY_ROUTE, FAILURE_ROUTE, and BRANCH_ROUTE.

Example 1.22. http_connect() usage

...
modparam("http_client", "httpcon", "apiserver=>http://kamailio.org/api/");
...
# POST Request
$var(res) = http_connect("apiserver", "/mailbox", "application/json", "{ ok, {200, ok}}", "$avp(gurka)");
xlog("L_INFO", "API-server HTTP connection: $avp(gurka) Result code $var(res)\n");

$var(res) = http_connect("apiserver", "/callroute", "application/json", "$var(jsondata)", "$avp(route)");
xlog("L_INFO", "API-server HTTP connection: $avp(route) Result code $var(res)\n");
...

4.2.  http_connect_raw(connection, url, content_type, data, result)

Sends HTTP POST request to a given connection. Similar to http_connect. The only difference is that the data parameter will not be parsed for pseudo variables, therefore it can safely be used for content that may contain "$" character like JSON.

  • connection - the name of an existing HTTP connection, defined by a httpcon modparam.

    url - the part of the URL to add to the predefined URL in the connection definition.

    content_type - Used only when posting data with HTTP POST. An Internet Media type, like "application/json" or "text/plain". Will be added to the HTTP request as a header.

    data - Data or a pseudo variable holding data to be posted. (will not be parsed for pseudo variable)

    result - The name of a pseudo variable that will have the data of the response from the HTTP server.

The return value is the HTTP return code (if >=100) or the CURL error code if below 100. See the $curlerror pseudovariable below for more information about CURL error codes.

This function can be used from REQUEST_ROUTE, ONREPLY_ROUTE, FAILURE_ROUTE, and BRANCH_ROUTE.

Example 1.23. http_connect_raw() usage

...
modparam("http_client", "httpcon", "apiserver=>http://kamailio.org/api/");
...
# POST Request
$var(res) = http_connect_raw("apiserver", "/mailbox", "application/json", "{ ok, {200, ok}}", "$avp(gurka)");
xlog("L_INFO", "API-server HTTP connection: $avp(gurka) Result code $var(res)\n");

$var(res) = http_connect_war("apiserver", "/callroute", "application/json", "$var(jsondata)", "$avp(route)");
xlog("L_INFO", "API-server HTTP connection: $avp(route) Result code $var(res)\n");
...

4.3.  http_get_redirect(connection, result)

When a http connection gets a redirect and the connection is configured to follow redirects (301,302) then the target URL, the result of the redirects can be retrieved with this function after a successful connection.

  • connection - the name of an existing HTTP connection, defined by a httpcon modparam.

    result - The name of a pseudo variable that will contain the last used URL.

Example 1.24. http_get_redirect() usage

...
modparam("http_client", "httpredirect", 1);
...
http_get_redirect("apiserver", "$var(targeturl)");
...

4.4.  http_client_query(url, [post-data], [hdrs], result)

Sends HTTP GET or POST request according to URL given in url parameter, which is a string that may contain pseudo variables.

If you want to make a POST-Request, you have to define the post-data, that should be submitted in that request as the second parameter.

Custom headers may be specified via hdrs parameter (e.g., Content-Type).

Either of post-data or hdrs can be also set to empty string in order to be ignored.

If HTTP server returns a class 2xx, 3xx or 4xx reply, the first line or the entire reply body (if any) is stored in result parameter, which must be a writable pseudo variable. See the query_result parameter for controling what value to be stored in the result variable.

Function returns reply code of HTTP reply or -1 if something went wrong.

This function can be used from ANY_ROUTE.

Note that this function is based on the http_query function in the utils module. It is changed to use the same base library and settings as the rest of the functions in this module.

Example 1.25. http_client_query() usage

...
# GET-Request
http_client_query("http://api.com/index.php?r_uri=$(ru{s.escape.param})&f_uri=$(fu{s.escape.param})",
	   "$var(result)");
switch ($rc) {
    ...
}
...
# POST-Request
http_client_query("http://api.com/index.php",
    "r_uri=$(ru{s.escape.param})&f_uri=$(fu{s.escape.param})",
    "$var(result)");
}
...
# POST-Request
http_client_query("http://api.com/index.php", "src=$si",
    "Content-Type: text/plain", "$var(result)");
...

5. Pseudovariables

5.1.  $curlerror(error)

The cURL library returns error codes from the protocol used. If an error happens, a cURL specific error code below 100 is returned. The $curlerror pv returns a text string representing the error. For more information on cURL error codes, please visit http://curl.haxx.se/libcurl/c/libcurl-errors.html

6. RPC Commands

6.1. httpclient.listcon

Lists all defined httpcon connections

Parameters:

  • No parameters

7. Counters

7.1.  httpclient.connections

The number of connection definitions that are in-memory.

7.2.  httpclient.connok

The number of successful connections since Kamailio start

7.3.  httpclient.connfail

The number of failed connections since Kamailio start

8. Remarks

Note: libcurl leak in CentOS 6 - this module uses libcurl library and in case if you are using CentOS 6, be aware that standard libcurl-7.19.7-52 has a memory leak. To fix this memory, install libcurl from city-fan repository. More details at: https://www.digitalocean.com/community/questions/how-to-upgrade-curl-in-centos6

Chapter 2. Developer Guide

1. Introduction

This module provides a set of API functions that other modules can use in order to integrate with HTTP services.

2. Available Functions

2.1.  int http_connect(msg, connection, url, result, content_type, post)

Sends HTTP GET or POST request to a given connection. If content_type and post are NULL GET will be used. If post is not null the data will be POSTed using the specified content_type.

Returns the status code of the HTTP response (if >= 100), or a curl error code (if < 100)

Meaning of the parameters is as follows:

  • struct sip_msg *msg

    The current sip message structure.

  • const str *connection

    The name of a preset http_con connection to use for this query.

  • const str *url

    A string that will be appended to the base URL specified in the connection. This parameter can be NULL, which means nothing will be appended to the base URL.

  • str *result

    A pointer to a string that will contain the response body. On success, the data is allocated in pkg memory by the http_client module and must be freed by the caller.

  • const char *content_type

    A null-terminated string specifying the content type to place in a Content-Type header. Use NULL when a message body is not required.

  • const str *post

    A string containing the message body to send. Use NULL when a message body is not required.

2.2.  int http_connection_exists(str *connection)

Check if a connection definition exists. Connections are defined as modparam's in the http_client modules.

Returns 1 if the connection exists, 0 if a connection with the given name can't be found.

2.3.  int http_query(msg, url, dest, post)

Sends a HTTP GET or POST request to a given connection. If post data is defined, POST will be used, otherwise GET. The default settings defined as module params of the http_client module will be used for the connection.

Meaning of the parameters is as follows:

  • struct sip_msg *msg

    The current SIP message structure.

  • const char *url

    A string that will be used as the URL specified in the connection.

  • str *dest

    A pointer to a string that will contain the first line of the response body. On success, the data is allocated in pkg memory by the http_client module and must be freed by the caller.

  • const char *post

    If not null, the data will be posted to the URL.

2.4.  http_get_content_type(str connection)

Get the content-type of the last result for this connection. This will be something like "text/html" for html or "application/json" for JSON data.

Result will be a pointer to a char string (char *). This is per process, so the connection will have to be done in the same process. Returns a NULL pointer if the connection does not exist or there's no previous connection data delivered.