Tutorials

These tutorials will help you understand the basics quickly and also covers some advanced topics that you may need later. First, please read [getting started](dev.html) with gSOAP and check out an online [example.](examples/calc/index.html) See [resources](resources.html) for frequently asked questions and our [advisories](advisory.html) if you encounter bugs. The full [documentation](docs.html) explains XML data bindings, XML/XPath, XML-RPC, JSON/JSONPath, WS-Security, and more. Understanding XML SOAP, REST, WSDL, and XML schema {#xml} --- The **wsdl2h** and **soapcpp2** tools simplify the development of applications by autocoding. Which means that the wsdl2h tool auto-generates the data binding interface file in familiar C/C++ syntax. The soapcpp2 tool then takes the interface file and auto-generates the data binding implementation code. This is the important "glue" that binds your C/C++ application data to XML and to use XML SOAP/REST services. You can also generate WSDL and XSD schema files from a C/C++ header file with soapcpp2. This means that you can start developing XML services directly from C/C++ without writing any WSDL and XSD files. We recommend to read [getting started](dev.html) with gSOAP first to understand the basics of SOAP, WSDL, XML schema, and XML. The wsdl2h and soapcpp2 tools are also used to develop [XML REST APIs.](dev.html#client-rest-cpp) Learn more about the wsdl2h and soapcpp2 tools in [XML data bindings](doc/databinding/html) and the [gSOAP user guide.](docs.html) To learn more about our new **domcpp** tool, see [XML DOM and XPath.](doc/dom/html/index.html) The domcpp tool produces high-quality, readable and reusable XML DOM source code to parse, search, manipulate, and write XML. The generated code can be readily used in your projects, thereby saving you substantial time and effort. [![To top](images/go-up.png) To top](tutorials.html) How to use JSON and JSONPath with gSOAP {#json} --- While gSOAP is primarily designed for XML, the gSOAP Web services engine and auto-coding tools are more than sufficiently powerful to develop other protocol frameworks such as JSON and XML-RPC. The `gsoap/samples/xml-rpc-json` directory in the gSOAP download package includes the JSON API for C and C++. Several JSON and XML-RPC examples are included. We also offer extensive documentation on [XML-RPC and JSON/JSONPath](doc/xml-rpc-json/html/index.html) in addition to this tutorial. A new code generation tool **jsoncpp** is included (gSOAP 2.8.26 and later). The jsoncpp tool produces high-quality, readable and reusable source code. The generated code can be readily used in your projects to populate JSON data and retrieve data, thereby saving you substantial time and effort. You may not have to write any C or C++ code to manipulate JSON data with your application's code base when taking full advantage of the jsoncpp autocoding tool. The gSOAP JSON C++ and C APIs are intuitive to use. You simply declare variables that contain JSON-based values. The values can be set by parsing JSON from streams, from files, from text buffers, or by directly assigning them. You can write values back to streams in JSON format: [codetab.C++] #include "json.h" // create a JSON value managed by context 'ctx': value v(ctx); // create a JSON "book" object with title, author, and price v["book"]["title"] = "Moby Dick"; v["book"]["author"] = "Herman Melville"; v["book"]["price"] = 22.99; // write JSON object to cout: std::cout << v; if (ctx->error) ... // handle IO error // parse JSON data from cin into 'v' managed by the context: std::cin >> v; if (ctx->error) ... // handle IO error (error info stored in 'v') [codetab.C] #include "json.h" // create a JSON value managed by context 'ctx': struct value *v = new_value(ctx); // create a JSON "book" object with title, author, and price *string_of(value_at(value_at(v, "book"), "title")) = "Moby Dick"; *string_of(value_at(value_at(v, "book"), "author")) = "Herman Melville"; *double_of(value_at(value_at(v, "book"), "price")) = 22.99; // write JSON object to stdout (or to any other open fd): ctx->sendfd = 1; // stdout fd json_write(ctx, v); if (ctx->error) ... // handle IO error // parse JSON data from stdin into 'v' managed by the context: ctx->recvfd = 0; // stdin fd json_read(ctx, v); if (ctx->error) ... // handle IO error (error info stored in 'v') The JSON object created by this code is: [json] { "book": { "title": "Moby Dick", "author": "Herman Melville", "price": 22.99 } } Memory management is automatic. You just need to set up a context and delete it later, when JSON values are no longer used: [codetab.C++] #include "json.h" struct soap *ctx = soap_new1(SOAP_C_UTFSTRING|SOAP_XML_INDENT); // new context ... value v(ctx); ... soap_destroy(ctx); // delete values soap_end(ctx); // delete temp data soap_free(ctx); // free context [codetab.C] #include "json.h" struct soap *ctx = soap_new1(SOAP_C_UTFSTRING|SOAP_XML_INDENT); // new context ... struct value *v = new_value(ctx); ... ... soap_end(ctx); // delete values and temp data soap_free(ctx); // free context If memory conservation is critical then we suggest to intermittently delete all values by calling `soap_destroy` and `soap_end` without deleting the context. If certain values must be kept then deep copy them into another managing context with `soap_dup_value` (with some conditions, see our full JSON documentation). The JSON values that you can assign are bool, int, double, string, array, and object: [codetab.C++] value v(ctx); v = 12345LL; // 64 bit int or narrower ints v = 12.34; // double float or float v = "abc"; // string (may contain UTF-8) v = string("abc"); // std::string (may contain UTF-8) v = L"xyz"; // wide string (converted to UTF-8) v = wstring(L"xyz"); // std::wstring (converted to UTF-8) v = false; // Boolean // create an array [24, 99.99, "abc"]: v[0] = 24; v[1] = 99.99; v[2] = "abc"; // create a struct (JSON object) {"name": "gsoap", "major": 2.8, "©": 2015}: v["name"] = "gsoap"; v["major"] = 2.8; v[L"©"] = 2015; // wide string tags are OK [codetab.C] struct value *v = new_value(ctx); *int_of(v) = 12345LL; // 64 bit int or narrower ints *double_of(v) = 12.34; // double float or float *string_of(v) = "abc"; // string (may contain UTF-8) *string_of(v) = soap_wchar2s(ctx, L"xyz"); // wide string (converted to UTF-8) *bool_of(v) = 0; // Boolean 0 is false *bool_of(v) = 1; // Boolean 1 is true // create an array [24, 99.99, "abc"]: *int_of(nth_value(v, 0)) = 24; *double_of(nth_value(v, 1)) = 99.99; *string_of(nth_value(v, 2)) = "abc"; // create a struct (JSON object) {"name": "gsoap", "major": 2.8, "©": 2015}: *string_of(value_at(v, "name")) = "gsoap"; *double_of(value_at(v, "major")) = 2.8; *int_of(value_atw(v, L"©")) = 2015; // wide string tags are OK Some special values for dateTime and base64 are also supported, which are not discussed here. See the full [XML-RPC and JSON/JSONPath](doc/xml-rpc-json/html/index.html) documentation. To convert a JSON value to bool, int, double, or string in C++, simply cast the value to one of these types, e.g. with `static_cast<double>(v)` to get the double float value when `v.is_double()` is true. For C only: to convert a JSON value to bool, int, double, or string, simply use one of the `bool_of`, `int_of`, `double_of`, and `string_of` functions. These always return a non-NULL pointer to a value. The internal value is converted to the target type when needed. For C++ only: you can use the `std::copy` algorithm to copy a container into a JSON array and vice versa, for example: int myints[] = { 1, 2, 3 }; v.size(3); // make value v a conformant array std::copy(myints, myints + 3, v.begin()); // copy ints into array Before casting a value or accessing an array element or object field, you may want to check the type of the value first and also obtain the array size and check the presence of fields in an object: [codetab.C++] v.is_null() // true if v is not set (JSON null) v.is_bool() // true if v is a Boolean "true" or "false" value v.is_true() // true if v is Boolean "true" v.is_false() // true if v is Boolean "false" v.is_int() // true if v is a 32 or a 64 bit int v.is_double() // true if v is a 64 bit double floating point v.is_string() // true if v is a string or wide string v.is_array() // true if v is an array v.is_struct() // true if v is a JSON object v.has(int) // true if index is within v's array bounds v.has(const char*) // true if v has the named field v.has(const wchar_t*) // true if v has the named field v.empty() // true if v is an empty array or struct v.size() // returns array or struct size or 0 v.size(int) // (re)set array size [codetab.C] is_null(v) // nonzero if v is not set (JSON null) is_bool(v) // nonzero if v is a Boolean "true" or "false" value is_true(v) // nonzero if v is Boolean "true" is_false(v) // nonzero if v is Boolean "false" is_int(v) // nonzero if v is a 32 or a 64 bit int is_double(v) // nonzero if v is a 64 bit double floating point is_string(v) // nonzero if v is a string or wide string is_array(v) // nonzero if v is an array is_struct(v) // nonzero if v is a JSON object nth_at(const char*) // returns index >= 0 if v has the named field nth_atw(const wchar_t*) // returns index >= 0 if v has the named field has_size(v) // returns array or struct size of v or 0 set_size(v, int) // (re)set array size of v As we pointed out earlier, the context manages the memory of the values that you create and parse from streams. The context also manages IO and socket connections for JSON RPC/REST services over HTTP and HTTPS: [codetab.C++] struct soap *ctx = soap_new1(SOAP_C_UTFSTRING|SOAP_XML_INDENT); value v(ctx); json_call(ctx, "https://api.github.com/orgs/Genivia/repos", NULL, &v); if (ctx->error) ... // handle IO error std::cout << v; // display JSON response for the REST GET ... soap_destroy(ctx); // delete values soap_end(ctx); // delete temp data soap_free(ctx); // free context [codetab.C] struct soap *ctx = soap_new1(SOAP_C_UTFSTRING|SOAP_XML_INDENT); struct value *v = new_value(ctx); json_call(ctx, "https://api.github.com/orgs/Genivia/repos", NULL, v); if (ctx->error) ... // handle IO error ctx->sendfd = 1; // stdout (or open() a file) json_write(ctx, v); // display JSON response of the REST GET ... soap_end(ctx); // delete values and temp data soap_free(ctx); // free context As you can see, a `json_call` takes a context, an endpoint URL (with URL query string parameters as needed), and optional `in` and `out` values to send and receive, respectively. To use the JSON REST POST method, pass both `in` and `out` values to the `json_call`. For the GET method, pass a NULL to `in`. For the PUT method, pass a NULL to `out`. For the DELETE method, pass both NULL to `in` and `out`. To compile your JSON RPC/REST C++ code with HTTPS enabled: [command] cd gsoap/samples/xml-rpc-json soapcpp2 -CSL -Ecd xml-rpc.h c++ -DWITH_OPENSSL -I../.. -o myapp myapp.cpp json.cpp xml-rpc.cpp \ soapC.cpp ../../stdsoap2.cpp -lssl -lcrypto To compile your JSON RPC/REST C code with HTTPS enabled: [command] cd gsoap/samples/xml-rpc-json soapcpp2 -c -CSL -Ecd xml-rpc.h cc -DWITH_OPENSSL -I../.. -o myapp myapp.c json.c xml-rpc.c \ soapC.c ../../stdsoap2.c -lssl -lcrypto A new code generation tool called "jsoncpp" is bundled with 2.8.27 (and later). You can use the gSOAP jsoncpp auto-coding tool to generate high-quality readable source code that uses the gSOAP JSON C or C++ API to populate JSON data and execute JSONPath queries on JSON data. You can embed C/C++ code in the JSONPath query and specify options to channel the JSONPath results to output, or collect results in a JSON array, or to specify your own C/C++ code to execute on the results. Let's use jsoncpp to generate code to retrieve the project names of a GitHub repository specified by its URL: [command] jsoncpp -m -p'$[:].name' This command generates the following C++ code for JSONPath `$[:].name`: #include "json.h" struct Namespace namespaces[] = {{NULL,NULL,NULL,NULL}}; int main(int argc, char **argv) { /* Generated by jsoncpp -m -p$[:].name */ struct soap *ctx = soap_new1(SOAP_C_UTFSTRING | SOAP_XML_INDENT); ctx->double_format = "%lG"; value x(ctx); std::cin >> x; if (x.soap->error) exit(EXIT_FAILURE); // error parsing JSON // JSONPath: $[:].name #define QUERY_YIELD(v) std::cout << v << std::endl if (x.is_array()) { int j, k = x.size()-1; for (j = 0; j <= k; j += 1) { value& r = x[j]; if (r.has("name")) { QUERY_YIELD(r["name"]); } } } soap_destroy(ctx); soap_end(ctx); soap_free(ctx); return 0; } By replacing the `std::cin >> x` JSON parsing in the above by a JSON REST call, you can run this code as a standalone command-line application that takes the URL of the repository and prints the project names by executing the JSONPath query: json_call(ctx, argv[1], NULL, &x); if (ctx->error) exit(EXIT_FAILURE); The new jsoncpp command-line tool is located in `gsoap/samples/xml-rpc-json` in the gSOAP download package. It is built together with the examples, but if you need to (re)build it then execute: [command] soapcpp2 -CSL -Ecd xml-rpc.h c++ -I../.. -o jsoncpp jsoncpp.cpp json.cpp xml-rpc.cpp \ soapC.cpp ../../stdsoap2.cpp The jsoncpp tool has several options to control the generated content, which we will not discuss here further. Finally, you can use arithmetic operators and comparisons on JSON numerical and string values in C++. You can also concatenate arrays and JSON objects: #include <sstream> std::stringstream ss("[0,1,2]"); // parse JSON array into 'x' value x(ctx); ss >> x; // x = [0,1,2] x[3] = x[1] + x[2]; // x = [0,1,2,3] value y(ctx); y[0] = "abc"; // y = ["abc"] y[1] = y[0] + 123; // y = ["abc","abc123"] std::cout << x + y; // [0,1,2,3] + ["abc","abc123"] is [0,1,2,3,"abc","abc123"] This requires gSOAP 2.8.28 or later. Arithmetic and concatenation operations incur memory overhead due to temporaries, type conversions (when applicable), and managed heap storage, so use them when memory usage is not critical. To learn more about JSON, JSONPath, the jsoncpp command, and how to compile the JSON API files together with other gSOAP XML data binding files, see the full [XML-RPC and JSON/JSONPath](doc/xml-rpc-json/html/index.html) documentation. [![To top](images/go-up.png) To top](tutorials.html) How to connect through HTTP proxies and use HTTP basic/digest authentication, NTLM authentication and WS-Security authentication {#auth} --- Suppose we have a C++ client application that uses a client proxy `calcProxy` object to talk to the server, as was discussed in [getting started.](dev.html) On Windows you can also use the [WinInet plugin](doc/wininet/html/index.html) to automate proxy settings, authentication, and other connection settings with the Internet Options of the control panel on Windows. ### HTTP digest authentication When the client must HTTP authenticate to the server, use [the gSOAP httpda plugin](doc/httpda/html/httpda.html) as follows: #include "calc.nsmap" #include "soapcalcProxy.h" #include "httpda.h" calcProxy calc("server-endpoint-URL"); double sum; soap_register_plugin(calc.soap, http_da); if (calc.add(1.23, 4.56, sum) == 401) // HTTP 401 Unauthorized { struct http_da_info authinfo; // store HTTP basic/digest auth info http_da_save(calc.soap, &authinfo, calc.soap->authrealm, "user-id", "user-pw"); if (calc.add(1.23, 4.56, sum) == SOAP_OK) // should be OK now ... http_da_restore(calc.soap, &authinfo); if (calc.add(4.56, 7.89, sum) == SOAP_OK) // should be OK ... http_da_release(calc.soap, &authinfo); // free auth info } calc.destroy(); The same mechanism works in C by setting the `soap` context as shown above. Note that `soap->authrealm` is set by the server with the fixed name of the realm that the client has to authenticate to. The digest authentication state is maintained in `authinfo` and has to be restored for each new call, otherwise the call will be rejected again with an HTTP 401 error. To compile, include the `httpda` plugin and compile the gSOAP portable mutex and threads library `threads.c` and `md5evp.c` (`smdevp.c` for gSOAP 2.8.29 and later that replaces `md5evp.c`) and the OpenSSL library as follows: [command] c++ -Iplugin -o calcclient calcclient.cpp stdsoap2.cpp \ plugin/httpda.c plugin/md5evp.c plugin/threads.c \ soapC.cpp soapcalcProxy.cpp -lssl -lcrypto Basic authentication is not recommended, unless you are communicating securely with the server over HTTPS. ### HTTP Proxy authentication Proxy authentication works the same way, but with HTTP error 407 and `http_da_proxy_X` plugin functions: if (calc.add(1.23, 4.56, sum) == 407) // HTTP 407 Proxy Authentication Required { struct http_da_info authproxy; // store HTTP basic/digest auth info http_da_proxy_save(calc.soap, &authproxy, "proxy-authrealm", "proxy-id", "proxy-pw"); if (calc.add(1.23, 4.56, sum) == SOAP_OK) // should be OK now ... http_da_proxy_restore(calc.soap, &authproxy); if (calc.add(4.56, 7.89, sum) == SOAP_OK) // should be OK ... http_da_proxy_release(calc.soap, &authproxy); // free auth info } calc.destroy(); Proxy authentication with only basic authentication (when supported by the proxy) is much simpler and does not require the `httpda` plugin or OpenSSL: calc.soap->proxy_host = "proxy-domain-or-IP"; calc.soap->proxy_port = 3128; calc.soap->proxy_userid = "proxy-id"; calc.soap->proxy_passwd = "proxy-pw"; if (calc.add(1.23, 4.56, sum) == SOAP_OK) // should be OK ... ### NTLM authentication NTLM authentication is enabled in the gSOAP engine by compiling `stdsoap2.cpp` with `-DWITH_NTLM`. The client code below illustrates NTLM authentication against a HTTP server with this option enabled: if (calc.add(1.23, 4.56, sum) == 401) // HTTP 401 Unauthorized { calc.soap->userid = "user-id"; calc.soap->passwd = "user-pw"; calc.soap->authrealm = "authrealm"; if (calc.add(1.23, 4.56, sum) == SOAP_OK) // should be OK now ... } The client code below illustrates NTLM authentication against a proxy: if (calc.add(1.23, 4.56, sum) == 407) // HTTP 407 Proxy Authentication Required { calc.soap->proxy_host = "ntlm-proxy-domain-or-IP"; calc.soap->proxy_port = 80; calc.soap->proxy_userid = "proxy-id"; calc.soap->proxy_passwd = "proxy-pw"; calc.soap->authrealm = "authrealm"; if (calc.add(1.23, 4.56, sum) == SOAP_OK) // should be OK now ... } To avoid the overhead of the initial call that fails, when we know for sure that we have to authenticate anyway, use the following trick: calc.soap->ntlm_challenge = ""; // pretend we've been challenged calc.soap->userid = "user-id"; // and/or set proxy_userid calc.soap->passwd = "user-pw"; // and/or set proxy_passwd calc.soap->authrealm = "authrealm"; if (calc.add(1.23, 4.56, sum) == SOAP_OK) // should be OK ... To compile the client, link the code against the [libntlm library](http://www.nongnu.org/libntlm/): [command] c++ -DWITH_NTLM -o calcclient calcclient.cpp soapC.cpp \ soapcalcProxy.cpp stdsoap2.cpp -lntlm ### WS-Security authentication The wsse-lite plugin is a simplified wsse plugin to support WS-Security authentication over HTTPS only. The plugin requires HTTPS to ensure that credentials are encrypted. You can use the full range of WS-Security capabilities, including digest authentication over clear HTTP, with the wsse plugin. The wsse-lite API is located in the gSOAP download package: - `gsoap/plugin/wsseapi-lite.h` include this file. - `gsoap/plugin/wsseapi-lite.c` compile and link this file (C and C++). - compile all sources with `-DWITH_OPENSSL` to enable HTTPS. - if you have zlib installed, compile all sources also with `-DWITH_GZIP`. - link with `-lssl -lcrypto -lz -gsoapssl++` (or `-lgsoapssl` for C, or compile `stdsoap2.c[pp]`). The gSOAP header file for soapcpp2 should import `wsse.h` (or the older 2002 version `wsse2.h`): #import "wsse.h" The wsdl2h tool adds the necessary imports to the generated header file if the WSDL declares the use of WS-Security. If not, you may have to add this import manually before running soapcpp2. To add a user name token with clear text password on the client side, use: #include "wsseapi-lite.h" soap_wsse_add_UsernameTokenText(soap, NULL, "username", "password"); It is strongly recommended to use `soap_wsse_add_UsernameTokenText` only in combination with HTTPS encrypted transmission or not at all because the credentials are send in the clear without HTTPS. Passwords are verified on the receiving side with `soap_wsse_verify_Password`, for example in a service operation `ns__myMethod`: int ns__myMethod(struct soap *soap, ...) { const char *username = soap_wsse_get_Username(soap); const char *password; if (!username) return soap->error; // no username: return FailedAuthentication (from soap_wsse_get_Username) password = ...; // lookup password of username if (soap_wsse_verify_Password(soap, password)) { int err = soap->error; soap_wsse_delete_Security(soap); // remove old security headers return err; // password verification failed: return FailedAuthentication } return SOAP_OK; } The `soap_wsse_get_Username` functions sets the wsse:FailedAuthentication fault upon failure. It is common for the API functions functions to return `SOAP_OK` or a wsse fault that should be passed to the sender by returning `soap->error` from service operations. See also [WS-Security Lite.](doc/wsse-lite/html/wsse.html) With plain HTTP (not HTTPS) you should use password digests instead of clear text passwords! To do so, you will need the **full WS-Security API and wsse plugin** that requires OpenSSL installed to compute digests: - `gsoap/plugin/wsseapi.h` include this file. - `gsoap/plugin/wsseapi.c` compile and link this file (C and C++). - `gsoap/plugin/smdevp.c` compile and link this file (C and C++). - `gsoap/plugin/mecevp.c` compile and link this file (C and C++). - `gsoap/plugin/threads.c` for portable threads (C and C++). - compile all sources with `-DWITH_OPENSSL -DWITH_DOM`. - if you have zlib installed, compile all sources also with `-DWITH_GZIP`. - link with `-lssl -lcrypto -lz -gsoapssl++` (or `-lgsoapssl` for C, or compile `stdsoap2.c[pp]` and `dom.c[pp]`). For example, to compile a client `myapp` application: [command] c++ -Iplugin -DWITH_OPENSSL -DWITH_DOM -DWITH_GZIP -o myapp myapp,cpp \ soapC.cpp soapClient.cpp stdsoap2.cpp dom.cpp plugin/wsseapi.c \ plugin/smdevp.c plugin/mevevp.c plugin/threads.c -lssl -lcrypto -lz To add a user name token with digest password on the client side, use: #include "wsseapi.h" soap_wsse_add_UsernameTokenDigest(soap, NULL, "username", "password"); The digest password verification on the receiving side uses the same code as shown above for `ns_myMethod`, so the verification applies to both text password authentication and digest password authentication. For full WS-Security functions with the wsse API you must register the wsse plugin with an optional security token handler: #include "wsseapi.h" soap_register_plugin(soap, soap_wsse, (void*)token_handler); For full WS-Security coverage that includes digest authentication, signatures and encryption, see [WS-Security.](doc/wsse/html/wsse.html) [![To top](images/go-up.png) To top](tutorials.html) How to enable HTTP access control (CORS) headers --- HTTP access control (CORS) is supported at the client and server sides. CORS is automatic at the server side. The server internally calls the `int fopt(struct soap*)` callback (built-in as `int http_200(struct soap*)` in stdsoap2.c and stdsoap2.cpp) to serve the OPTION method CORS requests. The built-in callback returns an HTTP 200 OK with CORS headers. The callback can be changed to point to your own server-side OPTIONS logic if needed. The default value of the CORS `Access-Control-Allow-Origin` is `*`. This default value can be changed by: struct soap *soap = soap_new(); // initialize context etc. ... soap->cors_allow = "http://example.com"; // set Access-Control-Allow-Origin header value to "http://example.com" ... soap_serve(soap); // serve requests, returning header Access-Control-Allow-Origin: "http://example.com" which means the the resource owners wish to restrict access to the resource to be only from `http://example.com`. At the client side CORS requires the OPTIONS method with CORS headers. Use the following code to send OPTIONS with CORS headers to a server: const char *server = "server-endpoint-URL"; soap->origin = "http://example.com"; soap->cors_method = "POST"; // request method is POST soap->cors_header = "..."; // list of comma-separated request headers (may be omitted) if (soap_connect_command(soap, SOAP_OPTIONS, server, NULL) || soap_end_send(soap) || soap_recv_empty_response(soap)) { soap_print_fault(soap, stderr); // an error occurred } else { soap_call_ns__myMethod(soap, server, NULL, ...); // SOAP call with POST method } See also MDN article [HTTP access control (CORS)](https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS) for details about CORS. [![To top](images/go-up.png) To top](tutorials.html) How to use HTTPS TLS/SSL with clients and stand-alone gSOAP servers {#tls} --- You must either have [OpenSSL](http://www.openssl.org) or [GNUTLS](http://www.gnutls.org) installed to use TLS/SSL for HTTPS clients and stand-alone servers. ### HTTPS client Suppose we have a C++ client application that uses a client proxy `calcProxy` object to talk to the server, as was discussed in [getting started.](dev.html) Change the code as follows to enable the client to communicate with TLS/SSL over HTTPS: #include "calc.nsmap" #include "soapcalcProxy.h" calcProxy calc("https-server-endpoint-URL"); double sum; soap_ssl_init(); // init SSL (just need to do this once) if (soap_ssl_client_context(calc.soap, SOAP_SSL_DEFAULT, NULL, // no keyfile NULL, // no keyfile password "cacerts.pem", // trusted certificates (or use self-signed cacert.pem) NULL, // no capath to trusted certificates NULL // no random data to seed randomness )) { calc.soap_stream_fault(std::cerr); exit(1); } if (calc.add(1.23, 4.56, sum) == SOAP_OK) ... To compile the client, enable OpenSSL with `-DWITH_OPENSSL`: [command] c++ -DWITH_OPENSSL -o calcclient calcclient.cpp soapC.cpp \ soapcalcProxy.cpp stdsoap2.cpp -lssl -lcrypto or enable GNUTLS with `-DWITH_GNUTLS` instead of OpenSSL: [command] c++ -DWITH_GNUTLS -o calcclient calcclient.cpp soapC.cpp \ soapcalcProxy.cpp stdsoap2.cpp -lgnutls -lgcrypt This example does not authenticate the client to the server. To authenticate clients to servers, please see the [gSOAP user guide](docs.html) and the `gsoap/samples/ssl` example in the gSOAP download package. In fact, without the `soap_ssl_init()` and `soap_ssl_client_context()` calls the engine will still connect to HTTPS services and the messages will be encrypted, but the server certificate is not authenticated by the client. ### HTTPS stand-alone server Suppose we have a C++ stand-alone application that uses a server class `calcService` object, as was discussed in [getting started.](dev.html) We change the code as follows to enable the iterative server to communicate with TLS/SSL over HTTPS by binding a port and accepting TLS/SSL connections to serve: #include "calc.nsmap" #include "soapcalcService.h" int main() { calcService calc; calc.soap->accept_timeout = 24*60*60; // quit after 24h of inactivity calc.soap->send_timeout = calc.soap->recv_timeout = 10; soap_ssl_init(); // init SSL (just need to do this once) if (soap_ssl_server_context(calc.soap, SOAP_SSL_DEFAULT, "server.pem", // server keyfile (cert+key) "password", // password to read the private key in the keyfile NULL, // no cert to authenticate clients NULL, // no capath to trusted certificates NULL, // DH/RSA: use 2048 bit RSA (default with NULL) NULL, // no random data to seed randomness NULL // no SSL session cache )) { calc.soap_stream_fault(std::cerr); exit(1); } if (soap_valid_socket(calc.bind(NULL, 8080, 100))) { while (soap_valid_socket(calc.accept())) { if (calc.ssl_accept() || calc.serve()) calc.soap_stream_fault(std::cerr); calc.destroy(); // free deserialized data } } calc.soap_stream_fault(std::cerr); calc.destroy(); return 0; } Newer versions of gSOAP include a `ssl_run()` method, so the above conditional loop can be rewritten to: while (calc.ssl_run(8080) != SOAP_OK) calc.soap_stream_fault(std::cerr); To compile the server, enable OpenSSL with `-DWITH_OPENSSL`: [command] c++ -DWITH_OPENSSL -o calcserver calcserver.cpp soapC.cpp \ soapcalcService.cpp stdsoap2.cpp -lssl -lcrypto or enable GNUTLS with `-DWITH_GNUTLS` instead of OpenSSL: [command] c++ -DWITH_GNUTLS -o calcserver calcserver.cpp soapC.cpp \ soapcalcService.cpp stdsoap2.cpp -lgnutls -lgcrypt Multi-threaded servers require mutex locks for OpenSSL, see the [gSOAP user guide](docs.html) and the `gsoap/samples/ssl` example in the gSOAP download package. ### HTTP TLS/SSL options The `SOAP_SSL_DEFAULT` flag used with `soap_ssl_client_context` and `soap_ssl_server_context` enables TLS v1.0, v1.1, and v1.2. It also requires server authentication to clients by means of the certificate included with the `server.pem` parameter of `soap_ssl_server_context`. Additional client-side and server-side options to set the TLS/SSL context are: - `SOAP_SSL_NO_AUTHENTICATION` disables peer authentication - `SOAP_SSL_REQUIRE_SERVER_AUTHENTICATION` requires servers to authenticate to the client (default) - `SOAP_SSL_REQUIRE_CLIENT_AUTHENTICATION` requires clients to authenticate to the server - `SOAP_SSL_SKIP_HOST_CHECK` disables checking of the common name of the host in certificate - `SOAP_SSL_ALLOW_EXPIRED_CERTIFICATE` disables checking of the expiration date of the certificate and omit CRL checks - `SOAP_SSL_NO_DEFAULT_CA_PATH` disables `default_verify_paths` (OpenSSL) - `SOAP_TLSv1` enables TLS v1.0/1.1/1.2 (default) - `SOAP_SSLv3_TLSv1` enables SSL v3 and TLS v1.0/1.1/1.2 - `SOAP_SSLv3` restricts SSL to SSL v3 only - `SOAP_TLSv1_0` restricts TLS to TLS v1.0 only - `SOAP_TLSv1_1` restricts TLS to TLS v1.1 only - `SOAP_TLSv1_2` restricts TLS to TLS v1.2 only Multiple options can be combined with `|` as in `SOAP_SSL_SKIP_HOST_CHECK | SOAP_SSL_ALLOW_EXPIRED_CERTIFICATE`, except for the TLS/SSL version restrictions of which only one of these options can be used. The DH/RSA server-side parameter of `soap_ssl_server_context` specifies a Diffie-Hellman (DH) key. When `NULL`, an ephemeral RSA key is generated. If this string parameter is the name of a file, say `"dh2048.pem"`, then the file is accessed to obtain the DH key from that PEM file. If the string parameter is a number, say `"4096"`, then an ephemeral DH key of that bit size will be generated. When the DH/RSA parameter is`NULL`, RSA keys of 2048 bits will be used (number of bits is defined by the `SOAP_SSL_RSA_BITS` macro). ### CRL checking CRL checking requires OpenSSL 0.9.8 or higher, or GNUTLS. To enable CRL checking, use `soap_ssl_crl` as follows: if (soap_ssl_crl(soap, "crl.pem")) // load CRLs from file in PEM format ... // error loading CRLs This verifies certificates against the revocation list. Use an empty string argument with `soap_ssl_crl()` to enable checking of CRLs that are provided by CAs and embedded in certificates: soap_ssl_crl(soap, "") If you enable a CRL on a context with OpenSSL, any certificate whose CA does not have a CRL will be rejected. Setting a verification callback can work around these issues. ### OpenSSL verification callback To set a cerfificate verification callback for OpenSSL, define a callback function as follows. The function returns `ok = 1` to attempt to override the error, for example when no CRL is provided: soap->fsslverify = verify_callback; int ssl_verify_callback(int ok, X509_STORE_CTX *store) { if (!ok) { int err = X509_STORE_CTX_get_error(store); switch (err) { case X509_V_ERR_UNABLE_TO_GET_CRL: X509_STORE_CTX_set_error(store, X509_V_OK); ok = 1; } } return ok; } The callback can also be used to report issues with certificates, such as self-signed certificates, and possibly accept them anyway if the application's security policy permits. ### OpenSSL and MT-Safety OpenSSL requires locks to support multi-threading. Use the gSOAP portable threads in `plugin/threads.h` and `plugin/threads.c` located in the gSOAP package to add OpenSSL locks as follows: #include "plugin/threads.h" int CRYPTO_thread_setup(); void CRYPTO_thread_cleanup(); soap_ssl_init(); if (CRYPTO_thread_setup()) // OpenSSL thread mutex setup ... // error ... // code goes here CRYPTO_thread_cleanup(); // OpenSSL thread mutex cleanup exit(0); // exit program The OpenSSL-specific code for these setup and cleanup functions is not shown here. Get it from [`thread_setup.c`](files/thread_setup.c) to add to your code base. [![To top](images/go-up.png) To top](tutorials.html) How to create self-signed certificates with OpenSSL and gSOAP {#cert} --- To generate a self-signed root certificate to sign client and server certificates, first create a new private directory, say `CA` for "Certificate Authority", to store private keys and certificates. Next, copy [`openssl.cnf`](files/openssl.cnf.txt), [`root.sh`](files/root.sh.txt), and [`cert.sh`](files/cert.sh.txt) to this directory (these files are also included in the gSOAP download package under `gsoap/samples/ssl`). Edit `openssl.cnf` in the `[req_distinguished_name]` section and change the following items: [command] [ req_distinguished_name ] countryName_default = US stateOrProvinceName_default = Your-State localityName_default = Your-City 0.organizationName_default = Your-Company-Name emailAddress_default = your-email@address If you are going to use these settings often, then we suggest to add this line to your `.cshrc` or `.tcshrc`: [command] setenv OPENSSL_CONF $HOME/CA/openssl.cnf To generate the root CA, execute: [command] ./root.sh When prompted, choose a passphrase to protect the CA's private key that you are about to generate. You need the passphrase again when you sign certificates with the CA's private key. Now you have a new `root.pem` with the CA's private key. Save the generated `root.pem` keyfile and the passphrase in a safe place (don't distribute!). Use the generated `cacert.pem` certificate of the CA for distribution so that peers (web browsers and other clients) can authenticate the service's certificates that are signed with it. The `root.pem` and `cacert.pem` are valid for three years (1095 days as set in `openssl.cnf`). You don't need to repeat these steps until the certificate expires. Next, generate the `server.pem` keyfile and sign it with the root CA: [command] ./cert.sh server Enter a password when prompted and enter the host name or simply `localhost` for the domain of the server application. The password is used to lock the private key of the server and will therefore be needed by your server application to unlock the private key in the `server.pem` keyfile when needed for secure TLS/SSL communications. Use the root CA passphrase when prompted to sign the server certificate. When applicable, repeat the procedure for the client (use a fresh password and select a host for the client application): [command] ./cert.sh client The `server.pem` and `client.pem` keys are valid for one year. Do not distribute them (they include the private key, which is encrypted with the passwords you selected which is not very secure). They are used only locally by the TLS/SSL application. Only distribute the CA certificate `cacert.pem` which is needed by peers to authenticate services (that internally use `server.pem`) and clients (that internally use `client.pem`). To print the contents of a PEM file: [command] openssl x509 -text -in file.pem To generate parameters for DH (Diffie Hellman) key exchange with OpenSSL, use: [command] openssl dhparam -out dh2048.pem -2 2048 To summarize, the files you need are: [command] openssl.cnf root.sh cert.sh Files generated: [command] cacert.pem root's certificate for distribution, for authentication root.pem root CA (to sign client/server key files, do not distribute!) rootkey.pem private key (do not distribute!) rootreq.pem sign request root.srl serial number client.pem client key with private key and certificate (do not distribute!) clientcert.pem client certificate signed by root CA (public) clientkey.pem private key (do not distribute!) clientreq.pem sign request server.pem server key with private key and certificate (do not distribute!) servercert.pem server certificate, signed by root CA, for distribution serverkey.pem private key (do not distribute!) serverreq.pem sign request [![To top](images/go-up.png) To top](tutorials.html) How to convert certificates in PEM format to CER format for MS Windows {#pem} --- To convert certificate `cacert.pem` to CER format: [command] openssl x509 -in cacert.pem -outform der -out cacert.cer Install the certificate on MS Windows by opening it and then select "Install Certificate". Client applications running on MS Windows can now connect to the server. The server authenticates to the client by means of the certificate. [![To top](images/go-up.png) To top](tutorials.html) How to create self-signed certificates with GNUTLS {#gnutls} --- Use the [GNUTLS `certtool`](http://www.gnutls.org/manual/html_node/certtool-Invocation.html#certtool-Invocation) command (or the `gnutls-certtool` command) to create keys and certificates as follows. First, generate a private key (for a client or server): [command] certtool --generate-privkey --outfile privkey.pem Make sure to use GNUTLS `certtool`, sometimes renamed to `gnutls-certtool` to avoid confusion with the system `certtool` command. Then self-sign the certificate: [command] certtool --generate-self-signed --load-privkey privkey.pem --outfile cert.pem When prompted, the following options are recommended for the certificate: - The certificate MUST NOT be marked as a CA certificate. - The DNS name MUST match the FQDN that clients will use to access the server. Use the server domain name here. One name suffices. - The certificate MUST be marked as usable for encryption. - The certificate MUST be marked as usable for a TLS server (or client when appropriate). The `client.pem` and `server.pem` keyfiles used in the `soap_ssl_client_context()` and `soap_ssl_server_context()` gSOAP API functions are a combination of the private key and the certificate: [command] cat privkey.pem cert.pem > server.pem The client can use `cert.pem` to authenticate the server. The private key file and server.pem are for the server only and SHOULD NEVER be shared. Note that the `server.pem` file generated above is NOT encrypted with a password, so the password parameter of `soap_ssl_server_context()` is not used. Neither is the `capath` parameter used for the fact that GNUTLS does not search for loadable certificates. The PEM files produced by GNUTLS can be used with OpenSSL. The PEM key files created with OpenSSL (such as `server.pem` and `client.pem`) CANNOT be used with GNUTLS, because they contain encrypted private keys that GNUTLS cannot decrypt to read ("SSL/TLS error: Can't read key file"). You can also use the GNUTLS `certtool` to create a Certificate Authority (CA) to sign client and server certificates. [command] certtool --generate-privkey --outfile cakey.pem Then self-sign the CA certificate: [command] certtool --generate-self-signed --load-privkey cakey.pem --outfile cacert.pem When prompted, the following options are recommended for the CA certificate: - The CA certificate SHOULD be marked to belong to an authority. - The CA certificate MUST be marked as a CA certificate. - The CA certificate MUST be usable for signing other certificates. - The CA certificate MUST be usable for signing Certificate Revocation Lists (CRLs). Now create a server key and use the CA to sign the server's certificate: [command] certtool --generate-privkey --outfile serverkey.pem certtool --generate-request --load-privkey serverkey.pem --outfile server.csr certtool --generate-certificate --load-request server.csr --load-ca-certificate cacert.pem --load-ca-privkey cakey.pem --outfile servercert.pem Use the recommended options discussed earlier for creating the certificate. The `client.pem` and `server.pem` keyfiles used in the `soap_ssl_client_context()` and `soap_ssl_server_context()` gSOAP API functions is a combination of the private key and the certificate: [command] cat serverkey.pem servercert.pem > server.pem The procedure above can be repeated to create a key and signed certificate for clients and other servers. All clients and servers can be authenticated with the CA certificate `cacert.pem`. That is, `cacert.pem` is to be used by all peers that require the other party to authenticate (e.g. the client uses cacert.pem CA cert to authenticate the server, who uses the `server.pem` keyfile). To generate parameters for DH (Diffie Hellman) key exchange with GNUTLS, use: [command] certtool --generate-dh-params --bits 2048 --outfile dh2048.pem [![To top](images/go-up.png) To top](tutorials.html)