Membrane Talk

New Documentation Site

October 2, 2023 tpolley Leave a comment

A new website has been setup covering Membrane API Gateway has been set up. (“Membrane Service Proxy” was renamed to “Membrane API Gateway” in the mean time.)

Categories: General

Protect your REST resources using Membrane’s OAuth2 feature

July 22, 2013 tpolley 2 comments

Summary

With the soon-to-be-published Membrane 4.0.7 you will be able to use OAuth2 with Google’s Authentication Service to restrict HTTP access to any REST resources.

Note

This setup takes 5 minutes if you have a Google Account and Membrane Service Proxy already downloaded. Please note that Membrane Service Proxy 4.0.7 has not yet been released as of writing, but as it is Apache-licensed open source, you are free to get the source from http://github.com/membrane/service-proxy/ and build it yourself (simply use mvn package -DskipTests) .

The Workflow

(If you want, you can skip straight to the “Brief Setup How-To” section.)

As the full OAuth2 workflow is a bit complex, we explain a simplified use case:

The user requests the secret resource. (Think GET http://localhost:8080/secret/resource.) As she is not yet authorized, access is denied.
The “denied” response contains a link to a login form (a redirect to https://www.google.com/...).
The user tells Google that she wants Membrane Service Proxy to be able to authenticate her using her email address.
Google returns an authorization token and a redirect to the original secret resource (http://localhost:8080/secret/resource).
The user re-requests the secret resource. In contrast to step 1, the request now contains an authorization token.
Membrane’s oauth2-google adapter verifies the authorization token.
Google returns the user’s email address.
As the user is now authorized, Membrane forwards the user’s HTTP request to the server hosting the secret resource.
The secret response is received by Membrane and
forwarded to the user without any further processing.

Brief Setup How-To

Google Project setup
1. Go to https://code.google.com/apis/console/.
2. Create a new project.
3. Under “API Access”, create an “OAuth2 Client ID…”.
4. Fill in the “Branding Information”.
5. Choose “Web Application” and enter the Membrane’s URL ( in our example “http://localhost:8080/“).
6. Remember the “Client Id” (without the trailing “.apps.googleusercontent.com“) as well as the “Client Secret”.

Membrane setup

Download and unpack Membrane 4.0.7 or above.
Change conf/proxies.xml to contain

<spring:beans xmlns="http://membrane-soa.org/proxies/1/"
  xmlns:spring="http://www.springframework.org/schema/beans"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
    http://membrane-soa.org/proxies/1/ http://membrane-soa.org/schemas/proxies-1.xsd">

  <router>

    <serviceProxy name="localhost" port="8080" >
      <oauth2Resource publicURL="http://localhost:8080/">
        <google clientId="396[...]897-3g7[...]jkb" clientSecret="dTs[...]agb" />
      </oauth2Resource>

      <groovy>
        def email = exc.request.header.getFirstValue("X-Authenticated-Email")
        exc.response = Response.ok("Hello " + email + ".").build()
        RETURN
      </groovy>
    </serviceProxy>

  </router>

</spring:beans>

Replace the publicURL in line 10 by the URL Membrane can be reached on. In our case this is already correct: http://localhost:8080/.
Replace the clientId and clientSecret with the ones from the Google API console.
(Note: the clientId might be numbers-only in your case.)
Start Membrane Service Proxy by executing service-proxy.bat or service-proxy.sh.
Open http://localhost:8080/ in a browser.

Closing comments

You have seen how quickly OAuth2 authorization can be set up using Membrane Service Proxy.

In our setup, we used a simple, dynamically generated “Hello <your-gmail-address>.” page as the secret resource. The secret resource does not have to be hosted within Membrane, but can reside on any other HTTP server Membrane has access to, including localhost.

We did not perform any further checking of the user’s email address to protect our secret resource. In practice, you should do that, as getting possession of a valid gmail login is not really a challenge. ;)
The web server hosting the secret resource should probably be protected by a firewall so that the user cannot access the secret resource directly by circumventing Membrane.
You should use HTTPS for all untrusted network connections. Membrane Service Proxy automatically takes care of using HTTPS for steps 3, 4, 6 and 7. Membrane’s administrator should install SSL certificates and adjust the proxies.xml for all other steps, if necessary.
If the user (identified by a session ID cookie) issues any further requests, only steps 5, 8, 9 and 10 will take place.
If the session timed out, steps 3-5 might occur automatically, if Google decides that the user already has granted trust to access the user’s email address to Membrane.

Thank you for following us. If you are interested, you are welcome to read about all the other features of Membrane Service Proxy.

Categories: Router

Versioning SOAP Web Services using a CBR and XSLT

November 21, 2012 tpolley Leave a comment

Summary

In this blog post we discuss the evolution of a web service, and how multiple versions of it using the same endpoint can be handled by configuring Membrane ESB to act as an Adapter using XSLT (Extensible Stylesheet Language Transformations).

The Scenario

The scenario is the same as in our previous post, so you should read its first three sections,

The Web Service
Modifying the Web Service
The Core of the Question,

to get to know our “ContactService” as well as the “email” property we added to it after its initial publication.

We again answer the questions posed when we modified the Web Service,

Did we change the published endpoint URL?
Did we change the SOAP namespace URI?

with “no, yes”.

But this time, we take a different approach implementing a solution.

Adapting old Web Service Requests to the New Interface

To avoid having to maintain the old (“v11” or “version 1”) implementation of our Web Service on the backend, we make use of the possibility to transform old requests into new ones:

Given the request

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:con="http://predic8.com/contactService/v11">
  <soapenv:Header/>
  <soapenv:Body>
    <con:addContact>
      <firstname>John</firstname>
      <lastname>Doe</lastname>
    </con:addContact>
  </soapenv:Body>
</soapenv:Envelope>

we add an empty element “email” and change the namespace to “v20”:

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:con="http://predic8.com/contactService/v20">
  <soapenv:Header/>
  <soapenv:Body>
    <con:addContact>
      <firstname>John</firstname>
      <lastname>Doe</lastname>
      <email></email>
    </con:addContact>
  </soapenv:Body>
</soapenv:Envelope>

Using XSLT

This transformation can be accomplished automatically using a few lines of XSLT, which we write into a file called v11-to-v20.xslt:

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

	<xsl:output omit-xml-declaration="yes" />
	
	<!-- map all old elements to new ones, add add 'email' as child to 'addContact' -->
	<xsl:template match="*[namespace-uri() = 'http://predic8.com/contactService/v11']">
		<xsl:element name="{name()}" namespace="http://predic8.com/contactService/v20">
			<xsl:apply-templates select="node()|@*"/>
			<xsl:if test="local-name() = 'addContact'">
				<xsl:element name="email" />
			</xsl:if>
		</xsl:element>
	</xsl:template>

	<!-- map all old attributes to new ones -->
	<xsl:template match="@*[namespace-uri() = 'http://predic8.com/contactService/v11']">
		<xsl:attribute name="{name()}" namespace="http://predic8.com/contactService/v20">
			<xsl:value-of select="." />
		</xsl:attribute>
	</xsl:template>

	<!-- leave other elements alone -->
	<xsl:template match="*|@*|text()">
		<xsl:copy><xsl:apply-templates select="*|@*|text()"/></xsl:copy>
	</xsl:template>

</xsl:stylesheet>

Set up Membrane ESB as CBR with XSLT

We now set up Membrane ESB to first distinguish between old and new requests (the same Content Based Router setup as in the previous tutorial). Additionally we tell Membrane to perform the XSLT transformation on the old requests:

For this, we download the distribution from the website, and change conf/proxies.xml to

<proxies xmlns="http://membrane-soa.org/schemas/proxies/v1/"
	 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      	 xsi:schemaLocation="http://membrane-soa.org/schemas/proxies/v1/ http://membrane-soa.org/schemas/proxies/v1/proxies.xsd">

	<soapProxy port="80" wsdl="http://localhost:8080/ContactService/v20?wsdl">
		<path>/ContactService</path>
	    <switch>
	        <case xPath="//*[contains(namespace-uri(), '/contactService/v11')]"
	                service="v11-to-v20" />
	    </switch>
	</soapProxy>
	 
	<soapProxy port="80" wsdl="http://localhost:8080/ContactService/v20?wsdl"
			name="v11-to-v20">
		<path>/ContactService</path>
	    <request>
	        <transform xslt="conf/v11-to-v20.xslt"/>
	    </request>
	</soapProxy>

</proxies>

and put v11-to-v20.xslt into the conf directory.

Done!

(Starting Membrane with memrouter.bat will open port 80 and forward incoming requests to the specified URLs: We assume here that the server implementing the new interfaces is running on localhost:8080/ContactService/v20.

A full working example of the versioned web service together with the CBR+XSLT configuration can be found in examples/versioning/xslt in the distribution archive of Membrane ESB.

Categories: General, Router

Converting JSON REST Requests to SOAP

August 30, 2012 tpolley Leave a comment

Summary

Today, we make a SOAP Web Service accessible as a JSON REST resource using Membrane ESB.

This can, for example, help to make your SOA available to mobile devices, as JSON processing usually uses less resources than fully-fledged SOAP.

The SOAP Web Service

The sample Web Service is called ArticleService and is used to maintain a list of Articles identified by their ID. Each article has a name, a description and a price (amount and currency).

Its WSDL contains, among other things, the list of the service’s operations:

create: creates a new Article

A new article is described using the following format:

<article>
  <id>AR-00000</id>
  <name>?XXX?</name>
  <description>?XXX?</description>
  <price>
    <amount>?999.99?</amount>
    <!-- possible value: EUR, possible value: USD -->
    <currency>???</currency>
  </price>
</article>

An ID will be automatically generated for the new article and is returned in the SOAP response.

get: retrieves information about one Article, given its ID.
The ID is specified using the following format:
```

<id>???</id>
```
getAll: retrieves information about all articles.
delete: deletes information of one Article, given its ID.
The request format is the same as for the get operation.

Translation to REST

For the advertised functionality, we define the following HTTP methods, to manage the REST-style resources, “articles“.

POST /articles
Creates a new article. The message entity uses JSON encoding to describe the article:
```
{
  "article": {
    "name":"Darjeeling",
    "description":"FTGFOP1, 1kg",
    "price": {"amount":21.70, "currency":"EUR"},
    "id":"AR-00000"
  }
}
```
HTTP status code “204 No Content” is returned and the URL of the new resource is returned in the “Location” HTTP header field.
GET /articles/AR-00001
Retrieves the article resource specified by the ID. HTTP status code “200 OK” and the JSON-encoded resource are returned. If no article with the given ID exists, HTTP status code “404 Not found” is returned.
GET /articles
Retrieves a list of all article resources. Returns HTTP status code “200 OK” and the JSON-encoded list.
DELETE /articles/AR-00001
Deletes the specified article. HTTP status code “204 No content” is returned, even if no article with the given ID existed.

Any HTTP status code >=400 indicates an error, as per the HTTP specification.

Using Membrane’s rest2Soap Feature

Membrane ESB offers out-of-the-box a rest2Soap feature with JSON support.

Monitoring the Progression of a Sample Request

For example, we send a REST request to Membrane to create an article. (A request of “REST Type 1”, which we just defined.)

POST /articles/ HTTP/1.1
User-Agent: curl/7.23.1 (x86_64-pc-win32) libcurl/7.23.1 OpenSSL/0.9.8r zlib/1.2.5
Host: localhost:2000
Accept: */*
Content-Type: application/json
Content-Length: 102
X-Forwarded-For: 127.0.0.1

{"article":{"name":"Test","description":"?","price":{"amount":1.00,"currency":"EUR"},"id":"AR-00000"}}

Membrane automatically translates the request into an XML format representing the HTTP request, called the “in-between XML document”:

<?xml version="1.0" ?>
<http:request xmlns:http="http://membrane-soa.org/schemas/http/v1/"
	method="POST" http-version="1.1">
	<uri value="/articles/">
		<path>
			<component>articles</component>
		</path>
	</uri>
	<headers>
		<header name="User-Agent">curl/7.23.1 (x86_64-pc-win32) libcurl/7.23.1
			OpenSSL/0.9.8r zlib/1.2.5</header>
		<header name="Host">localhost:2000</header>
		<header name="Accept">*/*</header>
		<header name="Content-Type">application/json</header>
		<header name="Content-Length">102</header>
		<header name="X-Forwarded-For">127.0.0.1</header>
	</headers>
	<body type="json">
		<root type="o">
			<article type="o">
				<name type="s">Test</name>
				<description type="s">?</description>
				<price type="o">
					<amount type="f">1.00</amount>
					<currency type="s">EUR</currency>
				</price>
				<id type="s">AR-00000</id>
			</article>
		</root>
	</body>
</http:request>

This document fully describes the request at the point in the diagram marked by the asterisk ‘*‘. As you can see, every piece of data from the HTTP request was mapped to some element or attribute. The JSON body of the request was also parsed and translated into XML. (The same also automagically works with XML bodies – although no “translation” is really needed in this case: The XML document from the request body would simply be inserted below the <body> node, with type="xml".)

This XML document is then used as input for the XSL Transformation. The fine-grained structure now makes it very easy to access specific data, which keeps the stylesheets very brief and concise, as well as easy to develop.

Configuring the XSLT component

At this point, to comply with the REST interface described above, we manually need to configure the XSLT component of the rest2soap feature, mapping REST requests to SOAP operations. We create a file “request.xsl” describing the translation of the “in-between XML document” to SOAP:

<pre><?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
							  xmlns:s11="http://schemas.xmlsoap.org/soap/envelope/"
							  xmlns:as="http://predic8.com/wsdl/material/ArticleService/1/"
							  xmlns:http="http://membrane-soa.org/schemas/http/v1/">
							  
	<xsl:template match="/">
		<s11:Envelope >
			<s11:Body>
				<!-- switch over operations -->
				<xsl:choose>
					<xsl:when test="/*/@method = 'GET' and count(//path/component) = 1">
						<as:getAll />
					</xsl:when>
					<xsl:when test="/*/@method = 'GET'">
						<as:get>
							<id><xsl:value-of select="(//path/component)[last()]"/></id>
						</as:get>
					</xsl:when>
					<xsl:when test="/*/@method = 'DELETE'">
						<as:delete>
							<id><xsl:value-of select="(//path/component)[last()]"/></id>
						</as:delete>
					</xsl:when>
					<xsl:when test="/*/@method = 'POST'">
						<as:create>
							<xsl:apply-templates select="/*/body/root/*" />
						</as:create>
					</xsl:when>
				</xsl:choose>
			</s11:Body>
		</s11:Envelope>	
	</xsl:template>

	<!-- remove 'type' attribute: it's an artifact of the JSON2XML converter -->
	<xsl:template match="@type" />
	
	<!-- leave other elements alone -->
	<xsl:template match="*|@*|text()">
		<xsl:copy><xsl:apply-templates select="*|@*|text()"/></xsl:copy>
	</xsl:template>
	
</xsl:stylesheet>

Basically each type of REST request is mapped to a SOAP operation.

We then create a second file called “response.xsl” with the following content to handle SOAP responses:

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
							  xmlns:s11="http://schemas.xmlsoap.org/soap/envelope/"
							  xmlns:http="http://membrane-soa.org/schemas/http/v1/">
	
	<xsl:template match="/s11:Envelope">
		<http:response>
			<xsl:choose>

			<!-- status 500 on SOAP Faults -->
			<xsl:when test="/*/*/*[local-name() = 'Fault']">
				<status code="500">Internal SOAP Server Error</status>
				<body type="xml">
					<response><xsl:apply-templates select="/*/*/*/*" /></response>
				</body>
			</xsl:when>

			<xsl:when test="/*/*/*/*[local-name() = 'createResponse']">
				<status code="201">Created</status>
				<headers>
					<!-- use ID from SOAP response for HTTP "Location" header -->
					<header name="Location">/articles/<xsl:value-of select="/*/*/*/id" /></header>
					<header name="Content-Type" remove="true" />
				</headers>
				<body type="plain"></body>
			</xsl:when>

			<xsl:when test="//*[local-name() = 'deleteResponse']">
				<status code="204">No Content</status>
				<headers>
					<header name="Content-Type" remove="true" />
				</headers>
				<body type="plain"></body>
			</xsl:when>

			<xsl:when test="//*[local-name() = 'getResponse']">
				<!-- set status 404, if response is empty -->
				<xsl:if test="count(/*/*/*/*) = 0">
					<status code="404">Not found</status>
				</xsl:if>
				<body type="xml">
					<response><xsl:apply-templates select="/*/*/*/*" /></response>
				</body>
			</xsl:when>

			<xsl:otherwise>
				<!-- unwrap the SOAP message content -->
				<body type="xml">
					<response><xsl:apply-templates select="/*/*/*/*" /></response>
				</body>
			</xsl:otherwise>

			</xsl:choose>
		</http:response>
	</xsl:template>

	<!-- leave other elements alone -->
	<xsl:template match="*|@*|text()">
		<xsl:copy><xsl:apply-templates select="*|@*|text()"/></xsl:copy>
	</xsl:template>
	
</xsl:stylesheet>

Again, we basically just translated the rules we defined above (in the SOAP and REST section) concerning the HTTP responses (status codes, headers, etc.) into XSLT.

Download and set up Membrane ESB

We now download the Membrane ESB distribution (3.5.2 or later) from the website, and change conf/proxies.xml to

<proxies xmlns="http://membrane-soa.org/schemas/proxies/v1/"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="http://membrane-soa.org/schemas/proxies/v1/ http://membrane-soa.org/schemas/proxies/v1/proxies.xsd">
 
 	<serviceProxy port="2000">
 		<rest2Soap>
 			<mapping 
 				regex="/articles.*" 
 				soapURI="/material/ArticleService"
 				requestXSLT="conf/request.xsl"
 				responseXSLT="conf/response.xsl"
 				soapAction=""
 				responseType="json" />
 		</rest2Soap>
 		<target host="www.predic8.com" />
 	</serviceProxy>
 
</proxies>

We put request.xsl and response.xsl into the conf directory of the distribution.

We then start Membrane using memrouter.bat.

Done!

Testing it

Using the curl utility on Windows, we can issue the following commands:

C:\ > curl http://localhost:2000/articles/
{"response":[{"name":"Black Tea","description":"English Breakfast Black Tea","price":{"amount":4,"currency":"EUR"},"id":"AR-00001"},{"name": "Assam Tea","description":"Finest Assam Tea","price":{"amount":2.5,"currency":"EUR"},"id":"AR-00002"}]}

C:\ > curl --header "Content-Type: application/json" -d "{\"article\":{\"name\":\"Test\",\"description\":\"?\",\"price\":{\"amount\":1.00,\"currency\":\"EUR\"},\"id\":\"AR-00000\"}}" http://localhost:2000/articles/
{"response":{"id":"AR-00043"}}

C:\ > curl http://localhost:2000/articles/AR-00043
{"response":{"article":{"name":"Test","description":"?","price":{"amount":1.00,"currency":"EUR"},"id":"AR-00043"}}}

C:\ > curl -X DELETE http://localhost:2000/articles/AR-00043
C:\ >

Mapping REST to Your SOAP Web Service

To realize a REST interface for another SOAP web service, or to proxy other operations, some tweeks to the stylesheets above are needed.

A good way to develop the REST-to-SOAP and SOAP-to-REST XSLT documents, is to set

log4j.logger.com.predic8=debug, stdout

in “conf/log4j.properties“. This turns on debug output. (You need to restart Membrane for the setting to take effect.) It allows you to see the “in-between XML document” created internally: When passing the arrow marked with a star * in the diagram, this document will be written to the log (standard output in our case).

Categories: General, Router

Easy Proxying of SOAP Web Services

August 16, 2012 tpolley Leave a comment

Proxying SOAP Web Services

To proxy a SOAP Web Service with a HTTP endpoint, two types of requests need to be handled:

Retrieval of the WSDL document, e.g. a
“GET http://www.thomas-bayer.com/axis2/services/BLZService?wsdl“
The SOAP request, e.g. a
“POST http://www.thomas-bayer.com/axis2/services/BLZService“.

The document returned by the first request, the WSDL document, is a technical description of the web service, its data structures and operations. Amongst other things, it describes the structure of the SOAP requests and their responses. The SOAP requests, the second type of requests, actually contain data, initiate actions and actually perform the tasks of the web service.

Requests of both types are forwarded by the proxy to the backend service, as by any generic HTTP proxy.

Additionally, the WSDL document needs to be slightly modified by the proxy before it is returned to the caller: It contains the SOAP endpoint’s address (“http://www.thomas-bayer.com/axis2/services/BLZService” in the example) and therefore influences the second type of requests. As we want these requests to be sent from the client to the proxy instead of directly to the backend service, we have to change this address to point to the proxy instead of the backend service:

Suppose our SOAP proxy is running on http://soapproxy.mycompany.example/. We then have to change “http://www.thomas-bayer.com/axis2/services/BLZService” to “http://soapproxy.mycompany.example/axis2/services/BLZService“.

A Concrete Example

For example, look at http://www.thomas-bayer.com/axis2/services/BLZService?wsdl. At the very bottom you will the three endpoint addresses for this web service. All of them will have to be changed by the proxy.

Action! – The Proxy Configuration

Doing all this is a piece of cake using Membrane ESB as SOAP proxy:

We download the Membrane ESB distribution (3.5 or later) from the website, and change conf/proxies.xml to

<proxies xmlns="http://membrane-soa.org/schemas/proxies/v1/"
	 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      	 xsi:schemaLocation="http://membrane-soa.org/schemas/proxies/v1/ http://membrane-soa.org/schemas/proxies/v1/proxies.xsd">

	<soapProxy port="80" wsdl="http://www.thomas-bayer.com/axis2/services/BLZService?wsdl">
	</soapProxy>
	
</proxies>

The new soapProxy element will handle requests of both types!

We then start Membrane using memrouter.bat.

Done!

Inspecting the result

From the machine itself, where you are running Membrane, you can retrieve the modified WSDL document using this URL: http://localhost/axis2/services/BLZService?wsdl
You will see that “http://www.thomas-bayer.com” was changed to “localhost“. If you use your qualified hostname instead of “localhost“, this will also work.

Of course, you can use any SOAP client, for example soapUI, to access the service via the proxy: Simply specify the WSDL location using the proxy’s address (http://localhost/axis2/services/BLZService?wsdl or use the proxy’s qualified hostname).

The Next Steps

Adding additional features to the proxy is also very simple. We can, for example, change conf/proxies.xml to

<proxies xmlns="http://membrane-soa.org/schemas/proxies/v1/"
	 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      	 xsi:schemaLocation="http://membrane-soa.org/schemas/proxies/v1/ http://membrane-soa.org/schemas/proxies/v1/proxies.xsd">

	<soapProxy port="80" wsdl="http://www.thomas-bayer.com/axis2/services/BLZService?wsdl">
		<validator />
	</soapProxy>
	
</proxies>

The change will be automatically be picked up by Membrane within a few seconds. With the validator interceptor, Membrane will perform XML Schema validation on both requests and responses, ensuring that all requests and responses adhere to the data structures and rules documented in the WSDL document and the referenced XML Schema documents. This can, for example, help you protect your web service against certain kinds of attacks.

You can find a wide range of features in Membrane’s configuration reference.

For a step-by-step tutorial, follow the SOAP quick start guide.

Categories: General

Versioning SOAP Web Services using a Content Based Router

June 19, 2012 tpolley Leave a comment

Summary

In this blog post we discuss the evolution of a web service, and how multiple versions of it using the same endpoint can be handled by configuring Membrane ESB to act as a Content Based Router.

The Web Service

Suppose we have a very simple SOAP web service providing data about our customers. To keep it simple, we decided to just track the first and last name of each customer.

The web service is deployed and clients in the field are using it, for example to add new customers:

To add a customer, a client might send a message like this one:

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:con="http://predic8.com/contactService/v11">
  <soapenv:Header/>
  <soapenv:Body>
    <con:addContact>
      <firstname>John</firstname>
      <lastname>Doe</lastname>
    </con:addContact>
  </soapenv:Body>
</soapenv:Envelope>

The structure used to describe customers was included in the WSDL:

...
  <xs:complexType name="addContact">
    <xs:sequence>
      <xs:element name="firstname" type="xs:string" minOccurs="0" />
      <xs:element name="lastname" type="xs:string" minOccurs="0" />
    </xs:sequence>
  </xs:complexType>
...

Modifying the Web Service

After the publication of the service, we noticed that we also wanted to track the email addresses of our customers.

We therefore changed the XML Schema used to describe customers to include a field “email”. A client using the new message format might now send:

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:con="http://predic8.com/contactService/v20">
  <soapenv:Header/>
  <soapenv:Body>
    <con:addContact>
      <firstname>John</firstname>
      <lastname>Doe</lastname>
      <email>john@doe.local</email>
    </con:addContact>
  </soapenv:Body>
</soapenv:Envelope>

But now, we run into a problem: Some clients out there might still be using the old version of our service:

What do we do with them?

The Core of the Question

There are several possible answers to this question.

But firstly, there are also several “dimensions”, several design aspects to this question: When we added the “email” field, we basically created a new version of the web service:

Did we change the published endpoint URL?
Did we change the SOAP namespace URI?

Whether you answer (or should answer) “yes, yes”, “yes, no” or “no, yes” is up to you, and this blog post should help you decide that. (You should not answer “no, no”: At least in the long term, this creates trouble.)

In the rest of this blog post, we discuss “no, yes”.

Versioning with a Content Based Router

As both the old clients and the new clients are using the same endpoint URL, we need to distinguish between old and new requests. In our case, as we changed the namespace, we use the namespace URL to distinguish between old and new requests.

SOAP requests using the old namespace (http://predic8.com/contactService/v11) are sent to the old server, while requests usind the new namespace (http://predic8.com/contactService/v20) are sent to the new server.

Of course, the easiest way to realize the “old server” is to implement both the old and new server interface using updated and/or new methods internally.

Membrane Setup

We now setup Membrane ESB to act as the Content Based Router (CBR).

For this, we download the distribution from the website, and change conf/proxies.xml to:

<proxies xmlns="http://membrane-soa.org/schemas/proxies/v1/"
		 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      	 xsi:schemaLocation="http://membrane-soa.org/schemas/proxies/v1/ http://membrane-soa.org/schemas/proxies/v1/proxies.xsd">

	<serviceProxy port="80">
		<switch>
			<case xPath="//*[namespace-uri()='http://predic8.com/contactService/v11']" url="http://localhost:8080/ContactService/v11" />
			<case xPath="//*[namespace-uri()='http://predic8.com/contactService/v20']" url="http://localhost:8080/ContactService/v20" />
		</switch>
	</serviceProxy>

</proxies>

Voilà!

(Starting Membrane with memrouter.bat will open port 80 and forward incoming requests to the specified URLs: We assume here that the server implementing both interfaces is running on localhost:8080 and the old (resp. new) interface is available on /ContactService/v11 (resp. /ContactService/v20).

A full working example of the versioned web service together with the CBR configuration can be found in examples/versioning/routing in the distribution archive of Membrane ESB.

Categories: General, Router Tags: EAI patterns, esb, Membrane, service design, soap, versioning

How to Remove Links to Twitter, Facebook, … Using a Proxy

June 17, 2011 sjayaratna 7 comments

Hi,

you can use the RegExReplaceInterceptor to remove links for example to Twitter and Facebook. It simply replaces parts of the response that matches a regular expression with a replacement string.

On the following site there are links and buttons to Twitter, Facebook and DZone located at the top right corner:

http://predic8.com/rest-webservices.htm

The Twitter and Facebook links use a tags while the DZone markup is generated by a script contained in a script tag. We will use the RegExReplaceInterceptor to remove those tags.

For each tag we want to remove we create a RegExReplaceInterceptor bean definition. You can do it with only one bean definition but than the pattern you use will be more complex. So we add the following bean definitions into our monitor-beans.xml document.

For Facebook:

<bean id="stripFacebook" class="com.predic8.membrane.core.interceptor.RegExReplaceInterceptor">
<property name="displayName" value="Strip Facebook"/>
<property name="pattern" value='&lt;a [^&gt;]*http://www.facebook\.com/sharer\.php.*?&lt;/a&gt;'/>
<property name="replacement" value=""/>
</bean>

For Twitter:

<bean id="stripTwitter" class="com.predic8.membrane.core.interceptor.RegExReplaceInterceptor">
<property name="displayName" value="Strip Twitter"/>
<property name="pattern" value='&lt;a [^&gt;]*http://twitter.com/share.*?&lt;/a&gt;'/>
<property name="replacement" value=""/>
</bean>

For DZone:

<bean id="stripDzone" class="com.predic8.membrane.core.interceptor.RegExReplaceInterceptor">
<property name="displayName" value="Strip DZone"/>
<property name="pattern" value="&lt;script [^&gt;]*http://widgets\.dzone\.com/links/widgets/zoneit\.js.*?&lt;/script&gt;"/>
<property name="replacement" value=""/>
</bean>

The pattern property defines the regular expression that is used. It follows standard Java RegEx Pattern syntax. Every part in the response that matches the regulare expression is replaced by the string defined by the replacement property.
We choose the patterns so that they match the a tag that points to facebook and twitter and the script tag that contains the script for DZone.

Now we need a rule configuration so that the RegExReplaceInterceptors are used. You can use the Membrane Monitor to create a rule and add the interceptors we created above to it or you can use the following rule.xml document:

<configuration>
  <rules>
    <forwarding-rule name="stripFacebookAndCo" port="2000">
      <targetport>80</targetport>
      <targethost>predic8.com</targethost>
      <interceptors>
        <interceptor id="stripDzone"/>
        <interceptor id="stripFacebook"/>
        <interceptor id="stripTwitter"/>
      </interceptors>
    </forwarding-rule>
  </rules>
</configuration>

Start the Router or Monitor with the rules.xml given above and the modifications to the monitor-beans.xml. Than open the following URL:

http://localhost:2000/rest-webservices.htm

You will see the same article as before but without the Twitter, Facebook and DZone links and buttons.

Categories: Router Tags: facebook, filter, http, interceptor, proxy, regex, router, twitter, web 2.0

Membrane Router as JBoss Service (MBean)

May 24, 2011 sjayaratna Leave a comment

I had to deploy Membrane Router at a data center but the only deployment options I had were EAR, WAR or SAR (JBoss Service) archives. So I created a JBoss services that wrapped the router. Listing 1 shows the MBean interface for the router service. There are two getter/setter pairs for the routers Spring configuration and the routing rules configuration.

package com.predic8.membrane.core;

public interface RouterServiceMBean {
    public void start() throws Exception;
    public void stop() throws Exception;
	public void setRulesXml(String rulesXml);
	public String getRulesXml();
	public void setMonitorBeansXml(String monitorBeansXml);
	public String getMonitorBeansXml();
}

RouterService implements the RouterServiceMBean. All the magic is in the start method that starts the router.

package com.predic8.membrane.core;

import org.apache.commons.logging.*;

public class RouterService implements RouterServiceMBean {

	private static Log log = LogFactory.getLog(RouterService.class.getName());

	Router router;
	
	private String monitorBeansXml="classpath:monitor-beans.xml";
	private String rulesXml="classpath:rules.xml";
	
    public void start() throws Exception {
    	router = Router.init(monitorBeansXml);
		router.getConfigurationManager().loadConfiguration(rulesXml);
        log.info("Router started");
    }

	public void stop() throws Exception {
        router.getTransport().closeAll();
        log.info("Router stopped");
    }

	public String getMonitorBeansXml() {
		return monitorBeansXml;
	}

	public void setMonitorBeansXml(String monitorBeansXml) {
		this.monitorBeansXml = monitorBeansXml;
	}

	public String getRulesXml() {
		return rulesXml;
	}

	public void setRulesXml(String rulesXml) {
		this.rulesXml = rulesXml;
	}

}

What’s left is a jboss-service.xml that configures the RouterService.

<server>
		<mbean code="com.predic8.membrane.core.RouterService" name="com.predic8:service=RouterService">
			 <attribute name="MonitorBeansXml">classpath:monitor-beans.xml</attribute>			
			 <attribute name="RulesXml">classpath:rules.xml</attribute>			
		</mbean>
</server>

At last we have to put all together with the Membrane libraries into a SAR archive that can be deployed at a JBoss server instance.

Categories: General

JSON Beautifier

May 12, 2011 tbayer Leave a comment

Membrane Monitor 2 now features a JSON beautifier that helps to debug AJAX applications and REST resources.

JSON Formatter

Categories: Monitor Tags: JSON, Membrane, Monitor, REST

Tutorial: Mapping SOAP Operations to REST Resources

May 11, 2011 sjayaratna 2 comments

REST 2 SOAP Mapping

In this Tutorial we will see how to use the REST2SOAPInterceptor. It will allow us to call a SOAP Web Service by using REST resource URIs. For example take a look at the BLZService at the following location:

http://www.thomas-bayer.com/axis2/services/BLZService?wsdl

To call the getBank operation you have to send a SOAP message like the following:

<soapenv:Envelope xmlns:soapenv=”http://schemas.xmlsoap.org/soap/envelope/”
   xmlns:blz=”http://thomas-bayer.com/blz/”>
  <soapenv:Header/>
  <soapenv:Body>
    <blz:getBank>
      <blz:blz>37050198</blz:blz>
    </blz:getBank>
  </soapenv:Body>
</soapenv:Envelope>

Now let us imagine that there is a REST resource for banks. Probably the URI to get a representation of a bank will look like this:

http://localhost:2000/bank/37050198

This mapping can be done by the REST2SOAPInterceptor without modifing the SOAP service.

REST2SOAPInterceptor

First we will take a look at how the interceptor works. It is configured by a list of mappings. Each of them contains the information to map HTTP requests to SOAP operations. This allows us to map many SOAP operations with one interceptor. Finding the right mapping for a given HTTP request is done by a regular expression. Every mapping contains a regular expression that is matched against the URL of the HTTP request. The mapping which regular expression matches first is taken. With that mapping the interceptor creates a XML document from the HTTP request.

For example for the following HTTP request

GET /bank/37050198 HTTP/1.1
Host: www.thomas-bayer.com
User-Agent: Jakarta Commons-HttpClient/3.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: de-de,de;q=0.8,en-us;q=0.5,en;q=0.3
Accept-Encoding: gzip,deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Connection: keep-alive

the interceptor will create the following XML document:

<request method="GET" http-version="1.1">
  <uri value="/bank/37050198">
    <host>www.thomas-bayer.com</host>
    <path>
      <component>bank</component>
      <component>37050198</component>
    </path>
  </uri>
  <headers>
    <header name="User-Agent">Jakarta Commons-HttpClient/3.1</header>
    <header name="Host">www.thomas-bayer.com</header>
  </headers>
</request>

Than a XSLT transformation is applied to the XML document. The transformation creates the SOAP message for the SOAP operation we want to call. Finally the transformed message is send to the endpoint of the SOAP service. After receiving the response the intercetor can by configured to apply another XSLT transformation to strip the SOAP envelope of the response. The service endpoint and the references to the XSLT stylesheets are defined in the mapping.

RESTfull BLZ Service

Create a stylesheet called blz-httpget2soap-request.xsl. It will be used by the interceptor to create the SOAP message. It should have the following content:

<xsl:stylesheet version=”1.0″ xmlns:xsl=”http://www.w3.org/1999/XSL/Transform”
  xmlns:s11=”http://schemas.xmlsoap.org/soap/envelope/”
  xmlns:blz=”http://thomas-bayer.com/blz/”>

  <xsl:template match=”/”>
    <s11:Envelope >
      <s11:Body>
        <blz:getBank>
          <blz:blz><xsl:value-of select=”//path/component[2]“/></blz:blz>
        </blz:getBank>
      </s11:Body>
    </s11:Envelope>
  </xsl:template>
</xsl:stylesheet>

Create a stylesheet called strip-soap-envelope.xslto strip the SOAP envelope from the response. It should have the following content:

<xsl:stylesheet version=”1.0″ xmlns:xsl=”http://www.w3.org/1999/XSL/Transform”
  xmlns:s11=”http://schemas.xmlsoap.org/soap/envelope/”>

  <xsl:template match=”/”>
    <xsl:apply-templates select=”//s11:Body/*”/>
  </xsl:template>

  <xsl:template match=”@*|node()”>
    <xsl:copy>
      <xsl:apply-templates select=”@*”/>
      <xsl:apply-templates/>
    </xsl:copy>
  </xsl:template>
</xsl:stylesheet>

Copy the stylesheets into the configuration folder of your monitor or router installation directory.

Open monitor-beans.xml and add a bean for the REST2SOAP interceptor to the list of interceptors in the transport bean.

<bean id="transport" class="com.predic8.membrane.core.transport.http.HttpTransport">
  [...]
  <property name="interceptors">
    <list>
      [...]
      <bean class="com.predic8.membrane.core.interceptor.rest.REST2SOAPInterceptor">
        <property name="mappings">
          <map>
            <entry key="/bank/.*">
              <map>
                <entry key="SOAPAction" value=""/>
          	<entry key="SOAPURL" value="/axis2/services/BLZService" />
          	<entry key="requestXSLT" value="configuration/blz-httpget2soap-request.xsl" />
          	<entry key="responseXSLT" value="configuration/strip-soap-envelope.xsl" />
              </map>
             </entry>
          </map>
        </property>
      </bean>
      [...]
    </list>
  </property>
  [...]
</bean>

Start the Membrane monitor.
Create a rule that listens on port 2000 and redirect requests to thomas-bayer.com:80.
Open the link http://localhost:2000/bank/37050198 in your browser.

Categories: Monitor, Router

Older Entries

Summary

Note

The Workflow

Brief Setup How-To

Closing comments

Summary

The Scenario

Adapting old Web Service Requests to the New Interface

Using XSLT

Set up Membrane ESB as CBR with XSLT

Summary

The SOAP Web Service

Translation to REST

Using Membrane’s rest2Soap Feature

Monitoring the Progression of a Sample Request

Configuring the XSLT component

Download and set up Membrane ESB

Testing it

Mapping REST to *Your* SOAP Web Service

Proxying SOAP Web Services

A Concrete Example

Action! – The Proxy Configuration

Inspecting the result

The Next Steps

Summary

The Web Service

Modifying the Web Service

The Core of the Question

Versioning with a Content Based Router

Membrane Setup

Random Posts

Tag Cloud

Categories

Blogroll

Archives

Meta

Mapping REST to Your SOAP Web Service