Friday, 3 March 2017

iPaaS. What is it exactly? is it on-premise software running on IaaS?

As cloud adoption continues to rise, the so called 'second wave' of cloud computing becomes less of a prediction and rather a reality we have to deal with. In the past 2 years or so for example, almost in every customer engagement I've had, 'the cloud' has been at the very least a topic of discussion. In most cases it has actually been within the scope of our activities.

This is not surprising of course as the term 'cloud' itself can mean ten different things to ten different people. The sad part is though, that is has been years since the first wave of cloud (started by Amazon) and there's still a fair degree of confusion in the topic.

In fact, I still often refer to the NIST definition of cloud to explain what cloud computing and PaaS actually is and how traditional on-premises middleware installed on IaaS isn't PaaS or iPaaS. This is in fact one of the main motivators of this post.

iPaaS

The term Integration Platform as a Service, or just iPaaS, is generally used when referring to integration capabilities delivered entirely on PaaS cloud infrastructure.

In terms of integration capabilities, iPaaS can deliver the same (and in many cases more) capabilities than the ones available in traditional on-premise middleware. Such capabilities should be sufficient to satisfy the main types of integration requirements:

Types of Integrations
It is perhaps because of such similarities that, in my experience, there still is a fair amount of misunderstanding of what iPaaS actually is, what it brings to the table and how it's different to traditional on-premise integration middleware.

Some for example wrongly believe that installing a traditional integration product (i.e. IBM BPM or Mule ESB) on IaaS infrastructure will make it iPaaS.

Well this is far from truth. Let me elaborate on why:

iPaaS characteristics

The following diagram is a summary of NIST definition of PaaS, it characteristics and how iPaaS relates to it:

iPaaS Characteristics
As the diagram suggests, for an integration capability (aka integration platform) to be truly iPaaS, it must comply with NIST's 'essential characteristics of cloud computing'. Following my own interpretation of such:
  1. On-demand and self-service: it should be possible, at any given point in time, to go to a cloud vendor website (i.e. https://cloud.oracle.com/en_US/integration), browse through the different iPaaS offerings, select the one wanted/needed, purchase it online, using a credit card of course, and in minutes get an email with all details of the instance (already installed and running), how to access it and even use it. No need to talk with a sales representative, negotiate license costs, provision infrastructure, install/configure the software, and so on.
  2. Broad network access: This is perhaps one of the most important characteristic. Network connectivity to/from iPaaS to other applications has to be fast, reliable and secured. Open internet connection can be unpredictable therefore the cloud vendor must provide alternative means to deliver dedicated high-speed connectivity. For example Oracle Cloud provides a service called Fast Connect, and Amazon a similar one called Direct Connect.
  3. Resource pooling: Compute resources (i.e. cpu, ram memory, disk space) should be allocated on-demand (without human-intervention) based on resource utilisation or via configuration if desired. It should be possible to increase or reduce resources either on demand and/or based on pre-configured rules.
  4. Rapid elasticity: During periods of high demand (i.e. black Friday), predicting resource usage can be almost imposable. A true iPaaS platform should be able to autoscale based on configurable rules and resource demand. Idea is that transaction peaks can be handled by scaling horizontally or vertically -without human intervention. When transaction throughput becomes stable, platform resources should reduce automatically to its original size. It should not be the case that it's possible to rapidly add more capacity, but not the other way around... 
  5. Measured service: pay-per-use / subscription-based charging model with complete transparency and visibility over usage and billing. For example, if a given iPaaS is charged based on number of connections to other applications, it should be possible to know exactly how many connections are being used -at any given point in time. If the number of connections is reaching the limit, a notification should be sent so it's possible to allocate more connections. If on the other hand, the number of connections is in average less than subscribed for, a notification should be sent suggesting to reduce the number of subscribed connections.
  6. Automatic application patching and upgrades: Although not explicitly mentioned, this is another key characteristic and possibly the one that makes iPaaS more distinctive from traditional on-premises middleware. Cloud vendors should be fully responsible for periodically applying patches and/or software upgrades to the purchased cloud infrastructure. It is the vendors responsibility to ensure that patches and upgrades applied are backward compatible and won't break any application.
Considering the aforementioned characteristics, it should hopefully be clear that iPaaS can't be simply delivered by installing traditional on-premise software on cloud infrastructure. 

Be watchful though: Some vendors might try to sell you 'the wooden bicycle' by simply rebranding their on-premises software as iPaaS when all they're doing is provisioning exactly the same on-premises software but on cloud infrastructure (i.e. Amazon EC2). This is known as Cloud Washing.

iPaaS & on-premise integration platforms co-existance

Even though iPaaS can independently satisfy cloud-to-cloud integration needs, it doesn't actually mean that on-premises middleware is no longer required. Quite the contrary.  Most organisations have already made considerable investments in on-premises integration capabilities. In such organisations, even the thought of replacing such capabilities will raise a few eyebrows. 

Instead, on-premise capabilities can continue to be leveraged not only to satisfy on-premises only integration use cases, but also to co-exist with an iPaaS solution to satisfy cloud to/from on-premises integration requirements.

The following diagram illustrates how typical integration patterns can be satisfied with a combination of iPaaS and on-premises integration capabilities.

Hybrid Integration Patterns
Another thing to bear in mind though, given that most software vendors are moving towards a cloud-first delivery model, new products, features, capabilities and even bug-fixes will be made available first to cloud applications and then (if so) to on-premise software.

Therefore in scenarios where for example a new integration capability (i.e. API management) is required which is not already available on-premises, it makes complete sense to consider the adoption of an iPaaS solution instead. Specially those that deliver flexible deployment models whereby runtime engines can be deployed on any infrastructure (cloud and/or on-premises) whilst the management console remains on the cloud.

Conclusion

The adoption of cloud (in all of its flavours: SaaS, PaaS or IaaS) combined with need for organisations to become more digital and user centric, mean that data is not only becoming more and more federated but accessing it in real time and from virtually anywhere is an absolute must. 

However there is no need to 'boil the ocean'. Organisations should continue to leverage their existing integration investments but in parallel define new integration strategies that identify what new capabilities are or will be needed in order to satisfy emerging integration requirements resulting from the adoption of cloud computing and digital transformation.

When possible iPaaS capabilities should be considered not only because they provide more cost-effective licensing models, but also because capability wise, they are better equipped to handle modern and emerging integration requirements. iPaaS also requires less effort to install, configure and run -as most of the work is done by the cloud vendor, including on-going patching and upgrades.

Lastly, avoid cloud washing by ensuring that the selected iPaaS platform satisfies all NIST's 'essential characteristics of cloud computing'. Read my previous blog for a nice comparison of different iPaaS vendors.

Wednesday, 24 August 2016

Comparing Oracle ICS connectors with Workday, Mule, Boomi and Azure

As SaaS adoption continue to increase in organisations of any size, it's only expected that different cloud vendors will stretch their cloud capabilities to try and increase their SaaS/PaaS/IaaS footprints in a clients landscape. This is particularly true for iPaaS related capabilities, as it seems that every cloud vendor has its own related offering and they are pushing it really hard to customers even if there really isn't a one to one fit.

The challenge is though, that organisations that don't carefully elaborate a cloud integration strategy and properly think this through, will almost certainly end up implementing point solutions using whichever iPaaS capability is available for the individual project.  This not only results in vendor lock-in but also increases the complexity and cost of integration.



To avoid this, the first step is to of course create a carefully thought cloud integration strategy with a clear objective in mind. This should be delivering a platform capable of supporting all integration needs (cloud-to-cloud, cloud-to-on-premise) in a seamless and consistent fashion without redundant infrastructures and capabilities. The concept should be something like this:



However the devil is in the detail, or so it goes.

One of the key reasons vendor lock-in occurs is simply because of misinformation. From my point of view, SaaS integration is all about connectivity. So without properly understanding 1) what are the connectivity requirements and 2) what platform best fits these connectivity needs in the short and long term, there is high probability that a solution will not be the right one (may be it will solve the short-term needs, but in the long-run it will probably add more complexity, cost hence risk).

The following table compares Oracle's Integration Cloud Service (ICS) connectors to other major iPaaS vendors (or at least the ones I've come across recently). The table is specially useful if you're in an Oracle SaaS adoption project and want to explain/justify why using ICS will make life easier.


# Connections Oracle Integration Cloud Workday Integration Cloud Mulesoft Anypoint Dell Boomi Microsoft Azure Connectors
1 Adobe eSign (in Using the Adobe eSign Adapter) Y N Y N N
2 Advanced Queuing (AQ) (in Using the Advanced Queuing (AQ) Adapter) Y N N N N
3 Concur (in Using the Concur Adapter) Y N Y N N
4 DocuSign (in Using the DocuSign Adapter) Y N Y N N
5 Email (SMTP, POP3 or IMAP) Y N Y Y Y
6 Eventbrite (in Using the Eventbrite Adapter) Y N N N N
7 Evernote (in Using the Evernote Adapter) Y N N N N
8 Facebook (in Using the Facebook Adapter) Y N Y N N
9 File (in Using the File Adapter) Y N Y N Y
10 FTP (in Using the FTP Adapter) Y N Y Y Y
11 Gmail (in Using the Gmail Adapter) Y N N N N
12 Google Calendar (in Using the Google Calendar Adapter) Y N Y Y N
13 Google Task (in Using the Google Task Adapter) Y N Y Y N
14 Integration Cloud Service Messaging Y N N N N
15 JMS (in Using the JMS Adapter) Y N Y Y N
16 LinkedIn (in Using the LinkedIn Adapter) Y N Y N N
17 MailChimp (in Using the MailChimp Adapter) Y N N Y N
18 Microsoft Calendar (in Using the Microsoft Calendar Adapter) Y N Y N Y
19 Microsoft Contact (in Using the Microsoft Contact Adapter) Y N Y N Y
20 Microsoft Email (in Using the Microsoft Email Adapter) Y N Y N Y
21 Microsoft SQL Server (in Using the Microsoft SQL Server Adapter) Y N Y Y Y
22 MS Dynamics (via REST or Native) Y N Y N Y
23 MySQL (in Using the MySQL Adapter) Y N Y N N
24 NetSuite Adapter Capabilities Y N Y Y N
25 On-premise integration agent  (not just an ESB onpremise) Y N N Y N
26 Oracle Commerce Cloud (in Using the Oracle Commerce Adapter) Y N N N N
27 Oracle CPQ Cloud Capabilities Y N N N N
28 Oracle Database (in Using the Oracle Database Adapter) Y N Y Y Y
29 Oracle E-Business Suite (in Using the Oracle E-Business Suite Adapter) Y N Y N N
30 Oracle Eloqua Cloud Y N N N N
31 Oracle ERP Cloud Capabilities Y N Y N N
32 Oracle Field Service (in Using the Oracle Field Service Adapter) Y N N N N
33 Oracle HCM Cloud (excluding Taleo) Y N N N N
34 Oracle JD Edwards EnterpriseOne (in Using the JD Edwards EnterpriseOne Adapter) Y N Y N N
35 Oracle Messaging Cloud Service Y N N N N
36 Oracle RightNow Cloud Y N Y N N
37 Oracle Sales Cloud Y N N N N
38 Oracle Siebel (in Using the Oracle Siebel Adapter) Y N Y N N
39 Oracle Taleo Y N Y Y N
40 Responsys (in Using the Responsys Adapter) Y N N N N
41 REST Adapter Capabilities (not just HTTP) Y Y Y N N
42 Salesforce Y Y Y Y Y
43 SAP (in Using the SAP Adapter) Y N Y Y Y
44 SAP Ariba (in Using the SAP Ariba Adapter) Y N Y Y N
45 SOAP Adapter Capabilities Y Y Y Y Y
46 SurveyMonkey (in Using the SurveyMonkey Adapter) Y N N N N
47 Twilio (in Using the Twilio Adapter) Y N N Y Y
48 Twitter (in Using the Twitter Adapter) Y N Y N Y
49 Workday (via REST or Native) Y Y Y N N

Below the sources of information I used in the comparison:

Saturday, 4 June 2016

A Microservice Approach for Legacy Modernisation

Very large portion of the world’s business critical systems are considered to be ‘legacy’ –and so is the code underpinning them (ie COBOL, PASCAL, C, to name a few).  Although in many cases it is the case that these systems are robust, stable and fit for the main purpose they were originally built, they aren’t flexible and scalable enough to support emerging requirements mainly derived from a more demanding ‘always on the move’ and ‘always connected’ user.

These systems struggle to meet these demands mainly because of the ‘monolithic’ approach on which they were built and the complexity hidden in millions of lines of code that is only understood a very few hand-full of people that still remain active from the teams that several years ago developed these systems.

In almost an equal amount there have also been thousands of failed attempts to modernise these legacy systems. The ‘eating the elephant’ in one go approach certainly didn’t work, and the traditional SOA approach alone although worked till certain extend, it also fell short when it came down to addressing specific requirements around scalability and platform/service inter-dependencies.

In this presentation I talk about how a legacy modernisation framework based on Microservice Architecture (MSA) in conjunction with some other known SOA patterns (ie. ESB, API Gateway), can be applied to ‘eat the elephant one piece at the time’ but most importantly ‘without getting indigestion’.



I did this presentation at the AMIS Beyond the Horizon conference that took place near Amsterdam in June 2-3 2016. Thank you AMIS and Oracle ACE Director Lucas Jellema for the opportunity to present in what was a very unique and wonderful conference full of great people, knowledge and fun!

Friday, 13 May 2016

Converting ADLs to implement end to end JSON in SOA Suite 12.2.1 -PART I

There is no doubt that web [Rest] APIs have become extremely popular and its usage has gone well beyond just building APIs in support of mobile apps. We can see the adoption of resource-oriented architectures (ROA) by probably all SaaS vendors who provide out-of-the-box APIs as the means to connect and interact with their cloud applications. Take for example the Oracle Cloud. To discover and consume publicly available Oracle SaaS APIs, all one need to do is browse the Oracle API Catalog Cloud Service (which is publicly accessible) and just select the Swagger definition for any given API.


But (as you probably already know) the adoption of web APIs hasn't stopped there.  With the increased popularity of Microservice Architectures , initiatives such as Open Legacy ,  and node.js based frameworks like loopback and sails (to name a few), API-enabling system of records is becoming a lot easier.

This is putting a lot of pressure in software vendors to quickly modernise their integration suites to natively support the technology-stacks and patterns prevalent in these type of architectures. For example, if an organisations mobile application needs to interact with a system of record (on premise or the cloud) that already exposes a web API, the integration stack should be capable of supporting JSON over HTTP end-to-end without having to convert to XML back and forth. Not only is this impractical but introduces more processing burden to the core stack...

Luckily for many Oracle's customers and Oracle Fusion Middleware / Oracle PaaS practitioners like myself, with the latest release of Oracle SOA Suite (12.2.1) , one of the many new features introduced is the support for handing JSON end-to-end.  I don't want to understate the importance of this as with such feature it is possible to use BPEL for example to orchestrate several APIs (all in native JSON and also in-memory with the new SOA in-memory feature) and therefore deliver coarse grained business APIs that actually perform.

For me this represents an important milestone for Oracle SOA Suite as it shows the departure from traditional SOA tech-stack and into SOA 2.0 (as I like to call it) as the suite is now better suited to support the adoption of ROA, microservices, IoT, and so on. Having worked with SOA Suite since 10.1.3.1 this is very exiting.

The catch

Unlike traditional SOAP services that make use of web service description language (WSDL) as the main mechanism to define a service interface (yes one can argue that there are different types like doc-literal, rpc-encoded and versions like 1.1, 2.0, broadly speaking they are similar and there is vast support for all of them), web APIs don't enjoy this mono-linguistic nature, in fact quite the opposite.

There is at least 8 different API description languages (ADL's) that am aware of. Some more popular than others but still in general all with substantial communities and vendor support:


Note: Suggest you go through this presentation for a good overview of the main ADL's

Oracle on the other hand didn't opt for endorsing or fully adopting any particular ADL. Instead it seems that conway's law came into action and different products currently support different ADLs.


Although I am sure that this inconsistency will go away in future releases, for us practitioners this sort of represents an immediate challenge. Even though all of the aforementioned do support REST/JSON in its purest sense, not being able to interpret an ADL automatically using an adapter basically means that one has to manually configure the URIs, resources, methods and parameters in a REST adapter in order to successfully consume the API.

Let's take for example the following Swagger ADL for the Oracle Field Service Cloud activities API (for which there isn't yet an out-of-the-box adapter in SOA Suite or ICS) :

https://apicatalog.oraclecloud.com/v1/services/oracle-public/ofsc-core/v1/apis/activities/canonical

If you clicked in the above you'll notice that tons of resources and methods are available (82 methods and 13 resources approximately). So imagine having to manually enter all of these in JDeveloper to configure the SOA Suite 12.2.1 REST adapter.. Yes possible but by any means practical. Far from it.

The solution

There is however a work around to this and its called API-Transformer


Developed by APIMATIC , API transformer is an online tool for converting ADL's and it supports the most popular formats (including all of the aforementioned ADLs):

Best of all is the fact that API transformer is also an API on its own right. Meaning that one could use this public API to build some sort of automation to facilitate the conversion between the Oracle supported formats.


Conclusions

APIs are great but the technologies and standards around it are still evolving. However the industry doesn't seem intimidated by this as the adoption of APIs continue to increase exponentially. Meaning that we practitioners are left having to deal with the complexity of adopting evolving paradigms without jeopardising the benefits that can be gained from their adoption. Luckily there are tons of open solutions out there that can be used to work around some of these complexities and API transformer is a good example of such.

In part II of this article I will show step by step how to consume an API defined in Swagger 2.0 from SOA Suite 12.2.1. For this I will transform the Swagger ADL to WADL, import it into a composite application using JDeveloper and the REST adapter and then implement end to end JSON to expose a generic API.